retoor/ad
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4268b1ec9e3c4e447da9e91ea6d677bc044d78b2
ad/molohttp_deploy.md
T

9.5 KiB
Raw Blame History

molohttp deploy and operations

Production server molodetz. molohttp 0.9.1. Facts below were read from the live system on 2026-10-05. This is a production box: follow the backup specification before every change.

Access

Operator account is retoor. Root login is never needed.

  • retoor is in the sudo group with (ALL) NOPASSWD: ALL (verified with sudo -n true and sudo -n -l).
  • Every privileged command in this guide runs under sudo and also works non-interactively in scripts.
  • /etc/molohttp is molohttp:molohttp 750: reads and edits both require sudo.

How it works

molohttp is the single front door on ports 80 and 443. One systemd unit runs one process as user molohttp (24+ days uptime at time of writing).

  • Binary: /usr/local/bin/molohttp (root:root 755).
  • Unit: /etc/systemd/system/molohttp.service (root:root 644), enabled and active.
  • Start: molohttp run --config /etc/molohttp/config.json --env /etc/molohttp/.env.json, working directory /etc/molohttp.
  • Reload: ExecReload=/bin/kill -HUP $MAINPID, so sudo systemctl reload molohttp applies config without dropping connections. There is no config-validate subcommand; validate JSON by hand (see change procedure).
  • Privileged ports are bound via CAP_NET_BIND_SERVICE; the process itself is unprivileged with ProtectSystem=strict and ProtectHome=true.
  • config.json holds 94 sites, all enabled, all reverse_proxy to http://127.0.0.1:<port> with one exception: a single upstream on http://pravda.education:10500. One site carries a basic-auth middleware. site-094 is staging.app.molodetz.nl, the Molodetz docker dev server (see Staging below).
  • TLS: minimum TLS 1.2, HSTS one year with subdomains. Certificates live per hostname as /etc/molohttp/certs/<host>.pem plus <host>.key.
  • ACME: production Let's Encrypt, account retoor@molodetz.nl, renewal check every 12 hours starting 30 days before expiry, state in /var/lib/molohttp/acme.
  • Logging: JSON to stdout, collected by journald (SyslogIdentifier=molohttp). The access_log_path/error_log_path settings are not materialized as live files.
  • Admin: the admin UI is deliberately not exposed. admin.molodetz.nl is not hosted, routed, or resolvable: no site entry, no DNS record. Exposing it would be a security flaw. The only admin path is the local CLI: molohttp admin user-list, molohttp admin hash. molohttp install/uninstall must run as root and are out of scope for routine work.

Files and owners

Canonical state. Verify with sudo find /etc/molohttp /var/lib/molohttp /var/log/molohttp -maxdepth 2 -exec stat -c '%U:%G %a %n' {} +.

Path Owner Mode Notes
/etc/molohttp molohttp:molohttp 750 config root, sudo to enter
/etc/molohttp/config.json molohttp:molohttp 640 site table, exact path read
/etc/molohttp/.env.json molohttp:molohttp 640 global settings, holds password hash
/etc/molohttp/certs molohttp:molohttp 700 per-host pairs
/etc/molohttp/certs/*.key molohttp:molohttp 600 private keys
/etc/molohttp/certs/*.pem molohttp:molohttp 644 certificates
/etc/molohttp/*.bak-* root:root 640 operator backups, this spec only
/var/lib/molohttp molohttp:molohttp 750 state root
/var/lib/molohttp/acme molohttp:molohttp 750 live ACME state
/var/log/molohttp molohttp:molohttp 750 empty dir, daemon logs to journal
/var/www/.well-known molohttp:molohttp 750 ACME challenges
/etc/systemd/system/molohttp.service root:root 644 unit file
/usr/local/bin/molohttp root:root 755 binary

Known anomalies, observed 2026-10-05, left untouched pending a decision:

  • Six key pairs under certs/ are owned caddy:caddy (the pravda.education hosts plus mail.molodetz.nl). The .key files at mode 600 are unreadable by the molohttp user; confirm those hosts serve TLS correctly, then normalize with sudo chown molohttp:molohttp on the pair.
  • /var/lib/molohttp/analytics plus /var/lib/molohttp/stats (caddy:caddy, stale since Jul 27) are leftovers from a previous server. Nothing reads them. (A stale 2.5 GB caddy-owned access.log in /var/log/molohttp was already removed on 2026-10-05 after confirming nothing held it open.)
  • Older backups in /etc/molohttp use four different naming schemes and mixed owners. They are harmless history; all new backups follow the one format below.

Backup specification

Always back up before editing. One format, no variants:

<file>.bak-<reason>-<YYYYMMDDHHMMSS>, owned root:root, mode 640, next to the original. Siblings are safe: the daemon reads config.json and .env.json by exact path only, never by glob.

FILE=/etc/molohttp/config.json
REASON=site-add-example
STAMP=$(date +%Y%m%d%H%M%S)
sudo cp -p "$FILE" "$FILE.bak-$REASON-$STAMP"
sudo chown root:root "$FILE.bak-$REASON-$STAMP"
sudo chmod 640 "$FILE.bak-$REASON-$STAMP"
ls -la "$FILE"*

Rules: reason is a required lowercase slug (site-add-<host>, site-remove-<host>, upstream-<host>, tls-<host>, env-<what>). Never delete old backups; at about 52 KB each they are the audit trail. Never invent a second format.

Change procedure

  1. Back up per the specification above.
  2. Edit with sudo (for example sudo -e /etc/molohttp/config.json).
  3. Restore ownership, since root-written files lock the daemon out: sudo chown molohttp:molohttp /etc/molohttp/config.json, sudo chmod 640 /etc/molohttp/config.json.
  4. Validate: sudo -n cat /etc/molohttp/config.json | python3 -m json.tool > /dev/null.
  5. Apply: sudo systemctl reload molohttp.
  6. Verify: systemctl is-active molohttp, curl -s -o /dev/null -w "%{http_code}\n" https://<host>/, and sudo journalctl -u molohttp --no-pager -n 20.
  7. Roll back on any failure: sudo cp -p <backup> /etc/molohttp/config.json, sudo chown molohttp:molohttp /etc/molohttp/config.json, sudo chmod 640 /etc/molohttp/config.json, then reload again and re-verify.

Use restart instead of reload only when the process itself must be replaced. Never stop without start on this box.

Add or remove a site

Append one object to the sites array. Keep id unique (site-094, site-095, ...); name equals the primary hostname.

{
  "id": "site-094",
  "name": "new.molodetz.nl",
  "enabled": true,
  "matchers": {"hosts": ["new.molodetz.nl"], "headers": {}, "paths": []},
  "handler": {
    "type": "reverse_proxy",
    "upstreams": [{"url": "http://127.0.0.1:8XXX", "weight": 1}],
    "load_balancing": "round_robin",
    "websocket": false,
    "connection_timeout_ms": 5000,
    "read_timeout_ms": 30000
  }
}

Pick a free localhost port for the upstream and keep the app itself bound to 127.0.0.1. Certificates are provisioned automatically through ACME within seconds of the first HTTPS traffic; verify with curl and openssl s_client -connect <host>:443 -servername <host>, and check the journal on any TLS error. Per-host files under certs/ may materialize later on molohttp's own schedule (observed with staging: valid certificate served immediately, files not yet present), so their absence right after creation is not a failure. Removal is the reverse: back up, delete the object, validate, reload, verify the hostname no longer routes.

Staging

staging.app.molodetz.nl (site-094) proxies to http://127.0.0.1:19847, the Molodetz docker dev server from this repo (see Docker in README.md). It carries a normal Let's Encrypt certificate. Every saved change under ./molodetz is live on staging within seconds through uvicorn reload; app edits never need a molohttp change. Staging data is the file mount /var/lib/molodetz-staging, refreshed from holy production with make staging-refresh (see bin/staging-refresh.sh); production data is the file mount /var/lib/molodetz. Named volumes are never used.

Static publishing

static.molodetz.nl proxies to http://127.0.0.1:8117, served by rserver running as retoor with working directory /home/retoor/projects/static. Publishing static content needs no molohttp change: write files under that directory and they are live (verified end to end with /pizdetz/, which redirects to /pizdetz/index.html). The Molodetz screenshot gallery is produced this way; see the Screenshots section in README.md.

Ownership ensure

Run after any manual intervention to restore the canonical state. It does not touch the documented anomalies.

sudo chown molohttp:molohttp /etc/molohttp /etc/molohttp/config.json /etc/molohttp/.env.json
sudo chmod 750 /etc/molohttp
sudo chmod 640 /etc/molohttp/config.json /etc/molohttp/.env.json
sudo chown molohttp:molohttp /etc/molohttp/certs /var/lib/molohttp /var/log/molohttp /var/www/.well-known
sudo chmod 700 /etc/molohttp/certs
sudo chmod 750 /var/lib/molohttp /var/log/molohttp /var/www/.well-known
sudo find /etc/molohttp/certs -name '*.key' -user molohttp -exec chmod 600 {} +
sudo find /etc/molohttp/certs -name '*.pem' -user molohttp -exec chmod 644 {} +
sudo find /etc/molohttp -maxdepth 1 -name '*.bak-*' -exec chown root:root {} + -exec chmod 640 {} +

Quick commands

Task Command
Status systemctl is-active molohttp, molohttp service-status
Logs sudo journalctl -u molohttp -f, molohttp service-logs
Reload config sudo systemctl reload molohttp
Restart sudo systemctl restart molohttp
Site table sudo -n cat /etc/molohttp/config.json | python3 -m json.tool | less
Port owners sudo -n ss -tlnp
Version molohttp --version
Reference in New Issue View Git Blame Copy Permalink