This file documents the Container manager subsystem (devplacepy/services/containers/, plus its routers/projects/containers and routers/admin/containers.py HTTP surface, and the shared ppy Docker image). Claude Code loads it automatically whenever a file under devplacepy/services/containers/ is read or edited.

## Overview

Admin-only. Supervises container instances, all running one shared prebuilt image. Driven through the `docker` CLI via `asyncio.create_subprocess_exec` behind a pluggable **`Backend` ABC**, never a docker SDK. Reference: admin docs `Services -> Container Manager` and the `containers` API group.

## Backend seam

`backend/base.py` defines the `Backend` ABC. `DockerCliBackend` drives `docker` via `asyncio.create_subprocess_exec`, streaming run logs line-by-line (`_stream`). `FakeBackend` is the in-memory test double (its `image_exists` returns `True`). `runtime.get_backend()`/`set_backend(b)` selects the active backend - tests call `set_backend(FakeBackend())`. A future Kubernetes/remote backend implements the same ABC. The container OS workspace mount point is the single constant `backend/base.py` `WORKSPACE_MOUNT` (`/app`).

## Security (load-bearing)

Mounting the docker socket is root-equivalent on the host. Every run/exec/lifecycle/schedule operation is gated `require_admin` (HTTP) and `requires_admin=True` (Devii). `--privileged` is never passed. User input is never shell-interpolated - every call is an arg-list subprocess, never `shell=True`. Resource limits are applied via `--cpus`/`--memory`.

On top of the admin gate, **per-user container isolation** applies (see root CLAUDE.md "Project visibility (is_private) and read-only" convention): only the **primary administrator** (earliest-created Admin) and the instance owner (creator or project owner) may manage an instance; other admins get read-only access, and only when the instance's project is public. Enforced via `content.py` predicates `owns_instance`, `can_view_project_containers`, `can_view_instance`, `can_manage_instance` (all reusing `is_primary_admin`): the primary administrator sees and manages every container including those on private projects; any other admin can VIEW containers of others only when the instance's project is public (private-project containers of others are invisible; of private containers they see only their own), and can MANAGE (edit/lifecycle/exec/terminal/sync/delete/schedules) only instances they own - non-owners get 403 plus an audit row with `result="denied"` (WS close `1008`). This is enforced at every entrypoint: `routers/projects/containers/` (`project_for` + `manage_guard` + the exec WS), the admin Containers manager (`_decorate` filtering + per-row `can_manage`, `_viewable_instance_or_404`, `_manage_denied`, create + project-search), the Devii container tools (`ContainerController._project` + `_require_manage`), and the live view relay (broadcast topics never carry private-project instances).

## One shared image, no in-app builds (load-bearing)

Every instance runs `config.CONTAINER_IMAGE` (default `ppy:latest`, override `DEVPLACE_CONTAINER_IMAGE`), built ONCE from `ppy.Dockerfile` via `make ppy` (context `devplacepy/services/containers/files`). There are no per-project Dockerfiles, builds, `ContainerBuildService`, or build UI - all removed.

`service._launch` calls `api.run_spec_for(inst, config.CONTAINER_IMAGE)`. `api.create_instance(project, *, name, ...)` fails fast with `ContainerError` if `backend.image_exists(CONTAINER_IMAGE)` is `false` ("run 'make ppy'"); it no longer takes a dockerfile/build. `backend.image_exists(ref)` (`docker image inspect`) is part of the `Backend` ABC.

Migrating off the old per-project-build model: `devplace containers prune-builds` removes the legacy per-project images (`backend.remove_image`) and clears the `dockerfiles`/`dockerfile_versions`/`builds` tables (one-time); existing instances auto-repoint to `ppy` (their stored `build_uid` is ignored).

**Default idle:** `run_spec_for` runs `["sleep", "infinity"]` when an instance has no `boot_command`, so a bare instance stays up instead of exiting (the image `CMD` is the same, but the explicit command makes it independent of the image).

## Data model

`store.py`, indexed in `init_db`:
- `instances`
- `instance_events` (audit trail; also home of the status-history rows written by `_set_status`)
- `instance_metrics` (ring buffer, `METRICS_RING=720`; sweep keeps it bounded)
- `instance_schedules` (cron/interval/once)

## Reconciler (`ContainerService`, `service.py`)

A `BaseService` reconciler; the model is desired-vs-actual, NOT a task per container. Each tick:
1. `backend.ps(label=devplace.instance)` snapshots `docker ps` (label `devplace.instance=<uid>` is the join key).
2. Converge each instance to its `desired_state`.
3. Apply restart policies.
4. Reap orphan containers (labeled but no DB row -> no orphans, no lost state).
5. Fire due `instance_schedules` (reusing `devii/tasks/schedule.py` `cron_next`/`next_run`).
6. Sample `docker stats`.

Only the service lock owner reconciles; HTTP handlers only flip `desired_state` / write events. `docker run --name <slug>` collision is the double-launch guard.

## Workspace materialization

Workspace = `/app` (the `WORKSPACE_MOUNT` constant). `project_files.export_to_dir` materializes the project to `config.CONTAINER_WORKSPACES_DIR/<project>` (`DATA_DIR/container_workspaces`, default `data/`) once, bind-mounted RW; `project_files.import_from_dir` (the inverse) syncs it back on the sync action. Both run via `asyncio.to_thread`.

**Runtime data must live OUTSIDE the `devplacepy/` package and NOT under `/static`**: workspaces (and zips) live in `config.DATA_DIR` (configurable via `DEVPLACE_DATA_DIR`). Never put generated data under `STATIC_DIR` - it is served publicly AND watched by dev `--reload` (scoped to `--reload-dir devplacepy`). The docker daemon must be able to bind-mount `DATA_DIR` for `/app`.

Every exec passes `-w /app` explicitly (`DockerCliBackend.exec` and the PTY exec in `routers/projects/containers/instances.py`), so one-shot `container_exec`, the interactive shell, and tmux sessions all start in the project workspace. The agent is told this (system prompt + `container_exec` tool summary) so it never prefixes a command with `cd /app`.

## Shared operations

`api.py` holds the operations shared by `routers/projects/containers/instances.py` (and the rest of the `routers/projects/containers/` subpackage) and the Devii `ContainerController` (`services/devii/container/`, `handler="container"`, 7 instance tools). The reconciler service defaults **disabled** (needs docker); enable it on `/admin/services`.

## HTTP routing surface

- `/projects/{slug}/containers` - the `routers/projects/containers/` subpackage: `instances.py` (creation/lifecycle/exec/logs/metrics/sync plus the exec websocket), `schedules.py` (cron/interval/once schedules), shared helpers in `_shared.py`. Every instance runs the shared `ppy` image. Discoverable from the project detail page's admin-only **Containers** button (gated by `content.can_view_project_containers` via the `viewer_can_containers` context flag) and from the admin index.
- `/admin/containers` - `routers/admin/containers.py`: lists every instance across all projects with inline actions (start/stop/restart/terminal/edit/delete) plus a create modal (project search-select, run-as user search-select, boot language + source editor, restart policy, start-on-boot, env/ports/limits/ingress); `/admin/containers/{uid}` is the per-instance detail page (lifecycle, live logs/metrics, PTY terminal, schedules, ingress, sync, status history) and `/admin/containers/{uid}/edit` edits run-as user, boot language/script/command, restart policy, start-on-boot, and limits.

**Key reuse rule:** the admin Containers section adds NO lifecycle/logs/exec endpoints of its own - the instance carries its `project_uid`, so the admin detail route resolves the project and its frontend targets the existing `/projects/{slug}/containers/instances/{uid}/...` routes. Add any new instance operation to `routers/projects/containers/instances.py` only; the admin detail page picks it up for free.

## UI - two surfaces over one API set

Both are discoverable (an earlier version of the per-project page had zero links to it - fixed).

1. **Per-project manager**: `templates/containers.html` + `static/js/ContainerManager.js` handles instance creation through an app modal form (the `_macros.html` `modal()` macro + `ModalManager` `.visible` toggle); its instance list links out to the shared detail page. Reached from the project detail page's Containers button, and passes breadcrumbs so content clears the fixed nav.
2. **Admin Containers section**: `routers/admin/containers.py` (mounted `/admin/containers`, sidebar link in `admin_base.html`, `admin_section="containers"`). `GET /admin/containers` lists every instance via `store.all_instances()` (decorated with project title/slug from one `projects` lookup) in an `.admin-table`. `GET /admin/containers/data` is the poll JSON. `GET /admin/containers/{uid}` renders `templates/containers_instance.html` + `static/js/ContainerInstance.js` - a dedicated detail page (lifecycle, poll logs/metrics, schedules add/delete, ingress, sync, interactive exec over a PTY WebSocket gated on the lock owner).

All container CSS (`static/css/containers.css`) uses app design tokens (`--bg-card`, `--text-primary`, `--success`/`--danger`/`--warning`, `--radius`) and the shared `.card` recipe.

## Admin CRUD manager (`/admin/containers`)

`/admin/containers` is a full manager, not a read-only list. `routers/admin/containers.py` adds (all `require_admin`, all reuse `api.*` directly - no docker/exec backend duplicated):
- `POST /create` (`ContainerAdminCreateForm` -> `api.create_instance`, project search-select + run-as user + boot fields + restart policy + start_on_boot + env/ports/limits/ingress).
- `GET`/`POST /{uid}/edit` (`ContainerEditForm` -> `api.update_instance_config`, edits `run_as_uid`/`boot_language`/`boot_script`/`boot_command`/`restart_policy`/`start_on_boot`/`cpu_limit`/`mem_limit`).
- Stacked literal lifecycle routes `POST /{uid}/{start,stop,restart,pause,resume}` (the no-wildcard rule below).
- `POST /{uid}/sync` (bidirectional).
- `POST /{uid}/delete` (soft, owner-or-admin via the admin guard).
- Two JSON search endpoints `GET /{projects,users}/search` (back the create/edit search-selects via `db.query LIKE`).

**Route order:** `/{uid}/edit` and the literal verb routes are declared BEFORE the `/{uid}` catch-all.

Frontend: `static/js/ContainerList.js` (poll-render rows with inline action buttons + the create modal with two debounced search-select widgets and a boot-language toggle) and `static/js/ContainerEdit.js` (the edit page). Both route through `Http.send`/`Http.getJson` and surface errors via `app.toast`; the Terminal button reuses `app.containerTerminals.open(slug, uid, name)`. The instance detail page (`containers_instance.html`) has a config summary (boot language, run-as uid, start-on-boot) plus a **Status history** card rendering the server-side `instance_events`.

Devii: `container_create_instance` takes the boot/run-as/start_on_boot args; `container_configure_instance` (`controller._configure_instance` -> `api.update_instance_config`) edits the same fields; both admin-only, audited via `_DEVII_CONTAINER_EVENTS`.

## No wildcard verb route (load-bearing)

`routers/projects/containers/instances.py` registers every instance verb as a **literal** route - `start`/`stop`/`pause`/`resume`/`restart` (stacked `@router.post` decorators on one `instance_action` handler that reads the verb from `request.url.path`), plus `delete`/`exec`/`sync`/`schedules`. There is deliberately NO `POST /instances/{uid}/{action}` catch-all: a wildcard segment matches its literal siblings too (Starlette matches first-declared), so a catch-all would silently swallow `exec`/`sync`/`delete`/`schedules` as `{"error": "unknown action: exec"}` whenever it sat above them. Add any new instance verb as a literal route (and to the `instance_action` decorator stack if it just flips desired state) - never reintroduce a `{action}` wildcard.

## Host port allocation

Host ports are auto-allocated and globally unique. `parse_ports` accepts a bare container port (`8899`, host side `0` = auto) or a pinned `host:container`. `api.assign_host_ports` (called in `create_instance`) resolves every `host=0` to the lowest free port in `[HOST_PORT_MIN=20001, 65535]` that is neither in `used_host_ports()` (the union of every instance's published host ports from `ports_json`) nor currently bound on the host (`_host_port_free` test-binds `0.0.0.0:port`). A pinned host port already published by another instance is rejected up front. This guarantees no two instances ever collide on a host port - previously a silent `docker run` "port is already allocated" failure. The resolved host port is what gets stored in `ports_json` and what the ingress proxy reads.

## Floating terminals (interactive shell)

An instance's shell is NOT inline - it is a floating `<container-terminal>` (`static/js/components/ContainerTerminal.js`) over the exec WS (`/projects/{slug}/containers/instances/{uid}/exec/ws`, `pty.openpty()` + `docker exec -it`, admin + `service_manager.owns_lock()` gated). xterm.js + the fit addon are vendored under `static/vendor/xterm/` and lazy-loaded. Opened by the instance-page **Open terminal** button (`app.containerTerminals.open(slug, uid, name)`, `ContainerTerminalManager`) or by telling Devii to "attach &lt;container&gt;" - the admin-only `open_terminal` **client** action (`client_actions.py`, `requires_admin=True`) -> `DeviiClient._openTerminal` -> the same manager.

**`FloatingWindow` base.** It extends the generic `FloatingWindow` base (`static/js/components/FloatingWindow.js` + `static/css/floating-window.css`), which owns the Devii-style chrome: drag, native `resize:both`, maximize/fullscreen, geometry persistence. `app.windows` (`WindowManager`, `components/WindowManager.js`) assigns z-index on `pointerdown` so the last-clicked window is on top (below the avatar/toasts).

**The Devii terminal also extends `FloatingWindow`.** `devii/devii-terminal.js` reuses the base drag/geometry/preset/resize/persist plus `_window`/`_registerWindow` (z-order + context-menu) machinery, overriding only the parts that differ - `_setState` (Devii adds a `closed` state + launcher FAB), `_defaultGeometry` (bottom-right, 760x600), `_changeFont` (per-user `--devii-font-size` CSS var), `_contextItems`, and `_persistGeometry` (delegates to its richer `{state,geometry,focused}` blob under `devii-terminal-state`). Devii keeps native `resize:both` (no `.fw-resize` grip); `devii.css` is unchanged - the only base change required was a `get _dragClass()` getter (default `fw-dragging`) that Devii overrides to `devii-dragging`, so the base drag handlers toggle Devii's own class and its existing CSS drives everything.

**Resize protocol.** The exec WS treats a brace-leading JSON frame `{"type":"resize","cols","rows"}` as a control message: it `TIOCSWINSZ`-es the host pty (`fcntl.ioctl`) **and** signals the `docker exec` client (`proc.send_signal(SIGWINCH)`) so the new size forwards through to the container's exec tty (vim/top reflow). The SIGWINCH is required: with `start_new_session=True` the host slave is not the client's controlling terminal, so the kernel does not auto-deliver it; the pair shares its winsize, so the client reads the new size off its slave stdio and calls Docker's exec-resize API. Any non-resize frame is raw stdin (backward compatible). The fit addon drives it client-side, and the window has a dedicated bottom-right resize grip (`.fw-resize`, since xterm covers the native CSS resize corner - the base `FloatingWindow` adds a manual grip instead of relying on `resize:both`). xterm renders ANSI natively, so the old `cleanTerm` stripping is gone for the interactive shell (the one-shot exec box still strips output, since `docker exec` without `-t` is non-TTY).

**Persistence (tmux).** A `?session=<name>` query param (validated `^[A-Za-z0-9_-]{1,64}$`) makes the WS run the shell inside tmux (`tmux new-session -A -s <name>`, falling back to bash if tmux is absent), so the session survives WS disconnect/window close - `proc.kill` on disconnect kills the tmux *client*, the server daemon + session live on. The frontend keeps exactly **one persistent slot** at a time (`ContainerTerminalManager`, the last-opened terminal): it attaches to session `devplace`, **auto-reconnects** with capped backoff on unexpected WS close (not on code `1008`), and is remembered per-user in `localStorage` (`ct-persistent:<scope>`); `app.containerTerminals.restore()` (called once in `Application.js` after `window.app`) reopens it on the next page load. Opening another terminal demotes/calls `setPersistent(false)` on the previous one (its tmux session lingers in the container, but the frontend stops auto-reconnecting/remembering it); explicit window-close (`_onClose`) detaches + forgets (session still survives, reopen re-attaches).

**Right-click context menu.** Every window wires `app.contextMenu.attach(this.win, () => this._contextItems())` (right-click + long-press), opening the shared `dp-context-menu`. The base `FloatingWindow._contextItems` is Minimize/Normalize/Close; `ContainerTerminal` overrides it with Copy (xterm `getSelection`, disabled when `!term.hasSelection()`) / Paste (clipboard -> WS stdin) / Sync files / Restart / Terminate (`dp-dialog` confirm, then delete + close) / Minimize / Normalize / Project (-> `/projects/{slug}`) / Files (-> `/projects/{slug}/files`) / Close - the lifecycle items POST to the existing `instances/{uid}/{sync,restart,delete}` routes. Devii's `_contextItems` override returns Copy (live `window.getSelection()`, or the current input line when empty) / Paste (clipboard inserted into `.devii-input` at the caret, `selectionStart/End` splice, then refocus) / Minimize / Normalize / Close (it is not bound to a container or project). Copy/Paste both use the async Clipboard API with a hidden-`textarea` + `execCommand("copy")` write fallback.

**Minimize/Normalize.** Geometry presets (`_presetGeometry(w,h)`, smallest-usable / comfortable), exposed both as titlebar buttons (`data-win="minimize|normalize"`) and menu items on every window.

## Editor profile and branding (`workspace/editor.py`, `files/vscode/`)

The browser editor is a DevPlace product surface, not stock code-server. Three layers own it, and
the split is the design: the deterministic part is host-side and unit-testable without Docker, the
cosmetic part is an extension that fails soft.

| Layer | Owns | Fails how |
|---|---|---|
| **Host** `workspace/editor.py` | Resolving the profile, seeding `settings.json`, building the argv, setting container CPU and memory | Deterministic, unit-tested against a temp state dir, no container needed |
| **Image** `ppy.Dockerfile` + `files/vscode/` | Branding assets, patched `product.json`, the bundled extension | Verified by the build smoke test; an image that cannot brand cannot build green |
| **Extension** `files/vscode/devplace-workspace/` | Boot terminals, panel layout, status bar, walkthrough, `DevPlace:` commands | Each stage in its own try/catch to a `DevPlace` output channel. A failure costs the terminals, never the editor |

**One resolver, exactly like `quota.resolve`.** `editor.resolve(owner_uid, instance) -> EditorProfile`
is the only place a `workspace_editor_*` setting is read. Order is user preference row, then site
setting, then built-in default; the container size (`cpu_millicores`, `memory_mb`, `disk_quota_mb`)
comes from `quota.resolve` so all four "sizes" live on one object. `editor.view()` adds
`source_map()` so every surface can say where a value came from. Never read one of those settings at
a call site.

**Inherit sentinels are explicit, never truthiness.** `workspace_editor_prefs` stores `""` / `0` for
"inherit", but zoom level `0` is a real value, so its sentinel is `-99` (`INHERIT_ZOOM`), and
`boot_shell` uses `-1` (`INHERIT_FLAG`) because `0` means off. `_inherits(key, row)` is the single
predicate; a bare `if value:` here would silently ignore a member who wants zoom 0 or no shell.

**`merge_managed` is the contract that a member edit is never overwritten** and it is a pure
function, which is why it is exhaustively unit-tested. DevPlace writes a key only when it is absent
or still equal to the value DevPlace wrote last time, recorded in
`{state}/data/User/.devplace-managed.json`. So raising a site default reaches everyone who never
expressed a preference and nobody who did. Do not replace this with a plain merge or a full rewrite.

**Seeding runs at launch, in `run_spec_for`**, alongside `ensure_editor_password` - the one point
every workspace launch passes through, so a workspace created before this feature is seeded on its
next boot. `stamp_boot_marker` writes a fresh `instances.boot_marker` there too; it reaches the
container as `DEVPLACE_CONTAINER_BOOT` and is what makes the extension's boot terminals idempotent
across browser reloads. Because seeding happens at launch, a preference change applies on the **next
start**: the workspace page compares the resolved profile against `{state}/devplace-editor.json`
(`editor.restart_required`) and shows a restart banner rather than pretending it applied.

**Nothing DevPlace writes goes to `/app`.** `/app` round-trips into the member's project through
`sync_dir_bidirectional`, so a `.vscode/tasks.json` there would land in their repository. Every
artefact goes under `WORKSPACE_STATE_DIR`, which is a separate bind mount and never synced. This is
why boot terminals are an extension rather than a folder-open task.

**The agent's own working files are in `SYNC_SKIP_NAMES` for the same reason.** `dpc` writes `.dpc/`
and `dpc.log` into its working directory, which is `/app`. That was harmless while `dpc` only ran
when a member typed it; now that it starts on every workspace boot, those artefacts would be
imported into every project on the next sync. `project_files.SYNC_SKIP_NAMES` therefore carries
`.dpc`, `dpc.log` and `.devplace` (the tunnel manifest directory, which was already being written
and already leaking) alongside the `.devplace_boot.*` entries. Any future in-container tool that
writes state next to the member's code needs the same entry.

**The boot marker degrades to once-per-extension-host, never to "always".** `BootTerminals` is
guarded by `DEVPLACE_CONTAINER_BOOT`, but an instance created before that column existed injects an
empty value. The guard used to treat an empty marker as "not booted yet" and opened a fresh pair of
terminals on **every browser reload** - caught by driving one container with three consecutive
Playwright sessions and finding six terminal tabs. The fallback is now `host-${process.pid}` of the
extension host, which survives a browser reload and changes when the container restarts, which is
exactly the intended semantic.

**Trust is disabled at three layers** and gated by one kill switch, `workspace_editor_trust_all`
(default on): the `--disable-workspace-trust` flag, the seeded `security.workspace.trust.*` settings,
and the extension's `contributes.configurationDefaults`. The third is belt and braces only -
`security.workspace.trust.enabled` is application-scoped and VS Code restricts which scopes an
extension may re-default - so never let it be the only layer. Turning the switch off restores
Restricted Mode with no code change and no image rebuild. It also enables
`task.allowAutomaticTasks`, so a project's own `runOn: folderOpen` task will run; that consequence is
documented to members on `/docs/workspace-editor.html` and must stay documented.

**Panel height is a preset, not a pixel value, and that is a hard constraint.** VS Code stores part
sizes in the workbench grid inside `state.vscdb`, an undocumented and version-unstable internal
SQLite database. Writing it from the host is rejected. The extension drives
`workbench.action.toggleMaximizedPanel` / `increaseViewSize` instead, so the four presets are named
honestly as presets in the UI. Do not "improve" this by writing `state.vscdb`.

**The extension is a built-in, copied to
`/usr/local/lib/code-server/lib/vscode/extensions/devplace-workspace`.** Built-ins are always
enabled, cannot be uninstalled, need no install step and survive workspace recreation because they
live in the image. `--builtin-extensions-dir` is deliberately NOT used: it has a known upstream
defect where extensions loaded through it present as disabled. There is no build step, no npm and no
bundler - a VS Code extension is a directory with a `package.json` and an entry point, and it runs in
code-server's Node remote extension host, so `main` applies.

**`extension.js` is CommonJS, and it is the one file in this repository that may be.** The VS Code
extension host loads CommonJS; it is not frontend code and is never served to a browser. Every other
house rule applies unchanged. Four small classes (`Profile`, `BootTerminals`, `Layout`, `Presence`)
and an `activate` that runs each through `stage()`, which owns the try/catch and logs to a
`DevPlace` output channel.

**The activation stages are awaited in order, and `Layout` never opens a panel of its own.** Firing
them concurrently is what produced a stray third terminal in the first live build: `Layout` called
`workbench.action.focusPanel` before `BootTerminals` had created anything, and VS Code answered by
spawning its own default `bash`. `Layout.apply(panelIsOpen)` therefore resizes only when the boot
terminals actually opened the panel, and `activate` awaits `terminals` before `layout`. Verified by
driving a real container with Playwright: the tab list must read exactly
`pravda@workspace` + `DevPlace Code`.

**A workspace suppresses the editor's own AI assistant.** Recent VS Code ships a chat panel in the
secondary sidebar, which opened by default with Microsoft branding, "AI responses may be inaccurate"
copy, and a competing agent right beside `dpc`. `editor.FOREIGN_AI_SETTINGS` turns it off
(`chat.disableAIFeatures`, `chat.commandCenter.enabled`, `workbench.secondarySideBar.defaultVisibility`)
and `workbench.startupEditor` is `none` so a workspace opens straight onto the member's code with the
agent terminal ready, rather than onto a welcome page listing a "Get Started with VS Code"
walkthrough. Unknown keys are ignored by VS Code, so these stay safe across version bumps. The
DevPlace walkthrough is still contributed and reachable from Help and the command palette.

**Branding is six things**, all baked in: the `--app-name` / `--welcome-text` /
`--disable-getting-started-override` flags in `editor.argv`; the favicon and PWA icon set generated
from `static/icon-512.png` into `files/vscode/branding/`; `devplace-login.css` appended to
code-server's `login.css`; `product.patch.json` merged key-wise into `product.json` (additive, so an
unnamed upstream key survives a bump); the `DevPlace Dark` / `DevPlace Light` themes generated from
`static/css/variables.css`; and the walkthrough, status bar item and five `DevPlace:` commands. The
login stylesheet is the **single sanctioned exception to the no-colour-literals rule** - code-server
serves it outside the application and cannot read `variables.css`, so the tokens are restated as
literals with a comment naming each one. Do not spread that exception anywhere else.

**Theme token mapping** (`variables.css` -> VS Code), recorded here because a JSON theme cannot carry
a comment: `--bg-primary` -> `editor.background`; `--bg-secondary` -> `sideBar`/`activityBar`;
`--bg-card` -> `editorWidget`/`panel`; `--accent` -> `focusBorder`/`button.background`/`progressBar`;
`--accent-light` -> `list.activeSelectionBackground`; `--text-primary` -> `foreground`;
`--text-secondary` -> `descriptionForeground`; `--border` -> every `*.border`;
`--success`/`--warning`/`--danger`/`--info` -> the ANSI green/yellow/red/blue. `DevPlace Light` is a
derived light palette (DevPlace ships no light tokens); keep the two in step by construction.

**Adding an editor setting** touches five places: `editor.DEFAULTS` + `SETTING_KEYS`, a `ConfigField`
on `WorkspaceService` (group `Editor`), the `workspace_editor_prefs` ensure block and
`editor.PREF_COLUMNS`, `EditorPrefsForm` + `EditorProfileOut`, and the `settings_for` map or the
extension. A `select` `ConfigField` MUST use `options=[{"value":..., "label":...}]` - plain strings
crash `docs_api.build_services_group`, which `docs_search` indexes, which 500s the docs search page.


## Pravda image (load-bearing, workspace ownership)

The `ppy` image (`ppy.Dockerfile`, built by `make ppy`, context `devplacepy/services/containers/files`) is a `python:3.13-slim-bookworm` base with Playwright plus a broad set of common Python libraries preinstalled, plus CLI tools (`tmux`, `apache2-utils` for `ab`, `procps`/`htop`/`iftop`/`iotop`, `netcat-openbsd` for `nc`, `zip`/`unzip`, `fakeroot`, git/curl/wget/vim/ack).

The security hotpatch that used to run per build is now baked into `ppy.Dockerfile` once. The final stage:
- `COPY`s the **sudo superclone** (`files/sudo`) over `/usr/local/bin/sudo` (+ symlink `/usr/bin/sudo`; the real `sudo` package is not installed).
- `COPY`s the **`aptroot` fakeroot wrapper** (`files/aptroot`, symlinked over `apt`/`apt-get`/`dpkg` in `/usr/local/bin` so pravda installs system packages without root).
- `COPY`s **`pagent`** (`files/pagent`, the stdlib AI agent; reads `DEVPLACE_OPENAI_URL`+`DEVPLACE_API_KEY`, falling back to its public endpoint + `DEEPSEEK_API_KEY`) to `/usr/bin/pagent.py`, plus `files/.vimrc` to `/home/pravda/.vimrc` (whose AI helper - `AiEditSelection` - targets the same gateway as pagent via `DEVPLACE_OPENAI_URL`/`DEVPLACE_API_KEY`, with a public fallback, never `api.openai.com`).
- `COPY`s **`dpc`** (`files/dpc`, DevPlace Code, the Claude-Code-class coding agent) to `/usr/bin/dpc`, and **`bot.py`** to `/usr/bin/botje.py`. **`dpc` is the one prebuilt binary in this repository**: an ELF 64-bit x86-64 position-independent executable, 10850272 bytes, sha256 `0c0f980717deebed285dcde7968c249f77638a65d76af8ae5deeda3ac2b0f042`. Its source is NOT in this repository and there is no build recipe here, so unlike every other file in `files/` it cannot be reviewed before it is installed root-owned onto `PATH` in every user container. Record a new size and checksum here whenever it is replaced - this record is the only integrity check that exists on it, so a stale entry is worse than none. (The previous entry, 3578664 bytes / sha256 `24f7fbb0...`, described a build that is no longer the file on disk.)
- Evicts any pre-existing uid-1000 user, creates user **`pravda` at `1000:1000`**.
- Hands pravda ownership of the toolchain AND the OS package trees (`chown -R pravda` over `/usr/local/lib`, `/usr/local/bin`, `/usr/lib/python3`, `/opt`, `/app`, `/home/pravda`, plus `/usr/lib`, `/usr/bin`, `/usr/sbin`, `/usr/share`, `/usr/include`, `/etc`, `/var/lib`, `/var/cache`, `/var/log`, `/srv` so `apt`/`dpkg` can write; `~/.local/bin` on `PATH`).
- Ends on `USER pravda`.

**Why uid 1000.** `/app` is bind-mounted from the host, and under DooD a container's UID maps 1:1 to the host. A container running as **root** wrote root-owned files into the workspace; the app process (host `retoor`, uid 1000) then hit `[Errno 13] Permission denied` in `export_to_dir` on the next instance create - a permanent per-project brick. Pinning the container UID to `1000` makes every write land as `retoor`, and pravda owning the global site-packages means runtime `pip install` succeeds as pravda with no elevation.

**The sudo superclone (`files/sudo`, POSIX `sh`)** is sudo-CLI-compatible but **never swaps user**: it parses the full flag interface (`-u/-g/-E/-H/-n/-S/-i/-s/--`, leading `VAR=value` env assignments, `-V/-l/-v/-k/-K` short-circuits), exports `SUDO_USER`/`SUDO_UID`/`SUDO_GID`/`SUDO_COMMAND`, then `exec`s the command **as the current user (pravda)**. So even a blind `sudo apt ...` / `sudo -u root ...` runs as uid 1000 and **cannot create a root-owned file** - the residual risk is gone by construction.

**Rootless apt (`files/aptroot`).** Pravda installs system packages directly (`apt install <pkg>`, no sudo). `aptroot` is symlinked over `apt`/`apt-get`/`dpkg` in `/usr/local/bin` (ahead of `/usr/bin` on `PATH`) and execs the real tool under **`fakeroot`** with `APT::Sandbox::User=root`; dpkg's chown-to-root calls are virtualized while every file actually written lands owned by **pravda (uid 1000)**. Combined with pravda owning the system trees (`/usr`, `/etc`, `/var/lib`, `/var/cache`, `/var/log`, `/srv`, ...), `apt install <pkg>` works with no real euid 0 and still cannot brick the bind mount. Run `apt update` first (the image clears `/var/lib/apt/lists`); a package whose maintainer script needs genuinely privileged syscalls may still fail - bake those into `ppy.Dockerfile` (its `RUN apt-get` runs as root before `USER pravda`), then `make ppy`.

**Trade-off (intentional).** The only genuinely-root operation that still does NOT escalate is binding a port < 1024 - use a high port + `/p/<slug>` ingress instead. Enforcement lives entirely in the Dockerfile (no `--user` on `docker run`). The `export_to_dir` unlink-before-write fix remains as belt-and-suspenders (the app owns the workspace dir, so it may delete any stale file in it regardless of owner before rewriting it).

## `DEVPLACE_*` runtime env injection (function name `pravda_env`)

`api.run_spec_for` merges `api.pravda_env(instance)` over the instance's own `env_json` (PRAVDA keys win), so every running container gets these platform vars:
- `DEVPLACE_BASE_URL` - the `site_url` setting via `seo.public_base_url()`.
- `DEVPLACE_OPENAI_URL` - `{base}/openai/v1`, the OpenAI-compatible gateway base.
- `DEVPLACE_API_KEY` - the **creating** user's (or `run_as_uid`'s) `api_key`, resolved live.
- `DEVPLACE_USER_UID` - the project **owner**'s uid.
- `DEVPLACE_CONTAINER_NAME` - the instance name.
- `DEVPLACE_CONTAINER_UID` - the instance uid.
- `DEVPLACE_INGRESS_URL` - the instance's absolute public ingress URL `{base}/p/{ingress_slug}` when published and `site_url` is set, relative `/p/{slug}` if `site_url` is unset, empty if the instance has no `ingress_slug`.

These let code inside a container call back into the platform and the AI gateway authenticated as the user. **Injected at run** (`docker run -e`), never baked into the image: the image is shared by every instance, but name/uid/api-key are per-instance and base-url/key must stay fresh, so they are resolved each launch and never stored as secrets in the DB.

Resolution needs two instance columns set at `create_instance`: `created_by` (= `actor[1]` when the actor is a user) feeds `DEVPLACE_API_KEY`, and `owner_uid` (= `project["user_uid"]`) feeds `DEVPLACE_USER_UID`. Instances created before this change have neither and degrade to an empty key/uid (base-url and container name/uid still resolve) until recreated. `DEVPLACE_BASE_URL`/`DEVPLACE_OPENAI_URL` are empty when `site_url` is unset, so set the admin `site_url` for container-to-platform calls (and ingress URLs) to work.

## Ingress (`/p/<slug>`)

`routers/proxy.py` (registered at `/p`) implements the ingress reverse proxy. An instance opts in with `ingress_slug` + `ingress_port` (must be one of its mapped container ports; slug unique, validated in `api.validate_ingress`). `_resolve(slug)` -> `store.find_instance_by_ingress` -> `api.proxy_target(instance)` returns `(gateway, host_port)` (the instance's recorded docker bridge gateway + published host port); the route reverse-proxies HTTP (httpx) and WebSocket (`websockets` client) to `http://{host}:{port}/{path}`, stripping the `/p/<slug>` prefix. The reconciler records `container_ip`/`container_gateway` from `docker inspect` each running tick.

**`proxy_target` dials the container's docker bridge gateway + the published host port** (`gateway:host_port`), NOT loopback and NOT the container's own bridge IP. This is deliberate: `127.0.0.1` fails where docker's `nat OUTPUT` excludes `127.0.0.0/8` from DNAT and no `docker-proxy` binds loopback for a `0.0.0.0`-published port; the container's own bridge IP can be dropped by docker's bridge-isolation rule (`DOCKER ! -i docker0 -o docker0 -j DROP`); the gateway+published-port path survives both. `DEVPLACE_CONTAINER_PROXY_HOST` overrides the host (e.g. `host.docker.internal` for a containerized app) and still uses the published host port; before a gateway is recorded the proxy falls back to `127.0.0.1`.

Ingress is **public** (no auth) but SSRF-safe: host/port are derived from the instance row, never from user input. The Devii container tools return an absolute `ingress_url` built from `seo.public_base_url()` (the admin `site_url` setting), so the agent fetches the public production URL, not localhost (which web tools refuse); unset `site_url` yields a relative `/p/<slug>`. nginx needs a `/p/` location with WS upgrade + long timeouts (present in `nginx/nginx.conf.template`).

## Production wiring

The container manager drives the host docker daemon, which needs heavy wiring - all carried by the opt-in `docker-compose.containers.yml` overlay (docker socket mount, `INSTALL_DOCKER_CLI=true` build arg, `group_add` the docker gid). **`make docker-build`/`make docker-up` always apply this overlay** and self-derive its inputs: `DOCKER_GID` via `stat -c '%g' /var/run/docker.sock` (the socket's owning group) and `DEVPLACE_DATA_DIR` = `$(CURDIR)/data` (the project's own dir at its real host absolute path). This makes the manager work out of the box with no `sudo`, no `/srv`, no manual `.env` edits. A plain `docker compose up -d` drops the overlay (no CLI, no socket) and silently re-breaks it - always update through the make targets.

**The DooD bind-mount gotcha:** `docker run -v <path>:/app` resolves `<path>` on the HOST, so `DEVPLACE_DATA_DIR` must be mounted at an identical host+container path (the make targets use `$(CURDIR)/data` on both sides; manual `docker compose` users get a `/srv/devplace-data` default). Build contexts ship via the docker API tarball, so the container temp dir is fine. Set `DEVPLACE_CONTAINER_PROXY_HOST=host.docker.internal` only when a containerized app cannot route to the recorded gateway. See README "Container Manager wiring".

## Bidirectional newer-wins sync (load-bearing direction rule)

Sync is NOT one-directional import. `project_files.sync_dir_bidirectional(project_uid, workspace, user) -> {"exported", "imported"}` is the one helper that reconciles a project's virtual FS against an instance's `workspace_dir`: per file, the side with the newer timestamp wins (project `updated_at` epoch, via `datetime.fromisoformat(...).timestamp()`, vs filesystem `st_mtime`, with a 1s skew tolerance favouring export on ties), and a file present on only one side propagates to the other. It **NEVER deletes a file** - only creates/overwrites the older side. A **read-only** project (`is_readonly`) exports only, never imports (the read-only guard direction).

Both `api.sync_workspace` (HTTP/Devii sync action, returns `{exported, imported}`) and `api.sync_bidirectional_sync` (reconciler, non-blocking `record_event` system actor, logs only when non-zero) call the same helper. The reconciler runs it before every `_launch` AND on a ~60s wall-clock cadence over running instances (`SYNC_EVERY_SECONDS`, gated on `time.monotonic()` independent of the 5s reconcile tick). The per-instance boot-helper files (`.devplace_boot.py`/`.devplace_boot.sh`) are in `SYNC_SKIP_NAMES` so they never round-trip into the project.

## Run-as user = identity + API key ONLY (load-bearing constraint)

An instance's `run_as_uid` column selects WHICH DevPlace user's identity and `api_key` are injected (`DEVPLACE_API_KEY`, `DEVPLACE_USER_UID`), resolved in `api.pravda_env` ahead of the `created_by`/`owner_uid` fallback chain. It does **NOT** change the container OS user, which is ALWAYS `pravda` (uid 1000) - required for the bind-mounted `/app` (DooD uid maps 1:1 to host). Validate it against an existing user via `api.validate_run_as`.

**Running as someone else requires their consent (load-bearing).** `validate_run_as(run_as_uid, actor_uid)` refuses when the run-as user is not the actor and has not granted the `container_credentials` consent. This is not a policy nicety: `pravda_env` injects that user's real `DEVPLACE_API_KEY` into a container someone else operates, so without the gate an administrator could hand any member's platform credential to software that member never saw. `create_instance` and `update_instance_config` both pass `_actor_uid(actor)`, so the gate covers the admin UI, the per-project manager and the Devii container tools at once. Running as **yourself** needs no consent - you are the one handing over your own credential. The consent is granted and withdrawn from the member's own profile privacy tab like every other consent; see `devplacepy/services/moderation/CLAUDE.md`.

## Boot source precedence (load-bearing)

Columns `boot_language` (`none`|`python`|`bash`) + `boot_script` (multiline source) sit alongside the legacy `boot_command`. Precedence in `api.run_spec_for`: `boot_script` (by language) > `boot_command` > image CMD (`sleep infinity`). When a boot script is set, the reconciler writes it into the workspace (`api.materialize_boot_script`, `.devplace_boot.py`/`.devplace_boot.sh`, excluded from sync) before launch and runs `python|bash /app/.devplace_boot.<ext>`. `api.validate_boot` enforces the language set and a 100k char cap.

## start_on_boot is per-container

The `start_on_boot` integer flag (0/1) forces `desired_state=running` ONLY for flagged instances when `ContainerService` first runs (`_boot_pass`, one-time); all others keep their last `desired_state`.

## Status-change choke point

Every reconciler status mutation flows through `service._set_status(inst, changes, reason=)`, which diffs old vs new `status` and, on change, writes an `instance_events` `status_change` row AND an audit `container.instance.status` `record_system` event (`old_value`/`new_value`). `_reconcile`, `_launch`, and `_handle_exit` (terminal + policy_restart) all flow through it, so previously-silent transitions (`running->stopped`/`->paused`/exit) are now logged and visible on the admin detail page's **Status history** card. New code that changes an instance's status in the reconciler must use `_set_status`, never a raw `store.update_instance(... status ...)`.

## New columns are ensured in init_db and defaulted in store.create_instance

`run_as_uid`/`boot_language`/`boot_script`/`start_on_boot` are added to the `instances` ensure-block in `init_db` (filtered/queried columns must exist on the cached schema) and seeded in the `store.create_instance` base dict so pre-existing rows degrade gracefully.

## Testing without Docker

Use `FakeBackend` (its `image_exists` returns `True`) + `runtime.set_backend`, and monkeypatch `config.CONTAINER_WORKSPACES_DIR` to a tmp dir. See `tests/unit/services/containers.py` and `tests/api/containers.py` (argv, instance creation on `config.CONTAINER_IMAGE`, image-not-built guard, reconcile matrices, schedule firing, ingress validation + live HTTP proxy, HTTP admin gate).

## Vibe coding on-ramp (user-facing doc)

The container runtime is also the basis of "vibe coding": the public prose page `templates/docs/getting-started-vibing.html` (slug `getting-started-vibing`, `SECTION_GENERAL`, not admin-gated so the docs stay public while the feature is admin-only/Alpha) is the canonical user-facing guide. It teaches the Devii-driven flow project -> container -> start -> terminal (`create_project`, `container_create_instance`, `container_instance_action`, the `open_terminal` client action), documents the three baked-in agents (`dpc` = DevPlace Code at `/usr/bin/dpc`, the Claude-Code-class coding agent; `botje.py` = the copy of `services/containers/files/bot.py` at `/usr/bin/botje.py`; `pagent`), all metered through the container's own `DEVPLACE_API_KEY`, the full `DEVPLACE_*` env table (see `api.pravda_env`), and ingress at `/p/<slug>` via `ingress_slug`/`ingress_port`.

When the runtime, the agent binaries, or the `DEVPLACE_*`/ingress contract change, update this page alongside the source.

**The agents are gateway-only:** `dpc`/`d.py` and `botje.py`/`bot.py` use a single `molodetz` backend pointed at `DEVPLACE_OPENAI_URL` (the gateway); the former direct `api.deepseek.com` fallback backend was removed so every in-container AI call is ledgered under the run-as user and nothing bypasses `gateway_usage_ledger`. `pagent`/`.vimrc` already posted to the gateway URL (using `DEEPSEEK_API_KEY` only as a key fallback, never the DeepSeek endpoint). Rebuild the image (`make ppy`) for the change to reach running containers.

## Dev Workspaces (`workspace/`, `workspace_service.py`, `activity.py`, `forward.py`)

A **workspace** is a member-facing container running `code-server`, layered on this same runtime. The
admin container manager's authorization is unchanged; workspaces add their own narrower predicates.

**Two planes, deliberately distinct.**

| | Plane A: editor | Plane B: user's servers |
|---|---|---|
| Entry | `/projects/{slug}/containers/instances/{uid}/code/...` | `{port}-{name}.tunnel.pravda.education` |
| Auth | session + `can_manage_workspace` | none, public by design |
| Backend | code-server, `--auth password`, bound in-container | whatever the user runs |

Plane B routing is **one static molohttp site** `*.tunnel.pravda.education -> 127.0.0.1:10500`;
DevPlace resolves the instance from the `Host` header. There is no molohttp object per tunnel.
`TunnelDispatchMiddleware` (`main.py`) is ASGI-level and pre-empts the router for both HTTP and
WebSocket, so a tunnel host can never render the application. molohttp's `HostIndex` glob matches
**exactly one label**, which is why the pattern is `{port}-{name}`, never `{port}.{name}`.

**One forwarding core.** `forward.py` owns header filtering, prefix stripping, `Location` rewriting,
`<base>` injection, response streaming and the bidirectional WS pump. `/p/{slug}`, the editor route
and the tunnel route all call it. Never write a second proxy, and never let the two planes drift:
**both build their headers from the single `base_headers` core**, so anything a tunnelled app sees
over HTTP (`X-Real-IP`, `X-Forwarded-For`, `Accept-Language`, custom auth headers) it also sees over
a websocket. `forward_headers` adds only what is HTTP-specific (`Host`, `Accept-Encoding: identity`
so the internal hop is never compressed); `ws_headers` drops only `WS_HANDSHAKE_HEADERS`, which the
`websockets` client regenerates itself. Both planes therefore send the same `Host` (the public one),
the same `X-Forwarded-Host`/`-Proto`/`-Prefix`, and the same `X-Script-Name`.

**The websocket handshake carries exactly ONE `Host`, and it is the public one (load-bearing).**
Never put `Host` in `additional_headers`: the `websockets` client already writes its own for the real
TCP target and `Headers.update` *appends*, so a second one produces a handshake with two `Host`
lines. Node keeps the FIRST (the internal `gateway:port`) and Go's `net/http` rejects the request
outright. The fix is `websockets.connect(uri, host=..., port=...)` - the public host goes in the URI
(so it becomes the single `Host` header) while the connection still dials the container. code-server
resolves the request host as `Forwarded` -> `X-Forwarded-Host` -> `Host` and runs
`authenticateOrigin` on EVERY websocket (`wsRouter.ws(/.*/, ensureOrigin, ...)`), so with only the
losing duplicate it compared the browser's `Origin` (`devplace.net`) against the internal host and
answered **403**, and the editor died on *"The workbench failed to connect to the server (Error: Time
limit reached)"* - a blank page under the whole
`/projects/{slug}/containers/instances/{uid}/code/` sub-path while its HTTP assets loaded fine
(`forward_headers` already sent `X-Forwarded-Host`, which is why only the websocket plane broke).
This hit the tunnel plane identically. Reproduce with the real image, not a mock: `docker run ppy
code-server --auth none` and replay the header set - the origin check runs before authentication.

**Dial upstream BEFORE accepting the browser socket.** `proxy_ws` connects first and only then calls
`websocket.accept(subprotocol=upstream.subprotocol)`. Accepting first turns every upstream refusal
into a phantom `101` followed by an immediate `1011`, which is exactly what made the 403 above
present as an opaque client-side timeout instead of a handshake failure; it also makes it impossible
to echo the negotiated subprotocol, because the answer is not known yet. Subprotocols travel through
`subprotocols=` (never as a forwarded header) and come back on the `accept`.

**Responses stream; only the `<base>`-injected HTML is buffered.** `proxy_http` sends with
`stream=True` and returns a `StreamingResponse` over `aiter_raw()`, passing `content-length` and
`content-encoding` through untouched (`RESPONSE_HOP_HEADERS` strips hop-by-hop only), so a large
download through a tunnel costs a constant few hundred KiB of worker memory instead of its full size,
and SSE works. The HTML branch must buffer because injection needs the whole body, and it therefore
drops `content-length`/`content-encoding` and lets Starlette recompute. Byte accounting rides the
`on_complete(sent)` callback fired when the stream ends - `routers/tunnel.py` uses it for
`activity.touch` + `tunnels.record_hit`, which is ONE `touch` per request (`activity` accumulates
while throttled, so a second call would double the request counter). **Request bodies stay
buffered on purpose**: they are bounded by nginx `client_max_body_size`, and streaming them would
force chunked encoding onto arbitrary upstream apps.

**One keep-alive client, closed on shutdown.** `forward.client()` is a lazily-created module-level
`httpx.AsyncClient` with `httpx.Limits` (the `ChromeStealthClient` pattern), closed by
`forward.close_client()` in the `main.py` lifespan. A client per request meant a fresh pool and TCP
handshake for every one of the workbench's hundreds of assets.

**Path and query are forwarded byte-exactly.** The route-matched path is `quote()`d before it is
interpolated (a filename containing `#` or a space otherwise truncates or corrupts the upstream URL),
and the query comes from `raw_query(connection)` = the raw ASGI `scope["query_string"]`, NEVER
`request.url.query`/`request.query_params`. Starlette rebuilds `.url` by string-joining path and
query, so a literal `#` in the path makes the reconstructed URL treat the query as a fragment and
`.url.query` silently returns empty - which would drop code-server's `?reconnectionToken=`.

**Editor persistence.** code-server's user-data and extensions live in
`config.WORKSPACE_STATE_DIR/<instance uid>`, bind-mounted at `WORKSPACE_STATE_MOUNT`, so extensions
survive container recreation. `editor.argv` builds the argv; `run_spec_for` prefers it over
the boot-script/boot-command chain when `is_workspace` and `editor_port` are set.

**Activity and egress are the presence pattern.** `activity.py` keeps a per-worker monotonic dict and
writes at most once per `WORKSPACE_ACTIVITY_WRITE_SECONDS`, accumulating egress and request counts
into one atomic `COALESCE` UPDATE (`store.record_activity`). Both proxy planes call `touch`; because
plane B traverses DevPlace, public traffic is observed directly rather than inferred.

**`workspace/` package.** `quota.py` resolves limits instance -> user rule -> setting -> default
through ONE resolver (never read a workspace setting at a call site). `flags.py` is the abuse ledger
and is **idempotent per `(instance_uid, kind)` while a flag is open**, so a sustained condition is one
row, not one per tick. `naming.py` generates faker labels with collision retry and owns the hostname
patterns plus `is_tunnel_host`. `tunnels.py` is CRUD with revive-not-duplicate. `provision.py` is the
create/resume/stop/suspend/view surface and writes `/app/.devplace/tunnels.json`.

**`WorkspaceService`** is the only new service: lock-owner, `default_enabled=False`, four wrapped
synchronous phases (disk sample on its own slow cadence, flag evaluation, lifecycle, purge sweep)
plus the async `_issue_tunnel_certificates`. It is a reconciler, not a `JobService`.

**Tunnel certificates are per-host, and that is forced by molohttp, not a preference.** molohttp's
ACME client implements **`http-01` only** (`Sources/MoloHTTP/ACME/AcmeClient.swift`,
`AcmeRenewalService.swift` - there is no `dns-01` anywhere), and Let's Encrypt will not issue a
wildcard over `http-01`. So a `*.tunnel.pravda.education` **certificate** is impossible today and
`workspace_cert_mode=per_host` is the only working mode; the prose above about one static wildcard
site describes **routing**, which is correct and already in place (one enabled molohttp site
`*.tunnel.pravda.education -> http://127.0.0.1:10500`, no per-tunnel site object). Only the cert is
per-host. `tunnels.create` writes a row at `STATUS_PENDING` and nothing else; the row is inert until
something certifies it - `pending` is not in `SERVING_STATUSES`, so an un-provisioned tunnel 404s.

**One state machine, two callers.** `certs.certify(row, log=None)` owns the whole transition:
`provisioning` -> `certs.issue(hostname)` (`POST {workspace_molohttp_base_url}/api/v1/certs/issue`
with the `x-api-key` header, Basic as fallback) -> `active` or `failed` with `last_error`, then
`certs.announce(row, hostname)` sends the owner the live URL (plus the password for the editor
tunnel). Never re-implement those transitions at a call site - both paths call `certify`:

- **Eagerly, at workspace creation.** `provision.ensure` ends with
  `schedule_certificate(publish_editor_tunnel(...))`, which claims the row as `provisioning`
  **synchronously** and then spawns `certify` as a loop task, so the editor certificate is ordered
  the instant the workspace exists rather than up to one service tick later. It is fire-and-forget
  by necessity: an ACME `http-01` order takes ~10s and `certs.issue` allows up to 180s, so awaiting
  it would hang the `POST /projects/{slug}/workspace` response. The task is held in
  `provision._pending_certificates` and discarded on completion - a bare `create_task` reference can
  be garbage collected mid-flight. The synchronous claim is what closes the race with the tick: the
  service selects only `pending`, so a row already claimed can never be issued twice and burn a
  Let's Encrypt duplicate-certificate slot. `schedule_certificate` no-ops (leaving the row `pending`
  for the tick) when molohttp is unconfigured or there is no running loop.
- **On the tick, as the safety net.** `WorkspaceService._issue_tunnel_certificates` reads
  `tunnels.awaiting_certificate()` (`status=pending` + `desired_state=present` + not deleted) and
  calls `certify` per row. This covers user-created tunnels, rows predating the eager path, and any
  creation that happened with molohttp unconfigured.

The residual cost is the ACME round trip itself, which is irreducible under `http-01`. The tick
interval (`service_workspace_interval`, default 30s, floor `min_interval = 5`) no longer sits in
front of the editor tunnel, but still bounds how long a user-created tunnel waits.

**A tunnel serves before its certificate exists.** `SERVING_STATUSES = (provisioning, active)`, so
`routers/tunnel.py` routes the host the moment the row is claimed, while the order is still in
flight. The wildcard DNS already resolves, so molohttp terminates TLS with its default certificate
and the browser shows `ERR_CERT_COMMON_NAME_INVALID` for that window rather than failing to connect.
That is deliberate (a reachable host beats a dead one) and is the reason eager issuance matters: the
window is only as long as issuance takes. **Only `pending` is retried** - a `failed` row stays failed until the user recreates
the tunnel (`create` revives it to `pending`), which keeps a broken host from burning the Let's
Encrypt failure rate limit every 30s. **Renewal is molohttp's job, not DevPlace's**: once a host is
issued, `AcmeRenewalService` re-issues it against its own expiry threshold forever, so DevPlace never
schedules or tracks renewals. This whole phase did not exist - `tunnels` had no reader at all, and
the six `workspace_molohttp_*`/`workspace_cert_mode`/`workspace_acme_email` settings were admin
fields wired to nothing, which is why every tunnel sat at `pending` with no certificate.

**`provision.publish_tunnel` is the ONE way a user-created tunnel comes into existence** - the HTTP
route, the Devii tool and the editor all funnel through it. It owns the port check, the `max_tunnels`
quota, `tunnels.create`, `schedule_certificate` and `write_manifest`, and raises `WorkspaceError` for
every refusal. Before it existed, the route and the Devii controller each carried their own copy of
the quota check and **neither ordered a certificate**, so a user-created tunnel sat `pending` until
the (default-disabled) `WorkspaceService` happened to tick - which on an instance where that service
was never enabled is forever. `schedule_certificate` now also writes `provision.CERT_UNCONFIGURED`
into the row's `last_error` when molohttp is not configured, because a tunnel that can never be
certified must say so on the workspace page rather than sit at `pending` with a blank error.

**A tunnel reaches its port through `api.tunnel_target(instance, container_port)`, never through
`proxy_target`.** `proxy_target` answers for `/p/{slug}`, whose port is published by construction;
a tunnel's port is whatever the member decided to serve on and is almost never published, because a
workspace publishes only `editor_port`. `tunnel_target` therefore prefers the published host port
when the port happens to have one (`CONTAINER_PROXY_HOST` or the recorded gateway, exactly like
`proxy_target`) and otherwise dials `container_ip:container_port` directly. The direct leg is what
makes an arbitrary port tunnellable at all: docker cannot add a published port to a running
container, so publishing on demand would mean recreating the container and killing the very dev
server the member just asked to share.

**The direct leg needs the app on the same docker network as the instances, and that wiring cannot
live in compose.** Measured on this host: from the app container, `container_ip:port` times out
(docker's inter-network isolation) while `gateway:published_host_port` connects; from the host, and
from any container sharing the instances' network, `container_ip:port` connects. So `make dev` works
untouched and the containerized production app does not - it must be attached to the network the
instances run on. Compose cannot express that: it always sends network-scoped aliases, which the
default `bridge` rejects (`invalid endpoint settings: network-scoped aliases are only supported for
user-defined networks`). The attachment is therefore a `make docker-attach` step, run by
`docker-up` and `docker-reload` and idempotent, deriving its input like `DOCKER_GID` does
(`DEVPLACE_CONTAINER_NETWORK`, default `bridge`). A bare `docker compose up -d` skips it and
silently re-breaks every unpublished-port tunnel - one more reason the make targets are the only
supported path.

**Forwarding a port in the editor publishes it, and that is the whole point of `VSCODE_PROXY_URI`.**
`api.workspace_env` advertises `https://{{port}}-{name}.{domain}` to VS Code, so the Ports view shows
a DevPlace address for every forwarded port - but VS Code never tells DevPlace, so that address had
no `tunnels` row, was 404ed by `routers/tunnel.py` and never got a certificate. The editor promised a
URL the platform could not serve. The `Tunnels` stage in the workspace extension closes it: it
subscribes to `vscode.workspace.onDidChangeTunnels`, reads `vscode.workspace.tunnels`, and POSTs each
new `remoteAddress.port` (skipping `DEVPLACE_EDITOR_PORT`, which is already published) to
`{DEVPLACE_BASE_URL}/projects/{DEVPLACE_PROJECT_SLUG}/workspace/tunnels` with the container's own
`DEVPLACE_API_KEY` and `Accept: application/json`. Four things about it:

- **`tunnels` is a proposed API** (`checkProposedApiEnabled(extension, 'tunnels')`), so the extension
  declares `enabledApiProposals: ["tunnels"]` and `product.patch.json` names it under
  `extensionEnabledApiProposals`. code-server patches the check to always pass, so it works today
  either way; the declarations are what keep it working if that patch goes away. Because the patch
  adds a nested object, the Dockerfile's `product.json` merge now merges one level deep - a plain
  `dict.update` would wipe an upstream map of the same name on a version bump.
- **It only ever creates.** Un-forwarding a port leaves the tunnel standing, because deleting it
  would revoke the certificate and a re-forward would re-issue, churning against Let's Encrypt's
  duplicate-certificate limit. Removal stays the explicit act it already was.
- **A port is added to the in-memory `published` set before the POST and removed again on failure**,
  so a burst of change events cannot double-post and a refusal (quota, 403) can still retry on the
  next change. Refusals surface both in the `DevPlace` output channel and as a warning message.
- **It uses `http`/`https` from Node, not `fetch`**, and is wrapped in the same `stage()` try/catch as
  every other activation step - a workspace whose network is down must still open its editor.

Verified against the real image by driving code-server with Playwright and forwarding port 3000: the
Ports view lists the port, the extension POSTs `label=Port+3000&container_port=3000` with the API key,
and the output channel reports the public URL. Reproduce it that way, not with a mock - the Ports
view is the only trigger, there is no `Forward a Port` command in the palette in code-server.

**Two contracts that bite:**
- A `ConfigField` with `type="select"` needs `options=[{"value": ..., "label": ...}]`. Plain strings
  crash `docs_api.build_services_group`, which `docs_search` indexes, so the whole docs search page
  500s. No other service used `select` before this one.
- A route taking `Annotated[Form, Form()]` validates **before** the handler's auth guard, so a
  required field makes an anonymous request 422 instead of 401 and `tests/api/auth/matrix.py` fails.
  Give the field a default and validate it inside the handler after `require_user`.

**The editor opens through one shared partial.** The project detail page renders an inline **Editor**
button in `.project-detail-actions` via `templates/_editor_open.html` (`target="_blank"`, plus the
`data-editor-*` attributes `EditorLauncher` reads) straight to the code-server proxy
`/projects/{slug}/containers/instances/{uid}/code/`, built by `_editor_url` in
`routers/projects/index.py` and carried as `workspace_editor_url` on the context and
`ProjectDetailOut`. It is emitted ONLY when the viewer passes `can_open_workspace` AND their
workspace for that project exists, is not suspended, and is `store.ST_RUNNING` - the three states the
`editor_proxy` route itself refuses (403 suspended, 409 not running, 502 no port), so the button can
never open a dead editor. When there is no running workspace the button is absent and the Workspace
menu item below is the way in (create/start it there). The workspace page's own **Open editor** link
opens in a new tab too; keep both in step.

**Member entry point** is the project detail page's overflow menu (`project_detail.html`), gated by
the `viewer_can_workspace` context flag (`can_open_workspace(project, user)`, set in
`routers/projects/index.py` and declared on `ProjectDetailOut`) - exactly the pattern the admin-only
**Containers** item uses with `viewer_can_containers`. `can_open_workspace` folds in the
`workspace_enabled` master switch, so the item disappears for everyone while the feature is off and
the route's own `_guard` stays the authority. **A workspace surface with no context flag is
unreachable**: the whole feature shipped once with routes, Devii tools and docs but no link into
`/projects/{slug}/workspace`, so it was reachable only by typing the URL - and then 404'd anyway
because `workspace_enabled` defaults to `"0"`. Any new workspace surface needs both the flag on the
page that links to it and the setting turned on.

**Admin console** is `/admin/workspaces` (`routers/admin/workspaces.py`, `admin_workspaces.html`):
list, start/stop, suspend/unsuspend with a required reason, raise/resolve/dismiss flags, per-user
quota rules. `base_seo_context` takes `breadcrumbs`/`schemas`, not `canonical`/`schema`.

**Devii** has 17 tools under `handler="workspace"`; six are in `CONFIRM_REQUIRED` and every one of
them declares a `confirm` param (schemas are `additionalProperties: false`, so a gated tool without it
loops forever). `workspace_editor_get` / `workspace_editor_set` cover the editor profile;
`workspace_editor_set` is gated because it changes how every workspace the member opens behaves.

**Opening a workspace publishes its editor on a public tunnel, automatically.** `provision.ensure`
does three things after creating the instance: `api.ensure_editor_password`, then
`publish_editor_tunnel` (a `tunnels.create` for the editor port, so the hostname is
`{editor_port}-{name}.{domain}`), and the row lands at `pending`. The `WorkspaceService` cert phase
then issues the certificate and calls `_announce_tunnel`, which sends the owner a `workspace`
notification carrying the live `https://` URL **and the password**. The notification fires on
`pending -> active`, not at creation, so the user is only told about a URL that already serves TLS.
`provision.is_editor_tunnel(instance, tunnel)` distinguishes the auto-published editor tunnel (its
`container_port` equals `editor_port`) from user-created ones, which get a plainer message with no
password. This is a deliberate departure from the plane-A/plane-B split described above: the editor
is now reachable publicly, which is only acceptable **because** `--auth password` is on - never
reintroduce `--auth none` while the editor tunnel is auto-published.

**The editor is password-protected, per workspace.** `editor.argv` runs code-server with
`--auth password`, and `api.pravda_env` injects the secret as `PASSWORD` (the variable code-server
reads). The secret is an 8-character **pronounceable** token from `api.generate_editor_password()`,
built as four consonant-vowel pairs (`ronebamu`, `zipesodu`) so a user can read it once and retype it
from memory. That is ~24 bits of entropy, which is deliberately weak for convenience: it is
acceptable only because the alternative in practice was `--auth none`. If the editor tunnel is ever
exposed to untrusted traffic at scale, lengthen the pair count rather than switching alphabet, and
keep it pronounceable. `api.ensure_editor_password(instance)` is the single generator/persister: it returns the
stored value or mints and saves one, so the password is **stable** across restarts and recreations.
It is called from `api.run_spec_for` whenever `is_workspace` - the one point every workspace launch
passes through - so a legacy instance created before this column existed gets one on its next boot
rather than letting code-server invent an unknowable password of its own. The owner reads it off the
workspace page (`editor_password` on the context and on `WorkspaceOut`). **It is deliberately NOT on
`WorkspaceViewOut`**: that model is shared with `AdminWorkspacesOut`, which lists every workspace on
the instance, so putting it there would hand every user's editor password to any admin loading
`/admin/workspaces`. Keep per-workspace secrets on the owner-scoped model only. The column is
`instances.editor_password`, ensured in `init_db`. This closes the hole that made a tunnelled editor
an unauthenticated public shell: `--auth none` was safe only while the editor was reachable solely
through the session-authenticated proxy route, and a user can publish a tunnel to the editor port.

**code-server lives in the `ppy` image, and nothing else supplies it.** `editor.argv` makes
`code-server` the container's argv, so an image without it fails `docker run` with exit **127**
(`executable file not found in $PATH`) and the reconciler records `crashed` / `launch_failed` with an
empty `container_id` - the container process never existed. It is installed in `ppy.Dockerfile` in
the runtime stage **before `USER pravda`** and before the `chown -R pravda` block (so uid 1000 owns
it with no elevation): version pinned in the single `ARG CODE_SERVER_VERSION`, arch resolved from
`dpkg --print-architecture`, release tarball unpacked to `/usr/local/lib/code-server` with a symlink
at `/usr/local/bin/code-server`. `code-server --version` is in the build smoke-test loop and the
symlink is in the executable-check list, so an image that cannot run the editor can never build
green. Bumping the version is a one-line `ARG` change plus `make ppy`, and the branding smoke test
(below) re-verifies the four CLI flags, the media file names and the `product.json` keys, so a
version that breaks any of them cannot build green either. (`workspace_editor_version` was a
`ConfigField` read by nothing and has been removed: the version is a property of the shared image,
not a runtime setting, and a live control that changes nothing is a silent failure.)

**A container stuck in `created` is recreated, never retried forever.** The reconciler's
`desired=running` branch reaches `backend.start()` for `ps.state == "created"`. That call is wrapped:
on failure it logs, records an instance event `recreated`, `rm -f`s the container and calls
`_launch`, so the instance is rebuilt from the CURRENT image. Without this, a container created
against an older image that can no longer start is a permanent wedge - `docker start` re-resolves
nothing, so the reconciler loops on the same error every tick until someone removes it by hand (the
real failure: after code-server was added to `ppy`, the already-created workspace container kept
failing `docker start` with the old 127 even though the new image was correct). Recreation is not a
new retry class: a launch that fails already leaves `ps is None`, which `_launch`es again next tick.

**Toolchains in `ppy`.** Rust (rustup), Nim (choosenim), Swift (swiftly, then the toolchain is moved
to a fixed `/opt/swift/toolchain` because swiftly's proxy resolves against `$HOME` and breaks for
`pravda` at runtime). The choosenim installer **exits 1 even on success**, so its `RUN` ends with
`|| true` plus a real version check. The build smoke test must not pipe (`cmd | head -1` returns
`head`'s status and masks a broken toolchain - this hid a non-working Swift through a full build).
`/etc/profile.d/devplace-toolchains.sh` re-exports the PATH because a login shell resets it, which is
what the container terminal uses.
