Make dev workspaces serve a working browser IDE end to end
The workspace feature shipped its routes, agent tools and docs, but the editor was never reachable: the project page had no entry point, the ppy image had no code-server binary, no certificate was ever requested for a tunnel, and both nginx and the proxy dropped what the editor needs. - Add a VS Code button to the project action row and a Workspace item to the overflow menu, gated by can_open_workspace plus a running instance (viewer_can_workspace and workspace_editor_url on ProjectDetailOut). - Install a pinned code-server in ppy.Dockerfile before USER pravda and assert it in the build smoke test, so an image that cannot run the editor no longer builds green. - Run the editor with --auth password and a per workspace 8 character pronounceable secret, minted once at the ensure_editor_password choke point and injected as PASSWORD. Keep it off WorkspaceViewOut, which the admin listing shares. - Publish the editor tunnel when a workspace is created and issue its certificate from a new WorkspaceService phase against the molohttp admin API, then notify the owner with the live URL and the password. Only pending rows are retried, so a broken host cannot burn the ACME failure rate limit. Renewal stays molohttp's job. - Forward the original Host on proxied requests and the client cookie on proxied websockets, so code-server scopes its session cookie to the public hostname and authenticates the workbench socket. - Recreate a container stuck in the created state instead of retrying docker start forever against an image it can no longer run. - Return a JSON string from WorkspaceController.dispatch; raw dicts landed in a tool message and aborted the turn at the model endpoint. - Let the nginx catch-all carry websocket upgrades, keeping upstream keepalive, so tunnelled apps and the editor both connect.
This commit is contained in:
parent
21f6ae0615
commit
192df12b1d
@ -418,7 +418,7 @@ and its full configuration are documented automatically - including future servi
|
||||
|
||||
### Container manager (admin only)
|
||||
|
||||
`services/containers/` runs supervised container instances from the web UI, the HTTP API, and Devii. It drives the `docker` CLI via async subprocesses behind a pluggable `Backend` interface (a `DockerCliBackend` plus a `FakeBackend` for tests). **There is no in-app image building.** Every instance runs ONE shared prebuilt image, `ppy:latest` (override `DEVPLACE_CONTAINER_IMAGE`), built once with **`make ppy`** from `ppy.Dockerfile`: a Python + Playwright base with a broad set of common libraries preinstalled, the `pravda` (uid 1000) user, the sudo superclone, and `pagent` at `/usr/bin/pagent.py` all baked in. Creating an instance is then an instant `docker run` (it fails fast with a clear error if the `ppy` image has not been built yet). `ContainerService` reconciles desired instance state against `docker ps` each tick (containers are labeled `devplace.instance=<uid>`, so orphans are reaped and no state is lost), applies restart policies, fires cron/interval/one-time schedules, and samples metrics. The container's `/app` is bind-mounted to a persistent project workspace (materialized from the project files) and **stays in sync automatically**: the manager runs a bidirectional, newer-wins sync between the project files and the workspace on every start and roughly once a minute while running, so edits made inside the container and edits made in the project file editor converge without manual intervention (the sync only ever creates or overwrites the older copy of a file, never deletes one; a read-only project is export-only). Projects that need extra packages use runtime `pip install` (pravda owns the site-packages, no sudo needed) or `apt install` directly (the `pravda` user runs `apt`/`dpkg` through a fakeroot wrapper, so system packages install without root) or add the library to `ppy.Dockerfile` and rerun `make ppy`. Each instance can run a **boot script** in Python or Bash (written into the workspace and run on launch) or a plain boot command, can be set to **start automatically** whenever the container service starts, and can be configured to **run as** a chosen DevPlace user - which only selects whose identity and API key are injected into the container (the container always runs as the unprivileged `pravda` user). Every status change is recorded and shown as a status history on the instance page. A running instance can be **published** with an `ingress_slug`, making its service reachable (HTTP and WebSocket) at `/p/<slug>` through DevPlace. The manager is reached two ways: the admin **Containers** sidebar entry (`/admin/containers`) lists, creates, edits, and controls every instance across all projects and opens a dedicated detail page per instance, and each project page carries an admin-only **Containers** button to its own instance manager.
|
||||
`services/containers/` runs supervised container instances from the web UI, the HTTP API, and Devii. It drives the `docker` CLI via async subprocesses behind a pluggable `Backend` interface (a `DockerCliBackend` plus a `FakeBackend` for tests). **There is no in-app image building.** Every instance runs ONE shared prebuilt image, `ppy:latest` (override `DEVPLACE_CONTAINER_IMAGE`), built once with **`make ppy`** from `ppy.Dockerfile`: a Python + Playwright base with a broad set of common libraries preinstalled, the `pravda` (uid 1000) user, the sudo superclone, `pagent` at `/usr/bin/pagent.py`, and the `code-server` browser IDE that backs dev workspaces all baked in. Creating an instance is then an instant `docker run` (it fails fast with a clear error if the `ppy` image has not been built yet). `ContainerService` reconciles desired instance state against `docker ps` each tick (containers are labeled `devplace.instance=<uid>`, so orphans are reaped and no state is lost), applies restart policies, fires cron/interval/one-time schedules, and samples metrics. The container's `/app` is bind-mounted to a persistent project workspace (materialized from the project files) and **stays in sync automatically**: the manager runs a bidirectional, newer-wins sync between the project files and the workspace on every start and roughly once a minute while running, so edits made inside the container and edits made in the project file editor converge without manual intervention (the sync only ever creates or overwrites the older copy of a file, never deletes one; a read-only project is export-only). Projects that need extra packages use runtime `pip install` (pravda owns the site-packages, no sudo needed) or `apt install` directly (the `pravda` user runs `apt`/`dpkg` through a fakeroot wrapper, so system packages install without root) or add the library to `ppy.Dockerfile` and rerun `make ppy`. Each instance can run a **boot script** in Python or Bash (written into the workspace and run on launch) or a plain boot command, can be set to **start automatically** whenever the container service starts, and can be configured to **run as** a chosen DevPlace user - which only selects whose identity and API key are injected into the container (the container always runs as the unprivileged `pravda` user). Every status change is recorded and shown as a status history on the instance page. A running instance can be **published** with an `ingress_slug`, making its service reachable (HTTP and WebSocket) at `/p/<slug>` through DevPlace. The manager is reached two ways: the admin **Containers** sidebar entry (`/admin/containers`) lists, creates, edits, and controls every instance across all projects and opens a dedicated detail page per instance, and each project page carries an admin-only **Containers** button to its own instance manager.
|
||||
|
||||
**Security:** this requires mounting the Docker socket, which grants host root. Every run, exec, lifecycle, and schedule operation is administrator-only; `--privileged` is never used and all docker calls are argument-list subprocesses. Containers are additionally isolated per user: an instance is managed only by its owner (the administrator who created it, or the owner of its project) and by the primary administrator, who alone sees and manages every instance including those on private projects; other administrators get a read-only view of instances on public projects and none of another user's private-project instances (exec, terminals, schedules, edits, and lifecycle actions are all refused and audited as denied). The service is disabled by default; an admin enables **Containers** on `/admin/services`. CLI: `devplace containers list | reconcile | prune | prune-builds | gc-workspaces` (`prune-builds` is a one-time cleanup that removes legacy per-project images and the old dockerfiles/builds tables).
|
||||
|
||||
|
||||
@ -572,6 +572,7 @@ def init_db():
|
||||
("workspace_owner_uid", ""),
|
||||
("editor_port", 0),
|
||||
("editor_host_port", 0),
|
||||
("editor_password", ""),
|
||||
("tunnel_name", ""),
|
||||
("last_active_at", ""),
|
||||
("idle_warned_at", ""),
|
||||
|
||||
@ -9,7 +9,7 @@ This file documents the project detail page, the per-project virtual filesystem,
|
||||
|
||||
Each project card links to `/projects/{project_uid}` showing full project details, author info, platforms, star count, delete-for-owner, and (for the owner) Private/Read-only toggle buttons plus badges (see **Project visibility and read-only** below). The route is `GET /projects/{project_uid}` in `routers/projects/index.py` and 404s when the viewer cannot see a private project. The sitemap generator links to this URL (not the old `?user_uid=` query param). The detail page also links to the project filesystem at `/projects/{slug}/files`.
|
||||
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Containers, Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Workspace, Containers, Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
|
||||
|
||||
**Owner editing.** Mirrors post editing exactly: an owner-only **Edit** menu item (`data-modal="edit-project-modal"`) opens the `modal()` macro's `edit-project-modal`, a plain `POST` form to `/projects/edit/{slug}` (route `edit_project` in `routers/projects/index.py`, body `ProjectEditForm`, owner-gated through the shared `content.edit_content_item` which returns 403 JSON / redirect for non-owners and stamps `updated_at`). The modal is the create modal pre-filled from the `project` row (title, description, type/status radios pre-checked, dates via `format_date()` back to DD/MM/YYYY). `is_private`/`read_only` are NOT edited here - they stay on their dedicated toggles. The platforms tag widget reuses the create modal's `platforms-input`/`platforms`/`platforms-tags` ids; `ProfileEditor.initPlatformTags` now **seeds existing tags** from the hidden `#platforms` value on load, so both the empty create form and the pre-filled edit form work from the same code. Devii tool `edit_project`; documented in `docs_api.py` (`projects-edit`).
|
||||
|
||||
|
||||
@ -14,7 +14,7 @@ from devplacepy.models import TunnelForm
|
||||
from devplacepy.responses import action_result, json_error, respond
|
||||
from devplacepy.schemas import WorkspaceOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.containers import activity, forward, store
|
||||
from devplacepy.services.containers import activity, api, forward, store
|
||||
from devplacepy.services.containers.workspace import provision, quota, tunnels
|
||||
from devplacepy.services.containers.workspace.provision import WorkspaceError
|
||||
from devplacepy.utils import not_found, require_user
|
||||
@ -75,6 +75,9 @@ async def workspace_page(request: Request, slug: str):
|
||||
if instance
|
||||
else ""
|
||||
),
|
||||
"editor_password": (
|
||||
api.ensure_editor_password(instance) if instance else ""
|
||||
),
|
||||
"user": user,
|
||||
}
|
||||
return respond(request, "workspace.html", context, model=WorkspaceOut)
|
||||
|
||||
@ -39,6 +39,7 @@ from devplacepy.content import (
|
||||
is_owner,
|
||||
can_view_project,
|
||||
can_view_project_containers,
|
||||
can_open_workspace,
|
||||
get_project_devlog,
|
||||
)
|
||||
from devplacepy.utils import (
|
||||
@ -175,6 +176,19 @@ async def projects_page(
|
||||
model=ProjectsOut,
|
||||
)
|
||||
|
||||
def _editor_url(project: dict, user: dict) -> str:
|
||||
from devplacepy.services.containers import store
|
||||
from devplacepy.services.containers.workspace import provision
|
||||
|
||||
instance = provision.find_for_project(project["uid"], user["uid"])
|
||||
if not instance or instance.get("suspended_at"):
|
||||
return ""
|
||||
if instance.get("status") != store.ST_RUNNING:
|
||||
return ""
|
||||
slug = project["slug"] or project["uid"]
|
||||
return f"/projects/{slug}/containers/instances/{instance['uid']}/code/"
|
||||
|
||||
|
||||
@router.get("/{project_slug}", response_class=HTMLResponse)
|
||||
async def project_detail(request: Request, project_slug: str, before: str = None):
|
||||
user = get_current_user(request)
|
||||
@ -211,6 +225,10 @@ async def project_detail(request: Request, project_slug: str, before: str = None
|
||||
],
|
||||
schemas=[website_schema(base), software_application_schema(project, base)],
|
||||
)
|
||||
viewer_can_workspace = can_open_workspace(project, user)
|
||||
workspace_editor_url = (
|
||||
_editor_url(project, user) if viewer_can_workspace else ""
|
||||
)
|
||||
parent = get_fork_parent(project["uid"])
|
||||
forked_from = (
|
||||
{
|
||||
@ -256,6 +274,8 @@ async def project_detail(request: Request, project_slug: str, before: str = None
|
||||
"is_private": bool(project.get("is_private")),
|
||||
"read_only": bool(project.get("read_only")),
|
||||
"viewer_can_containers": can_view_project_containers(project, user),
|
||||
"viewer_can_workspace": viewer_can_workspace,
|
||||
"workspace_editor_url": workspace_editor_url,
|
||||
"forked_from": forked_from,
|
||||
"fork_count": count_forks(project["uid"]),
|
||||
"file_count": count_files(project["uid"]),
|
||||
|
||||
@ -163,6 +163,7 @@ class WorkspaceOut(_Out):
|
||||
workspace_count: int = 0
|
||||
max_workspaces: int = 0
|
||||
editor_url: str = ""
|
||||
editor_password: str = ""
|
||||
user: Optional[Any] = None
|
||||
|
||||
|
||||
|
||||
@ -163,6 +163,8 @@ class ProjectDetailOut(_Out):
|
||||
is_private: bool = False
|
||||
read_only: bool = False
|
||||
viewer_can_containers: bool = False
|
||||
viewer_can_workspace: bool = False
|
||||
workspace_editor_url: Optional[str] = None
|
||||
forked_from: Optional[dict] = None
|
||||
fork_count: int = 0
|
||||
file_count: int = 0
|
||||
|
||||
@ -211,7 +211,7 @@ admin container manager's authorization is unchanged; workspaces add their own n
|
||||
|---|---|---|
|
||||
| Entry | `/projects/{slug}/containers/instances/{uid}/code/...` | `{port}-{name}.tunnel.pravda.education` |
|
||||
| Auth | session + `can_manage_workspace` | none, public by design |
|
||||
| Backend | code-server, `--auth none`, bound in-container | whatever the user runs |
|
||||
| Backend | code-server, `--auth password`, bound in-container | whatever the user runs |
|
||||
|
||||
Plane B routing is **one static molohttp site** `*.tunnel.pravda.education -> 127.0.0.1:10500`;
|
||||
DevPlace resolves the instance from the `Host` header. There is no molohttp object per tunnel.
|
||||
@ -241,8 +241,29 @@ patterns plus `is_tunnel_host`. `tunnels.py` is CRUD with revive-not-duplicate.
|
||||
create/resume/stop/suspend/view surface and writes `/app/.devplace/tunnels.json`.
|
||||
|
||||
**`WorkspaceService`** is the only new service: lock-owner, `default_enabled=False`, four wrapped
|
||||
phases (disk sample on its own slow cadence, flag evaluation, lifecycle, purge sweep). It is a
|
||||
reconciler, not a `JobService`.
|
||||
synchronous phases (disk sample on its own slow cadence, flag evaluation, lifecycle, purge sweep)
|
||||
plus the async `_issue_tunnel_certificates`. It is a reconciler, not a `JobService`.
|
||||
|
||||
**Tunnel certificates are per-host, and that is forced by molohttp, not a preference.** molohttp's
|
||||
ACME client implements **`http-01` only** (`Sources/MoloHTTP/ACME/AcmeClient.swift`,
|
||||
`AcmeRenewalService.swift` - there is no `dns-01` anywhere), and Let's Encrypt will not issue a
|
||||
wildcard over `http-01`. So a `*.tunnel.pravda.education` **certificate** is impossible today and
|
||||
`workspace_cert_mode=per_host` is the only working mode; the prose above about one static wildcard
|
||||
site describes **routing**, which is correct and already in place (one enabled molohttp site
|
||||
`*.tunnel.pravda.education -> http://127.0.0.1:10500`, no per-tunnel site object). Only the cert is
|
||||
per-host. `tunnels.create` writes a row at `STATUS_PENDING` and nothing else; the row is inert until
|
||||
`_issue_tunnel_certificates` picks it up - `pending` is not in `SERVING_STATUSES`, so an
|
||||
un-provisioned tunnel 404s. The phase reads `tunnels.awaiting_certificate()` (`status=pending` +
|
||||
`desired_state=present` + not deleted), flips the row to `provisioning`, calls
|
||||
`certs.issue(hostname)` (`workspace/certs.py`, `POST {workspace_molohttp_base_url}/api/v1/certs/issue`
|
||||
with the `x-api-key` header, Basic as fallback), and lands on `active` or on `failed` with
|
||||
`last_error`. **Only `pending` is retried** - a `failed` row stays failed until the user recreates
|
||||
the tunnel (`create` revives it to `pending`), which keeps a broken host from burning the Let's
|
||||
Encrypt failure rate limit every 30s. **Renewal is molohttp's job, not DevPlace's**: once a host is
|
||||
issued, `AcmeRenewalService` re-issues it against its own expiry threshold forever, so DevPlace never
|
||||
schedules or tracks renewals. This whole phase did not exist - `tunnels` had no reader at all, and
|
||||
the six `workspace_molohttp_*`/`workspace_cert_mode`/`workspace_acme_email` settings were admin
|
||||
fields wired to nothing, which is why every tunnel sat at `pending` with no certificate.
|
||||
|
||||
**Two contracts that bite:**
|
||||
- A `ConfigField` with `type="select"` needs `options=[{"value": ..., "label": ...}]`. Plain strings
|
||||
@ -252,6 +273,28 @@ reconciler, not a `JobService`.
|
||||
required field makes an anonymous request 422 instead of 401 and `tests/api/auth/matrix.py` fails.
|
||||
Give the field a default and validate it inside the handler after `require_user`.
|
||||
|
||||
**The editor opens in a new tab, directly.** The project detail page renders an inline **VS Code**
|
||||
button in `.project-detail-actions` (`target="_blank"`) straight to the code-server proxy
|
||||
`/projects/{slug}/containers/instances/{uid}/code/`, built by `_editor_url` in
|
||||
`routers/projects/index.py` and carried as `workspace_editor_url` on the context and
|
||||
`ProjectDetailOut`. It is emitted ONLY when the viewer passes `can_open_workspace` AND their
|
||||
workspace for that project exists, is not suspended, and is `store.ST_RUNNING` - the three states the
|
||||
`editor_proxy` route itself refuses (403 suspended, 409 not running, 502 no port), so the button can
|
||||
never open a dead editor. When there is no running workspace the button is absent and the Workspace
|
||||
menu item below is the way in (create/start it there). The workspace page's own **Open editor** link
|
||||
opens in a new tab too; keep both in step.
|
||||
|
||||
**Member entry point** is the project detail page's overflow menu (`project_detail.html`), gated by
|
||||
the `viewer_can_workspace` context flag (`can_open_workspace(project, user)`, set in
|
||||
`routers/projects/index.py` and declared on `ProjectDetailOut`) - exactly the pattern the admin-only
|
||||
**Containers** item uses with `viewer_can_containers`. `can_open_workspace` folds in the
|
||||
`workspace_enabled` master switch, so the item disappears for everyone while the feature is off and
|
||||
the route's own `_guard` stays the authority. **A workspace surface with no context flag is
|
||||
unreachable**: the whole feature shipped once with routes, Devii tools and docs but no link into
|
||||
`/projects/{slug}/workspace`, so it was reachable only by typing the URL - and then 404'd anyway
|
||||
because `workspace_enabled` defaults to `"0"`. Any new workspace surface needs both the flag on the
|
||||
page that links to it and the setting turned on.
|
||||
|
||||
**Admin console** is `/admin/workspaces` (`routers/admin/workspaces.py`, `admin_workspaces.html`):
|
||||
list, start/stop, suspend/unsuspend with a required reason, raise/resolve/dismiss flags, per-user
|
||||
quota rules. `base_seo_context` takes `breadcrumbs`/`schemas`, not `canonical`/`schema`.
|
||||
@ -260,6 +303,62 @@ quota rules. `base_seo_context` takes `breadcrumbs`/`schemas`, not `canonical`/`
|
||||
them declares a `confirm` param (schemas are `additionalProperties: false`, so a gated tool without it
|
||||
loops forever).
|
||||
|
||||
**Opening a workspace publishes its editor on a public tunnel, automatically.** `provision.ensure`
|
||||
does three things after creating the instance: `api.ensure_editor_password`, then
|
||||
`publish_editor_tunnel` (a `tunnels.create` for the editor port, so the hostname is
|
||||
`{editor_port}-{name}.{domain}`), and the row lands at `pending`. The `WorkspaceService` cert phase
|
||||
then issues the certificate and calls `_announce_tunnel`, which sends the owner a `workspace`
|
||||
notification carrying the live `https://` URL **and the password**. The notification fires on
|
||||
`pending -> active`, not at creation, so the user is only told about a URL that already serves TLS.
|
||||
`provision.is_editor_tunnel(instance, tunnel)` distinguishes the auto-published editor tunnel (its
|
||||
`container_port` equals `editor_port`) from user-created ones, which get a plainer message with no
|
||||
password. This is a deliberate departure from the plane-A/plane-B split described above: the editor
|
||||
is now reachable publicly, which is only acceptable **because** `--auth password` is on - never
|
||||
reintroduce `--auth none` while the editor tunnel is auto-published.
|
||||
|
||||
**The editor is password-protected, per workspace.** `api.editor_command` runs code-server with
|
||||
`--auth password`, and `api.pravda_env` injects the secret as `PASSWORD` (the variable code-server
|
||||
reads). The secret is an 8-character **pronounceable** token from `api.generate_editor_password()`,
|
||||
built as four consonant-vowel pairs (`ronebamu`, `zipesodu`) so a user can read it once and retype it
|
||||
from memory. That is ~24 bits of entropy, which is deliberately weak for convenience: it is
|
||||
acceptable only because the alternative in practice was `--auth none`. If the editor tunnel is ever
|
||||
exposed to untrusted traffic at scale, lengthen the pair count rather than switching alphabet, and
|
||||
keep it pronounceable. `api.ensure_editor_password(instance)` is the single generator/persister: it returns the
|
||||
stored value or mints and saves one, so the password is **stable** across restarts and recreations.
|
||||
It is called from `api.run_spec_for` whenever `is_workspace` - the one point every workspace launch
|
||||
passes through - so a legacy instance created before this column existed gets one on its next boot
|
||||
rather than letting code-server invent an unknowable password of its own. The owner reads it off the
|
||||
workspace page (`editor_password` on the context and on `WorkspaceOut`). **It is deliberately NOT on
|
||||
`WorkspaceViewOut`**: that model is shared with `AdminWorkspacesOut`, which lists every workspace on
|
||||
the instance, so putting it there would hand every user's editor password to any admin loading
|
||||
`/admin/workspaces`. Keep per-workspace secrets on the owner-scoped model only. The column is
|
||||
`instances.editor_password`, ensured in `init_db`. This closes the hole that made a tunnelled editor
|
||||
an unauthenticated public shell: `--auth none` was safe only while the editor was reachable solely
|
||||
through the session-authenticated proxy route, and a user can publish a tunnel to the editor port.
|
||||
|
||||
**code-server lives in the `ppy` image, and nothing else supplies it.** `api.editor_command` makes
|
||||
`code-server` the container's argv, so an image without it fails `docker run` with exit **127**
|
||||
(`executable file not found in $PATH`) and the reconciler records `crashed` / `launch_failed` with an
|
||||
empty `container_id` - the container process never existed. It is installed in `ppy.Dockerfile` in
|
||||
the runtime stage **before `USER pravda`** and before the `chown -R pravda` block (so uid 1000 owns
|
||||
it with no elevation): version pinned in the single `ARG CODE_SERVER_VERSION`, arch resolved from
|
||||
`dpkg --print-architecture`, release tarball unpacked to `/usr/local/lib/code-server` with a symlink
|
||||
at `/usr/local/bin/code-server`. `code-server --version` is in the build smoke-test loop and the
|
||||
symlink is in the executable-check list, so an image that cannot run the editor can never build
|
||||
green. Bumping the version is a one-line `ARG` change plus `make ppy`. **`workspace_editor_version`
|
||||
(a `ConfigField` on `WorkspaceService`) is currently read by nothing** - the version is the image's,
|
||||
not a runtime setting; wire it up or drop it before relying on it.
|
||||
|
||||
**A container stuck in `created` is recreated, never retried forever.** The reconciler's
|
||||
`desired=running` branch reaches `backend.start()` for `ps.state == "created"`. That call is wrapped:
|
||||
on failure it logs, records an instance event `recreated`, `rm -f`s the container and calls
|
||||
`_launch`, so the instance is rebuilt from the CURRENT image. Without this, a container created
|
||||
against an older image that can no longer start is a permanent wedge - `docker start` re-resolves
|
||||
nothing, so the reconciler loops on the same error every tick until someone removes it by hand (the
|
||||
real failure: after code-server was added to `ppy`, the already-created workspace container kept
|
||||
failing `docker start` with the old 127 even though the new image was correct). Recreation is not a
|
||||
new retry class: a launch that fails already leaves `ps is None`, which `_launch`es again next tick.
|
||||
|
||||
**Toolchains in `ppy`.** Rust (rustup), Nim (choosenim), Swift (swiftly, then the toolchain is moved
|
||||
to a fixed `/opt/swift/toolchain` because swiftly's proxy resolves against `$HOME` and breaks for
|
||||
`pravda` at runtime). The choosenim installer **exits 1 even on success**, so its `RUN` ends with
|
||||
|
||||
@ -3,6 +3,7 @@
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
import secrets
|
||||
import socket
|
||||
from pathlib import Path
|
||||
|
||||
@ -487,6 +488,7 @@ def workspace_env(instance: dict, base_url: str) -> dict:
|
||||
"DEVPLACE_TUNNEL_MANIFEST": f"{WORKSPACE_MOUNT}/.devplace/tunnels.json",
|
||||
"DEVPLACE_TUNNEL_MAX": str(limits.max_tunnels),
|
||||
"VSCODE_PROXY_URI": naming.proxy_uri_template(name),
|
||||
"PASSWORD": instance.get("editor_password") or "",
|
||||
"DEVPLACE_EDITOR": "code-server",
|
||||
"DEVPLACE_EDITOR_PORT": str(editor_port),
|
||||
"DEVPLACE_EDITOR_URL": (
|
||||
@ -509,6 +511,28 @@ def workspace_env(instance: dict, base_url: str) -> dict:
|
||||
|
||||
|
||||
EDITOR_DEFAULT_PORT = 8443
|
||||
EDITOR_PASSWORD_LENGTH = 8
|
||||
EDITOR_PASSWORD_CONSONANTS = "bdfgkmnprstvz"
|
||||
EDITOR_PASSWORD_VOWELS = "aeiou"
|
||||
|
||||
|
||||
def generate_editor_password() -> str:
|
||||
pairs = EDITOR_PASSWORD_LENGTH // 2
|
||||
return "".join(
|
||||
secrets.choice(EDITOR_PASSWORD_CONSONANTS)
|
||||
+ secrets.choice(EDITOR_PASSWORD_VOWELS)
|
||||
for _ in range(pairs)
|
||||
)
|
||||
|
||||
|
||||
def ensure_editor_password(instance: dict) -> str:
|
||||
password = (instance.get("editor_password") or "").strip()
|
||||
if password:
|
||||
return password
|
||||
password = generate_editor_password()
|
||||
store.update_instance(instance["uid"], {"editor_password": password})
|
||||
instance["editor_password"] = password
|
||||
return password
|
||||
|
||||
|
||||
def editor_command(instance: dict) -> list[str]:
|
||||
@ -518,7 +542,7 @@ def editor_command(instance: dict) -> list[str]:
|
||||
"--bind-addr",
|
||||
f"0.0.0.0:{port}",
|
||||
"--auth",
|
||||
"none",
|
||||
"password",
|
||||
"--disable-telemetry",
|
||||
"--disable-update-check",
|
||||
"--user-data-dir",
|
||||
@ -530,6 +554,8 @@ def editor_command(instance: dict) -> list[str]:
|
||||
|
||||
|
||||
def run_spec_for(instance: dict, image_tag: str) -> RunSpec:
|
||||
if instance.get("is_workspace"):
|
||||
ensure_editor_password(instance)
|
||||
env = {**json.loads(instance.get("env_json") or "{}"), **pravda_env(instance)}
|
||||
ports = [
|
||||
PortMapping(p["host"], p["container"], p.get("proto", "tcp"))
|
||||
|
||||
@ -36,9 +36,10 @@ def forward_headers(request: Request, prefix: str = "") -> dict:
|
||||
if prefix:
|
||||
headers["X-Forwarded-Prefix"] = prefix
|
||||
headers["X-Script-Name"] = prefix
|
||||
headers["X-Forwarded-Host"] = request.headers.get(
|
||||
"host", request.url.hostname or ""
|
||||
)
|
||||
origin_host = request.headers.get("host", request.url.hostname or "")
|
||||
headers["X-Forwarded-Host"] = origin_host
|
||||
if origin_host:
|
||||
headers["Host"] = origin_host
|
||||
headers["X-Forwarded-Proto"] = request.headers.get(
|
||||
"x-forwarded-proto", request.url.scheme
|
||||
)
|
||||
@ -46,6 +47,21 @@ def forward_headers(request: Request, prefix: str = "") -> dict:
|
||||
return headers
|
||||
|
||||
|
||||
WS_FORWARD_HEADERS = ("cookie", "authorization", "user-agent", "origin")
|
||||
|
||||
|
||||
def ws_headers(websocket) -> dict:
|
||||
headers = {
|
||||
name: websocket.headers[name]
|
||||
for name in WS_FORWARD_HEADERS
|
||||
if name in websocket.headers
|
||||
}
|
||||
origin_host = websocket.headers.get("host", "")
|
||||
if origin_host:
|
||||
headers["Host"] = origin_host
|
||||
return headers
|
||||
|
||||
|
||||
def inject_base(body: bytes, prefix: str) -> bytes:
|
||||
lowered = body.lower()
|
||||
if b"<base" in lowered:
|
||||
@ -128,7 +144,10 @@ async def proxy_ws(
|
||||
await websocket.accept()
|
||||
try:
|
||||
async with websockets.connect(
|
||||
upstream_url, open_timeout=10, max_size=None
|
||||
upstream_url,
|
||||
open_timeout=10,
|
||||
max_size=None,
|
||||
additional_headers=ws_headers(websocket),
|
||||
) as upstream:
|
||||
await pump(websocket, upstream)
|
||||
except Exception as error:
|
||||
|
||||
@ -168,7 +168,16 @@ class ContainerService(BaseService):
|
||||
await backend.unpause(ps.container_id)
|
||||
_set_status(inst, {"status": store.ST_RUNNING}, reason="unpause")
|
||||
elif ps.state == "created":
|
||||
await backend.start(ps.container_id)
|
||||
try:
|
||||
await backend.start(ps.container_id)
|
||||
except Exception as exc:
|
||||
self.log(f"start {inst['name']} failed, recreating: {exc}")
|
||||
store.record_event(
|
||||
inst, "recreated", "reconciler", "", {"reason": str(exc)}
|
||||
)
|
||||
await backend.rm(ps.container_id, force=True)
|
||||
await self._launch(backend, inst)
|
||||
return
|
||||
_set_status(inst, {"status": store.ST_RUNNING}, reason="start")
|
||||
elif status == store.ST_RUNNING:
|
||||
await self._handle_exit(backend, inst, ps)
|
||||
|
||||
64
devplacepy/services/containers/workspace/certs.py
Normal file
64
devplacepy/services/containers/workspace/certs.py
Normal file
@ -0,0 +1,64 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from devplacepy.database import get_setting
|
||||
from devplacepy.stealth import stealth_async_client
|
||||
|
||||
API_PATH = "/api/v1"
|
||||
ISSUE_TIMEOUT_SECONDS = 180.0
|
||||
|
||||
|
||||
class CertError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _base_url() -> str:
|
||||
return (get_setting("workspace_molohttp_base_url", "") or "").rstrip("/")
|
||||
|
||||
|
||||
def _auth_headers() -> dict[str, str]:
|
||||
api_key = get_setting("workspace_molohttp_api_key", "") or ""
|
||||
if api_key:
|
||||
return {"x-api-key": api_key}
|
||||
return {}
|
||||
|
||||
|
||||
def _basic_auth() -> tuple[str, str] | None:
|
||||
username = get_setting("workspace_molohttp_username", "") or ""
|
||||
password = get_setting("workspace_molohttp_password", "") or ""
|
||||
if username and password:
|
||||
return (username, password)
|
||||
return None
|
||||
|
||||
|
||||
def configured() -> bool:
|
||||
return bool(_base_url()) and bool(_auth_headers() or _basic_auth())
|
||||
|
||||
|
||||
async def issue(hostname: str) -> None:
|
||||
if not hostname:
|
||||
raise CertError("no hostname")
|
||||
base = _base_url()
|
||||
if not base:
|
||||
raise CertError("workspace_molohttp_base_url is not configured")
|
||||
headers = _auth_headers()
|
||||
auth = _basic_auth()
|
||||
if not headers and not auth:
|
||||
raise CertError("no molohttp credentials configured")
|
||||
payload: dict[str, object] = {"domains": [hostname]}
|
||||
email = get_setting("workspace_acme_email", "") or ""
|
||||
if email:
|
||||
payload["email"] = email
|
||||
async with stealth_async_client(timeout=ISSUE_TIMEOUT_SECONDS) as client:
|
||||
response = await client.post(
|
||||
f"{base}{API_PATH}/certs/issue",
|
||||
json=payload,
|
||||
headers=headers,
|
||||
auth=auth,
|
||||
)
|
||||
if response.status_code >= 400:
|
||||
raise CertError(
|
||||
f"molohttp cert issue failed ({response.status_code}): "
|
||||
f"{response.text[:200]}"
|
||||
)
|
||||
@ -61,9 +61,24 @@ async def ensure(project: dict, user: dict) -> dict:
|
||||
"desired_state": "running",
|
||||
},
|
||||
)
|
||||
instance = store.get_instance(instance["uid"])
|
||||
api.ensure_editor_password(instance)
|
||||
publish_editor_tunnel(instance, owner_uid)
|
||||
return store.get_instance(instance["uid"])
|
||||
|
||||
|
||||
def publish_editor_tunnel(instance: dict, owner_uid: str) -> dict | None:
|
||||
port = int(instance.get("editor_port") or api.EDITOR_DEFAULT_PORT)
|
||||
if not port or not instance.get("tunnel_name"):
|
||||
return None
|
||||
return tunnels.create(instance, "Editor", port, owner_uid)
|
||||
|
||||
|
||||
def is_editor_tunnel(instance: dict, tunnel: dict) -> bool:
|
||||
editor_port = int(instance.get("editor_port") or api.EDITOR_DEFAULT_PORT)
|
||||
return int(tunnel.get("container_port") or 0) == editor_port
|
||||
|
||||
|
||||
def resume(instance: dict) -> dict:
|
||||
if instance.get("suspended_at"):
|
||||
raise WorkspaceError("this workspace is suspended; contact an administrator")
|
||||
|
||||
@ -114,6 +114,17 @@ def create(
|
||||
return table.find_one(uid=uid)
|
||||
|
||||
|
||||
def awaiting_certificate() -> list[dict]:
|
||||
return list(
|
||||
_table().find(
|
||||
status=STATUS_PENDING,
|
||||
desired_state="present",
|
||||
deleted_at=None,
|
||||
order_by=["created_at"],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def update(uid: str, changes: dict) -> None:
|
||||
_table().update({"uid": uid, "updated_at": _now(), **changes}, ["uid"])
|
||||
|
||||
|
||||
@ -11,7 +11,13 @@ from devplacepy import config
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.services.base import BaseService, ConfigField
|
||||
from devplacepy.services.containers import store
|
||||
from devplacepy.services.containers.workspace import flags, quota, tunnels
|
||||
from devplacepy.services.containers.workspace import (
|
||||
certs,
|
||||
flags,
|
||||
provision,
|
||||
quota,
|
||||
tunnels,
|
||||
)
|
||||
|
||||
DISK_SAMPLE_DEFAULT_MINUTES = 10
|
||||
|
||||
@ -161,6 +167,63 @@ class WorkspaceService(BaseService):
|
||||
phase(rows, cfg)
|
||||
except Exception as error:
|
||||
self.log(f"{phase.__name__} failed: {error}")
|
||||
try:
|
||||
await self._issue_tunnel_certificates()
|
||||
except Exception as error:
|
||||
self.log(f"_issue_tunnel_certificates failed: {error}")
|
||||
|
||||
async def _issue_tunnel_certificates(self) -> None:
|
||||
if not certs.configured():
|
||||
return
|
||||
for row in tunnels.awaiting_certificate():
|
||||
hostname = row.get("hostname", "")
|
||||
tunnels.update(
|
||||
row["uid"],
|
||||
{"status": tunnels.STATUS_PROVISIONING, "last_error": ""},
|
||||
)
|
||||
try:
|
||||
await certs.issue(hostname)
|
||||
except Exception as error:
|
||||
tunnels.update(
|
||||
row["uid"],
|
||||
{
|
||||
"status": tunnels.STATUS_FAILED,
|
||||
"last_error": str(error)[:500],
|
||||
},
|
||||
)
|
||||
self.log(f"tunnel certificate for {hostname} failed: {error}")
|
||||
continue
|
||||
tunnels.update(
|
||||
row["uid"], {"status": tunnels.STATUS_ACTIVE, "last_error": ""}
|
||||
)
|
||||
self.log(f"tunnel certificate issued for {hostname}")
|
||||
self._announce_tunnel(row, hostname)
|
||||
|
||||
def _announce_tunnel(self, row: dict, hostname: str) -> None:
|
||||
from devplacepy.utils import create_notification
|
||||
|
||||
instance = store.get_instance(row.get("instance_uid", ""))
|
||||
if not instance:
|
||||
return
|
||||
owner_uid = instance.get("workspace_owner_uid", "")
|
||||
if not owner_uid:
|
||||
return
|
||||
url = f"https://{hostname}"
|
||||
if provision.is_editor_tunnel(instance, row):
|
||||
password = instance.get("editor_password") or ""
|
||||
message = (
|
||||
f"Your VS Code workspace is live at {url} "
|
||||
f"(password: {password})."
|
||||
)
|
||||
else:
|
||||
message = f"Tunnel {row.get('label') or hostname} is live at {url}."
|
||||
create_notification(
|
||||
owner_uid,
|
||||
"workspace",
|
||||
message,
|
||||
instance["uid"],
|
||||
f"/projects/{instance.get('project_uid', '')}/workspace",
|
||||
)
|
||||
|
||||
def _sample_disk(self, rows: list[dict], cfg: dict) -> None:
|
||||
interval = int(cfg.get("workspace_disk_sample_minutes") or
|
||||
|
||||
@ -166,6 +166,10 @@ The Devii LLM-client and `PlatformClient` read timeout is `timeout_seconds` (fie
|
||||
|
||||
Config is all `config_fields` (AI url/model/key, base url, plan/verify toggles, max iterations, JS-execution toggle, user/guest 24h caps, guests toggle, pricing). `effective_config()` falls back the AI key to `DEVII_AI_KEY` and the base url to the instance origin.
|
||||
|
||||
## Every controller `dispatch` MUST return a JSON string (hard rule)
|
||||
|
||||
`Dispatcher.dispatch` is annotated `-> str` and its result is placed verbatim into the tool message as `{"role": "tool", "content": result}` (`agentic/loop.py`). A controller that returns a raw `dict`/`list`/`None` instead of a serialized string is never caught locally - `wrap_if_large` measures `len(result)`, which on a dict is the **key count**, so a small dict sails through unchunked and untruncated - and the request dies at the upstream with `400 ... content should be a string or a list`, killing the whole conversation mid-turn with no server-side traceback. This was a real production failure: `WorkspaceController.dispatch` was annotated `-> Any` and returned raw dicts from all 20 of its handlers, so any turn that called `workspace_list` (or any other workspace tool) 400'd. The same shape also silently disabled chunking for those payloads. Serialize at the controller's single `dispatch` choke point (`json.dumps(payload, ensure_ascii=False, default=str)` - `default=str` because DB rows carry datetimes), never per handler, and keep the `-> str` annotation so the contract is visible. `tests/api/projects/workspace.py::test_workspace_controller_always_returns_a_json_string` guards it.
|
||||
|
||||
## Backend confidentiality
|
||||
|
||||
Devii must never disclose the underlying model, provider, or any upstream URL - enforced in two layers. The `SYSTEM_PROMPT` (`agent.py`) forbids it even when such values appear inside a tool result. Structurally, `text.redact_backend()` runs on every JSON HTTP tool response in `format_response()` and scrubs the values of `REDACT_FIELD_KEYS` (`gateway_upstream_url`/`gateway_model`/`gateway_vision_url`/`gateway_vision_model`) and any stat labelled in `REDACT_STAT_LABELS` (`Model`) to `[hidden]`, so the admin-services tools cannot leak the gateway's upstream even though the admin settings UI still shows the real values. Add a config key to `REDACT_FIELD_KEYS` if a new field would expose backend infrastructure to the agent.
|
||||
|
||||
@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from devplacepy.database import get_table, resolve_by_slug
|
||||
@ -45,17 +46,21 @@ class WorkspaceController:
|
||||
raise WorkspaceError("no workspace exists for this project")
|
||||
return instance
|
||||
|
||||
async def dispatch(self, name: str, args: dict[str, Any]) -> Any:
|
||||
@staticmethod
|
||||
def _encode(payload: Any) -> str:
|
||||
return json.dumps(payload, ensure_ascii=False, default=str)
|
||||
|
||||
async def dispatch(self, name: str, args: dict[str, Any]) -> str:
|
||||
handler = getattr(self, f"_{name}", None)
|
||||
if handler is None:
|
||||
return {"error": f"unknown workspace action: {name}"}
|
||||
return self._encode({"error": f"unknown workspace action: {name}"})
|
||||
try:
|
||||
result = handler(args)
|
||||
if hasattr(result, "__await__"):
|
||||
return await result
|
||||
return result
|
||||
result = await result
|
||||
return self._encode(result)
|
||||
except WorkspaceError as error:
|
||||
return {"error": str(error)}
|
||||
return self._encode({"error": str(error)})
|
||||
|
||||
async def _workspace_open(self, args: dict) -> Any:
|
||||
user = self._user()
|
||||
|
||||
@ -45,6 +45,19 @@
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.workspace-password {
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
||||
.workspace-password code {
|
||||
color: var(--text-primary);
|
||||
background: var(--bg-input);
|
||||
font-family: var(--font-mono);
|
||||
padding: var(--space-xs) var(--space-sm);
|
||||
border-radius: var(--radius);
|
||||
user-select: all;
|
||||
}
|
||||
|
||||
.workspace-tunnel-list {
|
||||
list-style: none;
|
||||
padding: 0;
|
||||
|
||||
@ -66,6 +66,9 @@
|
||||
|
||||
<div class="project-detail-actions">
|
||||
<a href="/projects/{{ project['slug'] or project['uid'] }}/files" class="project-star-btn"><span class="icon">📁</span><span class="label"> Files ({{ file_count }} files)</span></a>
|
||||
{% if workspace_editor_url %}
|
||||
<a href="{{ workspace_editor_url }}" target="_blank" rel="noopener" class="project-star-btn"><span class="icon">💻</span><span class="label"> VS Code</span></a>
|
||||
{% endif %}
|
||||
<button type="button" class="project-star-btn" data-share="/projects/{{ project['slug'] or project['uid'] }}"><span class="icon">🔗</span><span class="label"> Share</span></button>
|
||||
{% if user %}
|
||||
{% set _type = "project" %}{% set _uid = project['uid'] %}{% set _my_vote = my_vote %}{% set _count = star_count %}{% set _btn_class = "project-star-btn" %}{% include "_star_vote.html" %}
|
||||
@ -75,6 +78,9 @@
|
||||
<button type="button" class="project-star-btn project-actions-more" aria-haspopup="menu" aria-expanded="false" aria-label="More actions"><span class="icon">⋯</span><span class="label"> More</span></button>
|
||||
|
||||
<div class="project-actions-overflow" hidden>
|
||||
{% if viewer_can_workspace %}
|
||||
<a href="/projects/{{ project['slug'] or project['uid'] }}/workspace" data-menu-action data-menu-icon="💻" data-menu-label="Workspace">Workspace</a>
|
||||
{% endif %}
|
||||
{% if viewer_can_containers %}
|
||||
<a href="/projects/{{ project['slug'] or project['uid'] }}/containers" data-menu-action data-menu-icon="🖥️" data-menu-label="Containers">Containers</a>
|
||||
{% endif %}
|
||||
|
||||
@ -60,7 +60,7 @@
|
||||
</p>
|
||||
<div class="workspace-actions">
|
||||
{% if workspace.status == "running" %}
|
||||
<a class="btn btn-primary" href="{{ editor_url }}">Open editor</a>
|
||||
<a class="btn btn-primary" href="{{ editor_url }}" target="_blank" rel="noopener">Open editor</a>
|
||||
<form method="post" action="/projects/{{ project.slug or project.uid }}/workspace/stop">
|
||||
<button type="submit" class="btn">Stop</button>
|
||||
</form>
|
||||
@ -69,6 +69,9 @@
|
||||
<button type="submit" class="btn btn-primary">Start</button>
|
||||
</form>
|
||||
{% endif %}
|
||||
{% if editor_password %}
|
||||
<span class="workspace-password">Editor password <code>{{ editor_password }}</code></span>
|
||||
{% endif %}
|
||||
<form method="post" action="/projects/{{ project.slug or project.uid }}/workspace/delete">
|
||||
<button type="submit" data-confirm="Delete this workspace?" data-confirm-danger class="btn btn-danger">Delete</button>
|
||||
</form>
|
||||
|
||||
@ -2,9 +2,11 @@ upstream app {
|
||||
server app:10500;
|
||||
}
|
||||
|
||||
# A non-upgrade request maps to an EMPTY Connection header, not "close", so the
|
||||
# catch-all can use this map and still keep the upstream connection alive.
|
||||
map $http_upgrade $connection_upgrade {
|
||||
default upgrade;
|
||||
'' close;
|
||||
'' '';
|
||||
}
|
||||
|
||||
map $uri $upload_disposition {
|
||||
@ -126,6 +128,23 @@ server {
|
||||
proxy_send_timeout 3600s;
|
||||
}
|
||||
|
||||
# Workspace editor: code-server HTTP assets AND its websocket share one prefix
|
||||
# (/projects/<slug>/containers/instances/<uid>/code/...), so this location must
|
||||
# carry both. The catch-all sets Connection "" and would break the editor.
|
||||
location ~ ^/projects/[^/]+/containers/instances/[^/]+/code(/.*)?$ {
|
||||
proxy_pass http://app;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $connection_upgrade;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_buffering off;
|
||||
proxy_read_timeout 3600s;
|
||||
proxy_send_timeout 3600s;
|
||||
}
|
||||
|
||||
# Real-time direct messages websocket (/messages/ws).
|
||||
location = /messages/ws {
|
||||
proxy_pass http://app;
|
||||
@ -200,11 +219,16 @@ server {
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Connection "";
|
||||
# Tunnel hosts (*.tunnel.<domain>) are dispatched by the app and carry
|
||||
# arbitrary user apps, which routinely use websockets. They arrive here,
|
||||
# so the catch-all must forward the upgrade instead of dropping it -
|
||||
# dropping it is what closes a tunnelled websocket with status 1006.
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $connection_upgrade;
|
||||
|
||||
proxy_connect_timeout 30s;
|
||||
proxy_read_timeout 60s;
|
||||
proxy_send_timeout 60s;
|
||||
proxy_read_timeout 3600s;
|
||||
proxy_send_timeout 3600s;
|
||||
|
||||
${NGINX_CACHE_CONFIG}
|
||||
}
|
||||
|
||||
@ -74,6 +74,19 @@ RUN pip install \
|
||||
RUN pip install playwright && playwright install --with-deps chromium
|
||||
|
||||
FROM deps AS runtime
|
||||
ARG CODE_SERVER_VERSION=4.131.0
|
||||
RUN set -eu; \
|
||||
case "$(dpkg --print-architecture)" in \
|
||||
amd64) arch=amd64 ;; \
|
||||
arm64) arch=arm64 ;; \
|
||||
*) echo "unsupported architecture: $(dpkg --print-architecture)"; exit 1 ;; \
|
||||
esac; \
|
||||
curl -fsSL -o /tmp/code-server.tar.gz \
|
||||
"https://github.com/coder/code-server/releases/download/v${CODE_SERVER_VERSION}/code-server-${CODE_SERVER_VERSION}-linux-${arch}.tar.gz"; \
|
||||
mkdir -p /usr/local/lib/code-server; \
|
||||
tar -xzf /tmp/code-server.tar.gz -C /usr/local/lib/code-server --strip-components=1; \
|
||||
rm -f /tmp/code-server.tar.gz; \
|
||||
ln -sf /usr/local/lib/code-server/bin/code-server /usr/local/bin/code-server
|
||||
COPY sudo /usr/local/bin/sudo
|
||||
COPY aptroot /usr/local/bin/aptroot
|
||||
COPY pagent /usr/bin/pagent.py
|
||||
@ -117,13 +130,14 @@ RUN printf '%s\n' \
|
||||
|
||||
RUN set -eu; \
|
||||
for tool in "python --version" "rustc --version" "cargo --version" \
|
||||
"nim --version" "nimble --version" "swift --version"; do \
|
||||
"nim --version" "nimble --version" "swift --version" \
|
||||
"code-server --version"; do \
|
||||
$tool > /tmp/toolcheck 2>&1 || { echo "TOOLCHAIN FAILED: $tool"; cat /tmp/toolcheck; exit 1; }; \
|
||||
head -1 /tmp/toolcheck; \
|
||||
done; \
|
||||
rm -f /tmp/toolcheck; \
|
||||
for b in /usr/local/bin/sudo /usr/local/bin/aptroot /usr/bin/pagent.py \
|
||||
/usr/bin/botje.py /usr/bin/d.py /usr/bin/dpc; do \
|
||||
/usr/bin/botje.py /usr/bin/d.py /usr/bin/dpc /usr/local/bin/code-server; do \
|
||||
[ -x "$b" ] || { echo "missing or not executable: $b"; exit 1; }; \
|
||||
done; \
|
||||
[ -f /home/pravda/.vimrc ] || { echo "missing /home/pravda/.vimrc"; exit 1; }
|
||||
|
||||
@ -421,6 +421,36 @@ def test_private_detail_visible_to_admin(app_server):
|
||||
)
|
||||
|
||||
|
||||
def _await_workspace_flag(url, key, expected, timeout=10.0):
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
body = requests.get(url, headers=_h_project_visibility(key)).json()
|
||||
if body["viewer_can_workspace"] is expected:
|
||||
return True
|
||||
time.sleep(0.2)
|
||||
return False
|
||||
|
||||
|
||||
def test_detail_json_exposes_viewer_can_workspace(app_server):
|
||||
_, _, owner_key = _signup_project_visibility()
|
||||
_, _, other_key = _signup_project_visibility()
|
||||
slug = _create_project_project_visibility(owner_key, "Workspace Flag")["slug"]
|
||||
url = f"{BASE_URL}/projects/{slug}"
|
||||
previous = get_table("site_settings").find_one(key="workspace_enabled")
|
||||
set_setting("workspace_enabled", "1")
|
||||
try:
|
||||
assert _await_workspace_flag(url, owner_key, True)
|
||||
other = requests.get(url, headers=_h_project_visibility(other_key)).json()
|
||||
assert other["viewer_can_workspace"] is False
|
||||
|
||||
set_setting("workspace_enabled", "0")
|
||||
assert _await_workspace_flag(url, owner_key, False)
|
||||
finally:
|
||||
set_setting(
|
||||
"workspace_enabled", previous.get("value") if previous else "0"
|
||||
)
|
||||
|
||||
|
||||
def test_project_uid_redirects_to_canonical_slug(app_server):
|
||||
slug, uid = _seed_project()
|
||||
r = requests.get(f"{BASE_URL}/projects/{uid}", allow_redirects=False)
|
||||
|
||||
@ -1,10 +1,12 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from devplacepy.content import can_manage_workspace, can_open_workspace
|
||||
from devplacepy.database import get_table, init_db, set_setting
|
||||
from devplacepy.services.containers import activity, store
|
||||
from devplacepy.services.containers import activity, api, store
|
||||
from devplacepy.services.containers.workspace import (
|
||||
flags,
|
||||
naming,
|
||||
@ -214,6 +216,171 @@ def test_devii_workspace_tools_are_role_gated():
|
||||
assert names <= admin
|
||||
|
||||
|
||||
def test_opening_a_workspace_publishes_the_editor_tunnel_automatically():
|
||||
project = _project()
|
||||
user = {"uid": OWNER, "username": "owner"}
|
||||
instance = run_async(provision.ensure(project, user))
|
||||
rows = tunnels.list_for_instance(instance["uid"])
|
||||
assert len(rows) == 1
|
||||
editor = rows[0]
|
||||
assert editor["container_port"] == api.EDITOR_DEFAULT_PORT
|
||||
assert editor["hostname"].startswith(f"{api.EDITOR_DEFAULT_PORT}-")
|
||||
assert editor["status"] == tunnels.STATUS_PENDING
|
||||
assert provision.is_editor_tunnel(instance, editor)
|
||||
assert instance["editor_password"]
|
||||
|
||||
|
||||
def test_active_editor_tunnel_notifies_the_owner_with_url_and_password():
|
||||
from devplacepy.services.containers.workspace_service import WorkspaceService
|
||||
|
||||
project = _project()
|
||||
instance = run_async(provision.ensure(project, {"uid": OWNER, "username": "o"}))
|
||||
row = tunnels.list_for_instance(instance["uid"])[0]
|
||||
try:
|
||||
WorkspaceService()._announce_tunnel(row, row["hostname"])
|
||||
sent = list(
|
||||
get_table("notifications").find(
|
||||
user_uid=OWNER, type="workspace", order_by=["-id"]
|
||||
)
|
||||
)
|
||||
assert sent, "owner was not notified"
|
||||
message = sent[0]["message"]
|
||||
assert f"https://{row['hostname']}" in message
|
||||
assert instance["editor_password"] in message
|
||||
assert sent[0]["target_url"].endswith("/workspace")
|
||||
finally:
|
||||
get_table("notifications").delete(user_uid=OWNER)
|
||||
|
||||
|
||||
def test_editor_password_is_pronounceable_and_eight_chars():
|
||||
seen = {api.generate_editor_password() for _ in range(50)}
|
||||
assert len(seen) > 40
|
||||
for password in seen:
|
||||
assert len(password) == 8
|
||||
assert password.isalpha() and password.islower()
|
||||
for index, char in enumerate(password):
|
||||
expected = (
|
||||
api.EDITOR_PASSWORD_CONSONANTS
|
||||
if index % 2 == 0
|
||||
else api.EDITOR_PASSWORD_VOWELS
|
||||
)
|
||||
assert char in expected
|
||||
|
||||
|
||||
def test_editor_runs_with_password_auth_and_an_eight_char_secret():
|
||||
from devplacepy.services.containers import api
|
||||
|
||||
instance = _instance()
|
||||
password = api.ensure_editor_password(instance)
|
||||
assert len(password) == api.EDITOR_PASSWORD_LENGTH == 8
|
||||
assert password.isalnum()
|
||||
|
||||
command = api.editor_command(store.get_instance(instance["uid"]))
|
||||
assert "--auth" in command
|
||||
assert command[command.index("--auth") + 1] == "password"
|
||||
assert "none" not in command
|
||||
|
||||
|
||||
def test_editor_password_is_stable_and_reaches_the_container_env():
|
||||
from devplacepy.services.containers import api
|
||||
|
||||
instance = _instance()
|
||||
first = api.ensure_editor_password(instance)
|
||||
again = api.ensure_editor_password(store.get_instance(instance["uid"]))
|
||||
assert first == again
|
||||
|
||||
env = api.pravda_env(store.get_instance(instance["uid"]))
|
||||
assert env["PASSWORD"] == first
|
||||
|
||||
|
||||
def test_every_workspace_launch_gets_a_password_even_if_never_provisioned():
|
||||
from devplacepy.services.containers import api
|
||||
|
||||
instance = _instance()
|
||||
store.update_instance(instance["uid"], {"editor_password": ""})
|
||||
spec = api.run_spec_for(store.get_instance(instance["uid"]), "ppy:latest")
|
||||
assert len(spec.env["PASSWORD"]) == 8
|
||||
assert store.get_instance(instance["uid"])["editor_password"] == spec.env["PASSWORD"]
|
||||
|
||||
|
||||
def test_non_workspace_instance_never_gets_an_editor_password():
|
||||
from devplacepy.services.containers import api
|
||||
|
||||
instance = _instance(is_workspace=0)
|
||||
spec = api.run_spec_for(store.get_instance(instance["uid"]), "ppy:latest")
|
||||
assert not spec.env.get("PASSWORD")
|
||||
|
||||
|
||||
def test_awaiting_certificate_only_returns_pending_present_tunnels():
|
||||
instance = _instance()
|
||||
tunnels.create(instance, "web", 8080, OWNER)
|
||||
tunnels.create(instance, "api", 9090, OWNER)
|
||||
active = tunnels.list_for_instance(instance["uid"])[0]
|
||||
tunnels.update(active["uid"], {"status": tunnels.STATUS_ACTIVE})
|
||||
waiting = [row["uid"] for row in tunnels.awaiting_certificate()]
|
||||
assert active["uid"] not in waiting
|
||||
assert len(waiting) == 1
|
||||
|
||||
absent = tunnels.get(waiting[0])
|
||||
tunnels.mark_absent(absent["uid"])
|
||||
assert not tunnels.awaiting_certificate()
|
||||
|
||||
|
||||
def test_certificate_phase_marks_active_on_success_and_failed_on_error(monkeypatch):
|
||||
from devplacepy.services.containers.workspace import certs
|
||||
from devplacepy.services.containers.workspace_service import WorkspaceService
|
||||
|
||||
instance = _instance()
|
||||
tunnels.create(instance, "web", 8080, OWNER)
|
||||
row = tunnels.awaiting_certificate()[0]
|
||||
service = WorkspaceService()
|
||||
|
||||
monkeypatch.setattr(certs, "configured", lambda: True)
|
||||
|
||||
async def _ok(hostname):
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(certs, "issue", _ok)
|
||||
run_async(service._issue_tunnel_certificates())
|
||||
assert tunnels.get(row["uid"])["status"] == tunnels.STATUS_ACTIVE
|
||||
|
||||
tunnels.update(row["uid"], {"status": tunnels.STATUS_PENDING})
|
||||
|
||||
async def _boom(hostname):
|
||||
raise certs.CertError("molohttp cert issue failed (503): upstream down")
|
||||
|
||||
monkeypatch.setattr(certs, "issue", _boom)
|
||||
run_async(service._issue_tunnel_certificates())
|
||||
failed = tunnels.get(row["uid"])
|
||||
assert failed["status"] == tunnels.STATUS_FAILED
|
||||
assert "503" in failed["last_error"]
|
||||
|
||||
|
||||
def test_certificate_phase_is_a_noop_without_molohttp_credentials(monkeypatch):
|
||||
from devplacepy.services.containers.workspace import certs
|
||||
from devplacepy.services.containers.workspace_service import WorkspaceService
|
||||
|
||||
instance = _instance()
|
||||
tunnels.create(instance, "web", 8080, OWNER)
|
||||
row = tunnels.awaiting_certificate()[0]
|
||||
monkeypatch.setattr(certs, "configured", lambda: False)
|
||||
run_async(WorkspaceService()._issue_tunnel_certificates())
|
||||
assert tunnels.get(row["uid"])["status"] == tunnels.STATUS_PENDING
|
||||
|
||||
|
||||
def test_workspace_controller_always_returns_a_json_string():
|
||||
from devplacepy.services.devii.registry import CATALOG
|
||||
from devplacepy.services.devii.workspace.controller import WorkspaceController
|
||||
|
||||
controller = WorkspaceController("user", OWNER, admin=True)
|
||||
names = [a.name for a in CATALOG.actions if a.handler == "workspace"]
|
||||
assert names
|
||||
for name in names + ["workspace_does_not_exist"]:
|
||||
result = run_async(controller.dispatch(name, {}))
|
||||
assert isinstance(result, str), f"{name} returned {type(result).__name__}"
|
||||
json.loads(result)
|
||||
|
||||
|
||||
def test_every_confirm_gated_tool_declares_a_confirm_param():
|
||||
from devplacepy.services.devii.actions.dispatcher import CONFIRM_REQUIRED
|
||||
from devplacepy.services.devii.registry import CATALOG
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import re
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
@ -46,6 +47,11 @@ def _project_for(owner_uid: str, title: str = "WS Project") -> dict:
|
||||
"title": title,
|
||||
"description": "workspace host project",
|
||||
"slug": slug,
|
||||
"project_type": "software",
|
||||
"status": "In Development",
|
||||
"platforms": "",
|
||||
"is_private": 0,
|
||||
"read_only": 0,
|
||||
"stars": 0,
|
||||
"created_at": "2026-01-01T00:00:00+00:00",
|
||||
"deleted_at": None,
|
||||
@ -70,6 +76,91 @@ def _workspace_for(project: dict, owner_uid: str, **overrides) -> dict:
|
||||
return store.create_instance(payload)
|
||||
|
||||
|
||||
def test_project_page_offers_the_workspace_entry_point_to_the_owner(alice):
|
||||
page, user = alice
|
||||
project = _project_for(_row_for(user)["uid"], "WS Entry")
|
||||
page.goto(
|
||||
f"{BASE_URL}/projects/{project['slug']}", wait_until="domcontentloaded"
|
||||
)
|
||||
page.locator(".project-actions-more").click()
|
||||
entry = page.locator(".context-menu-item:has-text('Workspace')")
|
||||
entry.wait_for(state="visible")
|
||||
entry.click()
|
||||
page.wait_for_url(
|
||||
f"{BASE_URL}/projects/{project['slug']}/workspace",
|
||||
wait_until="domcontentloaded",
|
||||
)
|
||||
page.locator(".workspace-page").wait_for(state="visible")
|
||||
|
||||
|
||||
def test_project_page_hides_the_workspace_entry_point_from_a_non_owner(bob):
|
||||
page, user = bob
|
||||
project = _project_for(str(uuid4()), "WS Entry Foreign")
|
||||
page.goto(
|
||||
f"{BASE_URL}/projects/{project['slug']}", wait_until="domcontentloaded"
|
||||
)
|
||||
page.locator(".project-actions-more").click()
|
||||
assert not page.locator(".context-menu-item:has-text('Workspace')").count()
|
||||
|
||||
|
||||
def test_project_page_shows_a_direct_vscode_button_for_a_running_workspace(alice):
|
||||
page, user = alice
|
||||
row = _row_for(user)
|
||||
project = _project_for(row["uid"], "WS Direct")
|
||||
instance = _workspace_for(project, row["uid"])
|
||||
page.goto(
|
||||
f"{BASE_URL}/projects/{project['slug']}", wait_until="domcontentloaded"
|
||||
)
|
||||
button = page.locator(".project-detail-actions a:has-text('VS Code')")
|
||||
button.wait_for(state="visible")
|
||||
expect(button).to_have_attribute("target", "_blank")
|
||||
expect(button).to_have_attribute(
|
||||
"href",
|
||||
f"/projects/{project['slug']}/containers/instances/{instance['uid']}/code/",
|
||||
)
|
||||
|
||||
|
||||
def test_direct_vscode_button_is_absent_while_the_workspace_is_stopped(alice):
|
||||
page, user = alice
|
||||
row = _row_for(user)
|
||||
project = _project_for(row["uid"], "WS Stopped")
|
||||
_workspace_for(project, row["uid"], status="stopped", desired_state="stopped")
|
||||
page.goto(
|
||||
f"{BASE_URL}/projects/{project['slug']}", wait_until="domcontentloaded"
|
||||
)
|
||||
page.locator(".project-detail-actions").wait_for(state="visible")
|
||||
assert not page.locator(".project-detail-actions a:has-text('VS Code')").count()
|
||||
|
||||
|
||||
def _await_workspace_flag(slug: str, key: str, expected: bool) -> bool:
|
||||
deadline = time.time() + 10.0
|
||||
url = f"{BASE_URL}/projects/{slug}"
|
||||
headers = {"Accept": "application/json", "X-API-KEY": key}
|
||||
while time.time() < deadline:
|
||||
body = requests.get(url, headers=headers).json()
|
||||
if body["viewer_can_workspace"] is expected:
|
||||
return True
|
||||
time.sleep(0.2)
|
||||
return False
|
||||
|
||||
|
||||
def test_project_page_hides_the_workspace_entry_point_when_disabled(alice):
|
||||
page, user = alice
|
||||
row = _row_for(user)
|
||||
project = _project_for(row["uid"], "WS Entry Off")
|
||||
set_setting("workspace_enabled", "0")
|
||||
try:
|
||||
assert _await_workspace_flag(project["slug"], row["api_key"], False)
|
||||
page.goto(
|
||||
f"{BASE_URL}/projects/{project['slug']}", wait_until="domcontentloaded"
|
||||
)
|
||||
page.locator(".project-actions-more").click()
|
||||
assert not page.locator(".context-menu-item:has-text('Workspace')").count()
|
||||
finally:
|
||||
set_setting("workspace_enabled", "1")
|
||||
assert _await_workspace_flag(project["slug"], row["api_key"], True)
|
||||
|
||||
|
||||
def test_workspace_page_offers_creation_to_owner(alice):
|
||||
page, user = alice
|
||||
project = _project_for(_row_for(user)["uid"])
|
||||
|
||||
Loading…
Reference in New Issue
Block a user