Make dev workspaces serve a working browser IDE end to end
The workspace feature shipped its routes, agent tools and docs, but the editor was never reachable: the project page had no entry point, the ppy image had no code-server binary, no certificate was ever requested for a tunnel, and both nginx and the proxy dropped what the editor needs. - Add a VS Code button to the project action row and a Workspace item to the overflow menu, gated by can_open_workspace plus a running instance (viewer_can_workspace and workspace_editor_url on ProjectDetailOut). - Install a pinned code-server in ppy.Dockerfile before USER pravda and assert it in the build smoke test, so an image that cannot run the editor no longer builds green. - Run the editor with --auth password and a per workspace 8 character pronounceable secret, minted once at the ensure_editor_password choke point and injected as PASSWORD. Keep it off WorkspaceViewOut, which the admin listing shares. - Publish the editor tunnel when a workspace is created and issue its certificate from a new WorkspaceService phase against the molohttp admin API, then notify the owner with the live URL and the password. Only pending rows are retried, so a broken host cannot burn the ACME failure rate limit. Renewal stays molohttp's job. - Forward the original Host on proxied requests and the client cookie on proxied websockets, so code-server scopes its session cookie to the public hostname and authenticates the workbench socket. - Recreate a container stuck in the created state instead of retrying docker start forever against an image it can no longer run. - Return a JSON string from WorkspaceController.dispatch; raw dicts landed in a tool message and aborted the turn at the model endpoint. - Let the nginx catch-all carry websocket upgrades, keeping upstream keepalive, so tunnelled apps and the editor both connect.
This commit is contained in:
@@ -572,6 +572,7 @@ def init_db():
|
||||
("workspace_owner_uid", ""),
|
||||
("editor_port", 0),
|
||||
("editor_host_port", 0),
|
||||
("editor_password", ""),
|
||||
("tunnel_name", ""),
|
||||
("last_active_at", ""),
|
||||
("idle_warned_at", ""),
|
||||
|
||||
@@ -9,7 +9,7 @@ This file documents the project detail page, the per-project virtual filesystem,
|
||||
|
||||
Each project card links to `/projects/{project_uid}` showing full project details, author info, platforms, star count, delete-for-owner, and (for the owner) Private/Read-only toggle buttons plus badges (see **Project visibility and read-only** below). The route is `GET /projects/{project_uid}` in `routers/projects/index.py` and 404s when the viewer cannot see a private project. The sitemap generator links to this URL (not the old `?user_uid=` query param). The detail page also links to the project filesystem at `/projects/{slug}/files`.
|
||||
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Containers, Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Workspace, Containers, Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
|
||||
|
||||
**Owner editing.** Mirrors post editing exactly: an owner-only **Edit** menu item (`data-modal="edit-project-modal"`) opens the `modal()` macro's `edit-project-modal`, a plain `POST` form to `/projects/edit/{slug}` (route `edit_project` in `routers/projects/index.py`, body `ProjectEditForm`, owner-gated through the shared `content.edit_content_item` which returns 403 JSON / redirect for non-owners and stamps `updated_at`). The modal is the create modal pre-filled from the `project` row (title, description, type/status radios pre-checked, dates via `format_date()` back to DD/MM/YYYY). `is_private`/`read_only` are NOT edited here - they stay on their dedicated toggles. The platforms tag widget reuses the create modal's `platforms-input`/`platforms`/`platforms-tags` ids; `ProfileEditor.initPlatformTags` now **seeds existing tags** from the hidden `#platforms` value on load, so both the empty create form and the pre-filled edit form work from the same code. Devii tool `edit_project`; documented in `docs_api.py` (`projects-edit`).
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from devplacepy.models import TunnelForm
|
||||
from devplacepy.responses import action_result, json_error, respond
|
||||
from devplacepy.schemas import WorkspaceOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.containers import activity, forward, store
|
||||
from devplacepy.services.containers import activity, api, forward, store
|
||||
from devplacepy.services.containers.workspace import provision, quota, tunnels
|
||||
from devplacepy.services.containers.workspace.provision import WorkspaceError
|
||||
from devplacepy.utils import not_found, require_user
|
||||
@@ -75,6 +75,9 @@ async def workspace_page(request: Request, slug: str):
|
||||
if instance
|
||||
else ""
|
||||
),
|
||||
"editor_password": (
|
||||
api.ensure_editor_password(instance) if instance else ""
|
||||
),
|
||||
"user": user,
|
||||
}
|
||||
return respond(request, "workspace.html", context, model=WorkspaceOut)
|
||||
|
||||
@@ -39,6 +39,7 @@ from devplacepy.content import (
|
||||
is_owner,
|
||||
can_view_project,
|
||||
can_view_project_containers,
|
||||
can_open_workspace,
|
||||
get_project_devlog,
|
||||
)
|
||||
from devplacepy.utils import (
|
||||
@@ -175,6 +176,19 @@ async def projects_page(
|
||||
model=ProjectsOut,
|
||||
)
|
||||
|
||||
def _editor_url(project: dict, user: dict) -> str:
|
||||
from devplacepy.services.containers import store
|
||||
from devplacepy.services.containers.workspace import provision
|
||||
|
||||
instance = provision.find_for_project(project["uid"], user["uid"])
|
||||
if not instance or instance.get("suspended_at"):
|
||||
return ""
|
||||
if instance.get("status") != store.ST_RUNNING:
|
||||
return ""
|
||||
slug = project["slug"] or project["uid"]
|
||||
return f"/projects/{slug}/containers/instances/{instance['uid']}/code/"
|
||||
|
||||
|
||||
@router.get("/{project_slug}", response_class=HTMLResponse)
|
||||
async def project_detail(request: Request, project_slug: str, before: str = None):
|
||||
user = get_current_user(request)
|
||||
@@ -211,6 +225,10 @@ async def project_detail(request: Request, project_slug: str, before: str = None
|
||||
],
|
||||
schemas=[website_schema(base), software_application_schema(project, base)],
|
||||
)
|
||||
viewer_can_workspace = can_open_workspace(project, user)
|
||||
workspace_editor_url = (
|
||||
_editor_url(project, user) if viewer_can_workspace else ""
|
||||
)
|
||||
parent = get_fork_parent(project["uid"])
|
||||
forked_from = (
|
||||
{
|
||||
@@ -256,6 +274,8 @@ async def project_detail(request: Request, project_slug: str, before: str = None
|
||||
"is_private": bool(project.get("is_private")),
|
||||
"read_only": bool(project.get("read_only")),
|
||||
"viewer_can_containers": can_view_project_containers(project, user),
|
||||
"viewer_can_workspace": viewer_can_workspace,
|
||||
"workspace_editor_url": workspace_editor_url,
|
||||
"forked_from": forked_from,
|
||||
"fork_count": count_forks(project["uid"]),
|
||||
"file_count": count_files(project["uid"]),
|
||||
|
||||
@@ -163,6 +163,7 @@ class WorkspaceOut(_Out):
|
||||
workspace_count: int = 0
|
||||
max_workspaces: int = 0
|
||||
editor_url: str = ""
|
||||
editor_password: str = ""
|
||||
user: Optional[Any] = None
|
||||
|
||||
|
||||
|
||||
@@ -163,6 +163,8 @@ class ProjectDetailOut(_Out):
|
||||
is_private: bool = False
|
||||
read_only: bool = False
|
||||
viewer_can_containers: bool = False
|
||||
viewer_can_workspace: bool = False
|
||||
workspace_editor_url: Optional[str] = None
|
||||
forked_from: Optional[dict] = None
|
||||
fork_count: int = 0
|
||||
file_count: int = 0
|
||||
|
||||
@@ -211,7 +211,7 @@ admin container manager's authorization is unchanged; workspaces add their own n
|
||||
|---|---|---|
|
||||
| Entry | `/projects/{slug}/containers/instances/{uid}/code/...` | `{port}-{name}.tunnel.pravda.education` |
|
||||
| Auth | session + `can_manage_workspace` | none, public by design |
|
||||
| Backend | code-server, `--auth none`, bound in-container | whatever the user runs |
|
||||
| Backend | code-server, `--auth password`, bound in-container | whatever the user runs |
|
||||
|
||||
Plane B routing is **one static molohttp site** `*.tunnel.pravda.education -> 127.0.0.1:10500`;
|
||||
DevPlace resolves the instance from the `Host` header. There is no molohttp object per tunnel.
|
||||
@@ -241,8 +241,29 @@ patterns plus `is_tunnel_host`. `tunnels.py` is CRUD with revive-not-duplicate.
|
||||
create/resume/stop/suspend/view surface and writes `/app/.devplace/tunnels.json`.
|
||||
|
||||
**`WorkspaceService`** is the only new service: lock-owner, `default_enabled=False`, four wrapped
|
||||
phases (disk sample on its own slow cadence, flag evaluation, lifecycle, purge sweep). It is a
|
||||
reconciler, not a `JobService`.
|
||||
synchronous phases (disk sample on its own slow cadence, flag evaluation, lifecycle, purge sweep)
|
||||
plus the async `_issue_tunnel_certificates`. It is a reconciler, not a `JobService`.
|
||||
|
||||
**Tunnel certificates are per-host, and that is forced by molohttp, not a preference.** molohttp's
|
||||
ACME client implements **`http-01` only** (`Sources/MoloHTTP/ACME/AcmeClient.swift`,
|
||||
`AcmeRenewalService.swift` - there is no `dns-01` anywhere), and Let's Encrypt will not issue a
|
||||
wildcard over `http-01`. So a `*.tunnel.pravda.education` **certificate** is impossible today and
|
||||
`workspace_cert_mode=per_host` is the only working mode; the prose above about one static wildcard
|
||||
site describes **routing**, which is correct and already in place (one enabled molohttp site
|
||||
`*.tunnel.pravda.education -> http://127.0.0.1:10500`, no per-tunnel site object). Only the cert is
|
||||
per-host. `tunnels.create` writes a row at `STATUS_PENDING` and nothing else; the row is inert until
|
||||
`_issue_tunnel_certificates` picks it up - `pending` is not in `SERVING_STATUSES`, so an
|
||||
un-provisioned tunnel 404s. The phase reads `tunnels.awaiting_certificate()` (`status=pending` +
|
||||
`desired_state=present` + not deleted), flips the row to `provisioning`, calls
|
||||
`certs.issue(hostname)` (`workspace/certs.py`, `POST {workspace_molohttp_base_url}/api/v1/certs/issue`
|
||||
with the `x-api-key` header, Basic as fallback), and lands on `active` or on `failed` with
|
||||
`last_error`. **Only `pending` is retried** - a `failed` row stays failed until the user recreates
|
||||
the tunnel (`create` revives it to `pending`), which keeps a broken host from burning the Let's
|
||||
Encrypt failure rate limit every 30s. **Renewal is molohttp's job, not DevPlace's**: once a host is
|
||||
issued, `AcmeRenewalService` re-issues it against its own expiry threshold forever, so DevPlace never
|
||||
schedules or tracks renewals. This whole phase did not exist - `tunnels` had no reader at all, and
|
||||
the six `workspace_molohttp_*`/`workspace_cert_mode`/`workspace_acme_email` settings were admin
|
||||
fields wired to nothing, which is why every tunnel sat at `pending` with no certificate.
|
||||
|
||||
**Two contracts that bite:**
|
||||
- A `ConfigField` with `type="select"` needs `options=[{"value": ..., "label": ...}]`. Plain strings
|
||||
@@ -252,6 +273,28 @@ reconciler, not a `JobService`.
|
||||
required field makes an anonymous request 422 instead of 401 and `tests/api/auth/matrix.py` fails.
|
||||
Give the field a default and validate it inside the handler after `require_user`.
|
||||
|
||||
**The editor opens in a new tab, directly.** The project detail page renders an inline **VS Code**
|
||||
button in `.project-detail-actions` (`target="_blank"`) straight to the code-server proxy
|
||||
`/projects/{slug}/containers/instances/{uid}/code/`, built by `_editor_url` in
|
||||
`routers/projects/index.py` and carried as `workspace_editor_url` on the context and
|
||||
`ProjectDetailOut`. It is emitted ONLY when the viewer passes `can_open_workspace` AND their
|
||||
workspace for that project exists, is not suspended, and is `store.ST_RUNNING` - the three states the
|
||||
`editor_proxy` route itself refuses (403 suspended, 409 not running, 502 no port), so the button can
|
||||
never open a dead editor. When there is no running workspace the button is absent and the Workspace
|
||||
menu item below is the way in (create/start it there). The workspace page's own **Open editor** link
|
||||
opens in a new tab too; keep both in step.
|
||||
|
||||
**Member entry point** is the project detail page's overflow menu (`project_detail.html`), gated by
|
||||
the `viewer_can_workspace` context flag (`can_open_workspace(project, user)`, set in
|
||||
`routers/projects/index.py` and declared on `ProjectDetailOut`) - exactly the pattern the admin-only
|
||||
**Containers** item uses with `viewer_can_containers`. `can_open_workspace` folds in the
|
||||
`workspace_enabled` master switch, so the item disappears for everyone while the feature is off and
|
||||
the route's own `_guard` stays the authority. **A workspace surface with no context flag is
|
||||
unreachable**: the whole feature shipped once with routes, Devii tools and docs but no link into
|
||||
`/projects/{slug}/workspace`, so it was reachable only by typing the URL - and then 404'd anyway
|
||||
because `workspace_enabled` defaults to `"0"`. Any new workspace surface needs both the flag on the
|
||||
page that links to it and the setting turned on.
|
||||
|
||||
**Admin console** is `/admin/workspaces` (`routers/admin/workspaces.py`, `admin_workspaces.html`):
|
||||
list, start/stop, suspend/unsuspend with a required reason, raise/resolve/dismiss flags, per-user
|
||||
quota rules. `base_seo_context` takes `breadcrumbs`/`schemas`, not `canonical`/`schema`.
|
||||
@@ -260,6 +303,62 @@ quota rules. `base_seo_context` takes `breadcrumbs`/`schemas`, not `canonical`/`
|
||||
them declares a `confirm` param (schemas are `additionalProperties: false`, so a gated tool without it
|
||||
loops forever).
|
||||
|
||||
**Opening a workspace publishes its editor on a public tunnel, automatically.** `provision.ensure`
|
||||
does three things after creating the instance: `api.ensure_editor_password`, then
|
||||
`publish_editor_tunnel` (a `tunnels.create` for the editor port, so the hostname is
|
||||
`{editor_port}-{name}.{domain}`), and the row lands at `pending`. The `WorkspaceService` cert phase
|
||||
then issues the certificate and calls `_announce_tunnel`, which sends the owner a `workspace`
|
||||
notification carrying the live `https://` URL **and the password**. The notification fires on
|
||||
`pending -> active`, not at creation, so the user is only told about a URL that already serves TLS.
|
||||
`provision.is_editor_tunnel(instance, tunnel)` distinguishes the auto-published editor tunnel (its
|
||||
`container_port` equals `editor_port`) from user-created ones, which get a plainer message with no
|
||||
password. This is a deliberate departure from the plane-A/plane-B split described above: the editor
|
||||
is now reachable publicly, which is only acceptable **because** `--auth password` is on - never
|
||||
reintroduce `--auth none` while the editor tunnel is auto-published.
|
||||
|
||||
**The editor is password-protected, per workspace.** `api.editor_command` runs code-server with
|
||||
`--auth password`, and `api.pravda_env` injects the secret as `PASSWORD` (the variable code-server
|
||||
reads). The secret is an 8-character **pronounceable** token from `api.generate_editor_password()`,
|
||||
built as four consonant-vowel pairs (`ronebamu`, `zipesodu`) so a user can read it once and retype it
|
||||
from memory. That is ~24 bits of entropy, which is deliberately weak for convenience: it is
|
||||
acceptable only because the alternative in practice was `--auth none`. If the editor tunnel is ever
|
||||
exposed to untrusted traffic at scale, lengthen the pair count rather than switching alphabet, and
|
||||
keep it pronounceable. `api.ensure_editor_password(instance)` is the single generator/persister: it returns the
|
||||
stored value or mints and saves one, so the password is **stable** across restarts and recreations.
|
||||
It is called from `api.run_spec_for` whenever `is_workspace` - the one point every workspace launch
|
||||
passes through - so a legacy instance created before this column existed gets one on its next boot
|
||||
rather than letting code-server invent an unknowable password of its own. The owner reads it off the
|
||||
workspace page (`editor_password` on the context and on `WorkspaceOut`). **It is deliberately NOT on
|
||||
`WorkspaceViewOut`**: that model is shared with `AdminWorkspacesOut`, which lists every workspace on
|
||||
the instance, so putting it there would hand every user's editor password to any admin loading
|
||||
`/admin/workspaces`. Keep per-workspace secrets on the owner-scoped model only. The column is
|
||||
`instances.editor_password`, ensured in `init_db`. This closes the hole that made a tunnelled editor
|
||||
an unauthenticated public shell: `--auth none` was safe only while the editor was reachable solely
|
||||
through the session-authenticated proxy route, and a user can publish a tunnel to the editor port.
|
||||
|
||||
**code-server lives in the `ppy` image, and nothing else supplies it.** `api.editor_command` makes
|
||||
`code-server` the container's argv, so an image without it fails `docker run` with exit **127**
|
||||
(`executable file not found in $PATH`) and the reconciler records `crashed` / `launch_failed` with an
|
||||
empty `container_id` - the container process never existed. It is installed in `ppy.Dockerfile` in
|
||||
the runtime stage **before `USER pravda`** and before the `chown -R pravda` block (so uid 1000 owns
|
||||
it with no elevation): version pinned in the single `ARG CODE_SERVER_VERSION`, arch resolved from
|
||||
`dpkg --print-architecture`, release tarball unpacked to `/usr/local/lib/code-server` with a symlink
|
||||
at `/usr/local/bin/code-server`. `code-server --version` is in the build smoke-test loop and the
|
||||
symlink is in the executable-check list, so an image that cannot run the editor can never build
|
||||
green. Bumping the version is a one-line `ARG` change plus `make ppy`. **`workspace_editor_version`
|
||||
(a `ConfigField` on `WorkspaceService`) is currently read by nothing** - the version is the image's,
|
||||
not a runtime setting; wire it up or drop it before relying on it.
|
||||
|
||||
**A container stuck in `created` is recreated, never retried forever.** The reconciler's
|
||||
`desired=running` branch reaches `backend.start()` for `ps.state == "created"`. That call is wrapped:
|
||||
on failure it logs, records an instance event `recreated`, `rm -f`s the container and calls
|
||||
`_launch`, so the instance is rebuilt from the CURRENT image. Without this, a container created
|
||||
against an older image that can no longer start is a permanent wedge - `docker start` re-resolves
|
||||
nothing, so the reconciler loops on the same error every tick until someone removes it by hand (the
|
||||
real failure: after code-server was added to `ppy`, the already-created workspace container kept
|
||||
failing `docker start` with the old 127 even though the new image was correct). Recreation is not a
|
||||
new retry class: a launch that fails already leaves `ps is None`, which `_launch`es again next tick.
|
||||
|
||||
**Toolchains in `ppy`.** Rust (rustup), Nim (choosenim), Swift (swiftly, then the toolchain is moved
|
||||
to a fixed `/opt/swift/toolchain` because swiftly's proxy resolves against `$HOME` and breaks for
|
||||
`pravda` at runtime). The choosenim installer **exits 1 even on success**, so its `RUN` ends with
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
import secrets
|
||||
import socket
|
||||
from pathlib import Path
|
||||
|
||||
@@ -487,6 +488,7 @@ def workspace_env(instance: dict, base_url: str) -> dict:
|
||||
"DEVPLACE_TUNNEL_MANIFEST": f"{WORKSPACE_MOUNT}/.devplace/tunnels.json",
|
||||
"DEVPLACE_TUNNEL_MAX": str(limits.max_tunnels),
|
||||
"VSCODE_PROXY_URI": naming.proxy_uri_template(name),
|
||||
"PASSWORD": instance.get("editor_password") or "",
|
||||
"DEVPLACE_EDITOR": "code-server",
|
||||
"DEVPLACE_EDITOR_PORT": str(editor_port),
|
||||
"DEVPLACE_EDITOR_URL": (
|
||||
@@ -509,6 +511,28 @@ def workspace_env(instance: dict, base_url: str) -> dict:
|
||||
|
||||
|
||||
EDITOR_DEFAULT_PORT = 8443
|
||||
EDITOR_PASSWORD_LENGTH = 8
|
||||
EDITOR_PASSWORD_CONSONANTS = "bdfgkmnprstvz"
|
||||
EDITOR_PASSWORD_VOWELS = "aeiou"
|
||||
|
||||
|
||||
def generate_editor_password() -> str:
|
||||
pairs = EDITOR_PASSWORD_LENGTH // 2
|
||||
return "".join(
|
||||
secrets.choice(EDITOR_PASSWORD_CONSONANTS)
|
||||
+ secrets.choice(EDITOR_PASSWORD_VOWELS)
|
||||
for _ in range(pairs)
|
||||
)
|
||||
|
||||
|
||||
def ensure_editor_password(instance: dict) -> str:
|
||||
password = (instance.get("editor_password") or "").strip()
|
||||
if password:
|
||||
return password
|
||||
password = generate_editor_password()
|
||||
store.update_instance(instance["uid"], {"editor_password": password})
|
||||
instance["editor_password"] = password
|
||||
return password
|
||||
|
||||
|
||||
def editor_command(instance: dict) -> list[str]:
|
||||
@@ -518,7 +542,7 @@ def editor_command(instance: dict) -> list[str]:
|
||||
"--bind-addr",
|
||||
f"0.0.0.0:{port}",
|
||||
"--auth",
|
||||
"none",
|
||||
"password",
|
||||
"--disable-telemetry",
|
||||
"--disable-update-check",
|
||||
"--user-data-dir",
|
||||
@@ -530,6 +554,8 @@ def editor_command(instance: dict) -> list[str]:
|
||||
|
||||
|
||||
def run_spec_for(instance: dict, image_tag: str) -> RunSpec:
|
||||
if instance.get("is_workspace"):
|
||||
ensure_editor_password(instance)
|
||||
env = {**json.loads(instance.get("env_json") or "{}"), **pravda_env(instance)}
|
||||
ports = [
|
||||
PortMapping(p["host"], p["container"], p.get("proto", "tcp"))
|
||||
|
||||
@@ -36,9 +36,10 @@ def forward_headers(request: Request, prefix: str = "") -> dict:
|
||||
if prefix:
|
||||
headers["X-Forwarded-Prefix"] = prefix
|
||||
headers["X-Script-Name"] = prefix
|
||||
headers["X-Forwarded-Host"] = request.headers.get(
|
||||
"host", request.url.hostname or ""
|
||||
)
|
||||
origin_host = request.headers.get("host", request.url.hostname or "")
|
||||
headers["X-Forwarded-Host"] = origin_host
|
||||
if origin_host:
|
||||
headers["Host"] = origin_host
|
||||
headers["X-Forwarded-Proto"] = request.headers.get(
|
||||
"x-forwarded-proto", request.url.scheme
|
||||
)
|
||||
@@ -46,6 +47,21 @@ def forward_headers(request: Request, prefix: str = "") -> dict:
|
||||
return headers
|
||||
|
||||
|
||||
WS_FORWARD_HEADERS = ("cookie", "authorization", "user-agent", "origin")
|
||||
|
||||
|
||||
def ws_headers(websocket) -> dict:
|
||||
headers = {
|
||||
name: websocket.headers[name]
|
||||
for name in WS_FORWARD_HEADERS
|
||||
if name in websocket.headers
|
||||
}
|
||||
origin_host = websocket.headers.get("host", "")
|
||||
if origin_host:
|
||||
headers["Host"] = origin_host
|
||||
return headers
|
||||
|
||||
|
||||
def inject_base(body: bytes, prefix: str) -> bytes:
|
||||
lowered = body.lower()
|
||||
if b"<base" in lowered:
|
||||
@@ -128,7 +144,10 @@ async def proxy_ws(
|
||||
await websocket.accept()
|
||||
try:
|
||||
async with websockets.connect(
|
||||
upstream_url, open_timeout=10, max_size=None
|
||||
upstream_url,
|
||||
open_timeout=10,
|
||||
max_size=None,
|
||||
additional_headers=ws_headers(websocket),
|
||||
) as upstream:
|
||||
await pump(websocket, upstream)
|
||||
except Exception as error:
|
||||
|
||||
@@ -168,7 +168,16 @@ class ContainerService(BaseService):
|
||||
await backend.unpause(ps.container_id)
|
||||
_set_status(inst, {"status": store.ST_RUNNING}, reason="unpause")
|
||||
elif ps.state == "created":
|
||||
await backend.start(ps.container_id)
|
||||
try:
|
||||
await backend.start(ps.container_id)
|
||||
except Exception as exc:
|
||||
self.log(f"start {inst['name']} failed, recreating: {exc}")
|
||||
store.record_event(
|
||||
inst, "recreated", "reconciler", "", {"reason": str(exc)}
|
||||
)
|
||||
await backend.rm(ps.container_id, force=True)
|
||||
await self._launch(backend, inst)
|
||||
return
|
||||
_set_status(inst, {"status": store.ST_RUNNING}, reason="start")
|
||||
elif status == store.ST_RUNNING:
|
||||
await self._handle_exit(backend, inst, ps)
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from devplacepy.database import get_setting
|
||||
from devplacepy.stealth import stealth_async_client
|
||||
|
||||
API_PATH = "/api/v1"
|
||||
ISSUE_TIMEOUT_SECONDS = 180.0
|
||||
|
||||
|
||||
class CertError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _base_url() -> str:
|
||||
return (get_setting("workspace_molohttp_base_url", "") or "").rstrip("/")
|
||||
|
||||
|
||||
def _auth_headers() -> dict[str, str]:
|
||||
api_key = get_setting("workspace_molohttp_api_key", "") or ""
|
||||
if api_key:
|
||||
return {"x-api-key": api_key}
|
||||
return {}
|
||||
|
||||
|
||||
def _basic_auth() -> tuple[str, str] | None:
|
||||
username = get_setting("workspace_molohttp_username", "") or ""
|
||||
password = get_setting("workspace_molohttp_password", "") or ""
|
||||
if username and password:
|
||||
return (username, password)
|
||||
return None
|
||||
|
||||
|
||||
def configured() -> bool:
|
||||
return bool(_base_url()) and bool(_auth_headers() or _basic_auth())
|
||||
|
||||
|
||||
async def issue(hostname: str) -> None:
|
||||
if not hostname:
|
||||
raise CertError("no hostname")
|
||||
base = _base_url()
|
||||
if not base:
|
||||
raise CertError("workspace_molohttp_base_url is not configured")
|
||||
headers = _auth_headers()
|
||||
auth = _basic_auth()
|
||||
if not headers and not auth:
|
||||
raise CertError("no molohttp credentials configured")
|
||||
payload: dict[str, object] = {"domains": [hostname]}
|
||||
email = get_setting("workspace_acme_email", "") or ""
|
||||
if email:
|
||||
payload["email"] = email
|
||||
async with stealth_async_client(timeout=ISSUE_TIMEOUT_SECONDS) as client:
|
||||
response = await client.post(
|
||||
f"{base}{API_PATH}/certs/issue",
|
||||
json=payload,
|
||||
headers=headers,
|
||||
auth=auth,
|
||||
)
|
||||
if response.status_code >= 400:
|
||||
raise CertError(
|
||||
f"molohttp cert issue failed ({response.status_code}): "
|
||||
f"{response.text[:200]}"
|
||||
)
|
||||
@@ -61,9 +61,24 @@ async def ensure(project: dict, user: dict) -> dict:
|
||||
"desired_state": "running",
|
||||
},
|
||||
)
|
||||
instance = store.get_instance(instance["uid"])
|
||||
api.ensure_editor_password(instance)
|
||||
publish_editor_tunnel(instance, owner_uid)
|
||||
return store.get_instance(instance["uid"])
|
||||
|
||||
|
||||
def publish_editor_tunnel(instance: dict, owner_uid: str) -> dict | None:
|
||||
port = int(instance.get("editor_port") or api.EDITOR_DEFAULT_PORT)
|
||||
if not port or not instance.get("tunnel_name"):
|
||||
return None
|
||||
return tunnels.create(instance, "Editor", port, owner_uid)
|
||||
|
||||
|
||||
def is_editor_tunnel(instance: dict, tunnel: dict) -> bool:
|
||||
editor_port = int(instance.get("editor_port") or api.EDITOR_DEFAULT_PORT)
|
||||
return int(tunnel.get("container_port") or 0) == editor_port
|
||||
|
||||
|
||||
def resume(instance: dict) -> dict:
|
||||
if instance.get("suspended_at"):
|
||||
raise WorkspaceError("this workspace is suspended; contact an administrator")
|
||||
|
||||
@@ -114,6 +114,17 @@ def create(
|
||||
return table.find_one(uid=uid)
|
||||
|
||||
|
||||
def awaiting_certificate() -> list[dict]:
|
||||
return list(
|
||||
_table().find(
|
||||
status=STATUS_PENDING,
|
||||
desired_state="present",
|
||||
deleted_at=None,
|
||||
order_by=["created_at"],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def update(uid: str, changes: dict) -> None:
|
||||
_table().update({"uid": uid, "updated_at": _now(), **changes}, ["uid"])
|
||||
|
||||
|
||||
@@ -11,7 +11,13 @@ from devplacepy import config
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.services.base import BaseService, ConfigField
|
||||
from devplacepy.services.containers import store
|
||||
from devplacepy.services.containers.workspace import flags, quota, tunnels
|
||||
from devplacepy.services.containers.workspace import (
|
||||
certs,
|
||||
flags,
|
||||
provision,
|
||||
quota,
|
||||
tunnels,
|
||||
)
|
||||
|
||||
DISK_SAMPLE_DEFAULT_MINUTES = 10
|
||||
|
||||
@@ -161,6 +167,63 @@ class WorkspaceService(BaseService):
|
||||
phase(rows, cfg)
|
||||
except Exception as error:
|
||||
self.log(f"{phase.__name__} failed: {error}")
|
||||
try:
|
||||
await self._issue_tunnel_certificates()
|
||||
except Exception as error:
|
||||
self.log(f"_issue_tunnel_certificates failed: {error}")
|
||||
|
||||
async def _issue_tunnel_certificates(self) -> None:
|
||||
if not certs.configured():
|
||||
return
|
||||
for row in tunnels.awaiting_certificate():
|
||||
hostname = row.get("hostname", "")
|
||||
tunnels.update(
|
||||
row["uid"],
|
||||
{"status": tunnels.STATUS_PROVISIONING, "last_error": ""},
|
||||
)
|
||||
try:
|
||||
await certs.issue(hostname)
|
||||
except Exception as error:
|
||||
tunnels.update(
|
||||
row["uid"],
|
||||
{
|
||||
"status": tunnels.STATUS_FAILED,
|
||||
"last_error": str(error)[:500],
|
||||
},
|
||||
)
|
||||
self.log(f"tunnel certificate for {hostname} failed: {error}")
|
||||
continue
|
||||
tunnels.update(
|
||||
row["uid"], {"status": tunnels.STATUS_ACTIVE, "last_error": ""}
|
||||
)
|
||||
self.log(f"tunnel certificate issued for {hostname}")
|
||||
self._announce_tunnel(row, hostname)
|
||||
|
||||
def _announce_tunnel(self, row: dict, hostname: str) -> None:
|
||||
from devplacepy.utils import create_notification
|
||||
|
||||
instance = store.get_instance(row.get("instance_uid", ""))
|
||||
if not instance:
|
||||
return
|
||||
owner_uid = instance.get("workspace_owner_uid", "")
|
||||
if not owner_uid:
|
||||
return
|
||||
url = f"https://{hostname}"
|
||||
if provision.is_editor_tunnel(instance, row):
|
||||
password = instance.get("editor_password") or ""
|
||||
message = (
|
||||
f"Your VS Code workspace is live at {url} "
|
||||
f"(password: {password})."
|
||||
)
|
||||
else:
|
||||
message = f"Tunnel {row.get('label') or hostname} is live at {url}."
|
||||
create_notification(
|
||||
owner_uid,
|
||||
"workspace",
|
||||
message,
|
||||
instance["uid"],
|
||||
f"/projects/{instance.get('project_uid', '')}/workspace",
|
||||
)
|
||||
|
||||
def _sample_disk(self, rows: list[dict], cfg: dict) -> None:
|
||||
interval = int(cfg.get("workspace_disk_sample_minutes") or
|
||||
|
||||
@@ -166,6 +166,10 @@ The Devii LLM-client and `PlatformClient` read timeout is `timeout_seconds` (fie
|
||||
|
||||
Config is all `config_fields` (AI url/model/key, base url, plan/verify toggles, max iterations, JS-execution toggle, user/guest 24h caps, guests toggle, pricing). `effective_config()` falls back the AI key to `DEVII_AI_KEY` and the base url to the instance origin.
|
||||
|
||||
## Every controller `dispatch` MUST return a JSON string (hard rule)
|
||||
|
||||
`Dispatcher.dispatch` is annotated `-> str` and its result is placed verbatim into the tool message as `{"role": "tool", "content": result}` (`agentic/loop.py`). A controller that returns a raw `dict`/`list`/`None` instead of a serialized string is never caught locally - `wrap_if_large` measures `len(result)`, which on a dict is the **key count**, so a small dict sails through unchunked and untruncated - and the request dies at the upstream with `400 ... content should be a string or a list`, killing the whole conversation mid-turn with no server-side traceback. This was a real production failure: `WorkspaceController.dispatch` was annotated `-> Any` and returned raw dicts from all 20 of its handlers, so any turn that called `workspace_list` (or any other workspace tool) 400'd. The same shape also silently disabled chunking for those payloads. Serialize at the controller's single `dispatch` choke point (`json.dumps(payload, ensure_ascii=False, default=str)` - `default=str` because DB rows carry datetimes), never per handler, and keep the `-> str` annotation so the contract is visible. `tests/api/projects/workspace.py::test_workspace_controller_always_returns_a_json_string` guards it.
|
||||
|
||||
## Backend confidentiality
|
||||
|
||||
Devii must never disclose the underlying model, provider, or any upstream URL - enforced in two layers. The `SYSTEM_PROMPT` (`agent.py`) forbids it even when such values appear inside a tool result. Structurally, `text.redact_backend()` runs on every JSON HTTP tool response in `format_response()` and scrubs the values of `REDACT_FIELD_KEYS` (`gateway_upstream_url`/`gateway_model`/`gateway_vision_url`/`gateway_vision_model`) and any stat labelled in `REDACT_STAT_LABELS` (`Model`) to `[hidden]`, so the admin-services tools cannot leak the gateway's upstream even though the admin settings UI still shows the real values. Add a config key to `REDACT_FIELD_KEYS` if a new field would expose backend infrastructure to the agent.
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from devplacepy.database import get_table, resolve_by_slug
|
||||
@@ -45,17 +46,21 @@ class WorkspaceController:
|
||||
raise WorkspaceError("no workspace exists for this project")
|
||||
return instance
|
||||
|
||||
async def dispatch(self, name: str, args: dict[str, Any]) -> Any:
|
||||
@staticmethod
|
||||
def _encode(payload: Any) -> str:
|
||||
return json.dumps(payload, ensure_ascii=False, default=str)
|
||||
|
||||
async def dispatch(self, name: str, args: dict[str, Any]) -> str:
|
||||
handler = getattr(self, f"_{name}", None)
|
||||
if handler is None:
|
||||
return {"error": f"unknown workspace action: {name}"}
|
||||
return self._encode({"error": f"unknown workspace action: {name}"})
|
||||
try:
|
||||
result = handler(args)
|
||||
if hasattr(result, "__await__"):
|
||||
return await result
|
||||
return result
|
||||
result = await result
|
||||
return self._encode(result)
|
||||
except WorkspaceError as error:
|
||||
return {"error": str(error)}
|
||||
return self._encode({"error": str(error)})
|
||||
|
||||
async def _workspace_open(self, args: dict) -> Any:
|
||||
user = self._user()
|
||||
|
||||
@@ -45,6 +45,19 @@
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.workspace-password {
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
||||
.workspace-password code {
|
||||
color: var(--text-primary);
|
||||
background: var(--bg-input);
|
||||
font-family: var(--font-mono);
|
||||
padding: var(--space-xs) var(--space-sm);
|
||||
border-radius: var(--radius);
|
||||
user-select: all;
|
||||
}
|
||||
|
||||
.workspace-tunnel-list {
|
||||
list-style: none;
|
||||
padding: 0;
|
||||
|
||||
@@ -66,6 +66,9 @@
|
||||
|
||||
<div class="project-detail-actions">
|
||||
<a href="/projects/{{ project['slug'] or project['uid'] }}/files" class="project-star-btn"><span class="icon">📁</span><span class="label"> Files ({{ file_count }} files)</span></a>
|
||||
{% if workspace_editor_url %}
|
||||
<a href="{{ workspace_editor_url }}" target="_blank" rel="noopener" class="project-star-btn"><span class="icon">💻</span><span class="label"> VS Code</span></a>
|
||||
{% endif %}
|
||||
<button type="button" class="project-star-btn" data-share="/projects/{{ project['slug'] or project['uid'] }}"><span class="icon">🔗</span><span class="label"> Share</span></button>
|
||||
{% if user %}
|
||||
{% set _type = "project" %}{% set _uid = project['uid'] %}{% set _my_vote = my_vote %}{% set _count = star_count %}{% set _btn_class = "project-star-btn" %}{% include "_star_vote.html" %}
|
||||
@@ -75,6 +78,9 @@
|
||||
<button type="button" class="project-star-btn project-actions-more" aria-haspopup="menu" aria-expanded="false" aria-label="More actions"><span class="icon">⋯</span><span class="label"> More</span></button>
|
||||
|
||||
<div class="project-actions-overflow" hidden>
|
||||
{% if viewer_can_workspace %}
|
||||
<a href="/projects/{{ project['slug'] or project['uid'] }}/workspace" data-menu-action data-menu-icon="💻" data-menu-label="Workspace">Workspace</a>
|
||||
{% endif %}
|
||||
{% if viewer_can_containers %}
|
||||
<a href="/projects/{{ project['slug'] or project['uid'] }}/containers" data-menu-action data-menu-icon="🖥️" data-menu-label="Containers">Containers</a>
|
||||
{% endif %}
|
||||
|
||||
@@ -60,7 +60,7 @@
|
||||
</p>
|
||||
<div class="workspace-actions">
|
||||
{% if workspace.status == "running" %}
|
||||
<a class="btn btn-primary" href="{{ editor_url }}">Open editor</a>
|
||||
<a class="btn btn-primary" href="{{ editor_url }}" target="_blank" rel="noopener">Open editor</a>
|
||||
<form method="post" action="/projects/{{ project.slug or project.uid }}/workspace/stop">
|
||||
<button type="submit" class="btn">Stop</button>
|
||||
</form>
|
||||
@@ -69,6 +69,9 @@
|
||||
<button type="submit" class="btn btn-primary">Start</button>
|
||||
</form>
|
||||
{% endif %}
|
||||
{% if editor_password %}
|
||||
<span class="workspace-password">Editor password <code>{{ editor_password }}</code></span>
|
||||
{% endif %}
|
||||
<form method="post" action="/projects/{{ project.slug or project.uid }}/workspace/delete">
|
||||
<button type="submit" data-confirm="Delete this workspace?" data-confirm-danger class="btn btn-danger">Delete</button>
|
||||
</form>
|
||||
|
||||
Reference in New Issue
Block a user