Seo tools

AGENTS.md

@@ -19,7 +19,7 @@ make locust-headless  # Locust in headless CLI mode (for CI)
 - **FastAPI** backend serving **Jinja2 templates** (SSR). Pure ES6 JS for interactivity.
 - **Database:** `dataset` (auto-syncs schema, uses `uid` for PKs). SQLite.
 - **Auth:** `session` cookie, plus `X-API-KEY` / `Authorization: Bearer <api_key>` / HTTP Basic (username-or-email:password) - all resolved in `get_current_user`. PBKDF2-SHA256 via passlib. No JWT. Every user has an `api_key` (uuid7), set at signup and backfilled in `init_db`/`devplace apikey backfill`. Docs site at `/docs` (`routers/docs/ package`, `DOCS_PAGES` registry; FastAPI's Swagger is moved to `/swagger` so `/docs` is free).
-- **Static:** `devplacepy/static/` mounted at `/static`
+- **Static:** `devplacepy/static/` mounted at `/static`; URLs are emitted boot-versioned (`/static/v<ts>/...`) via the `static_url`/`assetUrl` helpers and served immutable for a year (see "Static asset caching and versioning")
 - **Templates:** `devplacepy/templates/`. Shared `templates` instance from `devplacepy.templating` - all routers import from there, do NOT create their own.
 - **Ports:** 10500 (dev), 10501 (tests; the serial suite uses a single uvicorn subprocess)
 - **Username:** letters, numbers, hyphens, underscores only. 3-32 chars.
@@ -111,6 +111,16 @@ Single-host Docker Compose (`docker-compose.yml`): an **app** container (Uvicorn
 - **Code updates need no rebuild** (bind-mounted source); rebuild only on `pyproject.toml` dependency changes. `.env` is git-ignored; `.env.example` is the committed template. The app reads `DEVPLACE_DATABASE_URL`, never `DATABASE_URL`.
 - **nginx parity rules** (`nginx/nginx.conf.template`, rendered by `start.sh` via `envsubst` with an allow-list that preserves `$http_upgrade`): `/static/uploads/` must re-apply `nosniff` + a `Content-Disposition` via the `map $uri $upload_disposition` block (`inline` for safe image/video/audio extensions, `attachment` otherwise), mirroring `UploadStaticFiles.INLINE_MEDIA_EXTENSIONS` - an XSS control nginx would otherwise bypass, and the inline branch is what lets video play in production; `/devii/ws` needs the `map $http_upgrade $connection_upgrade` block and `Upgrade`/`Connection` headers or the Devii terminal cannot connect; `client_max_body_size` comes from `NGINX_MAX_BODY_SIZE` (default `50m`) and must be `>= max_upload_size_mb` or uploads 413. nginx serves `devplacepy/static` via a read-only bind mount, so assets stay current without an image rebuild.
 
+## Static asset caching and versioning (`config.STATIC_VERSION`, `static_url`, `assetUrl`)
+
+Static assets are served `public, immutable, max-age=31536000` (1 year) for the Lighthouse "efficient cache policy" / SEO score, while a restart still busts every browser. The reconciler is a **boot-time version path segment**: `config.STATIC_VERSION` (env `DEVPLACE_STATIC_VERSION`, else `int(time.time())` at process start) is injected after `/static`, so `/static/css/base.css` is emitted as `/static/v<ts>/css/base.css`. A deploy = a restart = a new timestamp = a new URL for every asset.
+
+- **Path segment, never a query string.** The frontend is unbundled ES6 with ~130 **relative** imports (`./Http.js`) and zero absolute ones. A `?v=` only versions the entry `<script>`; its transitive relative imports would resolve to unversioned URLs and stay frozen under `immutable`, so a deploy would not propagate JS for a year. A path segment is inherited automatically by the entire module graph and by relative CSS `url()`, so everything busts together. Do NOT switch this to a query param.
+- **Helpers (reuse, never hardcode a `/static/` literal):** templates use the `static_url(path)` Jinja global (`templating.py`); runtime JS that builds an absolute static URL uses `assetUrl(path)` from `static/js/assetVersion.js`, which reads `<meta name="asset-version">` rendered in `base.html` (also a `static_version` Jinja global). Both no-op for non-`/static/` paths and for `/static/uploads/`, so they are safe to wrap around dynamic values (`static_url(og_image or '/static/og-default.png')`). Relative ES6 imports and relative CSS `url()` need neither - they inherit the version from the importing URL.
+- **Serving:** `main.py` mounts a `CachedStaticFiles` at `/static/v{STATIC_VERSION}` (immutable 1y; `service-worker.js` -> `no-cache`) plus the plain `/static` mount as the unversioned fallback; nginx mirrors this with `location ~ ^/static/v\d+/` (immutable 1y) and a short `max-age=3600` on the unversioned `/static/`.
+- **Exclusions:** `/static/uploads/` (user content, stable DB paths) is never versioned; `service-worker.js` keeps a stable unversioned URL with `no-cache` (its `/service-worker.js` route already sets this) so its registration scope is stable and a new worker always deploys.
+- **Multi-worker:** the version is captured **once at launch** and shared via `DEVPLACE_STATIC_VERSION` (set in `make prod` = `$(date +%s)` and the Dockerfile's `sh -c` CMD), otherwise two workers could compute timestamps a second apart and serve mismatched versioned mounts (a 404 on the other worker). Unset in dev (`--reload`) so each reload refreshes it. Docs: `/docs/static-caching.html`.
+
 ## Multi-worker concurrency (preferred rules)
 
 `uvicorn --workers N` = N independent processes sharing only the filesystem and SQLite DB. Module-global caches/counters are per-process, so a local `clear()` is invisible to siblings. Full reference: admin docs `Production -> Multi-worker and concurrency` (`templates/docs/production-concurrency.html`). Enforce these:
@@ -143,6 +153,20 @@ Forking copies a source project into a brand-new project owned by the forking us
 - **Frontend:** `app.projectForker` (`static/js/ProjectForker.js`) wires `data-fork-project` (a Fork button on the project detail page, visible to any logged-in user): prompt for a name via `app.dialog.prompt` -> `Http.sendForm` POST -> `JobPoller.run("/forks/{uid}", ...)` -> on `done` redirect to `project_url`.
 - **Status route** `GET /forks/{uid}` (`routers/forks.py`, `ForkJobOut`) exposes `project_uid`/`project_url`/`source_project_uid` only once `status == done`. Devii tools `fork_project`/`fork_status`; docs `projects-fork`/`forks-status`.
 
+## SEO Diagnostics tool (`services/jobs/seo/`, `routers/tools/`)
+
+The public **Tools -> SEO Diagnostics** auditor crawls a URL or sitemap with a headless browser and runs a broad battery of SEO checks, on the **same async-job pattern as zip/fork** plus a live websocket. It is **public (guests included)**; abuse is bounded by the per-IP POST rate limit, a per-owner one-active-job cap, a page cap, and the shared SSRF guard.
+
+- **Surface:** a collapsible **Tools** dropdown in `base.html` (desktop center nav + a mobile section, visible to everyone) toggled by `MobileNav.initToolsDropdown`. `GET /tools` lists tools; `GET /tools/seo` is the auditor page (`static/js/SeoDiagnostics.js` -> `app.seoDiagnostics`, instantiated page-side in the template, not in `Application.js`).
+- **Enqueue:** `POST /tools/seo/run` (`routers/tools/seo.py`, body `SeoRunForm{url, mode: url|sitemap, max_pages 1-50}`). Owner is `("user", uid)` or `("guest", X-Real-IP)`. It rejects with `429` if the owner already has a pending/running `seo` job, then enqueues `{url, mode, max_pages, allow_private:False}` and returns `{uid, status_url, ws_url}`.
+- **`SeoService`** (kind `seo`, `service.py`) `process`: writes the payload to `config.SEO_REPORTS_DIR/{uid}/payload.json`, launches `python -m devplacepy.services.jobs.seo.worker <payload_json> <output_dir>` via `create_subprocess_exec` (high `limit=` so big lines never overflow the StreamReader), reads **NDJSON frames from stdout** line by line, forwards each into the in-process **`ProgressHub`** (`progress.py`), and on completion loads `output_dir/report.json` as the job result. `cleanup()` clears the hub buffer and removes the report dir.
+- **Worker** (`worker.py`, subprocess): `crawler.crawl_target` resolves the target (single URL, or sitemap `<loc>` URLs capped at `max_pages`) and fetches `robots.txt`/`sitemap.xml`/`llms.txt` with `httpx`. The audited target host is guarded once with `net_guard.guard_public_url`; candidate URLs **sharing that host are pre-approved** (no redundant per-URL `getaddrinfo` - a transient DNS failure or a self-hosted server resolving its own domain must not blank the whole crawl), and only cross-host sitemap entries are re-guarded. **In sitemap mode the crawler never falls back to auditing the sitemap document itself**: if no page URLs survive it raises a clear error (a stray `or [target]` fallback previously rendered the sitemap XML as one 56k-node "page" with no title/H1). For each page it launches one Playwright navigation: a single `page.evaluate(EXTRACT_SCRIPT)` returns the whole DOM contract (title/metas/canonical/headings/images/links/jsonld/og/twitter/semantic/mixed-content/word-count), an injected `PerformanceObserver` (`add_init_script(INIT_SCRIPT)`) captures LCP/CLS, navigation timing gives TTFB/FCP/transfer/protocol, a mobile-viewport pass measures overflow/tap-targets, a screenshot is saved, and a raw `httpx` GET supplies the SSR HTML for the rendered-vs-server parity check. It emits `stage|target|progress|page_loaded|page|site_checks|report_ready` frames and writes the full `report.json`.
+- **Check registry** (`checks/`): `base.py` defines the `Check`/`PageContext`/`SiteContext` dataclasses and the `@page_check`/`@site_check` decorators (collected into `PAGE_CHECKS`/`SITE_CHECKS`); one module per category (`crawl`, `meta`, `headings`, `links`, `structured_data`, `social`, `performance`, `mobile_a11y`, `security`, `ai_readiness`, `crosspage`). `registry.run_page_checks`/`run_site_checks` run them defensively (one failing check never aborts a page), and `compute_score` produces a severity-weighted overall score + grade and per-category subscores. **To add a check:** write a function decorated `@page_check`/`@site_check` in the right category module and import that module in `registry.py`.
+- **Live progress WS:** `WS /tools/seo/{uid}/ws` is served **only by the service-lock owner** (closes `4013` for a fast retry on a non-owner worker, like `/devii/ws`); it replays `hub.snapshot(uid)` then streams `hub.subscribe(uid)`. The frontend `SeoProgressSocket` mirrors the `DeviiSocket` 4013/reconnect pattern. **The `done` frame carries the full report inline** (and the router's late-join terminal branch reads it from `job.result`); the client renders from `frame.report` directly. This is load-bearing: `service.process` publishes `done` from inside `process()`, BEFORE the JobService base persists the result on the next reap tick (~2s later), so a client that fetched `/tools/seo/{uid}/report` on the `done` signal would race the DB write and get an empty (all-zero) report. Do not "simplify" this back into a fetch-on-done. After publishing `done`, `process()` calls `hub.clear(uid)` to bound the in-memory buffer (late reconnects fall back to the DB-backed terminal branch).
+- **Status/report routes:** `GET /tools/seo/{uid}` (`SeoJobOut`), `GET /tools/seo/{uid}/report` (`respond(..., SeoReportOut)`, HTML or JSON), `GET /tools/seo/{uid}/screenshot/{n}` (FileResponse from `SEO_REPORTS_DIR`, path-guarded). All are **capability URLs** scoped by the unguessable uuid7. Devii tools `seo_diagnostics`/`seo_status` (public); docs `tools-seo-*`; CLI `devplace seo prune|clear`; audit `seo.run.request|complete|failed` (category `tools`).
+- **SSRF guard is shared:** `devplacepy/net_guard.py` (`guard_public_url`, `is_blocked_address`, `effective_address`) was extracted from the Devii fetch controller, which now imports it; the crawler reuses it. **`playwright` is a core dependency** (Chromium installed by `make install` and the Docker image).
+- **Production nginx needs a dedicated WS location.** In `nginx/nginx.conf.template` the catch-all `location /` sets `Connection ""` (no upgrade) and a 60s timeout, so any websocket route that falls through to it fails the handshake (browser: `WebSocket connection failed`, no close code). The progress socket has its own `location ~ ^/tools/seo/[^/]+/ws$` block forwarding `Upgrade`/`Connection` with a 1h timeout, mirroring `/devii/ws`. **Any new websocket path must add its own upgrade `location` above `location /`** (see [[nginx-websocket-location]] and docs `Production -> nginx`).
+
 ## Container manager (`services/containers/`)
 
 Admin-only. Supervises container instances, all running one shared prebuilt image. Reference: admin docs `Services -> Container Manager` and the `containers` API group.
@@ -1239,14 +1263,79 @@ auto-opened on `/docs`. Opt-in (`default_enabled=False`).
   any per-user gateway limit, while guests fall back to the internal key. Guests get an unauthenticated
   sandbox client. Every turn is
   audited in `devii_turns`.
-- **Sessions & persistence.** `DeviiHub` keeps one `DeviiSession` per owner
-  `(owner_kind, owner_id)` - `user`/uid or `guest`/`devii_guest` cookie. A session holds a
-  `set` of WebSockets and **broadcasts** every frame to all of an owner's tabs/devices. After a
-  turn it persists `agent._messages` to `devii_conversations` (users only; rehydrated on
-  reconnect → survives restarts), appends a `devii_usage_ledger` row, and a `devii_turns` audit
-  row. On `attach` it sends a `{"type":"history"}` snapshot so a new tab renders the prior
-  conversation. Tasks persist in `devii_tasks` (owner-scoped; guests use an in-memory db via
-  `tasks.store.memory_db()`).
+- **Sessions & persistence.** `DeviiHub` keeps one `DeviiSession` per owner-and-channel
+  `(owner_kind, owner_id, channel)` - `user`/uid or `guest`/`devii_guest` cookie, channel
+  `main` (default) or `docs`. A session holds a `set` of WebSockets and **broadcasts** every
+  frame to all of that owner-channel's tabs/devices. After a turn it persists `agent._messages`
+  to `devii_conversations` (users only; rehydrated on reconnect → survives restarts), appends a
+  `devii_usage_ledger` row, and a `devii_turns` audit row. On `attach` it sends a
+  `{"type":"history"}` snapshot so a new tab renders the prior conversation. Tasks persist in
+  `devii_tasks` (owner-scoped; guests use an in-memory db via `tasks.store.memory_db()`).
+- **Channels (independent conversation thread only).** A `channel` separates the floating
+  terminal (`main`) from the in-page docs chat (`docs`) so each has its own conversation thread
+  for the same owner. The WS reads `?channel=` (`routers/devii.py`, allowlisted to
+  `{"main","docs"}`, falls back to `main`) and threads it `get_or_create(..., channel=)` →
+  `DeviiSession(channel=)`. **Only** `devii_conversations` is keyed `(owner_kind, owner_id,
+  channel)` (column added + NULL→`main` backfilled + index extended in `init_db`); lessons,
+  behavior, virtual tools, tasks, and the `devii_usage_ledger`/`devii_turns` 24h quota stay
+  owner-scoped (deliberately shared across channels). `find(...)`/`get_or_create(...)` default
+  `channel="main"` so existing callers (e.g. `/devii/adopt`) are unchanged. The `docs` channel
+  is driven only by `<dp-docs-chat>` (`static/js/components/AppDocsChat.js`): a light-DOM
+  component that calls `/devii/session` (guest cookie), opens `DeviiSocket(handlers, "docs")`,
+  renders user/agent bubbles in docs style (`static/css/docs-chat.css`) via the devii markdown
+  renderer, stubs any `avatar`/`client` request so the agent never hangs, and seeds its first
+  question from the `seed` attribute (the sidebar search box, intercepted). It is mounted on
+  `/docs/search.html` (`templates/docs/_chat.html`); the docs pages no longer carry
+  `data-devii-autoopen`.
+- **Docii (the `docs` channel persona).** The `docs` channel is a documentation-only assistant
+  branded **Docii**, built by specialising the same `DeviiSession` on `self.channel == "docs"`
+  (no separate service):
+  - **Prompt:** `session.py` sets `self._system_prompt = DOCS_SYSTEM_PROMPT` (instead of
+    `_system_prompt_for`) - it forces the agent to call `search_docs` first for every question,
+    to recursively refine-and-search (reading the returned section `content`, then searching
+    again with new keywords) until grounded, to answer ONLY from retrieved docs, and to do no
+    platform actions. `_compose_system_prompt()` returns it verbatim for docs (the owner
+    `BEHAVIOR_HEADER` is NOT appended, so a user's general Devii behavior rules can't derail it).
+  - **Tools:** `_builtin_tools()` filters `CATALOG.tool_schemas_for(...)` to the `DOCS_TOOLS`
+    allowlist (`{"search_docs"}`); `_refresh_tools()` for docs sets `self.tools[:] = _builtin_tools()`
+    with NO virtual tools. So the docs agent has exactly one tool and cannot wander.
+  - **Greeting:** `bootstrap_greeting()` returns `DOCS_GREETING` for docs.
+  - `search_docs` (`services/devii/docs/controller.py`) now delegates to the in-process page
+    index `docs_search.search_pages(...)` (no HTTP), returning per result `title`, `url`
+    (`/docs/{slug}.html`), `score` (BM25), and `content` (page text truncated to `CONTENT_CHARS`),
+    admin-filtered by the dispatcher's `is_admin`. So "visiting the search results" is repeated
+    `search_docs` calls; the default 40 `max_tool_iterations` leaves ample room to recurse.
+  - **References + inline linking:** because results carry real `url`s and `score`s, the prompt
+    REQUIRES the answer to (a) link inline to documented pages with markdown links to their `url`
+    and (b) end with a `## References` list of the pages used as clickable links with their score.
+    The devii markdown renderer makes `/docs/...` links clickable client-side.
+  - **Topic gate (follow-up vs new):** `_run_turn` runs `_docs_topic_gate(text)` before the main
+    loop (docs channel only). When there is prior history it calls `_classify_topic` (a no-tools
+    `LLMClient.complete_text` with `TOPIC_CLASSIFIER_PROMPT`, parsed by `_parse_topic`, defaulting
+    to `follow_up` on any failure). On `new` it resets `agent._messages` to `[system]`, clears the
+    persisted docs conversation, and emits `{type:"topic", decision, reason}`; the frontend clears
+    the log, re-shows the current question, and prints a reasoning note. On `follow_up` it keeps
+    context and shows the note.
+  - **Isolation from Devii (critical):** the docs channel is built with an EPHEMERAL `owned_db`
+    (`hub.get_or_create`: `db if owner_kind=="user" and channel=="main" else memory_db()`), so its
+    task/lesson/behavior/virtual-tool stores are private and empty - a Docii reflection never
+    pollutes the user's Devii memory and vice-versa. The **Scheduler only starts in the `main`
+    channel** (`attach`: `if not self._started and self.channel == "main"`), so Devii's scheduled
+    tasks never fire inside a docs session. (Regression fixed: previously the docs session's
+    scheduler ran owner tasks over the shared `devii_tasks` table, and the search-only Docii agent
+    tried to fulfil a `run_js` task by spamming `search_docs` - tasks are a `main`-only feature.)
+  - The `main` channel is untouched (full 90-tool catalog + behavior header + Devii greeting).
+- **Docs search mode + BM25 fallback.** The docs search surface is admin-configurable via the
+  `docs_search_mode` site setting (`agent` | `bm25`, default `agent`, on `/admin/settings`,
+  seeded in `init_db` `operational_defaults`, field in `AdminSettingsForm`). `routers/docs/views.py`
+  `docs_page` (slug `search`) resolves the effective mode: `_agent_search_state(request, user,
+  is_admin)` returns `disabled` (Devii off, or guests-disabled for a guest), `quota` (viewer
+  over their `daily_limit_for`/`spent_24h`), or `ok`. Agent renders **only** when
+  `mode=="agent"` and state `ok`; otherwise it runs the classic `docs_search.search(...)` (BM25,
+  `docs_search.py`) and renders `templates/docs/_search.html`. `docs_base.html` branches the
+  `kind=="search"` include on `search_mode`; on a quota fallback (`quota_fallback`) the BM25 page
+  shows a "reached your daily AI assistant limit" hint. So `docs_search.py`/`_search.html` are
+  live again (the bm25 path), not orphaned.
 - **Per-owner self-learning memory (privacy-critical).** The `LessonStore` (reflect/recall) is
   **owner-scoped, never shared**: `LessonStore(db, owner_kind, owner_id)` filters every read/write
   by owner. The hub builds one per session over the same `owned_db` as the task store - the main

Dockerfile

@@ -35,4 +35,4 @@ ENV DEVPLACE_WEB_WORKERS=2
 HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=10s \
     CMD curl -f http://localhost:10500/ || exit 1
 
-CMD ["uvicorn", "devplacepy.main:app", "--host", "0.0.0.0", "--port", "10500", "--workers", "2", "--backlog", "8192", "--proxy-headers", "--forwarded-allow-ips", "*"]
+CMD ["sh", "-c", "DEVPLACE_STATIC_VERSION=${DEVPLACE_STATIC_VERSION:-$(date +%s)} exec uvicorn devplacepy.main:app --host 0.0.0.0 --port 10500 --workers 2 --backlog 8192 --proxy-headers --forwarded-allow-ips '*'"]

Makefile

@@ -14,12 +14,13 @@ export PYTHONDONTWRITEBYTECODE
 
 install:
 	pip install -e .
+	python -m playwright install chromium
 
 dev:
 	uvicorn devplacepy.main:app --reload --reload-dir devplacepy --host 0.0.0.0 --port 10500 --backlog 4096
 
 prod:
-	DEVPLACE_WEB_WORKERS=2 uvicorn devplacepy.main:app --host 0.0.0.0 --port 10500 --workers 2 --backlog 8192 --proxy-headers --forwarded-allow-ips '*'
+	DEVPLACE_STATIC_VERSION=$$(date +%s) DEVPLACE_WEB_WORKERS=2 uvicorn devplacepy.main:app --host 0.0.0.0 --port 10500 --workers 2 --backlog 8192 --proxy-headers --forwarded-allow-ips '*'
 
 tree:
 	git ls-files | tree --fromfile --noreport

README.md

@@ -62,6 +62,7 @@ devplacepy/
 | `/projects/{slug}/files` | Per-project filesystem: directory and file CRUD, upload, inline editing, and line-range operations (`lines` read, `replace-lines`, `insert-lines`, `delete-lines`, `append`) for surgical edits to large text files (public read, owner write; all writes refused while the project is read-only) |
 | `/zips` | Zip job status (`/zips/{uid}`) and archive download (`/zips/{uid}/download`); archives are queued via `/projects/{slug}/zip` and `/projects/{slug}/files/zip` |
 | `/forks` | Fork job status (`/forks/{uid}`); forks are queued via `/projects/{slug}/fork`. Any signed-in user can fork a project they can view into a new project they own; once the job finishes the response carries the new project URL |
+| `/tools` | Public developer tools. `/tools/seo` is **SEO Diagnostics**: audit any URL or sitemap and stream live progress over a websocket. Queue with `POST /tools/seo/run`, poll `GET /tools/seo/{uid}`, read the full report at `GET /tools/seo/{uid}/report` |
 | `/projects/{slug}/containers` | Admin per-project container manager: Dockerfile CRUD with immutable versions, async image builds, and container instance creation. Reachable from the project page via the admin-only **Containers** button |
 | `/admin/containers` | Admin **Containers** section: a list of every container instance across all projects (`/admin/containers`), each linking to a dedicated instance detail page (`/admin/containers/{uid}`) with lifecycle controls, live logs and metrics, an interactive terminal, schedules, ingress, and workspace sync |
 | `/p/{slug}` | Public ingress proxy (HTTP + WebSocket) to a running container instance's published port, opt-in per instance via `ingress_slug` |
@@ -125,6 +126,7 @@ The log is **administrator-only**. `/admin/audit-log` is a paginated, filterable
 | `SECRET_KEY` | hardcoded fallback | Session signing key |
 | `DEVPLACE_VAPID_SUB` | `mailto:retoor@molodetz.nl` | Contact address in the VAPID JWT `sub` claim |
 | `DEVPLACE_INTERNAL_BASE_URL` | `http://localhost:10500` | Base URL the platform's own services dial for the AI gateway |
+| `DEVPLACE_STATIC_VERSION` | server boot unix timestamp | Cache-busting version stamped into every static asset URL (`/static/v<version>/...`). Set it at launch so multiple workers agree (the `prod` target and Docker image do this); leave unset in dev to refresh on each reload. See [Static asset caching](#static-asset-caching) |
 | `DEEPSEEK_API_KEY` / `OPENROUTER_API_KEY` | unset | Upstream provider keys; migrated into the gateway settings on first boot |
 
 ### Runtime settings
@@ -201,6 +203,17 @@ prose pages are curated markdown rendered through the shared content renderer (m
 highlight.js) with the current host and, when logged in, your own API key and username
 filled in.
 
+`/docs/search.html` search method is admin-configurable (`docs_search_mode` on
+`/admin/settings`, default `agent`). In **agent** mode it is **Docii**, a documentation-only
+assistant: an in-page chat (`<dp-docs-chat>`) rendered in the docs style where you ask
+questions in plain language and Docii repeatedly searches the documentation, reading the
+matching sections and refining its query until it has a grounded answer, with links to the
+relevant pages. Docii is the Devii engine on a dedicated `docs` conversation channel, but
+restricted to a single tool (documentation search) and driven by its own system prompt, so it
+cannot perform platform actions - it only reads the docs. In **bm25** mode it is the classic
+keyword results list. Either way, a viewer who has exceeded their daily AI limit (or a guest
+when Devii is disabled) falls back to keyword (BM25) search automatically.
+
 Every other endpoint is documented from a single source of truth, the `API_GROUPS` registry
 in `devplacepy/docs_api.py`. Each group becomes a reference page listing its endpoints with
 copy-paste cURL, JavaScript, and Python examples and an interactive "try it" panel
@@ -249,6 +262,8 @@ and its full configuration are documented automatically - including future servi
 
 `ForkService` copies a project into a new project owned by the forking user. The **Fork** button on the project page (any signed-in user) prompts for a name; the job creates the destination project, duplicates the entire virtual filesystem, and records a directional `project_forks` relation so each fork shows a "Forked from X" link. The frontend `app.projectForker` enqueues, polls `/forks/{uid}`, and redirects to the new project once it is done; on failure the partially created project is rolled back. The forked project is permanent, so retention removes only the job tracking row. CLI: `devplace forks prune` / `devplace forks clear` (job rows only).
 
+`SeoService` powers the public **Tools -> SEO Diagnostics** auditor. It runs a headless-browser (Playwright) crawl of a single URL or a sitemap (capped pages) in a subprocess and runs a broad battery of checks across eleven categories: crawlability and indexing (status, redirects, HTTPS/HSTS, canonical, robots/meta-robots, sitemap, URL hygiene, mixed content), on-page meta and content (title, description, headings, language, charset, viewport, favicon, content depth), links, structured data and rich results (JSON-LD validity and required properties, microdata/RDFa), social cards (Open Graph, Twitter), Core Web Vitals and performance (LCP, CLS, FCP, TTFB, page weight, requests, DOM size, compression, caching, image optimisation, console errors), mobile and accessibility (responsive layout, tap targets, image alt, form labels), security headers, and AI/LLM-search readiness (server-rendered-vs-JS content parity, `llms.txt`, semantic HTML). It produces a weighted score and grade with per-category subscores and a recommendation for every finding. Progress streams live over `WS /tools/seo/{uid}/ws`; the full report is available at `/tools/seo/{uid}/report` (HTML or JSON). CLI: `devplace seo prune` / `devplace seo clear`. Playwright is a core dependency; `make install` fetches the Chromium browser.
+
 ### Adding a service
 
 Create a class extending `BaseService`, declare its `config_fields`, override `run_once()` (read parameters via `self.get_config()`), and register in `main.py`:
@@ -743,6 +758,12 @@ Then enable **Container builds** and **Containers** on `/admin/services`. Builds
 - **Upload size:** `NGINX_MAX_BODY_SIZE` (default `50m`) must be **>= the admin-configurable `max_upload_size_mb`** (Admin -> Settings), or large uploads are rejected with HTTP 413 before reaching the app.
 - **Micro-cache:** off by default; enable with `NGINX_CACHE_ENABLED=true`.
 
+### Static asset caching
+
+Static assets (CSS, JS, vendored libraries) are served with a **one-year immutable cache** for the best Lighthouse "efficient cache policy" score, while deploys still take effect immediately. Every app-owned static URL carries a boot-time version path segment, `/static/v<timestamp>/...`, where `<timestamp>` is the unix time the server process started (`config.STATIC_VERSION`). A restart changes the segment, so every asset URL changes and returning browsers refetch on their next page load - no cache purge, no hashing build step.
+
+The version sits in the **path**, not a query string, because the frontend is unbundled ES6 modules wired with relative imports: a path segment is inherited automatically by every transitively imported module and relative CSS `url()`, so the whole graph busts on deploy. Templates emit URLs through the `static_url` Jinja global and runtime JavaScript through the `assetUrl` helper (`static/js/assetVersion.js`, reading `<meta name="asset-version">`). User uploads under `/static/uploads/` and the `service-worker.js` route are excluded. Set `DEVPLACE_STATIC_VERSION` at launch so multiple workers share one value (the `prod` target and Docker image do this). Full detail: `/docs/static-caching.html`.
+
 ### Bare-metal alternative
 
 `make prod` runs the same app without containers (`uvicorn ... --workers 2 --proxy-headers`) from the project root, sharing the identical database and files. Note it binds port 10500, so it conflicts with the Docker front door on the same port - run one, or set a different `PORT`.

devplacepy/cli.py

@@ -284,6 +284,46 @@ def cmd_forks_clear(args):
     print(f"Cleared {len(jobs)} fork job(s)")
 
 
+def _remove_seo_artifacts(job):
+    import shutil
+    from devplacepy.config import SEO_REPORTS_DIR
+
+    shutil.rmtree(SEO_REPORTS_DIR / job["uid"], ignore_errors=True)
+
+
+def cmd_seo_prune(args):
+    from datetime import datetime, timezone
+    from devplacepy.services.jobs import queue
+
+    now = datetime.now(timezone.utc)
+    removed = 0
+    for job in queue.list_jobs(kind="seo", status=queue.DONE):
+        expires_at = job.get("expires_at")
+        if not expires_at:
+            continue
+        try:
+            expiry = datetime.fromisoformat(expires_at)
+        except (ValueError, TypeError):
+            continue
+        if expiry < now:
+            _remove_seo_artifacts(job)
+            get_table("jobs").delete(uid=job["uid"])
+            removed += 1
+    _audit_cli("cli.seo.prune", f"CLI pruned {removed} expired SEO jobs", metadata={"count": removed})
+    print(f"Pruned {removed} expired SEO audit(s)")
+
+
+def cmd_seo_clear(args):
+    from devplacepy.services.jobs import queue
+
+    jobs = queue.list_jobs(kind="seo")
+    for job in jobs:
+        _remove_seo_artifacts(job)
+        get_table("jobs").delete(uid=job["uid"])
+    _audit_cli("cli.seo.clear", f"CLI cleared all SEO jobs ({len(jobs)})", metadata={"count": len(jobs)})
+    print(f"Cleared {len(jobs)} SEO audit(s) and their reports")
+
+
 def cmd_containers_list(args):
     from devplacepy.services.containers import store
 
@@ -661,6 +701,17 @@ def main():
     )
     forks_clear.set_defaults(func=cmd_forks_clear)
 
+    seo = sub.add_parser("seo", help="SEO Diagnostics job management")
+    seo_sub = seo.add_subparsers(title="action", dest="action")
+    seo_prune = seo_sub.add_parser(
+        "prune", help="Delete expired SEO audit reports and their job rows"
+    )
+    seo_prune.set_defaults(func=cmd_seo_prune)
+    seo_clear = seo_sub.add_parser(
+        "clear", help="Delete every SEO audit report and job row"
+    )
+    seo_clear.set_defaults(func=cmd_seo_clear)
+
     containers = sub.add_parser("containers", help="Container manager")
     containers_sub = containers.add_subparsers(title="action", dest="action")
     containers_sub.add_parser("list", help="List container instances").set_defaults(

devplacepy/config.py

@@ -1,5 +1,6 @@
 # retoor <retoor@molodetz.nl>
 
+import time
 from pathlib import Path
 from dotenv import load_dotenv
 from os import environ
@@ -23,6 +24,7 @@ CONTAINER_WORKSPACES_DIR = DATA_DIR / "container_workspaces"
 ZIPS_DIR = DATA_DIR / "zips"
 ZIP_STAGING_DIR = DATA_DIR / "zip_staging"
 FORK_STAGING_DIR = DATA_DIR / "fork_staging"
+SEO_REPORTS_DIR = DATA_DIR / "seo_reports"
 KEYS_DIR = DATA_DIR / "keys"
 BOT_DIR = DATA_DIR / "bot"
 LOCKS_DIR = DATA_DIR / "locks"
@@ -40,6 +42,12 @@ SESSION_MAX_AGE_REMEMBER = SECONDS_PER_DAY * 30
 PORT = 10500
 SITE_URL = environ.get("DEVPLACE_SITE_URL", "").rstrip("/")
 
+# Cache-busting version stamped onto every app-owned static URL. Computed once at
+# process start, so a restart/deploy changes it and busts every browser cache while
+# assets stay heavily cached (immutable, 1 year) between deploys. Set
+# DEVPLACE_STATIC_VERSION at launch so multiple prod workers share one value.
+STATIC_VERSION = environ.get("DEVPLACE_STATIC_VERSION") or str(int(time.time()))
+
 INTERNAL_BASE_URL = environ.get(
     "DEVPLACE_INTERNAL_BASE_URL", f"http://localhost:{PORT}"
 ).rstrip("/")
@@ -76,6 +84,7 @@ DATA_PATHS: dict[str, Path] = {
     "zips": ZIPS_DIR,
     "zip_staging": ZIP_STAGING_DIR,
     "fork_staging": FORK_STAGING_DIR,
+    "seo_reports": SEO_REPORTS_DIR,
     "keys": KEYS_DIR,
     "bot": BOT_DIR,
     "locks": LOCKS_DIR,

devplacepy/database.py

@@ -318,8 +318,21 @@ def init_db():
         db, "bug_comment_authors", "idx_bug_comment_authors_number", ["gitea_number"]
     )
     _index(db, "service_state", "idx_service_state_name", ["name"])
+    if "devii_conversations" in db.tables:
+        conversations = get_table("devii_conversations")
+        if not conversations.has_column("channel"):
+            conversations.create_column_by_example("channel", "main")
+        try:
+            db.query(
+                "UPDATE devii_conversations SET channel='main' WHERE channel IS NULL"
+            )
+        except Exception as e:  # noqa: BLE001
+            logger.warning(f"Could not backfill devii_conversations.channel: {e}")
     _index(
-        db, "devii_conversations", "idx_devii_conv_owner", ["owner_kind", "owner_id"]
+        db,
+        "devii_conversations",
+        "idx_devii_conv_owner",
+        ["owner_kind", "owner_id", "channel"],
     )
     _index(
         db,
@@ -528,6 +541,7 @@ def init_db():
             "customization_enabled": "1",
             "customization_js_enabled": "1",
             "audit_log_retention_days": "90",
+            "docs_search_mode": "agent",
         }
         for key, value in operational_defaults.items():
             existing = db["site_settings"].find_one(key=key)
@@ -2263,6 +2277,7 @@ def get_daily_topic():
                 "summary": desc,
                 "slug": article.get("slug", ""),
                 "url": article.get("url", ""),
+                "image_url": article.get("image_url", ""),
             }
     return {
         "title": "Welcome to DevPlace",

devplacepy/docs_api.py

@@ -3232,6 +3232,104 @@ Mutations flip desired state; a single reconciler converges containers to it.
         ),
         ],
     },
+    {
+        "slug": "tools",
+        "title": "Tools (SEO Diagnostics)",
+        "intro": """
+# Tools: SEO Diagnostics
+
+A public auditor that crawls a URL or sitemap with a headless browser and runs a broad battery of
+technical, on-page, structured-data, Core Web Vitals, accessibility and AI-readiness checks. It runs
+as a background job; poll the status URL (or watch the websocket) until `status` is `done`, then read
+the full report.
+
+Every endpoint follows the shared [Conventions & Errors](/docs/conventions.html). These are
+**capability URLs**: the job `uid` is an unguessable identifier, so anyone holding it can read the
+status and report.
+""",
+        "endpoints": [
+            endpoint(
+                id="tools-seo-run",
+                method="POST",
+                path="/tools/seo/run",
+                title="Queue an SEO audit",
+                summary="Start a background SEO audit of a URL or sitemap. Returns the job uid plus status and websocket URLs.",
+                auth="public",
+                params=[
+                    field("url", "body", "string", True, "https://example.com", "Page URL or sitemap.xml URL to audit."),
+                    field("mode", "body", "string", False, "url", "'url' (single page) or 'sitemap' (crawl)."),
+                    field("max_pages", "body", "integer", False, "10", "Max pages to crawl in sitemap mode (1-50)."),
+                ],
+                sample_response={
+                    "uid": "SEO_JOB_UID",
+                    "status_url": "/tools/seo/SEO_JOB_UID",
+                    "ws_url": "/tools/seo/SEO_JOB_UID/ws",
+                },
+            ),
+            endpoint(
+                id="tools-seo-status",
+                method="GET",
+                path="/tools/seo/{uid}",
+                title="SEO audit status",
+                summary="Poll an SEO audit. Once done, score, grade and report_url are populated.",
+                auth="public",
+                params=[
+                    field("uid", "path", "string", True, "SEO_JOB_UID", "SEO job uid returned when the audit was queued."),
+                ],
+                sample_response={
+                    "uid": "SEO_JOB_UID",
+                    "kind": "seo",
+                    "status": "done",
+                    "target": "https://example.com",
+                    "mode": "url",
+                    "ws_url": "/tools/seo/SEO_JOB_UID/ws",
+                    "report_url": "/tools/seo/SEO_JOB_UID/report",
+                    "score": 82,
+                    "grade": "B",
+                    "page_count": 1,
+                    "error": None,
+                    "created_at": "2026-06-14T10:00:00+00:00",
+                    "completed_at": "2026-06-14T10:00:18+00:00",
+                },
+            ),
+            endpoint(
+                id="tools-seo-report",
+                method="GET",
+                path="/tools/seo/{uid}/report",
+                title="SEO audit report",
+                summary="Full categorised report: overall score, per-category subscores, and every check with its recommendation. Negotiates HTML or JSON.",
+                auth="public",
+                params=[
+                    field("uid", "path", "string", True, "SEO_JOB_UID", "SEO job uid of a finished audit."),
+                ],
+                sample_response={
+                    "uid": "SEO_JOB_UID",
+                    "status": "done",
+                    "target": "https://example.com",
+                    "score": 82,
+                    "grade": "B",
+                    "page_count": 1,
+                    "counts": {"pass": 40, "warn": 8, "fail": 3, "info": 5, "skip": 0},
+                    "categories": {"crawlability": {"score": 90, "pass": 9, "warn": 1, "fail": 0}},
+                    "pages": [{"url": "https://example.com", "status": 200, "score": 82, "grade": "B"}],
+                    "checks": [
+                        {
+                            "id": "meta.title_present",
+                            "category": "meta",
+                            "title": "Title tag",
+                            "status": "pass",
+                            "severity": "high",
+                            "value": "Example Domain",
+                            "recommendation": "",
+                            "url": "https://example.com",
+                        }
+                    ],
+                    "site": {"robots": {"status": 200}, "sitemap": {"status": 200, "url_count": 12}},
+                    "generated_at": "2026-06-14T10:00:18+00:00",
+                },
+            ),
+        ],
+    },
     {
         "slug": "push",
         "title": "Web Push",

devplacepy/docs_search.py

@@ -172,10 +172,10 @@ def _snippet(text: str, terms: list, width: int = 280) -> str:
     return prefix + fragment + suffix
 
 
-def search(query: str, user=None, is_admin: bool = False, limit: int = 20) -> list:
+def _rank(query: str, user, is_admin: bool, limit: int) -> tuple:
     terms = _tokenize(query or "")
     if not terms:
-        return []
+        return [], None
     idx = get_index(user)
     scores: dict = {}
     for term in set(terms):
@@ -188,20 +188,49 @@ def search(query: str, user=None, is_admin: bool = False, limit: int = 20) -> li
             denom = tf + K1 * (1 - B + B * dl / idx["avgdl"])
             scores[i] = scores.get(i, 0.0) + idf_t * (tf * (K1 + 1)) / denom
     ranked = sorted(scores.items(), key=lambda kv: kv[1], reverse=True)
-    results = []
+    selected = []
     for i, score in ranked:
         doc = idx["docs"][i]
         if doc["admin"] and not is_admin:
             continue
-        results.append(
-            {
-                "slug": doc["slug"],
-                "title": doc["title"],
-                "score": round(score, 3),
-                "url": f"/docs/{doc['slug']}.html",
-                "snippet": _snippet(idx["doc_text"][i], terms),
-            }
-        )
-        if len(results) >= limit:
+        selected.append((i, doc, round(score, 3)))
+        if len(selected) >= limit:
             break
-    return results
+    return selected, idx
+
+
+def search(query: str, user=None, is_admin: bool = False, limit: int = 20) -> list:
+    selected, idx = _rank(query, user, is_admin, limit)
+    if not selected:
+        return []
+    terms = _tokenize(query)
+    return [
+        {
+            "slug": doc["slug"],
+            "title": doc["title"],
+            "score": score,
+            "url": f"/docs/{doc['slug']}.html",
+            "snippet": _snippet(idx["doc_text"][i], terms),
+        }
+        for i, doc, score in selected
+    ]
+
+
+def search_pages(
+    query: str,
+    user=None,
+    is_admin: bool = False,
+    limit: int = 6,
+    content_chars: int = 2400,
+) -> list:
+    selected, idx = _rank(query, user, is_admin, limit)
+    return [
+        {
+            "title": doc["title"],
+            "slug": doc["slug"],
+            "url": f"/docs/{doc['slug']}.html",
+            "score": score,
+            "content": idx["doc_text"][i][:content_chars],
+        }
+        for i, doc, score in selected
+    ]

devplacepy/main.py

@@ -13,6 +13,7 @@ from fastapi.staticfiles import StaticFiles
 from fastapi.exceptions import RequestValidationError
 from devplacepy.config import (
     STATIC_DIR,
+    STATIC_VERSION,
     UPLOADS_DIR,
     PORT,
     SERVICE_LOCK_FILE,
@@ -69,6 +70,7 @@ from devplacepy.routers import (
     zips,
     forks,
     proxy,
+    tools,
 )
 from devplacepy.services.manager import service_manager
 from devplacepy.services.news import NewsService
@@ -78,6 +80,7 @@ from devplacepy.services.devii import DeviiService
 from devplacepy.services.jobs.zip_service import ZipService
 from devplacepy.services.jobs.fork_service import ForkService
 from devplacepy.services.jobs.bug_create_service import BugCreateService
+from devplacepy.services.jobs.seo.service import SeoService
 from devplacepy.services.gitea.service import BugTrackerService
 from devplacepy.services.containers.service import ContainerService
 from devplacepy.services.audit import AuditService
@@ -155,6 +158,16 @@ class UploadStaticFiles(StaticFiles):
         return response
 
 
+class CachedStaticFiles(StaticFiles):
+    async def get_response(self, path, scope):
+        response = await super().get_response(path, scope)
+        if Path(path).name == "service-worker.js":
+            response.headers["Cache-Control"] = "no-cache"
+        else:
+            response.headers["Cache-Control"] = "public, max-age=31536000, immutable"
+        return response
+
+
 app = FastAPI(
     title="DevPlace",
     docs_url="/swagger",
@@ -166,6 +179,11 @@ app.mount(
     UploadStaticFiles(directory=str(UPLOADS_DIR), check_dir=False),
     name="uploads",
 )
+app.mount(
+    f"/static/v{STATIC_VERSION}",
+    CachedStaticFiles(directory=str(STATIC_DIR)),
+    name="static_versioned",
+)
 app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static")
 
 
@@ -305,6 +323,7 @@ app.include_router(media.router, prefix="/media")
 app.include_router(zips.router, prefix="/zips")
 app.include_router(forks.router, prefix="/forks")
 app.include_router(proxy.router, prefix="/p")
+app.include_router(tools.router, prefix="/tools")
 
 
 @app.middleware("http")
@@ -418,6 +437,7 @@ async def startup():
     service_manager.register(DeviiService())
     service_manager.register(ZipService())
     service_manager.register(ForkService())
+    service_manager.register(SeoService())
     service_manager.register(BugCreateService())
     service_manager.register(BugTrackerService())
     service_manager.register(ContainerService())

devplacepy/models.py

@@ -370,6 +370,20 @@ class PollVoteForm(BaseModel):
     option_uid: str = Field(min_length=1, max_length=36)
 
 
+class SeoRunForm(BaseModel):
+    url: str = Field(min_length=3, max_length=2000)
+    mode: Literal["url", "sitemap"] = "url"
+    max_pages: int = Field(default=10, ge=1, le=50)
+
+    @field_validator("url")
+    @classmethod
+    def url_scheme(cls, value):
+        text = value.strip()
+        if not text:
+            raise ValueError("A URL is required")
+        return text
+
+
 class AdminRoleForm(BaseModel):
     role: Literal["member", "admin"]
 
@@ -393,3 +407,4 @@ class AdminSettingsForm(BaseModel):
     registration_open: str = Field(default="", max_length=1)
     maintenance_mode: str = Field(default="", max_length=1)
     maintenance_message: str = Field(default="", max_length=300)
+    docs_search_mode: str = Field(default="", max_length=20)

devplacepy/net_guard.py

@@ -0,0 +1,62 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+import asyncio
+import ipaddress
+import socket
+from typing import Any
+from urllib.parse import urlparse
+
+NAT64_PREFIXES = (
+    ipaddress.ip_network("64:ff9b::/96"),
+    ipaddress.ip_network("64:ff9b:1::/48"),
+)
+
+
+class BlockedAddressError(ValueError):
+    pass
+
+
+def effective_address(address: Any) -> Any:
+    if isinstance(address, ipaddress.IPv6Address):
+        if address.ipv4_mapped is not None:
+            return address.ipv4_mapped
+        for prefix in NAT64_PREFIXES:
+            if address in prefix:
+                return ipaddress.IPv4Address(int(address) & 0xFFFFFFFF)
+    return address
+
+
+def is_blocked_address(address: Any) -> bool:
+    resolved = effective_address(address)
+    return (
+        resolved.is_private
+        or resolved.is_loopback
+        or resolved.is_link_local
+        or resolved.is_reserved
+        or resolved.is_multicast
+        or resolved.is_unspecified
+    )
+
+
+async def guard_public_url(url: str, *, allow_private: bool = False) -> str:
+    parsed = urlparse(url)
+    if parsed.scheme not in ("http", "https"):
+        raise BlockedAddressError("Only http and https URLs are allowed.")
+    host = parsed.hostname
+    if not host:
+        raise BlockedAddressError("URL has no host.")
+    if allow_private:
+        return host
+    try:
+        infos = await asyncio.to_thread(socket.getaddrinfo, host, None)
+    except socket.gaierror as exc:
+        raise BlockedAddressError(f"Could not resolve host: {host}") from exc
+    for info in infos:
+        address = ipaddress.ip_address(info[4][0])
+        if is_blocked_address(address):
+            raise BlockedAddressError(
+                f"Refusing to reach a private or local address ({effective_address(address)})."
+            )
+    return host

devplacepy/routers/devii.py

@@ -29,6 +29,7 @@ router = APIRouter()
 
 GUEST_COOKIE = DEVII_GUEST_COOKIE
 GUEST_COOKIE_MAX_AGE = 31536000
+CHANNELS = frozenset({"main", "docs"})
 
 
 def _service():
@@ -208,6 +209,9 @@ async def devii_ws(websocket: WebSocket):
     if not service_manager.owns_lock():
         await websocket.close(code=4013)
         return
+    channel = websocket.query_params.get("channel", "main")
+    if channel not in CHANNELS:
+        channel = "main"
     owner_kind, owner_id, username, api_key, owner_is_admin = _resolve_ws_owner(
         websocket
     )
@@ -225,6 +229,7 @@ async def devii_ws(websocket: WebSocket):
         api_key,
         svc.instance_base_url(),
         is_admin=owner_is_admin,
+        channel=channel,
     )
     session.attach(websocket)
     try:

devplacepy/routers/docs/pages.py

@@ -3,6 +3,7 @@
 from devplacepy.docs_api import api_doc_pages
 
 SECTION_GENERAL = "General"
+SECTION_TOOLS = "Tools"
 SECTION_COMPONENTS = "Components"
 SECTION_STYLES = "Styles"
 SECTION_API = "API"
@@ -22,7 +23,7 @@ AUDIENCE_CONTRIBUTE = "Contribute and internals"
 AUDIENCE_OPERATE = "Operate"
 
 AUDIENCES = [
-    (AUDIENCE_START, [SECTION_GENERAL]),
+    (AUDIENCE_START, [SECTION_GENERAL, SECTION_TOOLS]),
     (AUDIENCE_BUILD, [SECTION_API, SECTION_COMPONENTS, SECTION_STYLES]),
     (
         AUDIENCE_CONTRIBUTE,
@@ -72,6 +73,13 @@ DOCS_PAGES = [
         "kind": "prose",
         "section": SECTION_GENERAL,
     },
+    # Tools - public developer tools (everyone)
+    {
+        "slug": "tools-seo",
+        "title": "SEO Diagnostics",
+        "kind": "prose",
+        "section": SECTION_TOOLS,
+    },
     # Maintenance agents - the self-maintaining quality fleet (public)
     {
         "slug": "maintenance-agents",
@@ -525,6 +533,13 @@ DOCS_PAGES = [
         "admin": True,
         "section": SECTION_PROD,
     },
+    {
+        "slug": "static-caching",
+        "title": "Static asset caching and versioning",
+        "kind": "prose",
+        "admin": True,
+        "section": SECTION_PROD,
+    },
 ]
 
 PAGES_BY_SLUG = {page["slug"]: page for page in DOCS_PAGES}

devplacepy/routers/docs/views.py

@@ -8,7 +8,9 @@ from devplacepy.utils import get_current_user, not_found, is_admin as user_is_ad
 from devplacepy.seo import base_seo_context, site_url, website_schema
 from devplacepy.docs_api import render_group, build_services_group
 from devplacepy.services.manager import service_manager
-from devplacepy import docs_search, docs_export, docs_live, docs_prose
+from devplacepy.database import get_setting
+from devplacepy.constants import DEVII_GUEST_COOKIE
+from devplacepy import docs_export, docs_live, docs_prose, docs_search
 
 from devplacepy.routers.docs.pages import DOCS_PAGES, PAGES_BY_SLUG, nav_groups
 
@@ -53,6 +55,24 @@ async def docs_download_html(request: Request):
     )
 
 
+def _agent_search_state(request: Request, user, is_admin: bool) -> str:
+    svc = service_manager.get_service("devii")
+    if svc is None or not svc.is_enabled():
+        return "disabled"
+    if user:
+        owner_kind, owner_id = "user", user["uid"]
+    else:
+        owner_kind = "guest"
+        owner_id = request.cookies.get(DEVII_GUEST_COOKIE) or ""
+    if owner_kind == "guest" and not svc.guests_enabled():
+        return "disabled"
+    if owner_id:
+        limit = svc.daily_limit_for(owner_kind, is_admin)
+        if limit > 0 and svc.spent_24h(owner_kind, owner_id) >= limit:
+            return "quota"
+    return "ok"
+
+
 @router.get("/docs/{slug}.html", response_class=HTMLResponse)
 async def docs_page(request: Request, slug: str):
     user = get_current_user(request)
@@ -62,7 +82,11 @@ async def docs_page(request: Request, slug: str):
 
     if slug == "search":
         query = request.query_params.get("q", "")
-        results = docs_search.search(query, user=user, is_admin=is_admin)
+        mode = get_setting("docs_search_mode", "agent")
+        state = _agent_search_state(request, user, is_admin) if mode == "agent" else "off"
+        use_agent = mode == "agent" and state == "ok"
+        quota_fallback = mode == "agent" and state == "quota"
+        results = None if use_agent else docs_search.search(query, user=user, is_admin=is_admin)
         seo_ctx = base_seo_context(
             request,
             title="Search - Documentation",
@@ -86,6 +110,8 @@ async def docs_page(request: Request, slug: str):
                 "nav": nav_groups(visible_pages),
                 "current": "search",
                 "kind": "search",
+                "search_mode": "agent" if use_agent else "bm25",
+                "quota_fallback": quota_fallback,
                 "page_title_doc": "Search",
                 "search_query": query,
                 "search_results": results,

devplacepy/routers/tools/__init__.py

@@ -0,0 +1,6 @@
+# retoor <retoor@molodetz.nl>
+
+from . import index, seo
+
+router = index.router
+router.include_router(seo.router, prefix="/seo")

devplacepy/routers/tools/index.py

@@ -0,0 +1,45 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+
+from fastapi import APIRouter, Request
+from fastapi.responses import HTMLResponse
+
+from devplacepy.templating import templates
+from devplacepy.seo import list_page_seo
+from devplacepy.utils import get_current_user
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+TOOLS = [
+    {
+        "slug": "seo",
+        "name": "SEO Diagnostics",
+        "icon": "🔍",
+        "url": "/tools/seo",
+        "description": (
+            "Audit any URL or sitemap against a broad battery of technical, on-page, "
+            "structured-data, performance, accessibility and AI-readiness checks."
+        ),
+    },
+]
+
+
+@router.get("", response_class=HTMLResponse)
+async def tools_index(request: Request):
+    user = get_current_user(request)
+    seo_ctx = list_page_seo(
+        request,
+        title="Tools",
+        description="Developer tools on DevPlace, including the SEO Diagnostics auditor.",
+        breadcrumbs=[
+            {"name": "Home", "url": "/feed"},
+            {"name": "Tools", "url": "/tools"},
+        ],
+    )
+    return templates.TemplateResponse(
+        request,
+        "tools/index.html",
+        {**seo_ctx, "request": request, "user": user, "tools": TOOLS},
+    )

devplacepy/routers/tools/seo.py

@@ -0,0 +1,213 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+from pathlib import Path
+from typing import Annotated
+
+from fastapi import APIRouter, Form, Request, WebSocket, WebSocketDisconnect
+from fastapi.responses import FileResponse, HTMLResponse, JSONResponse
+
+from devplacepy.config import SEO_REPORTS_DIR
+from devplacepy.models import SeoRunForm
+from devplacepy.responses import respond
+from devplacepy.schemas import SeoJobOut, SeoReportOut
+from devplacepy.seo import list_page_seo
+from devplacepy.services.jobs import queue
+from devplacepy.services.jobs.seo.progress import hub
+from devplacepy.services.manager import service_manager
+from devplacepy.templating import templates
+from devplacepy.utils import get_current_user, not_found
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+ACTIVE_STATES = (queue.PENDING, queue.RUNNING)
+TERMINAL_STATES = (queue.DONE, queue.FAILED)
+
+
+def _owner(request: Request) -> tuple[str, str]:
+    user = get_current_user(request)
+    if user:
+        return "user", user["uid"]
+    ip = request.headers.get("X-Real-IP") or (
+        request.client.host if request.client else "anon"
+    )
+    return "guest", ip
+
+
+def _status_payload(job: dict) -> dict:
+    result = job.get("result", {})
+    done = job.get("status") == queue.DONE
+    uid = job.get("uid", "")
+    payload = job.get("payload", {})
+    return {
+        "uid": uid,
+        "kind": job.get("kind", ""),
+        "status": job.get("status", ""),
+        "target": result.get("target") or payload.get("url"),
+        "mode": result.get("mode") or payload.get("mode"),
+        "ws_url": f"/tools/seo/{uid}/ws",
+        "report_url": f"/tools/seo/{uid}/report" if done else None,
+        "score": result.get("score") if done else None,
+        "grade": result.get("grade") if done else None,
+        "page_count": int(result.get("page_count") or 0),
+        "error": job.get("error") or None,
+        "created_at": job.get("created_at"),
+        "completed_at": job.get("completed_at") or None,
+    }
+
+
+@router.get("", response_class=HTMLResponse)
+async def seo_page(request: Request):
+    user = get_current_user(request)
+    seo_ctx = list_page_seo(
+        request,
+        title="SEO Diagnostics",
+        description=(
+            "Run a deep SEO audit on any URL or sitemap: technical, on-page, structured "
+            "data, Core Web Vitals, accessibility and AI-readiness checks with live progress."
+        ),
+        breadcrumbs=[
+            {"name": "Home", "url": "/feed"},
+            {"name": "Tools", "url": "/tools"},
+            {"name": "SEO Diagnostics", "url": "/tools/seo"},
+        ],
+    )
+    return templates.TemplateResponse(
+        request,
+        "tools/seo.html",
+        {**seo_ctx, "request": request, "user": user},
+    )
+
+
+@router.post("/run")
+async def seo_run(request: Request, data: Annotated[SeoRunForm, Form()]):
+    owner_kind, owner_id = _owner(request)
+    active = [
+        job
+        for job in queue.list_jobs(kind="seo", owner=(owner_kind, owner_id))
+        if job.get("status") in ACTIVE_STATES
+    ]
+    if active:
+        return JSONResponse(
+            {
+                "error": {
+                    "status": 429,
+                    "message": "You already have an SEO audit running. Wait for it to finish.",
+                    "uid": active[0]["uid"],
+                }
+            },
+            status_code=429,
+        )
+    uid = queue.enqueue(
+        "seo",
+        {"url": data.url, "mode": data.mode, "max_pages": data.max_pages, "allow_private": False},
+        owner_kind,
+        owner_id,
+        f"SEO: {data.url}"[:64],
+    )
+    from devplacepy.services.audit import record as audit
+
+    audit.record(
+        request,
+        "seo.run.request",
+        summary=f"requested SEO diagnostics for {data.url}",
+        metadata={"target": data.url, "mode": data.mode, "max_pages": data.max_pages},
+        links=[audit.job(uid)],
+    )
+    return JSONResponse(
+        {
+            "uid": uid,
+            "status_url": f"/tools/seo/{uid}",
+            "ws_url": f"/tools/seo/{uid}/ws",
+        }
+    )
+
+
+@router.get("/{uid}")
+async def seo_status(request: Request, uid: str):
+    job = queue.get_job(uid)
+    if not job or job.get("kind") != "seo":
+        raise not_found("Audit not found")
+    return JSONResponse(
+        SeoJobOut.model_validate(_status_payload(job)).model_dump(mode="json")
+    )
+
+
+@router.get("/{uid}/report")
+async def seo_report(request: Request, uid: str):
+    job = queue.get_job(uid)
+    if not job or job.get("kind") != "seo":
+        raise not_found("Audit not found")
+    result = job.get("result", {})
+    report = result.get("report", {}) if job.get("status") == queue.DONE else {}
+    context = {
+        "uid": uid,
+        "status": job.get("status", ""),
+        "target": report.get("target") or job.get("payload", {}).get("url"),
+        "mode": report.get("mode") or job.get("payload", {}).get("mode"),
+        "score": report.get("score"),
+        "grade": report.get("grade"),
+        "page_count": report.get("page_count", 0),
+        "counts": report.get("counts", {}),
+        "categories": report.get("categories", {}),
+        "pages": report.get("pages", []),
+        "checks": report.get("checks", []),
+        "site": report.get("site", {}),
+        "generated_at": report.get("generated_at"),
+        "request": request,
+        "user": get_current_user(request),
+    }
+    return respond(request, "tools/seo_report.html", context, model=SeoReportOut)
+
+
+@router.get("/{uid}/screenshot/{index}")
+async def seo_screenshot(request: Request, uid: str, index: int):
+    job = queue.get_job(uid)
+    if not job or job.get("kind") != "seo":
+        raise not_found("Screenshot not available")
+    path = (SEO_REPORTS_DIR / uid / f"page-{index}.png").resolve()
+    if not path.is_relative_to(SEO_REPORTS_DIR.resolve()) or not path.is_file():
+        raise not_found("Screenshot not available")
+    return FileResponse(path, media_type="image/png")
+
+
+@router.websocket("/{uid}/ws")
+async def seo_ws(websocket: WebSocket, uid: str):
+    await websocket.accept()
+    if not service_manager.owns_lock():
+        await websocket.close(code=4013)
+        return
+    job = queue.get_job(uid)
+    if not job or job.get("kind") != "seo":
+        await websocket.close(code=1008)
+        return
+    listener = hub.subscribe(uid)
+    try:
+        for frame in hub.snapshot(uid):
+            await websocket.send_json(frame)
+        current = queue.get_job(uid)
+        if current and current.get("status") in TERMINAL_STATES:
+            done = current.get("status") == queue.DONE
+            report = current.get("result", {}).get("report", {}) if done else {}
+            await websocket.send_json(
+                {
+                    "type": "done" if done else "failed",
+                    "status": current.get("status"),
+                    "report_url": f"/tools/seo/{uid}/report",
+                    "error": current.get("error") or None,
+                    "report": report,
+                }
+            )
+            return
+        while True:
+            frame = await listener.get()
+            await websocket.send_json(frame)
+            if frame.get("type") in ("done", "failed"):
+                break
+    except WebSocketDisconnect:
+        pass
+    except Exception:  # noqa: BLE001 - never crash the worker on a socket error
+        logger.exception("SEO websocket loop failed for %s", uid)
+    finally:
+        hub.unsubscribe(uid, listener)

devplacepy/schemas.py

@@ -95,6 +95,38 @@ class ForkJobOut(_Out):
     completed_at: Optional[str] = None
 
 
+class SeoJobOut(_Out):
+    uid: str = ""
+    kind: str = ""
+    status: str = ""
+    target: Optional[str] = None
+    mode: Optional[str] = None
+    ws_url: Optional[str] = None
+    report_url: Optional[str] = None
+    score: Optional[int] = None
+    grade: Optional[str] = None
+    page_count: int = 0
+    error: Optional[str] = None
+    created_at: Optional[str] = None
+    completed_at: Optional[str] = None
+
+
+class SeoReportOut(_Out):
+    uid: str = ""
+    status: str = ""
+    target: Optional[str] = None
+    mode: Optional[str] = None
+    score: Optional[int] = None
+    grade: Optional[str] = None
+    page_count: int = 0
+    counts: dict = {}
+    categories: dict = {}
+    pages: list = []
+    checks: list = []
+    site: dict = {}
+    generated_at: Optional[str] = None
+
+
 # ---------- shared resource projections ----------
 
 

devplacepy/seo.py

@@ -333,6 +333,8 @@ def _build_sitemap(base_url):
         url_element(f"{base_url}/leaderboard", changefreq="daily", priority="0.7")
     )
     urlset.append(url_element(f"{base_url}/bugs", changefreq="daily", priority="0.6"))
+    urlset.append(url_element(f"{base_url}/tools", changefreq="monthly", priority="0.5"))
+    urlset.append(url_element(f"{base_url}/tools/seo", changefreq="monthly", priority="0.5"))
 
     if "posts" in db.tables:
         posts = _collect(
@@ -440,6 +442,7 @@ def _build_sitemap(base_url):
     _public_docs_pages = [
         "index",
         "devii",
+        "tools-seo",
         "media-gallery",
         "maintenance-agents",
         "maintenance-usage",

devplacepy/services/audit/categories.py

@@ -26,6 +26,7 @@ CATEGORY_BY_PREFIX: dict[str, str] = {
     "service": "service",
     "container": "container",
     "proxy": "ingress",
+    "seo": "tools",
     "ai": "ai",
     "devii": "devii",
     "cli": "cli",

devplacepy/services/devii/actions/catalog.py

@@ -566,6 +566,35 @@ ACTIONS: tuple[Action, ...] = (
         params=(path("uid", "Fork job uid returned by fork_project."),),
         requires_auth=False,
     ),
+    Action(
+        name="seo_diagnostics",
+        method="POST",
+        path="/tools/seo/run",
+        summary="Run an SEO audit on a URL or sitemap",
+        description=(
+            "Queues a background SEO diagnostics job and returns {uid, status_url}. Poll the "
+            "status_url with seo_status until status is 'done', then share the score, grade and "
+            "report_url. mode='url' audits a single page; mode='sitemap' crawls up to max_pages."
+        ),
+        params=(
+            body("url", "The page URL or sitemap.xml URL to audit.", required=True),
+            body("mode", "'url' for a single page or 'sitemap' to crawl a sitemap."),
+            body("max_pages", "Maximum pages to crawl in sitemap mode (1-50)."),
+        ),
+        requires_auth=False,
+    ),
+    Action(
+        name="seo_status",
+        method="GET",
+        path="/tools/seo/{uid}",
+        summary="Check an SEO audit and obtain its score once finished",
+        description=(
+            "Returns the audit status. When status is 'done', score, grade and report_url are "
+            "populated; while 'pending' or 'running', poll again shortly."
+        ),
+        params=(path("uid", "SEO job uid returned by seo_diagnostics."),),
+        requires_auth=False,
+    ),
     Action(
         name="search_users",
         method="GET",

devplacepy/services/devii/actions/dispatcher.py

@@ -248,7 +248,7 @@ class Dispatcher:
         self._owner_kind = owner_kind
         self._owner_id = owner_id
         self._fetch = FetchController(settings)
-        self._docs = DocsController(settings)
+        self._docs = DocsController(settings, is_admin=is_admin)
         self._cost = CostController(quota_provider=quota_provider)
         self._chunks = ChunkController(settings)
         self._rsearch = RsearchController(settings)

devplacepy/services/devii/docs/controller.py

@@ -2,81 +2,32 @@
 
 from __future__ import annotations
 
-import asyncio
 import json
 import logging
-import re
 from typing import Any
 
-import httpx
-
-from ..agentic.lessons import tokenize
 from ..config import Settings
-from ..errors import NetworkError, ToolInputError
-from ..fetch.controller import STEALTH_HEADERS
-from ..text import truncate
+from ..errors import ToolInputError
 
 logger = logging.getLogger("devii.docs")
 
-DOCS_PATH = "/docs/download.md"
-HEADING = re.compile(r"^(#{1,3})\s+(.*)$")
 DEFAULT_MAX_RESULTS = 5
-SECTION_CHARS = 1800
-HEADING_WEIGHT = 3
+CONTENT_CHARS = 2400
 
 
 class DocsController:
-    def __init__(self, settings: Settings) -> None:
+    def __init__(self, settings: Settings, is_admin: bool = False) -> None:
         self._settings = settings
-        self._sections: list[tuple[str, str]] | None = None
-        self._lock = asyncio.Lock()
+        self._is_admin = is_admin
 
     async def dispatch(self, name: str, arguments: dict[str, Any]) -> str:
         if name != "search_docs":
             raise ToolInputError(f"Unknown docs tool: {name}")
         return await self._search(arguments)
 
-    async def _load(self) -> list[tuple[str, str]]:
-        async with self._lock:
-            if self._sections is not None:
-                return self._sections
-            url = self._settings.base_url + DOCS_PATH
-            try:
-                async with httpx.AsyncClient(
-                    headers=STEALTH_HEADERS,
-                    follow_redirects=True,
-                    timeout=self._settings.fetch_timeout_seconds,
-                ) as client:
-                    response = await client.get(url)
-                    response.raise_for_status()
-                    markdown = response.text
-            except httpx.HTTPError as exc:
-                raise NetworkError(
-                    f"Could not load documentation: {exc}", url=url
-                ) from exc
-            self._sections = self._split(markdown)
-            logger.info("Loaded %d documentation sections", len(self._sections))
-            return self._sections
-
-    @staticmethod
-    def _split(markdown: str) -> list[tuple[str, str]]:
-        sections: list[tuple[str, str]] = []
-        heading = "Overview"
-        body: list[str] = []
-        for line in markdown.splitlines():
-            match = HEADING.match(line)
-            if match:
-                if body:
-                    sections.append((heading, "\n".join(body).strip()))
-                    body = []
-                heading = match.group(2).strip()
-            else:
-                body.append(line)
-        if body:
-            sections.append((heading, "\n".join(body).strip()))
-        return [(title, text) for title, text in sections if text]
-
     async def _search(self, arguments: dict[str, Any]) -> str:
+        from devplacepy import docs_search
+
         query = str(arguments.get("query", "")).strip()
         if not query:
             raise ToolInputError("search_docs requires a query.")
@@ -85,32 +36,13 @@ class DocsController:
         )
         max_results = max(1, min(max_results, 10))
 
-        sections = await self._load()
-        terms = tokenize(query)
-        scored: list[tuple[float, str, str]] = []
-        for heading, text in sections:
-            heading_tokens = set(tokenize(heading))
-            body_tokens = tokenize(text)
-            body_counts: dict[str, int] = {}
-            for token in body_tokens:
-                body_counts[token] = body_counts.get(token, 0) + 1
-            score = 0.0
-            for term in terms:
-                score += body_counts.get(term, 0)
-                if term in heading_tokens:
-                    score += HEADING_WEIGHT
-            if score > 0:
-                scored.append((score, heading, text))
-        scored.sort(key=lambda item: item[0], reverse=True)
-
-        results = [
-            {
-                "title": heading,
-                "score": round(score, 2),
-                "content": truncate(text, SECTION_CHARS),
-            }
-            for score, heading, text in scored[:max_results]
-        ]
+        results = docs_search.search_pages(
+            query,
+            user=None,
+            is_admin=self._is_admin,
+            limit=max_results,
+            content_chars=CONTENT_CHARS,
+        )
         return json.dumps(
             {
                 "status": "success",

devplacepy/services/devii/fetch/controller.py

@@ -13,6 +13,7 @@ from urllib.parse import urlparse
 
 import httpx
 
+from devplacepy.net_guard import effective_address, is_blocked_address
 from ..config import Settings
 from ..errors import NetworkError, ToolInputError, UpstreamError
 from ..text import html_to_text
@@ -49,21 +50,6 @@ TITLE = re.compile(r"<title[^>]*>(.*?)</title>", re.IGNORECASE | re.DOTALL)
 MIN_FETCH_CHARS = 1000
 ALLOWED_METHODS = ("GET", "POST", "PUT", "PATCH", "DELETE", "HEAD", "OPTIONS")
 
-NAT64_PREFIXES = (
-    ipaddress.ip_network("64:ff9b::/96"),
-    ipaddress.ip_network("64:ff9b:1::/48"),
-)
-
-
-def _effective_address(address: Any) -> Any:
-    if isinstance(address, ipaddress.IPv6Address):
-        if address.ipv4_mapped is not None:
-            return address.ipv4_mapped
-        for prefix in NAT64_PREFIXES:
-            if address in prefix:
-                return ipaddress.IPv4Address(int(address) & 0xFFFFFFFF)
-    return address
-
 
 class FetchController:
     def __init__(self, settings: Settings) -> None:
@@ -218,17 +204,10 @@ class FetchController:
         except socket.gaierror as exc:
             raise NetworkError(f"Could not resolve host: {host}", host=host) from exc
         for info in infos:
-            address = _effective_address(ipaddress.ip_address(info[4][0]))
-            if (
-                address.is_private
-                or address.is_loopback
-                or address.is_link_local
-                or address.is_reserved
-                or address.is_multicast
-                or address.is_unspecified
-            ):
+            address = ipaddress.ip_address(info[4][0])
+            if is_blocked_address(address):
                 raise ToolInputError(
-                    f"Refusing to fetch a private or local address ({address}). "
+                    f"Refusing to fetch a private or local address ({effective_address(address)}). "
                     "Set DEVII_FETCH_ALLOW_PRIVATE=1 to override."
                 )
 

devplacepy/services/devii/hub.py

@@ -30,7 +30,7 @@ class DeviiHub:
             "ledger": UsageLedger(),
             "turns": TurnAudit(),
         }
-        self._sessions: dict[tuple[str, str], DeviiSession] = {}
+        self._sessions: dict[tuple[str, str, str], DeviiSession] = {}
 
     @property
     def ledger(self) -> UsageLedger:
@@ -44,15 +44,20 @@ class DeviiHub:
         api_key: str,
         base_url: str,
         is_admin: bool = False,
+        channel: str = "main",
     ) -> DeviiSession:
-        key = (owner_kind, owner_id)
+        key = (owner_kind, owner_id, channel)
         session = self._sessions.get(key)
         if session is not None:
             return session
         settings, pricing = self._build(api_key, base_url, owner_kind, is_admin)
         llm = LLMClient(settings)
         # Persistent, owner-isolated stores for signed-in users; ephemeral in-memory for guests.
-        owned_db = db if owner_kind == "user" else memory_db()
+        # The `docs` channel (Docii) is a self-contained documentation assistant: it gets its own
+        # ephemeral stores so Devii's tasks, lessons, behavior and virtual tools never leak into it
+        # (and a Docii reflection never pollutes the user's Devii memory). Only the conversation
+        # thread persists, keyed per channel in the shared ConversationStore.
+        owned_db = db if (owner_kind == "user" and channel == "main") else memory_db()
         task_store = TaskStore(owned_db, owner_kind, owner_id)
         lessons = LessonStore(owned_db, owner_kind, owner_id)
         virtual_tool_store = VirtualToolStore(owned_db, owner_kind, owner_id)
@@ -70,22 +75,26 @@ class DeviiHub:
             behavior_store,
             self._stores,
             is_admin=is_admin,
+            channel=channel,
         )
         if owner_kind == "user":
-            saved = self._stores["conversations"].load(owner_kind, owner_id)
+            saved = self._stores["conversations"].load(owner_kind, owner_id, channel)
             if saved:
                 session.restore_history(saved)
         self._sessions[key] = session
         logger.info(
-            "Created session %s/%s (total %d)",
+            "Created session %s/%s [%s] (total %d)",
             owner_kind,
             owner_id,
+            channel,
             len(self._sessions),
         )
         return session
 
-    def find(self, owner_kind: str, owner_id: str) -> DeviiSession | None:
-        return self._sessions.get((owner_kind, owner_id))
+    def find(
+        self, owner_kind: str, owner_id: str, channel: str = "main"
+    ) -> DeviiSession | None:
+        return self._sessions.get((owner_kind, owner_id, channel))
 
     def active_sessions(self) -> int:
         return len(self._sessions)

devplacepy/services/devii/llm.py

@@ -71,6 +71,30 @@ class LLMClient:
         )
         return message
 
+    async def complete_text(
+        self, messages: list[dict[str, Any]], temperature: float = 0.0
+    ) -> str:
+        payload = {
+            "model": self._settings.ai_model,
+            "messages": messages,
+            "temperature": temperature,
+        }
+        try:
+            response = await self._client.post(self._settings.ai_url, json=payload)
+        except httpx.HTTPError as exc:
+            raise LLMError(f"Could not reach the model endpoint: {exc}") from exc
+        if response.status_code >= 400:
+            raise LLMError(
+                f"Model endpoint returned {response.status_code}: {self._reason(response)}"
+            )
+        try:
+            data = response.json()
+            content = data["choices"][0]["message"]["content"] or ""
+        except (ValueError, KeyError, IndexError) as exc:
+            raise LLMError("Model endpoint returned an unexpected response.") from exc
+        record_usage(data.get("usage"))
+        return content
+
     async def summarize(self, text: str) -> str:
         payload = {
             "model": self._settings.ai_model,

devplacepy/services/devii/session.py

@@ -85,9 +85,11 @@ class DeviiSession:
         behavior_store: Any,
         stores: dict[str, Any],
         is_admin: bool = False,
+        channel: str = "main",
     ) -> None:
         self.owner_kind = owner_kind
         self.owner_id = owner_id
+        self.channel = channel
         self.username = username
         self.settings = settings
         self.is_admin = is_admin
@@ -127,8 +129,10 @@ class DeviiSession:
             virtual_tools=self.virtual_tools,
             behavior=self.behavior,
         )
-        self.tools = CATALOG.tool_schemas_for(self.client.authenticated, is_admin)
-        self._system_prompt = _system_prompt_for(is_admin)
+        self.tools = self._builtin_tools()
+        self._system_prompt = (
+            DOCS_SYSTEM_PROMPT if channel == "docs" else _system_prompt_for(is_admin)
+        )
         self.agentic.bind(
             llm=llm,
             dispatcher=self.dispatcher,
@@ -224,7 +228,7 @@ class DeviiSession:
         self._disconnected.clear()
         self.avatar.bind(self._avatar_request)
         self.browser.bind(self._client_request)
-        if not self._started:
+        if not self._started and self.channel == "main":
             self.scheduler.start()
             self._started = True
         if self._buffer:
@@ -282,6 +286,8 @@ class DeviiSession:
         await self._llm.aclose()
 
     async def bootstrap_greeting(self) -> str:
+        if self.channel == "docs":
+            return DOCS_GREETING
         if self.client.authenticated:
             return f"Signed in as {self.username}. Devii is operating your DevPlace account. How can I help?"
         return LOGIN_REQUEST
@@ -307,7 +313,7 @@ class DeviiSession:
             self._buffer = []
             if self.persist_conversation:
                 try:
-                    self._conv.clear(self.owner_kind, self.owner_id)
+                    self._conv.clear(self.owner_kind, self.owner_id, self.channel)
                 except Exception:  # noqa: BLE001 - clearing storage must not break the socket
                     logger.exception(
                         "Failed to clear conversation for %s/%s",
@@ -354,7 +360,10 @@ class DeviiSession:
             if self.persist_conversation:
                 try:
                     self._conv.save(
-                        self.owner_kind, self.owner_id, self.agent._messages
+                        self.owner_kind,
+                        self.owner_id,
+                        self.agent._messages,
+                        self.channel,
                     )
                 except Exception:  # noqa: BLE001 - persistence must not break the socket
                     logger.exception(
@@ -376,6 +385,8 @@ class DeviiSession:
             async with self._lock:
                 self._refresh_tools()
                 self._refresh_system_prompt()
+                if self.channel == "docs":
+                    await self._docs_topic_gate(text)
                 reply = await self.agent.respond(text)
             if epoch == self._turn_epoch:
                 await self._emit({"type": "reply", "text": reply}, buffer=True)
@@ -391,12 +402,26 @@ class DeviiSession:
             if not cancelled and epoch == self._turn_epoch:
                 self._record_turn(turn_id, started_at, text, reply, error, before)
 
+    def _builtin_tools(self) -> list[dict[str, Any]]:
+        schemas = CATALOG.tool_schemas_for(self.client.authenticated, self.is_admin)
+        if self.channel == "docs":
+            return [
+                s
+                for s in schemas
+                if s.get("function", {}).get("name") in DOCS_TOOLS
+            ]
+        return schemas
+
     def _refresh_tools(self) -> None:
-        builtin = CATALOG.tool_schemas_for(self.client.authenticated, self.is_admin)
+        if self.channel == "docs":
+            self.tools[:] = self._builtin_tools()
+            return
         virtual = self._virtual_tool_store.tool_schemas()
-        self.tools[:] = builtin + virtual
+        self.tools[:] = self._builtin_tools() + virtual
 
     def _compose_system_prompt(self) -> str:
+        if self.channel == "docs":
+            return self._system_prompt
         body = self._behavior_store.text().strip()
         section = BEHAVIOR_HEADER if not body else f"{BEHAVIOR_HEADER}\n{body}"
         return f"{self._system_prompt}\n\n{section}"
@@ -406,6 +431,56 @@ class DeviiSession:
         if messages and messages[0].get("role") == "system":
             messages[0]["content"] = self._compose_system_prompt()
 
+    async def _docs_topic_gate(self, text: str) -> None:
+        prior = [
+            m
+            for m in self.agent._messages
+            if m.get("role") in ("user", "assistant") and m.get("content")
+        ]
+        if not prior:
+            return
+        decision, reason = await self._classify_topic(prior, text)
+        if decision == "new":
+            messages = self.agent._messages
+            if messages and messages[0].get("role") == "system":
+                system = messages[0]
+            else:
+                system = {"role": "system", "content": self._compose_system_prompt()}
+            self.agent._messages = [system]
+            self._buffer = []
+            if self.persist_conversation:
+                try:
+                    self._conv.clear(self.owner_kind, self.owner_id, self.channel)
+                except Exception:  # noqa: BLE001 - clearing storage must not break the turn
+                    logger.exception(
+                        "Failed to clear docs conversation for %s/%s",
+                        self.owner_kind,
+                        self.owner_id,
+                    )
+        await self._emit(
+            {"type": "topic", "decision": decision, "reason": reason}, buffer=False
+        )
+
+    async def _classify_topic(
+        self, prior: list[dict[str, Any]], text: str
+    ) -> tuple[str, str]:
+        recent = prior[-6:]
+        convo = "\n".join(
+            f"{m['role']}: {str(m['content'])[:300]}" for m in recent
+        )
+        messages = [
+            {"role": "system", "content": TOPIC_CLASSIFIER_PROMPT},
+            {
+                "role": "user",
+                "content": f"Prior conversation:\n{convo}\n\nNew message:\n{text}\n\nClassify.",
+            },
+        ]
+        try:
+            raw = await self._llm.complete_text(messages)
+        except Exception:  # noqa: BLE001 - on any failure, keep context (treat as follow-up)
+            return "follow_up", ""
+        return _parse_topic(raw)
+
     def _quota_snapshot(self) -> dict[str, Any]:
         spent = self._ledger.spent_24h(self.owner_kind, self.owner_id)
         turns = self._ledger.turns_24h(self.owner_kind, self.owner_id)
@@ -440,7 +515,12 @@ class DeviiSession:
         cost_delta = round(after["cost_usd"] - before["cost_usd"], 8)
         try:
             if self.persist_conversation:
-                self._conv.save(self.owner_kind, self.owner_id, self.agent._messages)
+                self._conv.save(
+                    self.owner_kind,
+                    self.owner_id,
+                    self.agent._messages,
+                    self.channel,
+                )
             self._ledger.record(
                 self.owner_kind,
                 self.owner_id,
@@ -615,6 +695,88 @@ NON_ADMIN_COST_RULE = (
     "quota used and the turn count."
 )
 
+DOCS_TOOLS = frozenset({"search_docs"})
+
+TOPIC_CLASSIFIER_PROMPT = (
+    "You are a routing classifier for a documentation assistant. Decide whether the "
+    "user's NEW message continues the PRIOR conversation (a follow-up: a clarification, "
+    "a refinement, or another question about the same subject) or starts a NEW, "
+    "unrelated topic that should begin from a clean slate.\n"
+    "Respond with ONLY a JSON object and nothing else: "
+    '{"decision": "follow_up" or "new", "reason": "<one short sentence explaining why>"}.'
+)
+
+DOCS_GREETING = (
+    "Hi, I am Docii, the DevPlace documentation assistant. Ask me anything about "
+    "DevPlace and I will search the documentation to find your answer."
+)
+
+DOCS_SYSTEM_PROMPT = (
+    "You are Docii, the documentation assistant for DevPlace, a social network for "
+    "software developers. You answer questions about DevPlace using ONLY its official "
+    "documentation.\n\n"
+    "YOU HAVE EXACTLY ONE TOOL: search_docs(query, max_results). It performs a keyword "
+    "search over the documentation and returns the matching pages. Each result is an object "
+    'with "title", "url" (e.g. /docs/authentication.html), "score" (BM25 relevance, higher '
+    'means more relevant), and "content" (the page text).\n\n'
+    "HARD RULES - follow every one, every turn:\n"
+    "1. Ground every CLAIM ABOUT DEVPLACE (routes, endpoints, parameters, auth methods, "
+    "settings, behavior, limits) in content returned by search_docs in THIS turn. Never state "
+    "a platform fact from memory or assumption.\n"
+    "2. For EVERY question, your FIRST action is to call search_docs with focused keywords "
+    "drawn from the question (feature names, endpoint paths, nouns).\n"
+    "3. READ the content in the results. If they do not fully and confidently answer the "
+    "question, call search_docs AGAIN with different or more specific keywords (synonyms, "
+    "related terms, the exact feature or route). Keep searching recursively - visiting the "
+    "results, refining the query, searching again - until you have gathered enough grounded "
+    "information to answer. Prefer several targeted searches over one broad search.\n"
+    "4. Do not invent routes, endpoints, parameters, settings, or behavior that the docs did "
+    "not state. The platform-specific facts in your answer must come from the docs.\n"
+    "5. If, after several distinct searches, the documentation genuinely does not cover a "
+    "needed FACT, say which part is undocumented - but still help as far as the docs allow "
+    "(for example, write the code using the endpoints you DID find).\n"
+    "6. Your only tool is search_docs and you perform no platform actions (no posting, "
+    "account changes, file edits, or web browsing). Writing code and examples in your reply "
+    "is allowed and encouraged.\n\n"
+    "WRITING CODE - when the user asks for a script, code example, API client, bot, or "
+    "snippet in ANY language, WRITE complete, runnable example code. Do NOT refuse merely "
+    "because the literal source is not published in the docs - synthesize it from the "
+    "documented API: take the base URL, endpoints, HTTP methods, parameters and "
+    "authentication from your search_docs results, and write all the ordinary programming "
+    "scaffolding (imports, language syntax, error handling, a main loop, comments) yourself. "
+    "Put it in a fenced code block with a language tag. Never invent endpoints or parameters "
+    "the docs do not describe; if a detail you need is missing, search for it first.\n\n"
+    "LINKING - this is mandatory:\n"
+    "- ACTIVE INLINE LINKING: whenever you mention a documented page, feature, endpoint, or "
+    "concept in your prose, write it as a markdown link to that page's `url`, for example "
+    "[authentication](/docs/authentication.html). Use the exact `url` from the search "
+    "results. Never paste a bare URL - always a markdown link with descriptive text.\n"
+    "- ALWAYS end every answer with a '## References' section: a markdown bullet list of the "
+    "documentation pages you actually used to answer, each as a clickable markdown link "
+    "followed by its relevance score, highest score first, e.g. "
+    "`- [Authentication](/docs/authentication.html) - score 12.3`. Include only pages you "
+    "used; do not invent links or scores.\n\n"
+    "STYLE: concise, practical, technical. Use markdown."
+)
+
+
+def _parse_topic(raw: str) -> tuple[str, str]:
+    import json
+    import re
+
+    match = re.search(r"\{.*\}", raw or "", re.S)
+    if not match:
+        return "follow_up", ""
+    try:
+        data = json.loads(match.group())
+    except (ValueError, TypeError):
+        return "follow_up", ""
+    decision = (
+        "new" if str(data.get("decision", "")).strip().lower() == "new" else "follow_up"
+    )
+    reason = str(data.get("reason", "")).strip()[:200]
+    return decision, reason
+
 
 def _system_prompt_for(is_admin: bool) -> str:
     if is_admin:

devplacepy/services/devii/store.py

@@ -28,11 +28,13 @@ def _iso(moment: datetime) -> str:
 
 
 class ConversationStore:
-    def load(self, owner_kind: str, owner_id: str) -> list[dict[str, Any]] | None:
+    def load(
+        self, owner_kind: str, owner_id: str, channel: str = "main"
+    ) -> list[dict[str, Any]] | None:
         if CONVERSATIONS not in db.tables:
             return None
         row = get_table(CONVERSATIONS).find_one(
-            owner_kind=owner_kind, owner_id=owner_id
+            owner_kind=owner_kind, owner_id=owner_id, channel=channel
         )
         if not row or not row.get("messages"):
             return None
@@ -43,18 +45,23 @@ class ConversationStore:
             return None
 
     def save(
-        self, owner_kind: str, owner_id: str, messages: list[dict[str, Any]]
+        self,
+        owner_kind: str,
+        owner_id: str,
+        messages: list[dict[str, Any]],
+        channel: str = "main",
     ) -> None:
         now = _iso(_now())
         record = {
             "owner_kind": owner_kind,
             "owner_id": owner_id,
+            "channel": channel,
             "messages": json.dumps(messages),
             "updated_at": now,
         }
         table = get_table(CONVERSATIONS)
         existing = (
-            table.find_one(owner_kind=owner_kind, owner_id=owner_id)
+            table.find_one(owner_kind=owner_kind, owner_id=owner_id, channel=channel)
             if CONVERSATIONS in db.tables
             else None
         )
@@ -64,9 +71,11 @@ class ConversationStore:
             record["created_at"] = now
             table.insert(record)
 
-    def clear(self, owner_kind: str, owner_id: str) -> None:
+    def clear(self, owner_kind: str, owner_id: str, channel: str = "main") -> None:
         if CONVERSATIONS in db.tables:
-            get_table(CONVERSATIONS).delete(owner_kind=owner_kind, owner_id=owner_id)
+            get_table(CONVERSATIONS).delete(
+                owner_kind=owner_kind, owner_id=owner_id, channel=channel
+            )
 
 
 class UsageLedger:

devplacepy/services/jobs/seo/__init__.py

@@ -0,0 +1 @@
+# retoor <retoor@molodetz.nl>

devplacepy/services/jobs/seo/checks/__init__.py

@@ -0,0 +1 @@
+# retoor <retoor@molodetz.nl>

devplacepy/services/jobs/seo/checks/ai_readiness.py

@@ -0,0 +1,90 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+import re
+
+from .base import (
+    HIGH,
+    MEDIUM,
+    LOW,
+    INFO,
+    Check,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+    site_check,
+)
+
+CATEGORY = "ai_readiness"
+
+_TAG = re.compile(r"<[^>]+>")
+_SCRIPT_STYLE = re.compile(r"<(script|style)[^>]*>.*?</\1>", re.IGNORECASE | re.DOTALL)
+
+
+def _text_words(html: str) -> int:
+    if not html:
+        return 0
+    stripped = _SCRIPT_STYLE.sub(" ", html)
+    stripped = _TAG.sub(" ", stripped)
+    return len([w for w in stripped.split() if w])
+
+
+@page_check
+def ssr_parity(page: PageContext, site: SiteContext) -> Check:
+    rendered = int(page.dom.get("wordCount", 0) or 0)
+    raw = _text_words(page.raw_html)
+    if rendered <= 0:
+        return None
+    ratio = raw / rendered if rendered else 0
+    passed = ratio >= 0.5
+    return ok_check(
+        "ai.ssr_parity",
+        CATEGORY,
+        "Content in initial HTML",
+        passed,
+        severity=HIGH,
+        value=f"{int(ratio * 100)}% server-rendered",
+        recommendation="Serve primary content in the initial HTML (SSR); JS-only content is invisible to many crawlers and AI agents.",
+        warn=ratio >= 0.25,
+        details={"raw_words": raw, "rendered_words": rendered},
+        url=page.url,
+    )
+
+
+@page_check
+def semantic_html(page: PageContext, site: SiteContext) -> Check:
+    semantic = page.dom.get("semantic", {}) or {}
+    has_main = int(semantic.get("main", 0) or 0) >= 1
+    has_landmarks = any(
+        int(semantic.get(tag, 0) or 0) >= 1 for tag in ("article", "nav", "header", "footer")
+    )
+    passed = has_main and has_landmarks
+    return ok_check(
+        "ai.semantic_html",
+        CATEGORY,
+        "Semantic HTML landmarks",
+        passed,
+        severity=LOW,
+        value="ok" if passed else "missing landmarks",
+        recommendation="Use <main>, <article>, <nav>, <header> and <footer> so machines can extract the main content.",
+        warn=True,
+        details=semantic,
+        url=page.url,
+    )
+
+
+@site_check
+def llms_txt(site: SiteContext) -> Check:
+    found = bool((site.llms_txt or {}).get("found"))
+    return ok_check(
+        "ai.llms_txt",
+        CATEGORY,
+        "llms.txt present",
+        found,
+        severity=LOW,
+        value="present" if found else "missing",
+        recommendation="Publish an /llms.txt manifest to guide AI crawlers to your key content (emerging standard).",
+        warn=True,
+    )

devplacepy/services/jobs/seo/checks/base.py

@@ -0,0 +1,128 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Callable, Optional
+
+PASS = "pass"
+WARN = "warn"
+FAIL = "fail"
+INFO = "info"
+SKIP = "skip"
+
+CRITICAL = "critical"
+HIGH = "high"
+MEDIUM = "medium"
+LOW = "low"
+INFORMATIONAL = "info"
+
+SEVERITY_WEIGHT = {
+    CRITICAL: 5.0,
+    HIGH: 3.0,
+    MEDIUM: 2.0,
+    LOW: 1.0,
+    INFORMATIONAL: 0.0,
+}
+
+STATUS_CREDIT = {PASS: 1.0, WARN: 0.5, FAIL: 0.0, INFO: 1.0, SKIP: 1.0}
+
+
+@dataclass
+class Check:
+    id: str
+    category: str
+    title: str
+    status: str
+    severity: str = MEDIUM
+    value: str = ""
+    recommendation: str = ""
+    details: dict = field(default_factory=dict)
+    url: str = ""
+
+    def to_dict(self) -> dict:
+        return {
+            "id": self.id,
+            "category": self.category,
+            "title": self.title,
+            "status": self.status,
+            "severity": self.severity,
+            "value": self.value,
+            "recommendation": self.recommendation,
+            "details": self.details,
+            "url": self.url,
+        }
+
+
+@dataclass
+class PageContext:
+    requested_url: str = ""
+    url: str = ""
+    status: int = 0
+    ok: bool = False
+    error: str = ""
+    elapsed_ms: int = 0
+    redirect_chain: list = field(default_factory=list)
+    headers: dict = field(default_factory=dict)
+    raw_html: str = ""
+    rendered_html: str = ""
+    dom: dict = field(default_factory=dict)
+    metrics: dict = field(default_factory=dict)
+    mobile: dict = field(default_factory=dict)
+    screenshot: str = ""
+
+
+@dataclass
+class SiteContext:
+    target_url: str = ""
+    mode: str = "url"
+    base_host: str = ""
+    base_scheme: str = "https"
+    robots: dict = field(default_factory=dict)
+    sitemap: dict = field(default_factory=dict)
+    llms_txt: dict = field(default_factory=dict)
+    pages: list = field(default_factory=list)
+
+
+PAGE_CHECKS: list[Callable] = []
+SITE_CHECKS: list[Callable] = []
+
+
+def page_check(func: Callable) -> Callable:
+    PAGE_CHECKS.append(func)
+    return func
+
+
+def site_check(func: Callable) -> Callable:
+    SITE_CHECKS.append(func)
+    return func
+
+
+def ok_check(
+    cid: str,
+    category: str,
+    title: str,
+    passed: bool,
+    *,
+    severity: str = MEDIUM,
+    value: str = "",
+    recommendation: str = "",
+    details: Optional[dict] = None,
+    url: str = "",
+    warn: bool = False,
+) -> Check:
+    if passed:
+        status = PASS
+    else:
+        status = WARN if warn else FAIL
+    return Check(
+        id=cid,
+        category=category,
+        title=title,
+        status=status,
+        severity=severity,
+        value=value,
+        recommendation=recommendation if not passed else "",
+        details=details or {},
+        url=url,
+    )

devplacepy/services/jobs/seo/checks/crawl.py

@@ -0,0 +1,302 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from urllib.parse import urlparse
+
+from .base import (
+    CRITICAL,
+    HIGH,
+    MEDIUM,
+    LOW,
+    PASS,
+    WARN,
+    FAIL,
+    INFO,
+    Check,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+    site_check,
+)
+
+CATEGORY = "crawlability"
+
+
+def _normalize(url: str) -> str:
+    return url.rstrip("/").split("#")[0]
+
+
+@page_check
+def http_status(page: PageContext, site: SiteContext) -> Check:
+    passed = 200 <= page.status < 300
+    return ok_check(
+        "crawl.http_status",
+        CATEGORY,
+        "HTTP status",
+        passed,
+        severity=CRITICAL,
+        value=str(page.status),
+        recommendation="Return a 200 OK status for indexable pages.",
+        url=page.url,
+    )
+
+
+@page_check
+def redirects(page: PageContext, site: SiteContext) -> Check:
+    hops = len(page.redirect_chain)
+    if hops == 0:
+        return Check(
+            "crawl.redirects",
+            CATEGORY,
+            "Redirect chain",
+            PASS,
+            LOW,
+            value="no redirect",
+            url=page.url,
+        )
+    status = WARN if hops <= 2 else FAIL
+    return Check(
+        "crawl.redirects",
+        CATEGORY,
+        "Redirect chain",
+        status,
+        MEDIUM,
+        value=f"{hops} hop(s)",
+        recommendation="Link directly to the final URL to avoid redirect latency and link-equity loss.",
+        details={"chain": page.redirect_chain},
+        url=page.url,
+    )
+
+
+@page_check
+def https(page: PageContext, site: SiteContext) -> Check:
+    passed = page.url.startswith("https://")
+    return ok_check(
+        "crawl.https",
+        CATEGORY,
+        "Served over HTTPS",
+        passed,
+        severity=HIGH,
+        value="https" if passed else "http",
+        recommendation="Serve every page over HTTPS.",
+        url=page.url,
+    )
+
+
+@page_check
+def hsts(page: PageContext, site: SiteContext) -> Check:
+    if not page.url.startswith("https://"):
+        return None
+    present = bool(page.headers.get("strict-transport-security"))
+    return ok_check(
+        "crawl.hsts",
+        CATEGORY,
+        "HSTS header",
+        present,
+        severity=LOW,
+        value="present" if present else "missing",
+        recommendation="Add a Strict-Transport-Security header to enforce HTTPS.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def canonical(page: PageContext, site: SiteContext) -> list:
+    href = page.dom.get("canonical", "")
+    count = int(page.dom.get("canonicalCount", 0) or 0)
+    results = [
+        ok_check(
+            "crawl.canonical_present",
+            CATEGORY,
+            "Canonical tag",
+            bool(href),
+            severity=MEDIUM,
+            value=href or "missing",
+            recommendation="Add a self-referential <link rel=canonical> to consolidate duplicates.",
+            warn=True,
+            url=page.url,
+        )
+    ]
+    if count > 1:
+        results.append(
+            Check(
+                "crawl.canonical_single",
+                CATEGORY,
+                "Single canonical",
+                FAIL,
+                MEDIUM,
+                value=f"{count} canonicals",
+                recommendation="Declare exactly one canonical URL per page.",
+                url=page.url,
+            )
+        )
+    if href:
+        same = _normalize(href) == _normalize(page.url)
+        results.append(
+            ok_check(
+                "crawl.canonical_self",
+                CATEGORY,
+                "Self-referential canonical",
+                same,
+                severity=LOW,
+                value="self" if same else href,
+                recommendation="Point the canonical at this page's own URL unless intentionally consolidating.",
+                warn=True,
+                url=page.url,
+            )
+        )
+    return results
+
+
+@page_check
+def indexability(page: PageContext, site: SiteContext) -> Check:
+    robots = str(page.dom.get("metaRobots", "")).lower()
+    header = str(page.headers.get("x-robots-tag", "")).lower()
+    blocked = "noindex" in robots or "noindex" in header
+    return ok_check(
+        "crawl.indexable",
+        CATEGORY,
+        "Indexable (no noindex)",
+        not blocked,
+        severity=HIGH,
+        value="noindex" if blocked else "indexable",
+        recommendation="Remove noindex from meta robots / X-Robots-Tag if this page should rank.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def url_hygiene(page: PageContext, site: SiteContext) -> Check:
+    parsed = urlparse(page.url)
+    path = parsed.path
+    issues = []
+    if len(page.url) > 100:
+        issues.append("long")
+    if any(c.isupper() for c in path):
+        issues.append("uppercase")
+    if "_" in path:
+        issues.append("underscores")
+    if parsed.query.count("&") >= 3:
+        issues.append("many params")
+    passed = not issues
+    return ok_check(
+        "crawl.url_hygiene",
+        CATEGORY,
+        "Clean URL",
+        passed,
+        severity=LOW,
+        value="clean" if passed else ", ".join(issues),
+        recommendation="Use short lowercase hyphenated paths with few query parameters.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def mixed_content(page: PageContext, site: SiteContext) -> Check:
+    if not page.url.startswith("https://"):
+        return None
+    items = page.dom.get("mixedContent", []) or []
+    passed = not items
+    return ok_check(
+        "crawl.mixed_content",
+        CATEGORY,
+        "No mixed content",
+        passed,
+        severity=HIGH,
+        value="clean" if passed else f"{len(items)} insecure resource(s)",
+        recommendation="Load every sub-resource over HTTPS to avoid mixed-content blocking.",
+        details={"resources": items[:20]},
+        url=page.url,
+    )
+
+
+@site_check
+def robots_txt(site: SiteContext) -> list:
+    robots = site.robots or {}
+    fetched = robots.get("status") == 200
+    results = [
+        ok_check(
+            "crawl.robots_txt",
+            CATEGORY,
+            "robots.txt present",
+            fetched,
+            severity=MEDIUM,
+            value=f"{robots.get('status', 'n/a')}",
+            recommendation="Publish a robots.txt at the site root.",
+            warn=True,
+        )
+    ]
+    if fetched:
+        results.append(
+            ok_check(
+                "crawl.robots_sitemap",
+                CATEGORY,
+                "Sitemap declared in robots.txt",
+                bool(robots.get("sitemap_urls")),
+                severity=LOW,
+                value=", ".join(robots.get("sitemap_urls", [])[:3]) or "none",
+                recommendation="Reference your sitemap with a Sitemap: directive in robots.txt.",
+                warn=True,
+            )
+        )
+        if robots.get("blocks_target"):
+            results.append(
+                Check(
+                    "crawl.robots_blocks_target",
+                    CATEGORY,
+                    "robots.txt blocks the audited URL",
+                    FAIL,
+                    HIGH,
+                    value="disallowed",
+                    recommendation="The audited URL is disallowed in robots.txt; allow it if it should be crawled.",
+                )
+            )
+    return results
+
+
+@site_check
+def sitemap_xml(site: SiteContext) -> list:
+    sm = site.sitemap or {}
+    fetched = sm.get("status") == 200
+    results = [
+        ok_check(
+            "crawl.sitemap_present",
+            CATEGORY,
+            "XML sitemap reachable",
+            fetched,
+            severity=MEDIUM,
+            value=f"{sm.get('url_count', 0)} url(s)" if fetched else f"{sm.get('status', 'n/a')}",
+            recommendation="Publish an XML sitemap and submit it to search engines.",
+            warn=True,
+        )
+    ]
+    if fetched:
+        results.append(
+            ok_check(
+                "crawl.sitemap_valid",
+                CATEGORY,
+                "Sitemap is valid XML",
+                bool(sm.get("valid_xml")),
+                severity=MEDIUM,
+                value="valid" if sm.get("valid_xml") else "invalid",
+                recommendation="Fix the sitemap XML so crawlers can parse it.",
+            )
+        )
+        results.append(
+            ok_check(
+                "crawl.sitemap_lastmod",
+                CATEGORY,
+                "Sitemap uses lastmod",
+                int(sm.get("lastmod_count", 0) or 0) > 0,
+                severity=LOW,
+                value=f"{sm.get('lastmod_count', 0)} entries",
+                recommendation="Add <lastmod> dates so crawlers prioritise fresh pages.",
+                warn=True,
+            )
+        )
+    return results

devplacepy/services/jobs/seo/checks/crosspage.py

@@ -0,0 +1,79 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from collections import defaultdict
+
+from .base import (
+    MEDIUM,
+    LOW,
+    INFO,
+    Check,
+    SiteContext,
+    site_check,
+)
+
+CATEGORY = "crosspage"
+
+
+def _duplicates(site: SiteContext, dom_key: str) -> dict:
+    groups: dict[str, list] = defaultdict(list)
+    for page in site.pages:
+        value = (page.dom.get(dom_key, "") or "").strip().lower()
+        if value:
+            groups[value].append(page.url)
+    return {value: urls for value, urls in groups.items() if len(urls) > 1}
+
+
+@site_check
+def duplicate_titles(site: SiteContext) -> Check:
+    if len(site.pages) < 2:
+        return None
+    dupes = _duplicates(site, "title")
+    passed = not dupes
+    return Check(
+        "crosspage.duplicate_titles",
+        CATEGORY,
+        "Unique page titles",
+        "pass" if passed else "warn",
+        MEDIUM,
+        value="unique" if passed else f"{len(dupes)} duplicated title(s)",
+        recommendation="" if passed else "Give every page a distinct <title>.",
+        details={"duplicates": {k: v for k, v in list(dupes.items())[:10]}},
+    )
+
+
+@site_check
+def duplicate_descriptions(site: SiteContext) -> Check:
+    if len(site.pages) < 2:
+        return None
+    dupes = _duplicates(site, "metaDescription")
+    passed = not dupes
+    return Check(
+        "crosspage.duplicate_descriptions",
+        CATEGORY,
+        "Unique meta descriptions",
+        "pass" if passed else "warn",
+        LOW,
+        value="unique" if passed else f"{len(dupes)} duplicated description(s)",
+        recommendation="" if passed else "Write a distinct meta description for every page.",
+        details={"duplicates": {k: v for k, v in list(dupes.items())[:10]}},
+    )
+
+
+@site_check
+def hreflang_usage(site: SiteContext) -> Check:
+    annotated = [p for p in site.pages if p.dom.get("hreflang")]
+    if not annotated:
+        return None
+    return Check(
+        "crosspage.hreflang",
+        CATEGORY,
+        "Hreflang annotations",
+        INFO,
+        INFO,
+        value=f"{len(annotated)} page(s) declare hreflang",
+        details={
+            "pages": {p.url: p.dom.get("hreflang") for p in annotated[:5]},
+        },
+    )

devplacepy/services/jobs/seo/checks/headings.py

@@ -0,0 +1,90 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from .base import (
+    MEDIUM,
+    LOW,
+    WARN,
+    Check,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+)
+
+CATEGORY = "content"
+
+THIN_CONTENT_WORDS = 250
+
+
+@page_check
+def single_h1(page: PageContext, site: SiteContext) -> list:
+    h1s = page.dom.get("h1", []) or []
+    results = [
+        ok_check(
+            "content.h1_present",
+            CATEGORY,
+            "H1 heading",
+            len(h1s) >= 1,
+            severity=MEDIUM,
+            value=(h1s[0][:80] if h1s else "missing"),
+            recommendation="Add a single descriptive H1 heading.",
+            url=page.url,
+        )
+    ]
+    if len(h1s) > 1:
+        results.append(
+            Check(
+                "content.h1_single",
+                CATEGORY,
+                "Single H1",
+                WARN,
+                LOW,
+                value=f"{len(h1s)} H1s",
+                recommendation="Use exactly one H1 per page.",
+                url=page.url,
+            )
+        )
+    return results
+
+
+@page_check
+def heading_order(page: PageContext, site: SiteContext) -> Check:
+    headings = page.dom.get("headings", []) or []
+    levels = [int(h.get("level", 0)) for h in headings if h.get("level")]
+    skipped = False
+    prev = 0
+    for level in levels:
+        if prev and level > prev + 1:
+            skipped = True
+            break
+        prev = level
+    return ok_check(
+        "content.heading_order",
+        CATEGORY,
+        "Heading hierarchy",
+        not skipped,
+        severity=LOW,
+        value="ordered" if not skipped else "skips a level",
+        recommendation="Do not skip heading levels (e.g. H2 to H4).",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def word_count(page: PageContext, site: SiteContext) -> Check:
+    words = int(page.dom.get("wordCount", 0) or 0)
+    passed = words >= THIN_CONTENT_WORDS
+    return ok_check(
+        "content.word_count",
+        CATEGORY,
+        "Content depth",
+        passed,
+        severity=MEDIUM,
+        value=f"{words} words",
+        recommendation=f"Thin content; aim for more than {THIN_CONTENT_WORDS} meaningful words.",
+        warn=True,
+        url=page.url,
+    )

devplacepy/services/jobs/seo/checks/links.py

@@ -0,0 +1,99 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from urllib.parse import urlparse
+
+from .base import (
+    MEDIUM,
+    LOW,
+    INFO,
+    Check,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+)
+
+CATEGORY = "links"
+
+GENERIC_ANCHORS = {
+    "click here",
+    "here",
+    "read more",
+    "more",
+    "link",
+    "this",
+    "this page",
+}
+
+
+@page_check
+def link_counts(page: PageContext, site: SiteContext) -> Check:
+    links = page.dom.get("links", []) or []
+    host = site.base_host
+    internal = 0
+    external = 0
+    for link in links:
+        href = link.get("href", "")
+        netloc = urlparse(href).netloc
+        if not netloc or netloc == host:
+            internal += 1
+        else:
+            external += 1
+    return Check(
+        "links.counts",
+        CATEGORY,
+        "Link profile",
+        INFO,
+        INFO,
+        value=f"{internal} internal, {external} external",
+        details={"internal": internal, "external": external},
+        url=page.url,
+    )
+
+
+@page_check
+def internal_links_present(page: PageContext, site: SiteContext) -> Check:
+    links = page.dom.get("links", []) or []
+    host = site.base_host
+    internal = [
+        link
+        for link in links
+        if not urlparse(link.get("href", "")).netloc
+        or urlparse(link.get("href", "")).netloc == host
+    ]
+    return ok_check(
+        "links.internal_present",
+        CATEGORY,
+        "Internal links",
+        len(internal) >= 1,
+        severity=LOW,
+        value=f"{len(internal)} internal link(s)",
+        recommendation="Add internal links so crawlers can discover related pages.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def generic_anchors(page: PageContext, site: SiteContext) -> Check:
+    links = page.dom.get("links", []) or []
+    generic = [
+        link
+        for link in links
+        if (link.get("text", "") or "").strip().lower() in GENERIC_ANCHORS
+    ]
+    passed = not generic
+    return ok_check(
+        "links.anchor_text",
+        CATEGORY,
+        "Descriptive anchor text",
+        passed,
+        severity=LOW,
+        value="descriptive" if passed else f"{len(generic)} generic anchor(s)",
+        recommendation="Replace generic anchors like 'click here' with descriptive text.",
+        warn=True,
+        details={"samples": [link.get("href", "") for link in generic[:10]]},
+        url=page.url,
+    )

devplacepy/services/jobs/seo/checks/meta.py

@@ -0,0 +1,169 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from .base import (
+    HIGH,
+    MEDIUM,
+    LOW,
+    PASS,
+    WARN,
+    Check,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+)
+
+CATEGORY = "meta"
+
+TITLE_MIN = 30
+TITLE_MAX = 60
+DESC_MIN = 70
+DESC_MAX = 160
+
+
+@page_check
+def title(page: PageContext, site: SiteContext) -> list:
+    text = (page.dom.get("title", "") or "").strip()
+    count = int(page.dom.get("titleCount", 0) or 0)
+    results = [
+        ok_check(
+            "meta.title_present",
+            CATEGORY,
+            "Title tag",
+            bool(text),
+            severity=HIGH,
+            value=text or "missing",
+            recommendation="Add a unique, descriptive <title>.",
+            url=page.url,
+        )
+    ]
+    if text:
+        length = len(text)
+        good = TITLE_MIN <= length <= TITLE_MAX
+        results.append(
+            ok_check(
+                "meta.title_length",
+                CATEGORY,
+                "Title length",
+                good,
+                severity=LOW,
+                value=f"{length} chars",
+                recommendation=f"Keep titles between {TITLE_MIN} and {TITLE_MAX} characters.",
+                warn=True,
+                url=page.url,
+            )
+        )
+    if count > 1:
+        results.append(
+            Check(
+                "meta.title_single",
+                CATEGORY,
+                "Single title tag",
+                WARN,
+                LOW,
+                value=f"{count} titles",
+                recommendation="Use exactly one <title> element.",
+                url=page.url,
+            )
+        )
+    return results
+
+
+@page_check
+def description(page: PageContext, site: SiteContext) -> list:
+    text = (page.dom.get("metaDescription", "") or "").strip()
+    results = [
+        ok_check(
+            "meta.description_present",
+            CATEGORY,
+            "Meta description",
+            bool(text),
+            severity=MEDIUM,
+            value=text[:120] or "missing",
+            recommendation="Write a compelling 70-160 character meta description.",
+            warn=True,
+            url=page.url,
+        )
+    ]
+    if text:
+        length = len(text)
+        good = DESC_MIN <= length <= DESC_MAX
+        results.append(
+            ok_check(
+                "meta.description_length",
+                CATEGORY,
+                "Description length",
+                good,
+                severity=LOW,
+                value=f"{length} chars",
+                recommendation=f"Keep meta descriptions between {DESC_MIN} and {DESC_MAX} characters.",
+                warn=True,
+                url=page.url,
+            )
+        )
+    return results
+
+
+@page_check
+def lang(page: PageContext, site: SiteContext) -> Check:
+    value = (page.dom.get("htmlLang", "") or "").strip()
+    return ok_check(
+        "meta.html_lang",
+        CATEGORY,
+        "Document language",
+        bool(value),
+        severity=LOW,
+        value=value or "missing",
+        recommendation="Declare the page language with <html lang=...>.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def charset(page: PageContext, site: SiteContext) -> Check:
+    value = (page.dom.get("charset", "") or "").strip()
+    return ok_check(
+        "meta.charset",
+        CATEGORY,
+        "Character encoding",
+        bool(value),
+        severity=LOW,
+        value=value or "missing",
+        recommendation="Declare a <meta charset> (UTF-8 recommended).",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def viewport(page: PageContext, site: SiteContext) -> Check:
+    present = bool(page.dom.get("hasViewport"))
+    return ok_check(
+        "meta.viewport",
+        CATEGORY,
+        "Mobile viewport",
+        present,
+        severity=HIGH,
+        value=page.dom.get("viewportContent", "") or ("present" if present else "missing"),
+        recommendation="Add <meta name=viewport content='width=device-width, initial-scale=1'>.",
+        url=page.url,
+    )
+
+
+@page_check
+def favicon(page: PageContext, site: SiteContext) -> Check:
+    present = bool(page.dom.get("favicon"))
+    return ok_check(
+        "meta.favicon",
+        CATEGORY,
+        "Favicon",
+        present,
+        severity=LOW,
+        value="present" if present else "missing",
+        recommendation="Provide a favicon for brand recognition in results and tabs.",
+        warn=True,
+        url=page.url,
+    )

devplacepy/services/jobs/seo/checks/mobile_a11y.py

@@ -0,0 +1,103 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from .base import (
+    HIGH,
+    MEDIUM,
+    LOW,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+)
+
+CATEGORY = "mobile_accessibility"
+
+
+@page_check
+def responsive(page: PageContext, site: SiteContext) -> PageContext:
+    mobile = page.mobile or {}
+    overflow = bool(mobile.get("hasHorizontalOverflow"))
+    return ok_check(
+        "mobile.no_overflow",
+        CATEGORY,
+        "No horizontal overflow (mobile)",
+        not overflow,
+        severity=MEDIUM,
+        value="ok" if not overflow else f"scrollWidth {mobile.get('scrollWidth', '?')}px",
+        recommendation="Eliminate horizontal scrolling at a 390px mobile viewport.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def tap_targets(page: PageContext, site: SiteContext):
+    mobile = page.mobile or {}
+    small = int(mobile.get("smallTapTargets", 0) or 0)
+    if "smallTapTargets" not in mobile:
+        return None
+    return ok_check(
+        "mobile.tap_targets",
+        CATEGORY,
+        "Tap target sizing",
+        small == 0,
+        severity=LOW,
+        value="ok" if small == 0 else f"{small} small target(s)",
+        recommendation="Make interactive targets at least 24x24 px (48 px recommended).",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def image_alt(page: PageContext, site: SiteContext):
+    images = page.dom.get("images", []) or []
+    if not images:
+        return None
+    missing = [img for img in images if not (img.get("alt", "") or "").strip()]
+    return ok_check(
+        "a11y.image_alt",
+        CATEGORY,
+        "Image alt coverage",
+        not missing,
+        severity=MEDIUM,
+        value="complete" if not missing else f"{len(missing)}/{len(images)} missing alt",
+        recommendation="Add descriptive alt text to every meaningful image.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def form_labels(page: PageContext, site: SiteContext):
+    missing = int(page.dom.get("formsMissingLabels", 0) or 0)
+    if not page.dom.get("formFieldCount"):
+        return None
+    return ok_check(
+        "a11y.form_labels",
+        CATEGORY,
+        "Form fields labelled",
+        missing == 0,
+        severity=LOW,
+        value="ok" if missing == 0 else f"{missing} unlabelled field(s)",
+        recommendation="Associate a <label> or aria-label with every form control.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def document_title(page: PageContext, site: SiteContext):
+    return ok_check(
+        "a11y.document_title",
+        CATEGORY,
+        "Accessible document title",
+        bool((page.dom.get("title", "") or "").strip()),
+        severity=LOW,
+        value="present" if (page.dom.get("title", "") or "").strip() else "missing",
+        recommendation="Every page needs a non-empty <title> for assistive tech.",
+        warn=True,
+        url=page.url,
+    )

devplacepy/services/jobs/seo/checks/performance.py

@@ -0,0 +1,283 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from .base import (
+    HIGH,
+    MEDIUM,
+    LOW,
+    INFO,
+    Check,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+)
+
+CATEGORY = "performance"
+
+LCP_GOOD_MS = 2500
+LCP_POOR_MS = 4000
+CLS_GOOD = 0.1
+CLS_POOR = 0.25
+FCP_GOOD_MS = 1800
+TTFB_GOOD_MS = 800
+PAGE_WEIGHT_BUDGET = 3_000_000
+REQUEST_BUDGET = 80
+DOM_NODE_BUDGET = 1500
+MODERN_FORMATS = (".webp", ".avif")
+
+
+def _ms(metrics: dict, key: str) -> float:
+    try:
+        return float(metrics.get(key) or 0)
+    except (TypeError, ValueError):
+        return 0.0
+
+
+@page_check
+def lcp(page: PageContext, site: SiteContext) -> Check:
+    value = _ms(page.metrics, "lcp")
+    if value <= 0:
+        return None
+    good = value <= LCP_GOOD_MS
+    return ok_check(
+        "perf.lcp",
+        CATEGORY,
+        "Largest Contentful Paint",
+        good,
+        severity=HIGH,
+        value=f"{value / 1000:.2f}s",
+        recommendation=f"Reduce LCP below {LCP_GOOD_MS / 1000:.1f}s (optimise hero image/render path).",
+        warn=value <= LCP_POOR_MS,
+        url=page.url,
+    )
+
+
+@page_check
+def cls(page: PageContext, site: SiteContext) -> Check:
+    value = _ms(page.metrics, "cls")
+    good = value <= CLS_GOOD
+    return ok_check(
+        "perf.cls",
+        CATEGORY,
+        "Cumulative Layout Shift",
+        good,
+        severity=HIGH,
+        value=f"{value:.3f}",
+        recommendation=f"Keep CLS under {CLS_GOOD} by reserving space for media and ads.",
+        warn=value <= CLS_POOR,
+        url=page.url,
+    )
+
+
+@page_check
+def fcp(page: PageContext, site: SiteContext) -> Check:
+    value = _ms(page.metrics, "fcp")
+    if value <= 0:
+        return None
+    return ok_check(
+        "perf.fcp",
+        CATEGORY,
+        "First Contentful Paint",
+        value <= FCP_GOOD_MS,
+        severity=LOW,
+        value=f"{value / 1000:.2f}s",
+        recommendation=f"Reduce FCP below {FCP_GOOD_MS / 1000:.1f}s.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def ttfb(page: PageContext, site: SiteContext) -> Check:
+    value = _ms(page.metrics, "ttfb")
+    if value <= 0:
+        return None
+    return ok_check(
+        "perf.ttfb",
+        CATEGORY,
+        "Time To First Byte",
+        value <= TTFB_GOOD_MS,
+        severity=MEDIUM,
+        value=f"{value:.0f}ms",
+        recommendation=f"Reduce server response time below {TTFB_GOOD_MS}ms.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def page_weight(page: PageContext, site: SiteContext) -> Check:
+    total = int(page.metrics.get("transferSize") or 0)
+    if total <= 0:
+        return None
+    return ok_check(
+        "perf.page_weight",
+        CATEGORY,
+        "Total page weight",
+        total <= PAGE_WEIGHT_BUDGET,
+        severity=MEDIUM,
+        value=f"{total / 1_000_000:.2f} MB",
+        recommendation=f"Trim total transfer below {PAGE_WEIGHT_BUDGET / 1_000_000:.0f} MB.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def request_count(page: PageContext, site: SiteContext) -> Check:
+    count = int(page.metrics.get("resourceCount") or 0)
+    if count <= 0:
+        return None
+    return ok_check(
+        "perf.requests",
+        CATEGORY,
+        "Request count",
+        count <= REQUEST_BUDGET,
+        severity=LOW,
+        value=str(count),
+        recommendation=f"Reduce HTTP requests below {REQUEST_BUDGET} (bundle, sprite, lazy-load).",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def dom_size(page: PageContext, site: SiteContext) -> Check:
+    nodes = int(page.metrics.get("domNodes") or 0)
+    if nodes <= 0:
+        return None
+    return ok_check(
+        "perf.dom_size",
+        CATEGORY,
+        "DOM size",
+        nodes <= DOM_NODE_BUDGET,
+        severity=LOW,
+        value=f"{nodes} nodes",
+        recommendation=f"Keep the DOM under {DOM_NODE_BUDGET} nodes for faster rendering.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def compression(page: PageContext, site: SiteContext) -> Check:
+    encoding = str(page.headers.get("content-encoding", "")).lower()
+    passed = any(token in encoding for token in ("gzip", "br", "zstd", "deflate"))
+    return ok_check(
+        "perf.compression",
+        CATEGORY,
+        "Text compression",
+        passed,
+        severity=MEDIUM,
+        value=encoding or "none",
+        recommendation="Enable gzip or brotli compression on text responses.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def caching(page: PageContext, site: SiteContext) -> Check:
+    cache = str(page.headers.get("cache-control", "")).lower()
+    passed = bool(cache) and "no-store" not in cache
+    return ok_check(
+        "perf.caching",
+        CATEGORY,
+        "Cache headers",
+        passed,
+        severity=LOW,
+        value=cache or "none",
+        recommendation="Set Cache-Control with sensible max-age for static assets.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def http_version(page: PageContext, site: SiteContext) -> Check:
+    version = str(page.metrics.get("protocol", "") or "")
+    if not version:
+        return None
+    modern = any(token in version.lower() for token in ("h2", "h3", "http/2", "http/3"))
+    return ok_check(
+        "perf.http_version",
+        CATEGORY,
+        "Modern HTTP protocol",
+        modern,
+        severity=LOW,
+        value=version,
+        recommendation="Serve over HTTP/2 or HTTP/3 for multiplexed delivery.",
+        warn=True,
+        url=page.url,
+    )
+
+
+@page_check
+def console_errors(page: PageContext, site: SiteContext) -> Check:
+    errors = int(page.metrics.get("consoleErrors") or 0)
+    return ok_check(
+        "perf.console_errors",
+        CATEGORY,
+        "No console errors",
+        errors == 0,
+        severity=LOW,
+        value="clean" if errors == 0 else f"{errors} error(s)",
+        recommendation="Resolve JavaScript console errors emitted on load.",
+        warn=True,
+        details={"samples": page.metrics.get("consoleErrorSamples", [])[:5]},
+        url=page.url,
+    )
+
+
+@page_check
+def image_optimization(page: PageContext, site: SiteContext) -> list:
+    images = page.dom.get("images", []) or []
+    if not images:
+        return []
+    no_dimensions = [
+        img for img in images if not img.get("width") or not img.get("height")
+    ]
+    legacy_format = [
+        img
+        for img in images
+        if (img.get("src", "") or "").lower().rsplit("?", 1)[0].endswith((".jpg", ".jpeg", ".png"))
+    ]
+    not_lazy = [img for img in images if (img.get("loading", "") or "") != "lazy"]
+    results = [
+        ok_check(
+            "perf.img_dimensions",
+            CATEGORY,
+            "Images have dimensions",
+            not no_dimensions,
+            severity=MEDIUM,
+            value="ok" if not no_dimensions else f"{len(no_dimensions)} without width/height",
+            recommendation="Set width and height on images to prevent layout shift (CLS).",
+            warn=True,
+            url=page.url,
+        ),
+        ok_check(
+            "perf.img_modern_format",
+            CATEGORY,
+            "Modern image formats",
+            not legacy_format,
+            severity=LOW,
+            value="ok" if not legacy_format else f"{len(legacy_format)} legacy image(s)",
+            recommendation="Serve WebP or AVIF instead of JPEG/PNG where possible.",
+            warn=True,
+            url=page.url,
+        ),
+        ok_check(
+            "perf.img_lazy",
+            CATEGORY,
+            "Off-screen images lazy-loaded",
+            len(not_lazy) <= 1,
+            severity=LOW,
+            value="ok" if len(not_lazy) <= 1 else f"{len(not_lazy)} eager image(s)",
+            recommendation="Add loading=lazy to below-the-fold images.",
+            warn=True,
+            url=page.url,
+        ),
+    ]
+    return results

devplacepy/services/jobs/seo/checks/registry.py

@@ -0,0 +1,131 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from .base import (
+    PAGE_CHECKS,
+    SITE_CHECKS,
+    SEVERITY_WEIGHT,
+    STATUS_CREDIT,
+    INFO,
+    SKIP,
+    Check,
+    PageContext,
+    SiteContext,
+)
+from . import crawl  # noqa: F401
+from . import meta  # noqa: F401
+from . import headings  # noqa: F401
+from . import links  # noqa: F401
+from . import structured_data  # noqa: F401
+from . import social  # noqa: F401
+from . import performance  # noqa: F401
+from . import mobile_a11y  # noqa: F401
+from . import security  # noqa: F401
+from . import ai_readiness  # noqa: F401
+from . import crosspage  # noqa: F401
+
+
+def _collect(result) -> list[Check]:
+    if result is None:
+        return []
+    if isinstance(result, Check):
+        return [result]
+    return [c for c in result if isinstance(c, Check)]
+
+
+def run_page_checks(page: PageContext, site: SiteContext) -> list[Check]:
+    checks: list[Check] = []
+    for func in PAGE_CHECKS:
+        try:
+            checks.extend(_collect(func(page, site)))
+        except Exception as exc:  # noqa: BLE001 - one bad check never aborts the page
+            checks.append(
+                Check(
+                    id=f"{func.__name__}.error",
+                    category="internal",
+                    title=f"Check {func.__name__} failed",
+                    status=INFO,
+                    severity="info",
+                    value=str(exc)[:200],
+                    url=page.url,
+                )
+            )
+    return checks
+
+
+def run_site_checks(site: SiteContext) -> list[Check]:
+    checks: list[Check] = []
+    for func in SITE_CHECKS:
+        try:
+            checks.extend(_collect(func(site)))
+        except Exception as exc:  # noqa: BLE001
+            checks.append(
+                Check(
+                    id=f"{func.__name__}.error",
+                    category="internal",
+                    title=f"Site check {func.__name__} failed",
+                    status=INFO,
+                    severity="info",
+                    value=str(exc)[:200],
+                )
+            )
+    return checks
+
+
+def compute_score(checks: list[Check]) -> dict:
+    by_category: dict[str, dict] = {}
+    earned = 0.0
+    weight = 0.0
+    counts = {"pass": 0, "warn": 0, "fail": 0, "info": 0, "skip": 0}
+    for check in checks:
+        counts[check.status] = counts.get(check.status, 0) + 1
+        bucket = by_category.setdefault(
+            check.category,
+            {"earned": 0.0, "weight": 0.0, "pass": 0, "warn": 0, "fail": 0, "info": 0, "skip": 0},
+        )
+        bucket[check.status] = bucket.get(check.status, 0) + 1
+        if check.status in (INFO, SKIP):
+            continue
+        w = SEVERITY_WEIGHT.get(check.severity, 1.0)
+        if w <= 0:
+            continue
+        credit = STATUS_CREDIT.get(check.status, 0.0)
+        earned += w * credit
+        weight += w
+        bucket["earned"] += w * credit
+        bucket["weight"] += w
+    categories = {}
+    for name, bucket in by_category.items():
+        cat_score = (
+            round(100 * bucket["earned"] / bucket["weight"])
+            if bucket["weight"] > 0
+            else None
+        )
+        categories[name] = {
+            "score": cat_score,
+            "pass": bucket["pass"],
+            "warn": bucket["warn"],
+            "fail": bucket["fail"],
+            "info": bucket["info"],
+            "skip": bucket["skip"],
+        }
+    score = round(100 * earned / weight) if weight > 0 else 0
+    return {
+        "score": score,
+        "grade": _grade(score),
+        "counts": counts,
+        "categories": categories,
+    }
+
+
+def _grade(score: int) -> str:
+    if score >= 90:
+        return "A"
+    if score >= 80:
+        return "B"
+    if score >= 70:
+        return "C"
+    if score >= 55:
+        return "D"
+    return "F"

devplacepy/services/jobs/seo/checks/security.py

@@ -0,0 +1,59 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from .base import (
+    MEDIUM,
+    LOW,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+)
+
+CATEGORY = "security"
+
+SECURITY_HEADERS = {
+    "content-security-policy": ("Content-Security-Policy", LOW),
+    "x-content-type-options": ("X-Content-Type-Options", LOW),
+    "x-frame-options": ("X-Frame-Options", LOW),
+    "referrer-policy": ("Referrer-Policy", LOW),
+}
+
+
+@page_check
+def security_headers(page: PageContext, site: SiteContext) -> list:
+    results = []
+    for key, (label, severity) in SECURITY_HEADERS.items():
+        present = bool(page.headers.get(key))
+        results.append(
+            ok_check(
+                f"security.{key}",
+                CATEGORY,
+                label,
+                present,
+                severity=severity,
+                value="present" if present else "missing",
+                recommendation=f"Add the {label} response header.",
+                warn=True,
+                url=page.url,
+            )
+        )
+    return results
+
+
+@page_check
+def tls_valid(page: PageContext, site: SiteContext):
+    if "tlsValid" not in page.metrics:
+        return None
+    valid = bool(page.metrics.get("tlsValid"))
+    return ok_check(
+        "security.tls",
+        CATEGORY,
+        "Valid TLS certificate",
+        valid,
+        severity=MEDIUM,
+        value="valid" if valid else "invalid",
+        recommendation="Serve a valid, unexpired TLS certificate.",
+        url=page.url,
+    )

devplacepy/services/jobs/seo/checks/social.py

@@ -0,0 +1,59 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from .base import (
+    MEDIUM,
+    LOW,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+)
+
+CATEGORY = "social"
+
+OG_REQUIRED = ("og:title", "og:description", "og:image", "og:url")
+TWITTER_REQUIRED = ("twitter:card",)
+
+
+@page_check
+def open_graph(page: PageContext, site: SiteContext) -> list:
+    og = page.dom.get("og", {}) or {}
+    missing = [tag for tag in OG_REQUIRED if not og.get(tag)]
+    results = [
+        ok_check(
+            "social.open_graph",
+            CATEGORY,
+            "Open Graph tags",
+            not missing,
+            severity=MEDIUM,
+            value="complete" if not missing else f"missing {', '.join(missing)}",
+            recommendation="Add og:title, og:description, og:image and og:url for rich social previews.",
+            warn=True,
+            details={"present": sorted(og.keys())},
+            url=page.url,
+        )
+    ]
+    return results
+
+
+@page_check
+def twitter_card(page: PageContext, site: SiteContext) -> list:
+    twitter = page.dom.get("twitter", {}) or {}
+    og = page.dom.get("og", {}) or {}
+    has_card = bool(twitter.get("twitter:card"))
+    return [
+        ok_check(
+            "social.twitter_card",
+            CATEGORY,
+            "Twitter Card",
+            has_card or bool(og),
+            severity=LOW,
+            value=twitter.get("twitter:card", "") or ("falls back to OG" if og else "missing"),
+            recommendation="Add a twitter:card meta tag (summary_large_image) for X/Twitter previews.",
+            warn=True,
+            details={"present": sorted(twitter.keys())},
+            url=page.url,
+        )
+    ]

devplacepy/services/jobs/seo/checks/structured_data.py

@@ -0,0 +1,165 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+import json
+
+from .base import (
+    MEDIUM,
+    LOW,
+    INFO,
+    PASS,
+    WARN,
+    FAIL,
+    Check,
+    PageContext,
+    SiteContext,
+    ok_check,
+    page_check,
+)
+
+CATEGORY = "structured_data"
+
+REQUIRED_PROPS = {
+    "Article": ("headline",),
+    "BlogPosting": ("headline",),
+    "NewsArticle": ("headline",),
+    "Product": ("name",),
+    "Offer": ("price", "priceCurrency"),
+    "Organization": ("name", "url"),
+    "LocalBusiness": ("name", "address"),
+    "BreadcrumbList": ("itemListElement",),
+    "FAQPage": ("mainEntity",),
+    "WebSite": ("name", "url"),
+    "Person": ("name",),
+    "Recipe": ("name", "recipeIngredient"),
+    "Event": ("name", "startDate"),
+    "VideoObject": ("name", "thumbnailUrl", "uploadDate"),
+    "SoftwareApplication": ("name",),
+}
+
+
+def _iter_objects(parsed):
+    if isinstance(parsed, list):
+        for item in parsed:
+            yield from _iter_objects(item)
+        return
+    if not isinstance(parsed, dict):
+        return
+    graph = parsed.get("@graph")
+    if isinstance(graph, list):
+        for item in graph:
+            yield from _iter_objects(item)
+    if parsed.get("@type"):
+        yield parsed
+
+
+def _types(node) -> list:
+    raw = node.get("@type")
+    if isinstance(raw, list):
+        return [str(item) for item in raw]
+    return [str(raw)] if raw else []
+
+
+@page_check
+def jsonld(page: PageContext, site: SiteContext) -> list:
+    blocks = page.dom.get("jsonld", []) or []
+    present = bool(blocks)
+    results = [
+        ok_check(
+            "structured.jsonld_present",
+            CATEGORY,
+            "Structured data (JSON-LD)",
+            present,
+            severity=MEDIUM,
+            value=f"{len(blocks)} block(s)" if present else "none",
+            recommendation="Add JSON-LD structured data so search engines can render rich results.",
+            warn=True,
+            url=page.url,
+        )
+    ]
+    if not present:
+        return results
+
+    invalid = 0
+    nodes = []
+    for block in blocks:
+        try:
+            parsed = json.loads(block)
+        except (ValueError, TypeError):
+            invalid += 1
+            continue
+        nodes.extend(_iter_objects(parsed))
+
+    results.append(
+        ok_check(
+            "structured.jsonld_valid",
+            CATEGORY,
+            "JSON-LD parses",
+            invalid == 0,
+            severity=MEDIUM,
+            value="valid" if invalid == 0 else f"{invalid} invalid block(s)",
+            recommendation="Fix JSON syntax errors in structured-data blocks.",
+            url=page.url,
+        )
+    )
+
+    found_types = sorted({t for node in nodes for t in _types(node)})
+    results.append(
+        Check(
+            "structured.types",
+            CATEGORY,
+            "Schema types",
+            INFO,
+            INFO,
+            value=", ".join(found_types) or "none",
+            details={"types": found_types},
+            url=page.url,
+        )
+    )
+
+    missing = []
+    for node in nodes:
+        for type_name in _types(node):
+            required = REQUIRED_PROPS.get(type_name)
+            if not required:
+                continue
+            absent = [prop for prop in required if not node.get(prop)]
+            if absent:
+                missing.append(f"{type_name}: {', '.join(absent)}")
+    if missing:
+        results.append(
+            Check(
+                "structured.required_props",
+                CATEGORY,
+                "Required schema properties",
+                WARN,
+                LOW,
+                value=f"{len(missing)} type(s) missing props",
+                recommendation="Add the required properties for each declared schema type.",
+                details={"missing": missing[:20]},
+                url=page.url,
+            )
+        )
+    return results
+
+
+@page_check
+def other_markup(page: PageContext, site: SiteContext) -> Check:
+    microdata = bool(page.dom.get("microdata"))
+    rdfa = bool(page.dom.get("rdfa"))
+    formats = []
+    if microdata:
+        formats.append("microdata")
+    if rdfa:
+        formats.append("RDFa")
+    return Check(
+        "structured.other_markup",
+        CATEGORY,
+        "Other structured-data formats",
+        INFO,
+        INFO,
+        value=", ".join(formats) or "none",
+        details={"microdata": microdata, "rdfa": rdfa},
+        url=page.url,
+    )

devplacepy/services/jobs/seo/crawler.py

@@ -0,0 +1,422 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+import asyncio
+from urllib.parse import urljoin, urlparse
+from xml.etree import ElementTree
+
+import httpx
+
+from devplacepy.net_guard import BlockedAddressError, guard_public_url
+from .checks.base import PageContext, SiteContext
+
+USER_AGENT = (
+    "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) "
+    "Chrome/131.0.0.0 Safari/537.36 DevPlaceSEOBot/1.0"
+)
+NAV_TIMEOUT_MS = 30000
+RAW_FETCH_TIMEOUT = 15.0
+MAX_RAW_BYTES = 3_000_000
+MOBILE_VIEWPORT = {"width": 390, "height": 844}
+DESKTOP_VIEWPORT = {"width": 1366, "height": 900}
+
+INIT_SCRIPT = """
+window.__seo = { lcp: 0, cls: 0, consoleErrors: 0, errorSamples: [] };
+try {
+  new PerformanceObserver((list) => {
+    for (const entry of list.getEntries()) {
+      window.__seo.lcp = Math.max(window.__seo.lcp, entry.startTime || entry.renderTime || 0);
+    }
+  }).observe({ type: 'largest-contentful-paint', buffered: true });
+} catch (e) {}
+try {
+  new PerformanceObserver((list) => {
+    for (const entry of list.getEntries()) {
+      if (!entry.hadRecentInput) window.__seo.cls += entry.value;
+    }
+  }).observe({ type: 'layout-shift', buffered: true });
+} catch (e) {}
+"""
+
+EXTRACT_SCRIPT = r"""
+() => {
+  const abs = (href) => { try { return new URL(href, document.baseURI).href; } catch (e) { return href || ''; } };
+  const metaByName = (name) => {
+    const el = document.querySelector(`meta[name="${name}"]`);
+    return el ? (el.getAttribute('content') || '') : '';
+  };
+  const og = {};
+  document.querySelectorAll('meta[property^="og:"]').forEach((m) => {
+    og[m.getAttribute('property')] = m.getAttribute('content') || '';
+  });
+  const twitter = {};
+  document.querySelectorAll('meta[name^="twitter:"]').forEach((m) => {
+    twitter[m.getAttribute('name')] = m.getAttribute('content') || '';
+  });
+  const headings = [];
+  document.querySelectorAll('h1,h2,h3,h4,h5,h6').forEach((h) => {
+    headings.push({ level: parseInt(h.tagName.substring(1), 10), text: (h.textContent || '').trim().slice(0, 160) });
+  });
+  const h1 = Array.from(document.querySelectorAll('h1')).map((h) => (h.textContent || '').trim().slice(0, 160));
+  const images = Array.from(document.querySelectorAll('img')).slice(0, 200).map((img) => ({
+    src: abs(img.getAttribute('src') || ''),
+    alt: img.getAttribute('alt'),
+    width: img.getAttribute('width') || (img.naturalWidth ? String(img.naturalWidth) : ''),
+    height: img.getAttribute('height') || (img.naturalHeight ? String(img.naturalHeight) : ''),
+    loading: img.getAttribute('loading') || ''
+  }));
+  const links = Array.from(document.querySelectorAll('a[href]')).slice(0, 500).map((a) => ({
+    href: abs(a.getAttribute('href') || ''),
+    rel: a.getAttribute('rel') || '',
+    text: (a.textContent || '').trim().slice(0, 120)
+  }));
+  const jsonld = Array.from(document.querySelectorAll('script[type="application/ld+json"]')).map((s) => s.textContent || '');
+  const hreflang = Array.from(document.querySelectorAll('link[rel="alternate"][hreflang]')).map((l) => ({
+    hreflang: l.getAttribute('hreflang'), href: abs(l.getAttribute('href') || '')
+  }));
+  const canonicalEls = document.querySelectorAll('link[rel="canonical"]');
+  const isHttps = location.protocol === 'https:';
+  const mixed = [];
+  if (isHttps) {
+    document.querySelectorAll('[src],[href]').forEach((el) => {
+      const v = el.getAttribute('src') || el.getAttribute('href') || '';
+      if (v.startsWith('http://')) mixed.push(v);
+    });
+  }
+  const viewportEl = document.querySelector('meta[name="viewport"]');
+  const charsetEl = document.querySelector('meta[charset]');
+  let formFieldCount = 0;
+  let formsMissingLabels = 0;
+  document.querySelectorAll('input,select,textarea').forEach((field) => {
+    const type = (field.getAttribute('type') || '').toLowerCase();
+    if (type === 'hidden' || type === 'submit' || type === 'button') return;
+    formFieldCount += 1;
+    const id = field.getAttribute('id');
+    const labelled = (id && document.querySelector(`label[for="${id}"]`)) ||
+      field.getAttribute('aria-label') || field.getAttribute('aria-labelledby') ||
+      field.closest('label');
+    if (!labelled) formsMissingLabels += 1;
+  });
+  const nav = performance.getEntriesByType('navigation')[0] || {};
+  const resources = performance.getEntriesByType('resource') || [];
+  let transfer = nav.transferSize || 0;
+  resources.forEach((r) => { transfer += (r.transferSize || 0); });
+  const bodyText = (document.body ? document.body.innerText || '' : '');
+  const wordCount = bodyText.split(/\s+/).filter(Boolean).length;
+  return {
+    title: (document.title || '').trim(),
+    titleCount: document.querySelectorAll('title').length,
+    metaDescription: metaByName('description'),
+    metaRobots: metaByName('robots'),
+    canonical: canonicalEls.length ? abs(canonicalEls[0].getAttribute('href') || '') : '',
+    canonicalCount: canonicalEls.length,
+    htmlLang: document.documentElement.getAttribute('lang') || '',
+    charset: charsetEl ? (charsetEl.getAttribute('charset') || '') : (document.characterSet || ''),
+    hasViewport: !!viewportEl,
+    viewportContent: viewportEl ? (viewportEl.getAttribute('content') || '') : '',
+    favicon: !!document.querySelector('link[rel~="icon"]'),
+    h1: h1,
+    headings: headings,
+    images: images,
+    links: links,
+    jsonld: jsonld,
+    hreflang: hreflang,
+    microdata: !!document.querySelector('[itemscope]'),
+    rdfa: !!document.querySelector('[vocab],[typeof],[property]'),
+    og: og,
+    twitter: twitter,
+    iframeCount: document.querySelectorAll('iframe').length,
+    semantic: {
+      main: document.querySelectorAll('main').length,
+      article: document.querySelectorAll('article').length,
+      nav: document.querySelectorAll('nav').length,
+      header: document.querySelectorAll('header').length,
+      footer: document.querySelectorAll('footer').length
+    },
+    mixedContent: mixed,
+    formFieldCount: formFieldCount,
+    formsMissingLabels: formsMissingLabels,
+    wordCount: wordCount,
+    metrics: {
+      ttfb: Math.max(0, (nav.responseStart || 0) - (nav.requestStart || 0)),
+      fcp: (performance.getEntriesByName('first-contentful-paint')[0] || {}).startTime || 0,
+      domContentLoaded: nav.domContentLoadedEventEnd || 0,
+      load: nav.loadEventEnd || 0,
+      transferSize: transfer,
+      resourceCount: resources.length,
+      domNodes: document.getElementsByTagName('*').length,
+      protocol: nav.nextHopProtocol || '',
+      lcp: window.__seo ? window.__seo.lcp : 0,
+      cls: window.__seo ? window.__seo.cls : 0
+    }
+  };
+}
+"""
+
+MOBILE_SCRIPT = r"""
+() => {
+  let small = 0;
+  document.querySelectorAll('a,button,[role="button"],input[type="submit"]').forEach((el) => {
+    const r = el.getBoundingClientRect();
+    if (r.width > 0 && r.height > 0 && (r.width < 24 || r.height < 24)) small += 1;
+  });
+  return {
+    scrollWidth: document.documentElement.scrollWidth,
+    clientWidth: document.documentElement.clientWidth,
+    hasHorizontalOverflow: document.documentElement.scrollWidth > document.documentElement.clientWidth + 4,
+    smallTapTargets: small
+  };
+}
+"""
+
+
+def _normalize_url(url: str) -> str:
+    url = (url or "").strip()
+    if not url:
+        return ""
+    if "://" not in url:
+        url = "https://" + url
+    return url
+
+
+async def _fetch_text(client: httpx.AsyncClient, url: str) -> dict:
+    try:
+        response = await client.get(url)
+        body = response.text[:MAX_RAW_BYTES]
+        return {"status": response.status_code, "text": body, "url": str(response.url)}
+    except httpx.HTTPError as exc:
+        return {"status": 0, "text": "", "url": url, "error": str(exc)[:200]}
+
+
+def _parse_robots(text: str, target_path: str) -> dict:
+    disallows = []
+    sitemap_urls = []
+    active = False
+    for raw in text.splitlines():
+        line = raw.split("#", 1)[0].strip()
+        if not line:
+            continue
+        key, _, value = line.partition(":")
+        key = key.strip().lower()
+        value = value.strip()
+        if key == "user-agent":
+            active = value == "*"
+        elif key == "sitemap":
+            sitemap_urls.append(value)
+        elif key == "disallow" and active and value:
+            disallows.append(value)
+    blocks_target = any(
+        target_path.startswith(rule) for rule in disallows if rule and rule != "/"
+    ) or "/" in [r for r in disallows]
+    return {
+        "disallows": disallows,
+        "sitemap_urls": sitemap_urls,
+        "blocks_target": blocks_target,
+    }
+
+
+def _parse_sitemap(text: str) -> dict:
+    result = {"valid_xml": False, "url_count": 0, "lastmod_count": 0, "urls": []}
+    try:
+        root = ElementTree.fromstring(text)
+    except ElementTree.ParseError:
+        return result
+    result["valid_xml"] = True
+    urls = []
+    lastmods = 0
+    for url_el in root.iter():
+        tag = url_el.tag.rsplit("}", 1)[-1]
+        if tag == "loc" and url_el.text:
+            urls.append(url_el.text.strip())
+        elif tag == "lastmod" and url_el.text:
+            lastmods += 1
+    result["url_count"] = len(urls)
+    result["lastmod_count"] = lastmods
+    result["urls"] = urls
+    return result
+
+
+async def _site_resources(
+    client: httpx.AsyncClient, base_url: str, sitemap_url: str, target_path: str
+) -> tuple[dict, dict, dict]:
+    parsed = urlparse(base_url)
+    root = f"{parsed.scheme}://{parsed.netloc}"
+    robots_raw = await _fetch_text(client, urljoin(root + "/", "robots.txt"))
+    robots = dict(robots_raw)
+    if robots_raw.get("status") == 200:
+        robots.update(_parse_robots(robots_raw.get("text", ""), target_path))
+    sm_target = sitemap_url or urljoin(root + "/", "sitemap.xml")
+    sitemap_raw = await _fetch_text(client, sm_target)
+    sitemap = dict(sitemap_raw)
+    if sitemap_raw.get("status") == 200:
+        sitemap.update(_parse_sitemap(sitemap_raw.get("text", "")))
+    llms_raw = await _fetch_text(client, urljoin(root + "/", "llms.txt"))
+    llms = {"found": llms_raw.get("status") == 200, "status": llms_raw.get("status")}
+    return robots, sitemap, llms
+
+
+async def _build_page(context, client, url: str, output_dir, index: int) -> PageContext:
+    page_ctx = PageContext(requested_url=url, url=url)
+    page = await context.new_page()
+    console_errors = []
+
+    def _on_console(message):
+        if message.type == "error":
+            console_errors.append(message.text[:200])
+
+    page.on("console", _on_console)
+    page.on("pageerror", lambda exc: console_errors.append(str(exc)[:200]))
+    await page.add_init_script(INIT_SCRIPT)
+    try:
+        response = await page.goto(url, wait_until="load", timeout=NAV_TIMEOUT_MS)
+        try:
+            await page.wait_for_load_state("networkidle", timeout=8000)
+        except Exception:
+            pass
+        page_ctx.status = response.status if response else 0
+        page_ctx.url = page.url
+        page_ctx.ok = bool(response and response.ok)
+        if response is not None:
+            page_ctx.headers = {k.lower(): v for k, v in response.headers.items()}
+            chain = []
+            req = response.request.redirected_from
+            while req is not None:
+                resp = await req.response()
+                chain.append({"url": req.url, "status": resp.status if resp else 0})
+                req = req.redirected_from
+            page_ctx.redirect_chain = list(reversed(chain))
+        dom = await page.evaluate(EXTRACT_SCRIPT)
+        page_ctx.metrics = dom.pop("metrics", {})
+        page_ctx.metrics["consoleErrors"] = len(console_errors)
+        page_ctx.metrics["consoleErrorSamples"] = console_errors[:5]
+        page_ctx.dom = dom
+        page_ctx.rendered_html = (await page.content())[:MAX_RAW_BYTES]
+        try:
+            await page.set_viewport_size(MOBILE_VIEWPORT)
+            page_ctx.mobile = await page.evaluate(MOBILE_SCRIPT)
+            await page.set_viewport_size(DESKTOP_VIEWPORT)
+        except Exception:
+            page_ctx.mobile = {}
+        if output_dir is not None:
+            shot = output_dir / f"page-{index}.png"
+            try:
+                await page.screenshot(path=str(shot), full_page=False)
+                page_ctx.screenshot = shot.name
+            except Exception:
+                page_ctx.screenshot = ""
+        raw = await _fetch_text(client, page_ctx.url)
+        page_ctx.raw_html = raw.get("text", "")
+    except Exception as exc:  # noqa: BLE001 - record the failure as page state
+        page_ctx.error = str(exc)[:300]
+        page_ctx.ok = False
+    finally:
+        await page.close()
+    return page_ctx
+
+
+async def crawl_target(payload: dict, emit, output_dir) -> SiteContext:
+    target = _normalize_url(payload.get("url", ""))
+    mode = payload.get("mode", "url")
+    allow_private = bool(payload.get("allow_private"))
+    max_pages = max(1, min(int(payload.get("max_pages", 1) or 1), 50))
+
+    await guard_public_url(target, allow_private=allow_private)
+    parsed = urlparse(target)
+    site = SiteContext(
+        target_url=target,
+        mode=mode,
+        base_host=parsed.netloc,
+        base_scheme=parsed.scheme,
+    )
+
+    from playwright.async_api import async_playwright
+
+    async with httpx.AsyncClient(
+        follow_redirects=True,
+        timeout=RAW_FETCH_TIMEOUT,
+        headers={"User-Agent": USER_AGENT},
+    ) as client:
+        emit({"type": "stage", "stage": "resources", "message": "Reading robots.txt and sitemap"})
+        sitemap_hint = target if mode == "sitemap" else ""
+        robots, sitemap, llms = await _site_resources(
+            client, target, sitemap_hint, parsed.path or "/"
+        )
+        site.robots = robots
+        site.sitemap = sitemap
+        site.llms_txt = llms
+
+        if mode == "sitemap":
+            candidates = sitemap.get("urls", [])[:max_pages]
+        else:
+            candidates = [target]
+
+        safe_urls = []
+        for url in candidates:
+            host = urlparse(url).netloc
+            if host and host == site.base_host:
+                # Same host as the audited target, already validated by the
+                # top-level guard above; skip the redundant per-URL DNS lookup.
+                safe_urls.append(url)
+                continue
+            try:
+                await guard_public_url(url, allow_private=allow_private)
+                safe_urls.append(url)
+            except BlockedAddressError:
+                continue
+
+        if not safe_urls:
+            if mode == "sitemap":
+                raise ValueError(
+                    "No crawlable page URLs found in the sitemap "
+                    f"({sitemap.get('url_count', 0)} entries; none reachable or all blocked)."
+                )
+            safe_urls = [target]
+
+        emit(
+            {
+                "type": "target",
+                "url": target,
+                "mode": mode,
+                "urls": safe_urls,
+                "sitemap_total": sitemap.get("url_count", 0) if mode == "sitemap" else 0,
+            }
+        )
+
+        async with async_playwright() as pw:
+            browser = await pw.chromium.launch(
+                headless=True, args=["--no-sandbox", "--disable-dev-shm-usage"]
+            )
+            context = await browser.new_context(
+                viewport=DESKTOP_VIEWPORT,
+                user_agent=USER_AGENT,
+                ignore_https_errors=False,
+            )
+            try:
+                total = len(safe_urls)
+                for index, url in enumerate(safe_urls):
+                    emit(
+                        {
+                            "type": "progress",
+                            "done": index,
+                            "total": total,
+                            "url": url,
+                            "message": f"Auditing {url}",
+                        }
+                    )
+                    page_ctx = await _build_page(
+                        context, client, url, output_dir, index
+                    )
+                    site.pages.append(page_ctx)
+                    yield_frame = {
+                        "type": "page_loaded",
+                        "url": page_ctx.url,
+                        "status": page_ctx.status,
+                        "done": index + 1,
+                        "total": total,
+                    }
+                    emit(yield_frame)
+            finally:
+                await context.close()
+                await browser.close()
+    return site

devplacepy/services/jobs/seo/progress.py

@@ -0,0 +1,43 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+import asyncio
+
+MAX_BUFFER = 2000
+
+
+class ProgressHub:
+    def __init__(self) -> None:
+        self._subscribers: dict[str, set[asyncio.Queue]] = {}
+        self._buffers: dict[str, list[dict]] = {}
+
+    def subscribe(self, uid: str) -> asyncio.Queue:
+        queue: asyncio.Queue = asyncio.Queue()
+        self._subscribers.setdefault(uid, set()).add(queue)
+        return queue
+
+    def unsubscribe(self, uid: str, queue: asyncio.Queue) -> None:
+        listeners = self._subscribers.get(uid)
+        if not listeners:
+            return
+        listeners.discard(queue)
+        if not listeners:
+            self._subscribers.pop(uid, None)
+
+    def publish(self, uid: str, frame: dict) -> None:
+        buffer = self._buffers.setdefault(uid, [])
+        buffer.append(frame)
+        if len(buffer) > MAX_BUFFER:
+            del buffer[: len(buffer) - MAX_BUFFER]
+        for queue in self._subscribers.get(uid, set()):
+            queue.put_nowait(frame)
+
+    def snapshot(self, uid: str) -> list[dict]:
+        return list(self._buffers.get(uid, []))
+
+    def clear(self, uid: str) -> None:
+        self._buffers.pop(uid, None)
+
+
+hub = ProgressHub()

devplacepy/services/jobs/seo/service.py

@@ -0,0 +1,155 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import shutil
+import sys
+from pathlib import Path
+
+from devplacepy.config import BASE_DIR, SEO_REPORTS_DIR
+from devplacepy.services.jobs.base import JobService
+from .progress import hub
+
+logger = logging.getLogger(__name__)
+
+WORKER_MODULE = "devplacepy.services.jobs.seo.worker"
+STREAM_LIMIT = 16 * 1024 * 1024
+
+
+class SeoService(JobService):
+    kind = "seo"
+    title = "SEO Diagnostics"
+    description = (
+        "Audits any URL or sitemap with a broad battery of technical, on-page, structured-data, "
+        "performance, accessibility and AI-readiness checks using a headless browser, streaming "
+        "live progress over a websocket and storing a categorised report."
+    )
+
+    def __init__(self):
+        super().__init__(name="seo", interval_seconds=2)
+
+    def report_dir(self, uid: str) -> Path:
+        return SEO_REPORTS_DIR / uid
+
+    async def process(self, job: dict) -> dict:
+        from devplacepy.services.audit import record as audit
+
+        uid = job["uid"]
+        payload = job.get("payload", {})
+        target = payload.get("url", "")
+        output_dir = self.report_dir(uid)
+        output_dir.mkdir(parents=True, exist_ok=True)
+        payload_path = output_dir / "payload.json"
+        payload_path.write_text(json.dumps(payload), encoding="utf-8")
+
+        actor_kind = "user" if job.get("owner_kind") == "user" else (
+            job.get("owner_kind") or "system"
+        )
+        actor_uid = job.get("owner_id") if job.get("owner_kind") == "user" else None
+
+        try:
+            summary = await self._run_worker(uid, payload_path, output_dir)
+        except Exception as exc:
+            hub.publish(uid, {"type": "failed", "message": str(exc)[:300]})
+            audit.record_system(
+                "seo.run.failed",
+                actor_kind=actor_kind,
+                actor_uid=actor_uid,
+                result="failure",
+                summary=f"SEO diagnostics for {target} failed",
+                metadata={"target": target, "error": str(exc)[:200]},
+                links=[audit.job(uid)],
+            )
+            raise
+
+        report = self._load_report(output_dir)
+        hub.publish(
+            uid,
+            {
+                "type": "done",
+                "score": summary.get("score"),
+                "grade": summary.get("grade"),
+                "page_count": summary.get("page_count"),
+                "report_url": f"/tools/seo/{uid}/report",
+                "report": report,
+            },
+        )
+        hub.clear(uid)
+        audit.record_system(
+            "seo.run.complete",
+            actor_kind=actor_kind,
+            actor_uid=actor_uid,
+            summary=f"SEO diagnostics for {target} scored {summary.get('score')}",
+            metadata={
+                "target": target,
+                "score": summary.get("score"),
+                "grade": summary.get("grade"),
+                "page_count": summary.get("page_count"),
+            },
+            links=[audit.job(uid)],
+        )
+        return {
+            "target": target,
+            "mode": payload.get("mode", "url"),
+            "score": summary.get("score", 0),
+            "grade": summary.get("grade", ""),
+            "page_count": summary.get("page_count", 0),
+            "counts": summary.get("counts", {}),
+            "categories": summary.get("categories", {}),
+            "report": report,
+            "report_url": f"/tools/seo/{uid}/report",
+            "bytes_in": 0,
+            "bytes_out": len(json.dumps(report)) if report else 0,
+            "item_count": summary.get("page_count", 0),
+        }
+
+    async def _run_worker(self, uid: str, payload_path: Path, output_dir: Path) -> dict:
+        proc = await asyncio.create_subprocess_exec(
+            sys.executable,
+            "-m",
+            WORKER_MODULE,
+            str(payload_path),
+            str(output_dir),
+            cwd=str(BASE_DIR),
+            stdout=asyncio.subprocess.PIPE,
+            stderr=asyncio.subprocess.PIPE,
+            limit=STREAM_LIMIT,
+        )
+        summary: dict = {}
+        worker_error = ""
+        while True:
+            line = await proc.stdout.readline()
+            if not line:
+                break
+            try:
+                frame = json.loads(line.decode("utf-8", "replace"))
+            except (ValueError, TypeError):
+                continue
+            hub.publish(uid, frame)
+            if frame.get("type") == "report_ready":
+                summary = frame
+            elif frame.get("type") == "error":
+                worker_error = frame.get("message", "worker error")
+        err = (await proc.stderr.read()).decode("utf-8", "replace")
+        await proc.wait()
+        if proc.returncode != 0 or not summary:
+            raise RuntimeError(
+                worker_error or err[:500] or f"seo worker exited {proc.returncode}"
+            )
+        return summary
+
+    def _load_report(self, output_dir: Path) -> dict:
+        report_path = output_dir / "report.json"
+        if not report_path.is_file():
+            return {}
+        try:
+            return json.loads(report_path.read_text(encoding="utf-8"))
+        except (ValueError, OSError):
+            return {}
+
+    def cleanup(self, job: dict) -> None:
+        hub.clear(job["uid"])
+        shutil.rmtree(self.report_dir(job["uid"]), ignore_errors=True)

devplacepy/services/jobs/seo/worker.py

@@ -0,0 +1,132 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+import asyncio
+import json
+import sys
+from datetime import datetime, timezone
+from pathlib import Path
+
+from .checks.registry import compute_score, run_page_checks, run_site_checks
+from .crawler import crawl_target
+
+
+def _emit(frame: dict) -> None:
+    sys.stdout.write(json.dumps(frame, ensure_ascii=False) + "\n")
+    sys.stdout.flush()
+
+
+def _trim_site(site) -> dict:
+    robots = site.robots or {}
+    sitemap = site.sitemap or {}
+    return {
+        "robots": {
+            "status": robots.get("status"),
+            "disallows": robots.get("disallows", [])[:50],
+            "sitemap_urls": robots.get("sitemap_urls", [])[:10],
+            "blocks_target": robots.get("blocks_target", False),
+        },
+        "sitemap": {
+            "status": sitemap.get("status"),
+            "valid_xml": sitemap.get("valid_xml", False),
+            "url_count": sitemap.get("url_count", 0),
+            "lastmod_count": sitemap.get("lastmod_count", 0),
+        },
+        "llms_txt": site.llms_txt or {},
+    }
+
+
+async def _run(payload: dict, output_dir: Path) -> dict:
+    site = await crawl_target(payload, _emit, output_dir)
+
+    all_checks = []
+    pages_summary = []
+    _emit({"type": "stage", "stage": "checks", "message": "Running SEO checks"})
+    for page in site.pages:
+        page_checks = run_page_checks(page, site)
+        all_checks.extend(page_checks)
+        page_score = compute_score(page_checks)
+        page_dicts = [c.to_dict() for c in page_checks]
+        pages_summary.append(
+            {
+                "url": page.url,
+                "requested_url": page.requested_url,
+                "status": page.status,
+                "ok": page.ok,
+                "error": page.error,
+                "screenshot": page.screenshot,
+                "score": page_score["score"],
+                "grade": page_score["grade"],
+                "counts": page_score["counts"],
+            }
+        )
+        _emit(
+            {
+                "type": "page",
+                "url": page.url,
+                "status": page.status,
+                "score": page_score["score"],
+                "grade": page_score["grade"],
+                "counts": page_score["counts"],
+                "checks": page_dicts,
+            }
+        )
+
+    site_checks = run_site_checks(site)
+    all_checks.extend(site_checks)
+    _emit(
+        {
+            "type": "site_checks",
+            "checks": [c.to_dict() for c in site_checks],
+        }
+    )
+
+    overall = compute_score(all_checks)
+    report = {
+        "target": site.target_url,
+        "mode": site.mode,
+        "generated_at": datetime.now(timezone.utc).isoformat(),
+        "page_count": len(site.pages),
+        "score": overall["score"],
+        "grade": overall["grade"],
+        "counts": overall["counts"],
+        "categories": overall["categories"],
+        "pages": pages_summary,
+        "checks": [c.to_dict() for c in all_checks],
+        "site": _trim_site(site),
+    }
+    if output_dir is not None:
+        (output_dir / "report.json").write_text(
+            json.dumps(report, ensure_ascii=False), encoding="utf-8"
+        )
+    _emit(
+        {
+            "type": "report_ready",
+            "score": report["score"],
+            "grade": report["grade"],
+            "counts": report["counts"],
+            "categories": report["categories"],
+            "page_count": report["page_count"],
+        }
+    )
+    return report
+
+
+def main(argv: list) -> int:
+    if len(argv) != 3:
+        sys.stderr.write("usage: seo.worker <payload_json> <output_dir>\n")
+        return 2
+    payload = json.loads(Path(argv[1]).read_text(encoding="utf-8"))
+    output_dir = Path(argv[2])
+    output_dir.mkdir(parents=True, exist_ok=True)
+    try:
+        asyncio.run(_run(payload, output_dir))
+    except Exception as exc:  # noqa: BLE001 - surface as a worker error line
+        _emit({"type": "error", "message": str(exc)[:500]})
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main(sys.argv))

devplacepy/static/css/base.css

@@ -36,6 +36,8 @@
     margin: 1rem;
     max-height: 90vh;
     overflow-y: auto;
+    border-radius: var(--radius-xl);
+    box-shadow: var(--shadow-lg);
 }
 .modal-card-wide {
     max-width: 700px;
@@ -361,7 +363,7 @@ input, textarea, select {
     color: var(--text-primary);
     background: var(--bg-input);
     border: 1px solid var(--border);
-    border-radius: var(--radius);
+    border-radius: var(--radius-input);
     padding: 0.625rem 0.875rem;
     outline: none;
     width: 100%;
@@ -411,13 +413,15 @@ img {
 }
 
 .btn-primary {
-    background: var(--accent);
+    background: var(--accent-gradient);
     color: var(--white);
+    box-shadow: 0 2px 12px rgba(255, 95, 70, 0.3);
 }
 
 .btn-primary:hover {
-    background: var(--accent-hover);
     color: var(--white);
+    box-shadow: var(--glow-accent);
+    transform: translateY(-1px);
 }
 
 .btn-secondary {
@@ -533,6 +537,9 @@ img {
 }
 .topnav-link:hover { color: var(--text-primary); background: var(--bg-card); }
 .topnav-link.active { color: var(--accent); background: var(--accent-light); }
+.topnav-link.topnav-link-feature { color: var(--accent); font-weight: 600; background: var(--accent-light); }
+.topnav-link.topnav-link-feature:hover { color: var(--accent); background: var(--accent-light); filter: brightness(1.2); }
+.topnav-link.topnav-link-feature.active { color: var(--white); background: var(--accent-gradient); box-shadow: var(--glow-accent); }
 .topnav-right { margin-left: auto; display: flex; align-items: center; gap: 1rem; flex-shrink: 0; }
 .topnav-icon { position: relative; padding: 0.375rem; color: var(--text-secondary); font-size: 1.25rem; background: none; border: none; cursor: pointer; font-family: inherit; line-height: 1; }
 .topnav-icon:hover { color: var(--text-primary); }
@@ -568,6 +575,26 @@ img {
 }
 .topnav-user-dropdown:hover .dropdown-menu,
 .topnav-user-dropdown.open .dropdown-menu { display: block; }
+
+.topnav-tools-dropdown { position: relative; }
+.topnav-tools-toggle { display: inline-flex; align-items: center; gap: 0.25rem; background: none; border: none; cursor: pointer; font-family: inherit; }
+.topnav-tools-dropdown.active .topnav-tools-toggle { color: var(--accent); background: var(--accent-light); }
+.topnav-caret { font-size: 0.625rem; }
+.topnav-tools-dropdown .dropdown-menu {
+    display: none;
+    position: absolute;
+    top: 100%;
+    left: 0;
+    min-width: 180px;
+    background: var(--bg-secondary);
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15);
+    padding: 0.25rem 0;
+    z-index: 1001;
+}
+.topnav-tools-dropdown:hover .dropdown-menu,
+.topnav-tools-dropdown.open .dropdown-menu { display: block; }
 .dropdown-item {
     display: block;
     padding: 0.5rem 0.875rem;
@@ -779,6 +806,7 @@ button.comment-form-submit:hover {
     background: var(--bg-card);
     border: 1px solid var(--border);
     border-radius: var(--radius-lg);
+    box-shadow: var(--shadow-sm);
     padding: 1rem;
 }
 

devplacepy/static/css/docs-chat.css

@@ -0,0 +1,318 @@
+/* retoor <retoor@molodetz.nl> */
+
+dp-docs-chat {
+    display: flex;
+    flex-direction: column;
+    gap: 0.75rem;
+    min-height: 60vh;
+}
+
+.docs-chat-log {
+    display: flex;
+    flex-direction: column;
+    gap: 0.875rem;
+    overflow-y: auto;
+    padding: 0.25rem;
+    flex: 1;
+    max-height: 65vh;
+}
+
+.docs-chat-msg {
+    display: flex;
+}
+
+.docs-chat-msg.user {
+    justify-content: flex-end;
+}
+
+.docs-chat-msg.agent,
+.docs-chat-msg.error {
+    justify-content: flex-start;
+}
+
+.docs-chat-bubble {
+    max-width: 85%;
+    padding: 0.75rem 1rem;
+    border-radius: var(--radius-lg);
+    border: 1px solid var(--border);
+    font-size: 0.9375rem;
+    line-height: 1.6;
+    color: var(--text-primary);
+    overflow-wrap: anywhere;
+}
+
+.docs-chat-msg.user .docs-chat-bubble {
+    background: var(--accent-light);
+    border-color: var(--accent);
+    border-bottom-right-radius: var(--radius);
+    white-space: pre-wrap;
+}
+
+.docs-chat-msg.agent .docs-chat-bubble {
+    background: var(--bg-card);
+    border-bottom-left-radius: var(--radius);
+}
+
+.docs-chat-msg.error .docs-chat-bubble {
+    background: var(--bg-card);
+    border-color: var(--danger);
+    color: var(--danger);
+}
+
+.docs-chat-bubble > :first-child {
+    margin-top: 0;
+}
+
+.docs-chat-bubble > :last-child {
+    margin-bottom: 0;
+}
+
+.docs-chat-bubble p {
+    margin: 0 0 0.5rem;
+}
+
+.docs-chat-bubble h1,
+.docs-chat-bubble h2,
+.docs-chat-bubble h3,
+.docs-chat-bubble h4 {
+    margin: 0.75rem 0 0.5rem;
+    font-size: 1rem;
+    color: var(--text-primary);
+}
+
+.docs-chat-bubble ul,
+.docs-chat-bubble ol {
+    margin: 0.5rem 0;
+    padding-left: 1.5rem;
+}
+
+.docs-chat-bubble li {
+    margin-bottom: 0.25rem;
+}
+
+.docs-chat-bubble a {
+    color: var(--accent);
+    text-decoration: none;
+}
+
+.docs-chat-bubble a:hover {
+    text-decoration: underline;
+}
+
+.docs-chat-bubble .md-code {
+    background: var(--bg-secondary);
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    padding: 0.05rem 0.3rem;
+    font-family: var(--font-mono, monospace);
+    font-size: 0.85em;
+}
+
+.docs-chat-bubble .md-codewrap {
+    position: relative;
+    margin: 0.5rem 0;
+}
+
+.docs-chat-bubble .md-pre {
+    background: var(--bg-secondary);
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    padding: 0.75rem 0.875rem;
+    overflow-x: auto;
+    font-size: 0.85rem;
+    line-height: 1.5;
+}
+
+.docs-chat-bubble .md-pre code {
+    background: none;
+    border: none;
+    padding: 0;
+    font-family: var(--font-mono, monospace);
+}
+
+.docs-chat-bubble .md-copy {
+    position: absolute;
+    top: 0.375rem;
+    right: 0.375rem;
+    opacity: 0;
+    padding: 0.125rem 0.5rem;
+    font-size: 0.75rem;
+    color: var(--text-secondary);
+    background: var(--bg-card);
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    cursor: pointer;
+    transition: opacity 0.15s ease;
+}
+
+.docs-chat-bubble .md-codewrap:hover .md-copy,
+.docs-chat-bubble .md-copy:focus-visible {
+    opacity: 1;
+}
+
+.docs-chat-bubble .md-copy:hover {
+    color: var(--text-primary);
+    border-color: var(--accent);
+}
+
+.docs-chat-bubble .md-table {
+    width: 100%;
+    border-collapse: collapse;
+    font-size: 0.85rem;
+    margin: 0.5rem 0;
+}
+
+.docs-chat-bubble .md-table th,
+.docs-chat-bubble .md-table td {
+    border: 1px solid var(--border);
+    padding: 0.375rem 0.5rem;
+    text-align: left;
+}
+
+.docs-chat-bubble .md-table th {
+    background: var(--bg-card-hover);
+    color: var(--text-primary);
+}
+
+.docs-chat-bubble .md-img {
+    max-width: 100%;
+    border-radius: var(--radius);
+}
+
+.docs-chat-step {
+    display: flex;
+    align-items: center;
+    gap: 0.5rem;
+    font-size: 0.8125rem;
+    color: var(--text-muted);
+    padding: 0.1rem 0.25rem;
+}
+
+.docs-chat-step-icon {
+    flex: 0 0 auto;
+    width: 0.85rem;
+    height: 0.85rem;
+    display: inline-flex;
+    align-items: center;
+    justify-content: center;
+    box-sizing: border-box;
+    font-size: 0.8rem;
+    line-height: 1;
+}
+
+.docs-chat-step.pending .docs-chat-step-icon {
+    border: 2px solid var(--border-light);
+    border-top-color: var(--accent);
+    border-radius: 50%;
+    animation: docs-chat-spin 0.7s linear infinite;
+}
+
+.docs-chat-step.done {
+    color: var(--text-secondary);
+}
+
+.docs-chat-step.done .docs-chat-step-icon::before {
+    content: "\2713";
+    color: var(--success);
+}
+
+.docs-chat-step.error .docs-chat-step-icon::before {
+    content: "\2717";
+    color: var(--danger);
+}
+
+.docs-chat-step-label {
+    overflow: hidden;
+    text-overflow: ellipsis;
+    white-space: nowrap;
+}
+
+@keyframes docs-chat-spin {
+    to {
+        transform: rotate(360deg);
+    }
+}
+
+.docs-chat-note {
+    font-size: 0.78rem;
+    font-style: italic;
+    color: var(--text-muted);
+    padding: 0.1rem 0.25rem;
+}
+
+.docs-chat-note.new {
+    color: var(--accent);
+    font-style: normal;
+}
+
+.docs-chat-status {
+    font-size: 0.8125rem;
+    color: var(--text-muted);
+    padding-left: 0.25rem;
+    min-height: 1rem;
+}
+
+.docs-chat-status:not(:empty)::after {
+    content: " …";
+    animation: docs-chat-blink 1.2s steps(1) infinite;
+}
+
+@keyframes docs-chat-blink {
+    50% {
+        opacity: 0.3;
+    }
+}
+
+.docs-chat-input {
+    display: flex;
+    gap: 0.5rem;
+    align-items: flex-end;
+    border-top: 1px solid var(--border);
+    padding-top: 0.75rem;
+}
+
+.docs-chat-field {
+    flex: 1;
+    resize: none;
+    background: var(--bg-input);
+    color: var(--text-primary);
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    padding: 0.625rem 0.75rem;
+    font: inherit;
+    font-size: 0.9375rem;
+    line-height: 1.5;
+    max-height: 160px;
+}
+
+.docs-chat-field:focus {
+    outline: none;
+    border-color: var(--accent);
+}
+
+.docs-chat-send {
+    background: var(--accent);
+    color: #fff;
+    border: none;
+    border-radius: var(--radius);
+    padding: 0.625rem 1.25rem;
+    font: inherit;
+    font-weight: 600;
+    cursor: pointer;
+    white-space: nowrap;
+}
+
+.docs-chat-send:hover {
+    filter: brightness(1.08);
+}
+
+.docs-chat-send:disabled {
+    opacity: 0.6;
+    cursor: default;
+}
+
+@media (max-width: 640px) {
+    .docs-chat-bubble {
+        max-width: 100%;
+    }
+}

devplacepy/static/css/feed.css

@@ -35,8 +35,9 @@
 }
 
 .feed-nav-btn.active {
-    background: var(--accent);
+    background: var(--accent-gradient);
     color: var(--white);
+    box-shadow: var(--glow-accent);
 }
 
 .feed-nav-actions {
@@ -67,12 +68,13 @@
     background: var(--bg-card);
     border: 1px solid var(--border);
     border-radius: var(--radius-lg);
+    box-shadow: var(--shadow-sm);
     padding: var(--space-xl);
 }
 
 .post-card:hover {
     border-color: var(--border-light);
-    box-shadow: var(--shadow-sm);
+    box-shadow: var(--shadow);
 }
 
 .post-header {
@@ -124,15 +126,16 @@
 }
 
 .post-title {
-    font-size: 1.125rem;
+    font-size: 1.25rem;
     font-weight: 700;
     margin-bottom: 0.5rem;
     color: var(--text-primary);
     line-height: 1.3;
+    letter-spacing: -0.01em;
 }
 
 .post-content {
-    font-size: 0.875rem;
+    font-size: 1rem;
     color: var(--text-secondary);
     line-height: 1.65;
     margin-bottom: 1rem;
@@ -225,6 +228,7 @@
     background: var(--bg-card);
     border: 1px solid var(--border);
     border-radius: var(--radius-lg);
+    box-shadow: var(--shadow-sm);
     padding: 1.25rem;
 }
 
@@ -234,12 +238,24 @@
     color: var(--accent);
     text-transform: uppercase;
     letter-spacing: 0.1em;
-    margin-bottom: 0.5rem;
+    margin-bottom: 0.75rem;
+}
+
+.daily-topic-image {
+    display: block;
+    width: 100%;
+    height: 140px;
+    object-fit: cover;
+    border-radius: var(--radius);
+    margin-bottom: 0.75rem;
+    border: 1px solid var(--border);
 }
 
 .daily-topic-card h4 {
-    font-size: 0.875rem;
-    margin-bottom: 0.375rem;
+    font-size: 1.0625rem;
+    font-weight: 600;
+    color: var(--text-primary);
+    margin-bottom: 0.5rem;
     line-height: 1.4;
 }
 
@@ -264,6 +280,7 @@
     background: var(--bg-card);
     border: 1px solid var(--border);
     border-radius: var(--radius-lg);
+    box-shadow: var(--shadow-sm);
     padding: 1.25rem;
 }
 
@@ -315,25 +332,26 @@
     position: fixed;
     bottom: 2rem;
     right: 2rem;
-    width: 56px;
-    height: 56px;
+    width: 64px;
+    height: 64px;
     border-radius: 50%;
-    background: var(--danger);
+    background: var(--accent-gradient);
     color: var(--white);
-    font-size: 1.5rem;
+    font-size: 2rem;
+    line-height: 1;
     display: flex;
     align-items: center;
     justify-content: center;
-    box-shadow: 0 4px 16px rgba(229, 57, 53, 0.4);
+    box-shadow: var(--glow-accent);
     z-index: 900;
     border: none;
     cursor: pointer;
 }
 
 .feed-fab:hover {
-    background: var(--danger);
-    transform: scale(1.05);
-    box-shadow: 0 6px 20px rgba(229, 57, 53, 0.5);
+    color: var(--white);
+    transform: scale(1.06);
+    box-shadow: 0 10px 32px rgba(255, 95, 70, 0.6);
 }
 
 @media (max-width: 1024px) {
@@ -453,6 +471,7 @@
     background: var(--bg-card);
     border: 1px solid var(--border);
     border-radius: var(--radius-lg);
+    box-shadow: var(--shadow-sm);
     padding: 1.25rem;
 }
 

devplacepy/static/css/post.css

@@ -40,13 +40,15 @@
 }
 
 .post-detail-title {
-    font-size: 1.5rem;
+    font-size: 2rem;
     font-weight: 700;
+    letter-spacing: -0.02em;
+    line-height: 1.2;
     margin-bottom: 0.75rem;
 }
 
 .post-detail-content {
-    font-size: 0.9375rem;
+    font-size: 1rem;
     color: var(--text-secondary);
     line-height: 1.7;
     margin-top: 0.625rem;

devplacepy/static/css/projects.css

@@ -23,8 +23,9 @@
 }
 
 .projects-tab.active {
-    background: var(--accent);
+    background: var(--accent-gradient);
     color: var(--white);
+    box-shadow: var(--glow-accent);
 }
 
 .projects-header {
@@ -56,27 +57,46 @@
 }
 
 .project-card {
+    position: relative;
+    overflow: hidden;
     background: var(--bg-card);
     border: 1px solid var(--border);
     border-radius: var(--radius-lg);
-    padding: 1.25rem;
+    box-shadow: var(--shadow-sm);
+    padding: 1.5rem 1.25rem 1.25rem;
     cursor: pointer;
+    transition: transform 0.15s ease, box-shadow 0.15s ease, border-color 0.15s ease;
+}
+
+.project-card::before {
+    content: "";
+    position: absolute;
+    top: 0;
+    left: 0;
+    right: 0;
+    height: 4px;
+    background: var(--accent-gradient);
 }
 
 .project-card:hover {
     border-color: var(--border-light);
+    box-shadow: var(--shadow);
+    transform: translateY(-3px);
 }
 
 .project-card-header {
     display: flex;
     align-items: flex-start;
     justify-content: space-between;
+    gap: 0.75rem;
     margin-bottom: 0.75rem;
 }
 
 .project-card-title {
-    font-size: 1.125rem;
+    font-size: 1.25rem;
     font-weight: 700;
+    letter-spacing: -0.01em;
+    line-height: 1.3;
 }
 
 .project-card-star {
@@ -123,28 +143,33 @@
 
 .platform-tag {
     display: inline-flex;
-    padding: 0.125rem 0.5rem;
-    border-radius: 4px;
+    padding: 0.1875rem 0.5rem;
+    border-radius: 6px;
     font-size: 0.6875rem;
     font-weight: 600;
-    background: var(--border);
-    color: var(--text-muted);
+    background: var(--overlay-light);
+    border: 1px solid var(--border);
+    color: var(--text-secondary);
 }
 
 .project-status {
     display: inline-flex;
     align-items: center;
     gap: 0.25rem;
-    font-size: 0.75rem;
-    font-weight: 600;
+    padding: 0.125rem 0.5rem;
+    border-radius: 999px;
+    font-size: 0.6875rem;
+    font-weight: 700;
 }
 
 .project-status.released {
     color: var(--success);
+    background: rgba(76, 175, 80, 0.14);
 }
 
 .project-status.dev {
     color: var(--warning);
+    background: rgba(255, 152, 0, 0.14);
 }
 
 @media (max-width: 1024px) {

devplacepy/static/css/seo-tool.css

@@ -0,0 +1,276 @@
+/* retoor <retoor@molodetz.nl> */
+
+.tools-page,
+.seo-tool {
+    max-width: 920px;
+    margin: 0 auto;
+    padding: 1.5rem 1rem 3rem;
+}
+
+.tools-header h1,
+.seo-header h1 {
+    font-size: 1.75rem;
+    margin: 0 0 0.5rem;
+}
+
+.tools-header p,
+.seo-header p {
+    color: var(--text-secondary);
+    margin: 0 0 1.5rem;
+    max-width: 60ch;
+}
+
+.tools-grid {
+    display: grid;
+    grid-template-columns: repeat(auto-fill, minmax(240px, 1fr));
+    gap: 1rem;
+}
+
+.tool-card {
+    display: flex;
+    flex-direction: column;
+    gap: 0.5rem;
+    padding: 1.25rem;
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    background: var(--bg-card);
+}
+
+.tool-card:hover {
+    border-color: var(--accent);
+}
+
+.tool-card-icon { font-size: 1.5rem; }
+.tool-card-name { font-weight: 600; color: var(--text-primary); }
+.tool-card-desc { font-size: 0.8125rem; color: var(--text-secondary); }
+
+.seo-form {
+    margin-bottom: 1.5rem;
+}
+
+.seo-form-row {
+    display: flex;
+    flex-wrap: wrap;
+    gap: 0.5rem;
+    align-items: center;
+}
+
+.seo-input,
+.seo-select {
+    padding: 0.625rem 0.75rem;
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    background: var(--bg-secondary);
+    color: var(--text-primary);
+    font-family: inherit;
+    font-size: 0.9375rem;
+}
+
+.seo-input[type="url"] { flex: 1 1 280px; }
+.seo-input-num { width: 5rem; }
+
+.seo-pages {
+    display: inline-flex;
+    align-items: center;
+    gap: 0.375rem;
+    font-size: 0.8125rem;
+    color: var(--text-secondary);
+}
+
+.seo-run-btn { flex-shrink: 0; }
+
+.seo-form-error {
+    margin: 0.625rem 0 0;
+    color: var(--danger, #e5484d);
+    font-size: 0.875rem;
+}
+
+.seo-live {
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    background: var(--bg-card);
+    padding: 1rem;
+    margin-bottom: 1.5rem;
+}
+
+.seo-live-head {
+    display: flex;
+    justify-content: space-between;
+    gap: 1rem;
+    margin-bottom: 0.625rem;
+    font-size: 0.875rem;
+}
+
+.seo-live-status { font-weight: 600; color: var(--text-primary); }
+.seo-live-count { color: var(--text-muted); }
+
+.seo-progress {
+    height: 6px;
+    border-radius: 3px;
+    background: var(--bg-secondary);
+    overflow: hidden;
+}
+
+.seo-progress-bar {
+    height: 100%;
+    width: 0;
+    background: var(--accent);
+    transition: width 0.3s ease;
+}
+
+.seo-log {
+    list-style: none;
+    margin: 0.75rem 0 0;
+    padding: 0;
+    max-height: 200px;
+    overflow-y: auto;
+    font-family: var(--font-mono, monospace);
+    font-size: 0.75rem;
+    color: var(--text-secondary);
+}
+
+.seo-log li {
+    padding: 0.125rem 0;
+    border-bottom: 1px solid var(--border);
+}
+
+.seo-score-card {
+    display: flex;
+    gap: 1.25rem;
+    align-items: center;
+    padding: 1.25rem;
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    background: var(--bg-card);
+    margin-bottom: 1.25rem;
+}
+
+.seo-gauge {
+    flex-shrink: 0;
+    width: 92px;
+    height: 92px;
+    border-radius: 50%;
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+    justify-content: center;
+    background: var(--bg-secondary);
+    border: 3px solid var(--accent);
+}
+
+.seo-gauge-score { font-size: 1.75rem; font-weight: 700; color: var(--text-primary); line-height: 1; }
+.seo-gauge-grade { font-size: 0.875rem; font-weight: 600; color: var(--text-muted); }
+
+.seo-grade-a { border-color: #30a46c; }
+.seo-grade-b { border-color: #5dbb7a; }
+.seo-grade-c { border-color: #e2a336; }
+.seo-grade-d { border-color: #e08c3b; }
+.seo-grade-f { border-color: #e5484d; }
+
+.seo-score-meta h2 { margin: 0 0 0.5rem; font-size: 1.0625rem; word-break: break-all; }
+.seo-report-sub { color: var(--text-muted); font-size: 0.8125rem; margin: 0.375rem 0 0; }
+
+.seo-counts { display: flex; flex-wrap: wrap; gap: 0.375rem; }
+
+.seo-chip {
+    font-size: 0.75rem;
+    font-weight: 600;
+    padding: 0.125rem 0.5rem;
+    border-radius: 999px;
+    border: 1px solid var(--border);
+}
+
+.seo-chip-pass { color: #30a46c; }
+.seo-chip-warn { color: #e2a336; }
+.seo-chip-fail { color: #e5484d; }
+.seo-chip-info { color: var(--text-muted); }
+
+.seo-report-link {
+    display: inline-block;
+    margin-top: 0.5rem;
+    font-size: 0.8125rem;
+    color: var(--accent);
+}
+
+.seo-categories {
+    display: flex;
+    flex-wrap: wrap;
+    gap: 0.5rem;
+    margin-bottom: 1.25rem;
+}
+
+.seo-cat-pill {
+    display: inline-flex;
+    align-items: center;
+    gap: 0.5rem;
+    padding: 0.375rem 0.75rem;
+    border: 1px solid var(--border);
+    border-radius: 999px;
+    background: var(--bg-card);
+    font-size: 0.8125rem;
+}
+
+.seo-cat-name { color: var(--text-secondary); }
+.seo-cat-score { font-weight: 700; color: var(--text-primary); }
+
+.seo-check-group { margin-bottom: 1.5rem; }
+.seo-check-group h2 { font-size: 1rem; margin: 0 0 0.625rem; text-transform: capitalize; }
+
+.seo-check {
+    display: flex;
+    gap: 0.75rem;
+    align-items: flex-start;
+    padding: 0.625rem 0.75rem;
+    border: 1px solid var(--border);
+    border-left-width: 3px;
+    border-radius: var(--radius);
+    margin-bottom: 0.5rem;
+    background: var(--bg-card);
+}
+
+.seo-check-pass { border-left-color: #30a46c; }
+.seo-check-warn { border-left-color: #e2a336; }
+.seo-check-fail { border-left-color: #e5484d; }
+.seo-check-info { border-left-color: var(--text-muted); }
+.seo-check-skip { border-left-color: var(--border); }
+
+.seo-check-status {
+    flex-shrink: 0;
+    text-transform: uppercase;
+    font-size: 0.625rem;
+    font-weight: 700;
+    letter-spacing: 0.04em;
+    padding: 0.1875rem 0.375rem;
+    border-radius: var(--radius);
+    background: var(--bg-secondary);
+    color: var(--text-secondary);
+}
+
+.seo-check-body { display: flex; flex-direction: column; gap: 0.1875rem; flex: 1; }
+.seo-check-title { font-weight: 600; color: var(--text-primary); font-size: 0.875rem; }
+.seo-check-url { font-weight: 400; color: var(--text-muted); font-size: 0.75rem; word-break: break-all; }
+.seo-check-value { font-size: 0.8125rem; color: var(--text-secondary); }
+.seo-check-rec { font-size: 0.8125rem; color: var(--accent); }
+
+.seo-check-severity {
+    flex-shrink: 0;
+    font-size: 0.6875rem;
+    color: var(--text-muted);
+    text-transform: uppercase;
+}
+
+.seo-sev-critical { color: #e5484d; }
+.seo-sev-high { color: #e08c3b; }
+
+.seo-empty {
+    padding: 1.5rem;
+    border: 1px dashed var(--border);
+    border-radius: var(--radius);
+    color: var(--text-secondary);
+}
+
+.seo-report-target { color: var(--text-muted); word-break: break-all; }
+
+@media (max-width: 600px) {
+    .seo-score-card { flex-direction: column; align-items: flex-start; }
+}

devplacepy/static/css/sidebar.css

@@ -6,6 +6,7 @@
     background: var(--bg-card);
     border: 1px solid var(--border);
     border-radius: var(--radius-lg);
+    box-shadow: var(--shadow-sm);
     padding: 1rem;
 }
 
@@ -75,9 +76,38 @@
 
 .sidebar-link.active,
 .sidebar-link.active:hover {
-    background: var(--accent-light);
-    color: var(--accent);
+    background: var(--accent-gradient);
+    color: var(--white);
     font-weight: 600;
+    box-shadow: var(--glow-accent);
+}
+
+.sidebar-more {
+    display: flex;
+    flex-direction: column;
+    gap: 0.125rem;
+}
+
+.sidebar-more-toggle {
+    list-style: none;
+    cursor: pointer;
+    user-select: none;
+}
+
+.sidebar-more-toggle::-webkit-details-marker {
+    display: none;
+}
+
+.sidebar-more-chevron {
+    margin-left: auto;
+    font-size: 1rem;
+    line-height: 1;
+    color: var(--text-muted);
+    transition: transform 0.15s ease;
+}
+
+.sidebar-more[open] > .sidebar-more-toggle .sidebar-more-chevron {
+    transform: rotate(90deg);
 }
 
 .sidebar-dot {

devplacepy/static/css/variables.css

@@ -1,49 +1,53 @@
 /* retoor <retoor@molodetz.nl> */
 
 :root {
-    --bg-primary: #0f0a1a;
-    --bg-secondary: #1a1028;
-    --bg-card: #221436;
-    --bg-card-hover: #2a1a42;
-    --bg-input: #1a1028;
-    --bg-modal: #1a1028;
+    --bg-primary: #080413;
+    --bg-secondary: #120821;
+    --bg-card: #1a1030;
+    --bg-card-hover: #241640;
+    --bg-input: #140b26;
+    --bg-modal: #1a1030;
 
     --accent: #ff6b35;
-    --accent-hover: #e55a2b;
-    --accent-light: rgba(255, 107, 53, 0.1);
+    --accent-hover: #ff7d4d;
+    --accent-light: rgba(255, 107, 53, 0.12);
+    --accent-2: #ff4f8b;
+    --accent-gradient: linear-gradient(135deg, #ff6b3d, #ff4f6d);
 
-    --text-primary: #f0e8f8;
+    --text-primary: #f4eefb;
     --text-secondary: #b8a8d0;
     --text-muted: #7a6a90;
 
-    --border: #2e2040;
-    --border-light: #3a2a50;
+    --border: rgba(255, 255, 255, 0.08);
+    --border-light: rgba(255, 255, 255, 0.14);
 
     --success: #4caf50;
     --warning: #ff9800;
     --danger: #e53935;
     --info: #42a5f5;
 
-    --radius: 8px;
-    --radius-lg: 12px;
-    --radius-xl: 16px;
+    --radius: 12px;
+    --radius-input: 14px;
+    --radius-lg: 18px;
+    --radius-xl: 24px;
 
-    --shadow: 0 4px 12px rgba(0, 0, 0, 0.3);
-    --shadow-lg: 0 8px 24px rgba(0, 0, 0, 0.4);
-    --shadow-sm: 0 2px 8px rgba(0, 0, 0, 0.2);
-    --shadow-md: 0 4px 12px rgba(0, 0, 0, 0.15);
+    --shadow-sm: 0 2px 8px rgba(0, 0, 0, 0.25);
+    --shadow: 0 4px 20px rgba(0, 0, 0, 0.28);
+    --shadow-md: 0 4px 20px rgba(0, 0, 0, 0.28);
+    --shadow-lg: 0 8px 32px rgba(0, 0, 0, 0.45);
+    --glow-accent: 0 6px 24px rgba(255, 95, 70, 0.45);
 
     --white: #fff;
     --overlay-dark: rgba(0, 0, 0, 0.7);
     --overlay-light: rgba(255, 255, 255, 0.05);
 
     --space-xs: 0.25rem;
-    --space-sm: 0.375rem;
+    --space-sm: 0.5rem;
     --space-base: 0.5rem;
     --space-md: 0.75rem;
     --space-lg: 1rem;
-    --space-xl: 1.25rem;
-    --space-2xl: 1.5rem;
+    --space-xl: 1.5rem;
+    --space-2xl: 2rem;
 
     --font-sans: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
     --font-mono: "SF Mono", Monaco, "Cascadia Code", monospace;
@@ -59,5 +63,5 @@
     --topic-fun: #ffab00;
     --topic-signals: #00bcd4;
 
-    --bg-gradient: linear-gradient(135deg, #0f0a1a 0%, #1a0a2e 50%, #0f0a1a 100%);
+    --bg-gradient: linear-gradient(135deg, #080413 0%, #160a28 50%, #080413 100%);
 }

devplacepy/static/js/ApiDocs.js

@@ -8,7 +8,8 @@ export class ApiDocs {
         this.copy = new CodeCopy();
         const mounts = document.querySelectorAll("[data-api-tester]");
         if (!mounts.length) return;
-        const ctx = window.DEVPLACE_DOCS || { base: location.origin, loggedIn: false, username: "", apiKey: "", isAdmin: false };
+        const ctx = window.DEVPLACE_DOCS || { loggedIn: false, username: "", apiKey: "", isAdmin: false };
+        ctx.base = location.origin;
         this.testers = [...mounts].map((mount) => new ApiTester(mount, ctx));
     }
 }

devplacepy/static/js/MobileNav.js

@@ -6,6 +6,7 @@ export class MobileNav {
         this.initMessagesResponsive();
         this.initMessageThread();
         this.initProfileDropdown();
+        this.initToolsDropdown();
     }
 
     initMobileNav() {
@@ -119,4 +120,31 @@ export class MobileNav {
             }
         });
     }
+
+    initToolsDropdown() {
+        const dropdown = document.querySelector(".topnav-tools-dropdown");
+        if (!dropdown) return;
+        const btn = dropdown.querySelector(".topnav-tools-toggle");
+        if (!btn) return;
+
+        btn.addEventListener("click", (e) => {
+            e.stopPropagation();
+            const open = dropdown.classList.toggle("open");
+            btn.setAttribute("aria-expanded", open ? "true" : "false");
+        });
+
+        document.addEventListener("click", (e) => {
+            if (!dropdown.contains(e.target)) {
+                dropdown.classList.remove("open");
+                btn.setAttribute("aria-expanded", "false");
+            }
+        });
+
+        document.addEventListener("keydown", (e) => {
+            if (e.key === "Escape") {
+                dropdown.classList.remove("open");
+                btn.setAttribute("aria-expanded", "false");
+            }
+        });
+    }
 }

devplacepy/static/js/SeoDiagnostics.js

@@ -0,0 +1,222 @@
+// retoor <retoor@molodetz.nl>
+
+import { Http } from "./Http.js";
+import { SeoProgressSocket } from "./SeoProgressSocket.js";
+
+const STATUS_LABEL = {
+    pass: "pass",
+    warn: "warn",
+    fail: "fail",
+    info: "info",
+    skip: "skip",
+};
+
+export class SeoDiagnostics {
+    constructor() {
+        this.root = document.querySelector("[data-seo-tool]");
+        if (!this.root) return;
+        this.form = this.root.querySelector("[data-seo-form]");
+        this.modeSelect = this.root.querySelector("[data-seo-mode]");
+        this.pagesField = this.root.querySelector("[data-seo-pages]");
+        this.errorBox = this.root.querySelector("[data-seo-error]");
+        this.live = this.root.querySelector("[data-seo-live]");
+        this.statusText = this.root.querySelector("[data-seo-status]");
+        this.countText = this.root.querySelector("[data-seo-count]");
+        this.bar = this.root.querySelector("[data-seo-bar]");
+        this.log = this.root.querySelector("[data-seo-log]");
+        this.report = this.root.querySelector("[data-seo-report]");
+        this.socket = null;
+        this.uid = null;
+        this._bind();
+    }
+
+    _bind() {
+        this.modeSelect.addEventListener("change", () => this._syncMode());
+        this._syncMode();
+        this.form.addEventListener("submit", (event) => {
+            event.preventDefault();
+            this._start();
+        });
+    }
+
+    _syncMode() {
+        const sitemap = this.modeSelect.value === "sitemap";
+        this.pagesField.hidden = !sitemap;
+    }
+
+    async _start() {
+        this._setError("");
+        const url = this.form.url.value.trim();
+        if (!url) return;
+        const params = {
+            url,
+            mode: this.modeSelect.value,
+            max_pages: this.form.max_pages ? this.form.max_pages.value : 10,
+        };
+        this._resetLive();
+        try {
+            const response = await fetch("/tools/seo/run", {
+                method: "POST",
+                headers: {
+                    "X-Requested-With": "fetch",
+                    "Content-Type": "application/x-www-form-urlencoded",
+                },
+                body: new URLSearchParams(params),
+            });
+            const data = await response.json().catch(() => ({}));
+            if (!response.ok) {
+                this._setError((data.error && data.error.message) || "Could not start the audit.");
+                this.live.hidden = true;
+                return;
+            }
+            this.uid = data.uid;
+            this._watch(data.uid);
+        } catch (error) {
+            this._setError("Could not start the audit.");
+            this.live.hidden = true;
+        }
+    }
+
+    _resetLive() {
+        if (this.socket) this.socket.close();
+        this.report.hidden = true;
+        this.report.querySelector("[data-seo-categories]").innerHTML = "";
+        this.report.querySelector("[data-seo-checks]").innerHTML = "";
+        this.live.hidden = false;
+        this.log.innerHTML = "";
+        this.bar.style.width = "0%";
+        this.countText.textContent = "";
+        this.statusText.textContent = "Starting…";
+        this.form.querySelector("[data-seo-run]").disabled = true;
+    }
+
+    _watch(uid) {
+        this.socket = new SeoProgressSocket(uid, {
+            onMessage: (frame) => this._frame(frame),
+            onClose: () => {},
+        });
+        this.socket.connect();
+    }
+
+    _frame(frame) {
+        const type = frame.type;
+        if (type === "stage") {
+            this.statusText.textContent = frame.message || "Working…";
+        } else if (type === "target") {
+            const total = (frame.urls || []).length;
+            this.countText.textContent = total ? `0 / ${total} pages` : "";
+            this.statusText.textContent = `Auditing ${frame.url}`;
+        } else if (type === "progress") {
+            const total = frame.total || 1;
+            const done = frame.done || 0;
+            this.bar.style.width = `${Math.round((done / total) * 100)}%`;
+            this.countText.textContent = `${done} / ${total} pages`;
+            if (frame.message) this.statusText.textContent = frame.message;
+        } else if (type === "page_loaded") {
+            this.bar.style.width = `${Math.round((frame.done / frame.total) * 100)}%`;
+            this.countText.textContent = `${frame.done} / ${frame.total} pages`;
+            this._appendLog(`${frame.status} ${frame.url}`);
+        } else if (type === "report_ready") {
+            this.statusText.textContent = "Compiling report…";
+        } else if (type === "done") {
+            this.bar.style.width = "100%";
+            this.statusText.textContent = "Done";
+            this.form.querySelector("[data-seo-run]").disabled = false;
+            const reportUrl = frame.report_url || `/tools/seo/${this.uid}/report`;
+            if (frame.report && Object.keys(frame.report).length) {
+                this._renderReport(frame.report, reportUrl);
+            } else {
+                this._loadReport(reportUrl);
+            }
+        } else if (type === "failed") {
+            this.statusText.textContent = "Failed";
+            this._setError(frame.message || frame.error || "The audit failed.");
+            this.form.querySelector("[data-seo-run]").disabled = false;
+        }
+    }
+
+    _appendLog(text) {
+        const item = document.createElement("li");
+        item.textContent = text;
+        this.log.prepend(item);
+    }
+
+    async _loadReport(url) {
+        this.form.querySelector("[data-seo-run]").disabled = false;
+        let report;
+        try {
+            report = await Http.getJson(url);
+        } catch (error) {
+            this._setError("Could not load the report.");
+            return;
+        }
+        this._renderReport(report, url);
+    }
+
+    _renderReport(report, url) {
+        this.report.hidden = false;
+        this.report.querySelector("[data-seo-score]").textContent = report.score ?? 0;
+        const gauge = this.report.querySelector("[data-seo-gauge]");
+        gauge.className = `seo-gauge seo-grade-${(report.grade || "f").toLowerCase()}`;
+        this.report.querySelector("[data-seo-grade]").textContent = report.grade || "";
+        this.report.querySelector("[data-seo-target]").textContent = report.target || "";
+        this.report.querySelector("[data-seo-report-link]").href = url.replace(/\?.*$/, "");
+        const counts = report.counts || {};
+        this.report.querySelector("[data-seo-counts]").innerHTML = ["pass", "warn", "fail", "info"]
+            .map((key) => `<span class="seo-chip seo-chip-${key}">${counts[key] || 0} ${key}</span>`)
+            .join("");
+        this._renderCategories(report.categories || {});
+        this._renderChecks(report.categories || {}, report.checks || [], report.page_count || 1);
+    }
+
+    _renderCategories(categories) {
+        const host = this.report.querySelector("[data-seo-categories]");
+        host.innerHTML = Object.entries(categories)
+            .map(([name, meta]) => {
+                const score = meta.score === null || meta.score === undefined ? "" : `<span class="seo-cat-score">${meta.score}</span>`;
+                return `<div class="seo-cat-pill"><span class="seo-cat-name">${this._title(name)}</span>${score}</div>`;
+            })
+            .join("");
+    }
+
+    _renderChecks(categories, checks, pageCount) {
+        const host = this.report.querySelector("[data-seo-checks]");
+        const groups = Object.keys(categories).map((cat) => {
+            const items = checks.filter((c) => c.category === cat);
+            const rows = items.map((check) => this._checkRow(check, pageCount)).join("");
+            return `<section class="seo-check-group"><h2>${this._title(cat)}</h2>${rows}</section>`;
+        });
+        host.innerHTML = groups.join("");
+    }
+
+    _checkRow(check, pageCount) {
+        const status = STATUS_LABEL[check.status] || check.status;
+        const url = check.url && pageCount > 1 ? `<span class="seo-check-url">${this._escape(check.url)}</span>` : "";
+        const value = check.value ? `<span class="seo-check-value">${this._escape(check.value)}</span>` : "";
+        const rec = check.recommendation ? `<span class="seo-check-rec">${this._escape(check.recommendation)}</span>` : "";
+        return `<div class="seo-check seo-check-${check.status}">
+            <span class="seo-check-status">${status}</span>
+            <div class="seo-check-body">
+                <span class="seo-check-title">${this._escape(check.title)} ${url}</span>
+                ${value}${rec}
+            </div>
+            <span class="seo-check-severity seo-sev-${check.severity}">${check.severity}</span>
+        </div>`;
+    }
+
+    _title(text) {
+        return String(text).replace(/_/g, " ").replace(/\b\w/g, (c) => c.toUpperCase());
+    }
+
+    _escape(text) {
+        const div = document.createElement("div");
+        div.textContent = text == null ? "" : String(text);
+        return div.innerHTML;
+    }
+
+    _setError(message) {
+        if (!this.errorBox) return;
+        this.errorBox.textContent = message;
+        this.errorBox.hidden = !message;
+    }
+}

devplacepy/static/js/SeoProgressSocket.js

@@ -0,0 +1,51 @@
+// retoor <retoor@molodetz.nl>
+
+const WRONG_WORKER_CODE = 4013;
+const WRONG_WORKER_RETRY_MS = 200;
+const RECONNECT_DELAY_MS = 1500;
+
+export class SeoProgressSocket {
+    constructor(uid, handlers) {
+        this.handlers = handlers || {};
+        this.ws = null;
+        this._shouldRun = true;
+        const scheme = location.protocol === "https:" ? "wss://" : "ws://";
+        this.url = `${scheme}${location.host}/tools/seo/${encodeURIComponent(uid)}/ws`;
+    }
+
+    connect() {
+        this._shouldRun = true;
+        this._open();
+    }
+
+    _open() {
+        this.ws = new WebSocket(this.url);
+        this.ws.addEventListener("message", (event) => {
+            let frame;
+            try {
+                frame = JSON.parse(event.data);
+            } catch {
+                return;
+            }
+            this._emit("onMessage", frame);
+        });
+        this.ws.addEventListener("close", (event) => {
+            if (event.code === WRONG_WORKER_CODE && this._shouldRun) {
+                window.setTimeout(() => this._open(), WRONG_WORKER_RETRY_MS);
+                return;
+            }
+            this._emit("onClose");
+        });
+        this.ws.addEventListener("error", () => this._emit("onError"));
+    }
+
+    close() {
+        this._shouldRun = false;
+        if (this.ws) this.ws.close();
+    }
+
+    _emit(name, payload) {
+        const handler = this.handlers[name];
+        if (typeof handler === "function") handler(payload);
+    }
+}

devplacepy/static/js/assetVersion.js

@@ -0,0 +1,11 @@
+// retoor <retoor@molodetz.nl>
+
+const META = document.querySelector('meta[name="asset-version"]');
+const VERSION = META ? META.content : "";
+
+export function assetUrl(path) {
+    if (!VERSION || !path.startsWith("/static/") || path.startsWith("/static/uploads/")) {
+        return path;
+    }
+    return `/static/v${VERSION}${path.slice("/static".length)}`;
+}

devplacepy/static/js/components/AppDocsChat.js

@@ -0,0 +1,344 @@
+// retoor <retoor@molodetz.nl>
+
+import { Component } from "./Component.js";
+import { assetUrl } from "../assetVersion.js";
+import DeviiSocket from "../devii/DeviiSocket.js";
+import Markdown from "../devii/markdown.js";
+
+const DOCS_CHAT_CSS_ID = "docs-chat-css";
+const CHANNEL = "docs";
+
+export class AppDocsChat extends Component {
+    constructor() {
+        super();
+        this.markdown = new Markdown();
+        this.socket = null;
+        this._seeded = false;
+        this._bootstrapped = false;
+        this._pending = {};
+        this._lastHistoryQuestion = null;
+    }
+
+    _ensureCss() {
+        if (document.getElementById(DOCS_CHAT_CSS_ID)) return;
+        const link = document.createElement("link");
+        link.id = DOCS_CHAT_CSS_ID;
+        link.rel = "stylesheet";
+        link.href = assetUrl("/static/css/docs-chat.css");
+        document.head.appendChild(link);
+    }
+
+    connectedCallback() {
+        if (this._built) return;
+        this._built = true;
+        this._build();
+        this._bindSidebar();
+        this._start();
+    }
+
+    disconnectedCallback() {
+        if (this.socket) this.socket.close();
+    }
+
+    _build() {
+        this.innerHTML = "";
+        this.log = document.createElement("div");
+        this.log.className = "docs-chat-log";
+        this.log.setAttribute("role", "log");
+        this.log.setAttribute("aria-live", "polite");
+
+        this.status = document.createElement("div");
+        this.status.className = "docs-chat-status";
+        this.status.hidden = true;
+
+        this.form = document.createElement("form");
+        this.form.className = "docs-chat-input";
+        this.field = document.createElement("textarea");
+        this.field.className = "docs-chat-field";
+        this.field.rows = 1;
+        this.field.placeholder = "Ask a question about DevPlace...";
+        this.field.setAttribute("aria-label", "Ask the documentation");
+        this.send = document.createElement("button");
+        this.send.type = "submit";
+        this.send.className = "docs-chat-send";
+        this.send.textContent = "Ask";
+        this.form.append(this.field, this.send);
+
+        this.append(this.log, this.status, this.form);
+
+        this.form.addEventListener("submit", (event) => {
+            event.preventDefault();
+            this._submit(this.field.value);
+        });
+        this.field.addEventListener("keydown", (event) => {
+            if (event.key === "Enter" && !event.shiftKey) {
+                event.preventDefault();
+                this._submit(this.field.value);
+            }
+        });
+        this.field.addEventListener("input", () => this._autosize());
+        this.log.addEventListener("click", (event) => this._onLogClick(event));
+    }
+
+    _bindSidebar() {
+        const search = document.querySelector("form.docs-search-form");
+        if (!search) return;
+        const input = search.querySelector("input[name='q']");
+        search.addEventListener("submit", (event) => {
+            event.preventDefault();
+            const value = input ? input.value : "";
+            this._submit(value);
+            if (input) input.value = "";
+        });
+    }
+
+    async _start() {
+        try {
+            await fetch("/devii/session", { credentials: "same-origin" });
+        } catch (error) {
+            // Best-effort: ensures the guest cookie exists before the socket opens.
+        }
+        this.socket = new DeviiSocket(
+            {
+                onReady: () => this._onReady(),
+                onMessage: (message) => this._onMessage(message),
+                onClose: () => this._setStatus("Disconnected. Reconnecting..."),
+                onError: () => this._setStatus("Connection error."),
+            },
+            CHANNEL,
+        );
+        this.socket.connect();
+    }
+
+    _onReady() {
+        this._setStatus("");
+    }
+
+    _onMessage(message) {
+        const kind = message.type;
+        if (kind === "reply") {
+            this._resolvePending();
+            this._agent(message.text);
+            if (!this._bootstrapped) {
+                this._bootstrapped = true;
+                this._maybeSeed();
+            }
+            this._setStatus("");
+        } else if (kind === "history") {
+            this._renderHistory(message.messages);
+            this._bootstrapped = true;
+            this._maybeSeed();
+        } else if (kind === "clear") {
+            this.log.innerHTML = "";
+        } else if (kind === "topic") {
+            this._topic(message);
+        } else if (kind === "error") {
+            this._resolvePending();
+            this._errorLine(message.text);
+            this._setStatus("");
+        } else if (kind === "status") {
+            this._setStatus(message.text);
+        } else if (kind === "trace") {
+            this._trace(message);
+        } else if (kind === "task") {
+            this._setStatus("Working...");
+        } else if (kind === "avatar" && message.id) {
+            this.socket.send({ type: "avatar_result", id: message.id, result: {} });
+        } else if (kind === "client" && message.id) {
+            this.socket.send({ type: "client_result", id: message.id, result: {} });
+        }
+    }
+
+    _maybeSeed() {
+        if (this._seeded) return;
+        this._seeded = true;
+        const seed = this.attr("seed").trim();
+        if (!seed) return;
+        if (seed === this._lastHistoryQuestion) return;
+        this._submit(seed);
+    }
+
+    _submit(raw) {
+        const text = (raw || "").trim();
+        if (!text) return;
+        this._pending = {};
+        this._lastQuestion = text;
+        this._userLine(text);
+        this.field.value = "";
+        this._autosize();
+        if (this.socket && this.socket.send({ type: "input", text })) {
+            this._setStatus("Docii is searching the docs");
+        } else {
+            this._errorLine("Not connected.");
+        }
+    }
+
+    _topic(message) {
+        const reason = (message.reason || "").trim();
+        if (message.decision === "new") {
+            this.log.innerHTML = "";
+            this._pending = {};
+            if (this._lastQuestion) this._userLine(this._lastQuestion);
+            this._noteLine(
+                reason ? `New topic - ${reason}` : "New topic - starting fresh",
+                "new",
+            );
+        } else {
+            this._noteLine(reason ? `Follow-up - ${reason}` : "Follow-up", "follow");
+        }
+    }
+
+    _noteLine(text, kind) {
+        const row = document.createElement("div");
+        row.className = `docs-chat-note ${kind}`;
+        row.textContent = text;
+        this._append(row);
+    }
+
+    _trace(message) {
+        const event = message.event;
+        const name = message.name || "";
+        const detail = message.detail || "";
+        if (event === "call") {
+            const row = this._stepRow(name, detail);
+            this._append(row);
+            (this._pending[name] = this._pending[name] || []).push(row);
+            this._setStatus("");
+        } else if (event === "ok" || event === "err") {
+            const queue = this._pending[name];
+            const row = queue && queue.shift();
+            if (row) {
+                row.classList.remove("pending");
+                row.classList.add(event === "ok" ? "done" : "error");
+            }
+        } else {
+            this._setStatus(this._eventLabel(event));
+        }
+    }
+
+    _resolvePending() {
+        Object.values(this._pending).forEach((queue) => {
+            (queue || []).forEach((row) => {
+                row.classList.remove("pending");
+                row.classList.add("done");
+            });
+        });
+        this._pending = {};
+    }
+
+    _stepRow(name, detail) {
+        const row = document.createElement("div");
+        row.className = "docs-chat-step pending";
+        const icon = document.createElement("span");
+        icon.className = "docs-chat-step-icon";
+        const label = document.createElement("span");
+        label.className = "docs-chat-step-label";
+        label.textContent = this._stepLabel(name, detail);
+        row.append(icon, label);
+        return row;
+    }
+
+    _stepLabel(name, detail) {
+        if (name === "search_docs") {
+            return detail ? `Searching the docs: ${detail}` : "Searching the docs";
+        }
+        return detail ? `${name}: ${detail}` : name;
+    }
+
+    _eventLabel(event) {
+        const labels = {
+            compact: "Condensing context",
+            "plan-gate": "Planning",
+            "reflect-trigger": "Reconsidering",
+            "verify-gate": "Verifying",
+        };
+        return labels[event] || "Working";
+    }
+
+    _userLine(text) {
+        const bubble = this._bubble("user");
+        bubble.textContent = text;
+        this._append(bubble.parentElement);
+    }
+
+    _agent(text) {
+        const bubble = this._bubble("agent");
+        bubble.innerHTML = this._renderMarkdown(text);
+        this._append(bubble.parentElement);
+    }
+
+    _errorLine(text) {
+        const bubble = this._bubble("error");
+        bubble.textContent = text;
+        this._append(bubble.parentElement);
+    }
+
+    _bubble(kind) {
+        const row = document.createElement("div");
+        row.className = `docs-chat-msg ${kind}`;
+        const bubble = document.createElement("div");
+        bubble.className = "docs-chat-bubble";
+        row.appendChild(bubble);
+        return bubble;
+    }
+
+    _renderMarkdown(text) {
+        const html = this.markdown.render(text);
+        if (window.DOMPurify && typeof window.DOMPurify.sanitize === "function") {
+            return window.DOMPurify.sanitize(html);
+        }
+        return html;
+    }
+
+    _renderHistory(messages) {
+        this.log.innerHTML = "";
+        (messages || []).forEach((message) => {
+            if (message.role === "user") {
+                this._userLine(message.content);
+                this._lastHistoryQuestion = (message.content || "").trim();
+            } else if (message.role === "assistant" && message.content) {
+                this._agent(message.content);
+            }
+        });
+    }
+
+    _onLogClick(event) {
+        const button = event.target.closest(".md-copy");
+        if (!button) return;
+        const wrap = button.closest(".md-codewrap");
+        const code = wrap && wrap.querySelector("code");
+        if (!code) return;
+        const label = button.textContent;
+        const restore = () => {
+            button.textContent = "Copied";
+            window.setTimeout(() => {
+                button.textContent = label;
+            }, 1200);
+        };
+        if (navigator.clipboard && navigator.clipboard.writeText) {
+            navigator.clipboard.writeText(code.textContent).then(restore, () => {});
+        }
+    }
+
+    _append(node) {
+        const pinned = this._atBottom();
+        this.log.appendChild(node);
+        if (pinned) this.log.scrollTop = this.log.scrollHeight;
+    }
+
+    _atBottom() {
+        return this.log.scrollHeight - this.log.scrollTop - this.log.clientHeight < 80;
+    }
+
+    _autosize() {
+        this.field.style.height = "auto";
+        this.field.style.height = `${Math.min(this.field.scrollHeight, 160)}px`;
+    }
+
+    _setStatus(text) {
+        this.status.textContent = text || "";
+        this.status.hidden = !text;
+    }
+}
+
+customElements.define("dp-docs-chat", AppDocsChat);

devplacepy/static/js/components/AppLightbox.js

@@ -1,6 +1,7 @@
 // retoor <retoor@molodetz.nl>
 
 import { Component } from "./Component.js";
+import { assetUrl } from "../assetVersion.js";
 
 const LB_CSS_ID = "lightbox-css";
 
@@ -22,7 +23,7 @@ export class AppLightbox extends Component {
         const link = document.createElement("link");
         link.id = LB_CSS_ID;
         link.rel = "stylesheet";
-        link.href = "/static/css/lightbox.css";
+        link.href = assetUrl("/static/css/lightbox.css");
         document.head.appendChild(link);
     }
 

devplacepy/static/js/components/Component.js

@@ -1,5 +1,7 @@
 // retoor <retoor@molodetz.nl>
 
+import { assetUrl } from "../assetVersion.js";
+
 const COMP_CSS_ID = "dp-components-css";
 
 export class Component extends HTMLElement {
@@ -13,7 +15,7 @@ export class Component extends HTMLElement {
         const link = document.createElement("link");
         link.id = COMP_CSS_ID;
         link.rel = "stylesheet";
-        link.href = "/static/css/components.css";
+        link.href = assetUrl("/static/css/components.css");
         document.head.appendChild(link);
     }
 

devplacepy/static/js/components/ContainerTerminal.js

@@ -3,10 +3,11 @@
 import FloatingWindow from "./FloatingWindow.js";
 import { Http } from "../Http.js";
 import { UserScope } from "../userScope.js";
+import { assetUrl } from "../assetVersion.js";
 
-const XTERM_JS = "/static/vendor/xterm/xterm.js";
-const XTERM_CSS = "/static/vendor/xterm/xterm.css";
-const XTERM_FIT = "/static/vendor/xterm/addon-fit.js";
+const XTERM_JS = assetUrl("/static/vendor/xterm/xterm.js");
+const XTERM_CSS = assetUrl("/static/vendor/xterm/xterm.css");
+const XTERM_FIT = assetUrl("/static/vendor/xterm/addon-fit.js");
 
 const FONT_MIN = 8;
 const FONT_MAX = 30;

devplacepy/static/js/components/FloatingWindow.js

@@ -1,6 +1,7 @@
 // retoor <retoor@molodetz.nl>
 
 import { Component } from "./Component.js";
+import { assetUrl } from "../assetVersion.js";
 
 const FW_CSS_ID = "floating-window-css";
 const MOBILE_QUERY = "(max-width: 640px)";
@@ -75,7 +76,7 @@ export default class FloatingWindow extends Component {
     const link = document.createElement("link");
     link.id = FW_CSS_ID;
     link.rel = "stylesheet";
-    link.href = "/static/css/floating-window.css";
+    link.href = assetUrl("/static/css/floating-window.css");
     document.head.appendChild(link);
   }
 

devplacepy/static/js/components/index.js

@@ -8,4 +8,5 @@ import "./AppToast.js";
 import "./AppDialog.js";
 import "./AppContextMenu.js";
 import "./AppLightbox.js";
+import "./AppDocsChat.js";
 import "./ContainerTerminal.js";

devplacepy/static/js/devii/DeviiSocket.js

@@ -5,10 +5,11 @@ const WRONG_WORKER_RETRY_MS = 200;
 const WRONG_WORKER_CODE = 4013;
 
 export default class DeviiSocket {
-  constructor(handlers) {
+  constructor(handlers, channel = "main") {
     this.handlers = handlers || {};
     this.ws = null;
-    this.url = (location.protocol === "https:" ? "wss://" : "ws://") + location.host + "/devii/ws";
+    const base = (location.protocol === "https:" ? "wss://" : "ws://") + location.host + "/devii/ws";
+    this.url = channel && channel !== "main" ? base + "?channel=" + encodeURIComponent(channel) : base;
     this._shouldRun = true;
     this._settled = false;
   }

devplacepy/static/js/devii/devii-avatar.js

@@ -1,5 +1,7 @@
 // retoor <retoor@molodetz.nl>
 
+import { assetUrl } from "../assetVersion.js";
+
 const CSS_ID = "devii-avatar-css";
 const DEFAULT_CHARACTER = "Clippy";
 const CORNER_MARGIN = 16;
@@ -36,13 +38,13 @@ export default class DeviiAvatarElement extends HTMLElement {
     const link = document.createElement("link");
     link.id = CSS_ID;
     link.rel = "stylesheet";
-    link.href = "/static/css/devii-avatar.css";
+    link.href = assetUrl("/static/css/devii-avatar.css");
     document.head.appendChild(link);
   }
 
   async _app() {
     if (!this._clippy) {
-      const module = await import("/static/vendor/md-clippy/index.js");
+      const module = await import(assetUrl("/static/vendor/md-clippy/index.js"));
       this._clippy = module.app;
     }
     return this._clippy;

devplacepy/static/js/devii/devii-terminal.js

@@ -2,6 +2,7 @@
 
 import FloatingWindow from "../components/FloatingWindow.js";
 import { Http } from "../Http.js";
+import { assetUrl } from "../assetVersion.js";
 import Markdown from "./markdown.js";
 import DeviiSocket from "./DeviiSocket.js";
 import DeviiClient from "./DeviiClient.js";
@@ -230,7 +231,7 @@ export default class DeviiTerminalElement extends FloatingWindow {
     const link = document.createElement("link");
     link.id = CSS_ID;
     link.rel = "stylesheet";
-    link.href = "/static/css/devii.css";
+    link.href = assetUrl("/static/css/devii.css");
     document.head.appendChild(link);
   }
 

devplacepy/templates/admin_audit_event.html

@@ -1,7 +1,7 @@
 {% extends "admin_base.html" %}
 {% block extra_head %}
 {{ super() }}
-<link rel="stylesheet" href="/static/css/audit.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/audit.css') }}">
 {% endblock %}
 {% block admin_content %}
 <a href="/admin/audit-log" class="audit-back-link"><span class="icon">&#x2190;</span> Back to audit log</a>

devplacepy/templates/admin_audit_log.html

@@ -1,7 +1,7 @@
 {% extends "admin_base.html" %}
 {% block extra_head %}
 {{ super() }}
-<link rel="stylesheet" href="/static/css/audit.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/audit.css') }}">
 {% endblock %}
 {% block admin_content %}
 <div class="admin-toolbar">

devplacepy/templates/admin_base.html

@@ -1,7 +1,7 @@
 {% extends "base.html" %}
 {% block extra_head %}
-<link rel="stylesheet" href="/static/css/admin.css">
-<link rel="stylesheet" href="/static/css/sidebar.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/admin.css') }}">
+<link rel="stylesheet" href="{{ static_url('/static/css/sidebar.css') }}">
 {% endblock %}
 {% block content %}
 <div class="admin-layout">

devplacepy/templates/admin_notifications.html

@@ -1,6 +1,6 @@
 {% extends "admin_base.html" %}
 {% block extra_head %}{{ super() }}
-<link rel="stylesheet" href="/static/css/profile.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/profile.css') }}">
 {% endblock %}
 {% block admin_content %}
 <div class="admin-toolbar">

devplacepy/templates/admin_settings.html

@@ -56,6 +56,15 @@
 
     <h3 style="font-size: 0.875rem; font-weight: 700; margin-bottom: 0.75rem; color: var(--text-secondary);">Operational</h3>
 
+    <div class="admin-field">
+        <label for="docs_search_mode">Docs Search</label>
+        <select id="docs_search_mode" name="docs_search_mode" style="max-width: 220px;">
+            <option value="agent" {% if settings.get('docs_search_mode', 'agent') == 'agent' %}selected{% endif %}>AI agent (conversational)</option>
+            <option value="bm25" {% if settings.get('docs_search_mode', 'agent') == 'bm25' %}selected{% endif %}>Keyword (BM25)</option>
+        </select>
+        <small class="hint-text">AI agent renders the in-page Devii chat on the docs search page. Keyword uses the classic BM25 results list. Users who exceed their daily AI limit (or guests when Devii is disabled) fall back to keyword search automatically.</small>
+    </div>
+
     <div class="admin-field">
         <label for="registration_open">Registration</label>
         <select id="registration_open" name="registration_open" style="max-width: 160px;">

devplacepy/templates/ai_usage.html

@@ -1,8 +1,8 @@
 {% extends "admin_base.html" %}
 {% block extra_head %}
 {{ super() }}
-<link rel="stylesheet" href="/static/css/services.css">
-<link rel="stylesheet" href="/static/css/ai_usage.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/services.css') }}">
+<link rel="stylesheet" href="{{ static_url('/static/css/ai_usage.css') }}">
 {% endblock %}
 {% block admin_content %}
 <div class="admin-toolbar">
@@ -31,7 +31,7 @@
 </div>
 {% endblock %}
 {% block extra_js %}
-<script type="module" src="/static/js/AiUsageMonitor.js"></script>
+<script type="module" src="{{ static_url('/static/js/AiUsageMonitor.js') }}"></script>
 <script type="module">
 new window.AiUsageMonitor().start();
 </script>

devplacepy/templates/base.html

@@ -4,6 +4,7 @@
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <meta name="theme-color" content="#0f0a1a">
+    <meta name="asset-version" content="{{ static_version }}">
     <title>{% if page_title %}{{ page_title }}{% else %}DevPlace - The Developer Social Network{% endif %}</title>
     <meta name="description" content="{% if meta_description %}{{ meta_description }}{% else %}Track industry shifts. Discover bold releases. Share what you're building in an open, uncensored environment.{% endif %}">
     <link rel="canonical" href="{{ canonical_url or request.url }}">
@@ -15,33 +16,33 @@
     <meta property="og:description" content="{{ og_description or meta_description or 'The Developer Social Network' }}">
     <meta property="og:type" content="{{ og_type or 'website' }}">
     <meta property="og:url" content="{{ canonical_url or request.url }}">
-    <meta property="og:image" content="{{ og_image or '/static/og-default.png' }}">
+    <meta property="og:image" content="{{ static_url(og_image or '/static/og-default.png') }}">
     <meta property="og:site_name" content="DevPlace">
     <meta property="og:locale" content="en_US">
 
     <meta name="twitter:card" content="summary_large_image">
     <meta name="twitter:title" content="{{ og_title or page_title or 'DevPlace' }}">
     <meta name="twitter:description" content="{{ og_description or meta_description or 'The Developer Social Network' }}">
-    <meta name="twitter:image" content="{{ og_image or '/static/og-default.png' }}">
+    <meta name="twitter:image" content="{{ static_url(og_image or '/static/og-default.png') }}">
 
     <link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>&#x1f4bb;</text></svg>">
-    <link rel="apple-touch-icon" href="/static/apple-touch-icon.png">
+    <link rel="apple-touch-icon" href="{{ static_url('/static/apple-touch-icon.png') }}">
     <link rel="manifest" href="/manifest.json">
     <meta name="mobile-web-app-capable" content="yes">
     <meta name="apple-mobile-web-app-capable" content="yes">
     <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
     <meta name="apple-mobile-web-app-title" content="DevPlace">
 
-    <link rel="stylesheet" href="/static/css/variables.css">
-    <link rel="stylesheet" href="/static/css/base.css">
-    <link rel="stylesheet" href="/static/css/components.css">
-    <link rel="stylesheet" href="/static/vendor/atom-one-dark.min.css">
-    <link rel="stylesheet" href="/static/css/markdown.css">
-    <link rel="stylesheet" href="/static/css/mention.css">
-    <link rel="stylesheet" href="/static/css/attachments.css">
-    <link rel="stylesheet" href="/static/css/lightbox.css">
-    <link rel="stylesheet" href="/static/css/media.css">
-    <link rel="stylesheet" href="/static/css/engagement.css">
+    <link rel="stylesheet" href="{{ static_url('/static/css/variables.css') }}">
+    <link rel="stylesheet" href="{{ static_url('/static/css/base.css') }}">
+    <link rel="stylesheet" href="{{ static_url('/static/css/components.css') }}">
+    <link rel="stylesheet" href="{{ static_url('/static/vendor/atom-one-dark.min.css') }}">
+    <link rel="stylesheet" href="{{ static_url('/static/css/markdown.css') }}">
+    <link rel="stylesheet" href="{{ static_url('/static/css/mention.css') }}">
+    <link rel="stylesheet" href="{{ static_url('/static/css/attachments.css') }}">
+    <link rel="stylesheet" href="{{ static_url('/static/css/lightbox.css') }}">
+    <link rel="stylesheet" href="{{ static_url('/static/css/media.css') }}">
+    <link rel="stylesheet" href="{{ static_url('/static/css/engagement.css') }}">
     {% block extra_head %}{% endblock %}
     {{ custom_css_tag(request) }}
 
@@ -60,8 +61,14 @@
                 <a href="/feed" class="topnav-link {% if request.url.path == '/feed' %}active{% endif %}"><span class="icon">📝</span> Posts</a>
                 <a href="/news" class="topnav-link {% if 'news' in request.url.path %}active{% endif %}"><span class="icon">📰</span> News</a>
                 <a href="/gists" class="topnav-link {% if 'gists' in request.url.path %}active{% endif %}"><span class="icon">📄</span> Gists</a>
-                <a href="/projects" class="topnav-link {% if 'projects' in request.url.path %}active{% endif %}"><span class="icon">🚀</span> Projects</a>
+                <a href="/projects" class="topnav-link topnav-link-feature {% if 'projects' in request.url.path %}active{% endif %}"><span class="icon">🚀</span> Projects</a>
                 <a href="/leaderboard" class="topnav-link {% if 'leaderboard' in request.url.path %}active{% endif %}"><span class="icon">🏆</span> Leaderboard</a>
+                <div class="topnav-tools-dropdown {% if request.url.path.startswith('/tools') %}active{% endif %}">
+                    <button type="button" class="topnav-link topnav-tools-toggle" aria-haspopup="true" aria-expanded="false"><span class="icon">🧰</span> Tools <span class="topnav-caret">▾</span></button>
+                    <div class="dropdown-menu">
+                        <a href="/tools/seo" class="dropdown-item"><span class="icon">🔍</span> SEO Diagnostics</a>
+                    </div>
+                </div>
                 {% if user %}
                 <a href="/messages" class="topnav-link {% if 'messages' in request.url.path %}active{% endif %}" data-counter="messages"><span class="icon">✉️</span> Messages{% set msg_unread = get_unread_messages(user["uid"]) %}<span class="nav-badge nav-badge-inline" data-counter-badge {% if msg_unread == 0 %}hidden{% endif %}>{{ msg_unread }}</span></a>
                 {% if is_admin(user) %}
@@ -112,6 +119,9 @@
             <a href="/gists" class="topnav-mobile-link {% if 'gists' in request.url.path %}active{% endif %}"><span class="icon">📄</span> Gists</a>
             <a href="/projects" class="topnav-mobile-link {% if 'projects' in request.url.path %}active{% endif %}"><span class="icon">🚀</span> Projects</a>
             <a href="/leaderboard" class="topnav-mobile-link {% if 'leaderboard' in request.url.path %}active{% endif %}"><span class="icon">🏆</span> Leaderboard</a>
+            <div class="topnav-mobile-divider"></div>
+            <div class="topnav-mobile-section-label">Tools</div>
+            <a href="/tools/seo" class="topnav-mobile-link {% if 'tools/seo' in request.url.path %}active{% endif %}"><span class="icon">🔍</span> SEO Diagnostics</a>
             {% if user %}
             <a href="/messages" class="topnav-mobile-link {% if 'messages' in request.url.path %}active{% endif %}" data-counter="messages"><span class="icon">✉️</span> Messages<span class="nav-badge nav-badge-inline" data-counter-badge {% if get_unread_messages(user["uid"]) == 0 %}hidden{% endif %}>{{ get_unread_messages(user["uid"]) }}</span></a>
             <a href="/notifications" class="topnav-mobile-link {% if 'notifications' in request.url.path %}active{% endif %}" data-counter="notifications"><span class="icon">🔔</span> Notifications<span class="nav-badge nav-badge-inline" data-counter-badge {% if get_unread_count(user["uid"]) == 0 %}hidden{% endif %}>{{ get_unread_count(user["uid"]) }}</span></a>
@@ -177,11 +187,11 @@
     </template>
     {% endif %}
 
-    <script defer src="/static/vendor/marked.umd.js"></script>
-    <script defer src="/static/vendor/highlight.min.js"></script>
-    <script defer src="/static/vendor/purify.min.js"></script>
-    <script type="module" src="/static/vendor/emoji-picker-element/index.js"></script>
-    <script type="module" src="/static/js/Application.js"></script>
+    <script defer src="{{ static_url('/static/vendor/marked.umd.js') }}"></script>
+    <script defer src="{{ static_url('/static/vendor/highlight.min.js') }}"></script>
+    <script defer src="{{ static_url('/static/vendor/purify.min.js') }}"></script>
+    <script type="module" src="{{ static_url('/static/vendor/emoji-picker-element/index.js') }}"></script>
+    <script type="module" src="{{ static_url('/static/js/Application.js') }}"></script>
     {% block extra_js %}{% endblock %}
     {{ custom_js_tag(request) }}
 </body>

devplacepy/templates/bug_detail.html

@@ -1,6 +1,6 @@
 {% extends "base.html" %}
 {% block extra_head %}
-<link rel="stylesheet" href="/static/css/bugs.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/bugs.css') }}">
 {% endblock %}
 {% block content %}
 <div class="bugs-layout bug-detail">

devplacepy/templates/bugs.html

@@ -1,7 +1,7 @@
 {% extends "base.html" %}
 {% from "_macros.html" import modal %}
 {% block extra_head %}
-<link rel="stylesheet" href="/static/css/bugs.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/bugs.css') }}">
 {% endblock %}
 {% block content %}
 <div class="bugs-layout">

devplacepy/templates/containers.html

@@ -1,7 +1,7 @@
 {% extends "base.html" %}
 {% from "_macros.html" import modal %}
 {% block extra_head %}
-<link rel="stylesheet" href="/static/css/containers.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/containers.css') }}">
 {% endblock %}
 {% block content %}
 <h1 class="sr-only">Container manager</h1>
@@ -76,7 +76,7 @@
 {% endblock %}
 {% block extra_js %}
 <script type="module">
-import { ContainerManager } from "/static/js/ContainerManager.js";
+import { ContainerManager } from "{{ static_url('/static/js/ContainerManager.js') }}";
 const root = document.getElementById("container-manager");
 if (root) {
     const manager = new ContainerManager(root);

devplacepy/templates/containers_admin.html

@@ -1,7 +1,7 @@
 {% extends "admin_base.html" %}
 {% block extra_head %}
 {{ super() }}
-<link rel="stylesheet" href="/static/css/containers.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/containers.css') }}">
 {% endblock %}
 {% block admin_content %}
 <div class="admin-toolbar">
@@ -48,7 +48,7 @@
 {% endblock %}
 {% block extra_js %}
 <script type="module">
-import { ContainerList } from "/static/js/ContainerList.js";
+import { ContainerList } from "{{ static_url('/static/js/ContainerList.js') }}";
 const list = new ContainerList(document.getElementById("cm-admin-list"));
 list.init();
 window.app.containerList = list;

devplacepy/templates/containers_instance.html

@@ -1,7 +1,7 @@
 {% extends "base.html" %}
 {% from "_macros.html" import modal %}
 {% block extra_head %}
-<link rel="stylesheet" href="/static/css/containers.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/containers.css') }}">
 {% endblock %}
 {% block content %}
 <div class="ci-page" id="container-instance"
@@ -128,7 +128,7 @@
 {% endblock %}
 {% block extra_js %}
 <script type="module">
-import { ContainerInstance } from "/static/js/ContainerInstance.js";
+import { ContainerInstance } from "{{ static_url('/static/js/ContainerInstance.js') }}";
 const root = document.getElementById("container-instance");
 const instance = new ContainerInstance(root);
 instance.init();

devplacepy/templates/docs/_chat.html

@@ -0,0 +1,5 @@
+<div class="docs-search docs-chat-page">
+    <h1>Ask Docii</h1>
+    <p class="docs-chat-intro">Docii is the documentation assistant. Ask a question in plain language and it searches the docs, reading the relevant pages until it has your answer.</p>
+    <dp-docs-chat seed="{{ search_query|default('', true) }}"></dp-docs-chat>
+</div>

devplacepy/templates/docs/_search.html

@@ -1,5 +1,8 @@
 <div class="docs-search">
     <h1>Search documentation</h1>
+    {% if quota_fallback %}
+    <p class="docs-search-hint">You have reached your daily AI assistant limit. Showing keyword search results instead.</p>
+    {% endif %}
     <form class="docs-search-bigform" method="get" action="/docs/search.html" role="search">
         <input type="search" name="q" value="{{ search_query|default('', true) }}" placeholder="Search the docs…" aria-label="Search documentation" class="docs-search-input" autofocus>
         <button type="submit" class="btn btn-primary">Search</button>

devplacepy/templates/docs/architecture-styling.html

@@ -60,7 +60,7 @@ There is no separate front-end application or client-side router: every page is
 
 **Production** (Docker Compose): an nginx front door sits in front of the app and serves the static layer directly from the bind-mounted `devplacepy/static` directory on disk, never touching the application:
 
-- `/static/` (CSS and JavaScript) is served from disk with an immutable 7-day cache.
+- `/static/v<version>/` (CSS and JavaScript) is served from disk with a one-year immutable cache; the `<version>` path segment is the server boot timestamp, so a deploy busts every asset URL while keeping returning visitors fully cached between deploys. The unversioned `/static/` fallback keeps a short one-hour cache. See [Static asset caching](/docs/static-caching.html).
 - `/static/uploads/` is served from disk, but nginx **re-applies** `Content-Disposition: attachment` and `X-Content-Type-Options: nosniff`, because the app's own `UploadStaticFiles` sets that download header and a direct nginx handler would otherwise bypass it (a stored-XSS defense for uploaded SVG/HTML).
 - `/avatar/` is proxied to the app, because avatars are generated SVGs, not files on disk.
 - Everything else (`/`, the rendered pages and the JSON API) is proxied to the app, with an optional micro-cache.

devplacepy/templates/docs/component-dp-avatar.html

@@ -33,5 +33,5 @@ No JavaScript is required - the element renders itself on connection.
     </div>
 </div>
 <script type="module">
-    import "/static/js/components/AppAvatar.js";
+    import "{{ static_url('/static/js/components/AppAvatar.js') }}";
 </script>

devplacepy/templates/docs/component-dp-code.html

@@ -37,5 +37,5 @@ On connection the element enhances its content once: highlighting, a copy button
 }</code></pre></dp-code>
 </div>
 <script type="module">
-    import "/static/js/components/AppCode.js";
+    import "{{ static_url('/static/js/components/AppCode.js') }}";
 </script>

devplacepy/templates/docs/component-dp-content.html

@@ -46,5 +46,5 @@ const answer = 42;
 </dp-content>
 </div>
 <script type="module">
-    import "/static/js/components/AppContent.js";
+    import "{{ static_url('/static/js/components/AppContent.js') }}";
 </script>

devplacepy/templates/docs/component-dp-context-menu.html

@@ -44,7 +44,7 @@ app.contextMenu.attach(myElement, (event) => [
     <dp-context-menu id="ctx-menu"></dp-context-menu>
 </div>
 <script type="module">
-    import "/static/js/components/AppContextMenu.js";
+    import "{{ static_url('/static/js/components/AppContextMenu.js') }}";
     const menu = document.getElementById("ctx-menu");
     const target = document.getElementById("ctx-target");
     const result = document.getElementById("ctx-result");

devplacepy/templates/docs/component-dp-dialog.html

@@ -48,7 +48,7 @@ if (ok) {
     <dp-dialog id="dialog-demo"></dp-dialog>
 </div>
 <script type="module">
-    import "/static/js/components/AppDialog.js";
+    import "{{ static_url('/static/js/components/AppDialog.js') }}";
     const dialog = document.getElementById("dialog-demo");
     const result = document.getElementById("dlg-result");
     document.getElementById("dlg-confirm").addEventListener("click", async () => {

devplacepy/templates/docs/component-dp-toast.html

@@ -36,7 +36,7 @@ The element renders nothing until `show` is called.
     <dp-toast id="toast-demo"></dp-toast>
 </div>
 <script type="module">
-    import "/static/js/components/AppToast.js";
+    import "{{ static_url('/static/js/components/AppToast.js') }}";
     const toast = document.getElementById("toast-demo");
     const messages = { info: "Heads up", success: "Saved", error: "Something went wrong" };
     document.querySelectorAll("[data-toast]").forEach((btn) => {

devplacepy/templates/docs/component-dp-upload.html

@@ -60,5 +60,5 @@ Set with the `mode` attribute:
     <dp-upload mode="field" field-name="demo" multiple label="Choose files"></dp-upload>
 </div>
 <script type="module">
-    import "/static/js/components/AppUpload.js";
+    import "{{ static_url('/static/js/components/AppUpload.js') }}";
 </script>

devplacepy/templates/docs/component-emoji-picker.html

@@ -43,7 +43,7 @@ picker.addEventListener("emoji-click", (event) => {
     </div>
 </div>
 <script type="module">
-    import "/static/vendor/emoji-picker-element/index.js";
+    import "{{ static_url('/static/vendor/emoji-picker-element/index.js') }}";
     const result = document.getElementById("emoji-demo-result");
     document.querySelector(".component-demo emoji-picker").addEventListener("emoji-click", (event) => {
         result.textContent = `Selected: ${event.detail.unicode}`;

devplacepy/templates/docs/production-nginx.html

@@ -1,7 +1,7 @@
 <div class="docs-content" data-render>
 # nginx and networking
 
-The nginx front door, how it serves each route, and the production-specific rules it enforces. See also [Production overview](/docs/production.html) and [Deploy and update](/docs/production-deploy.html).
+The nginx front door, how it serves each route, and the production-specific rules it enforces. See also [Production overview](/docs/production.html), [Deploy and update](/docs/production-deploy.html), and [Static asset caching and versioning](/docs/static-caching.html).
 
 ## Build and configuration
 
@@ -12,8 +12,10 @@ The nginx image (`nginx/Dockerfile`) renders `nginx/nginx.conf.template` at star
 | Location | Behavior |
 |----------|----------|
 | `/static/uploads/` | Served from disk with `Content-Disposition: attachment` and `X-Content-Type-Options: nosniff`. |
-| `/static/` | Served from disk, immutable, 7-day cache. |
+| `/static/v<version>/` | Versioned assets served from disk, `public, immutable, max-age=31536000` (1 year). The `<version>` is the server boot timestamp, so a deploy changes every asset URL. See [Static asset caching](/docs/static-caching.html). |
+| `/static/` | Unversioned fallback served from disk with a short `max-age=3600` (1 hour), not immutable. |
 | `/devii/ws` | Proxied as a WebSocket (`Upgrade`/`Connection` headers, 1h read/send timeout). |
+| `/tools/seo/<uid>/ws` | SEO Diagnostics live-progress WebSocket; matched by `location ~ ^/tools/seo/[^/]+/ws$` with the same `Upgrade`/`Connection` headers and 1h timeout. |
 | `/avatar/` | Proxied to the app (generated SVG avatars). |
 | `/` | Proxied to the app; optional micro-cache. |
 
@@ -34,6 +36,8 @@ map $http_upgrade $connection_upgrade {
 
 and the `/devii/ws` location forwards `Upgrade $http_upgrade` and `Connection $connection_upgrade` with a long read timeout. The app serves `/devii/ws` only from the worker holding the service lock; other workers close with code 1013 and the auto-reconnecting client lands on the owner, so a connection may take a brief reconnect to settle. See [Multi-worker and concurrency](/docs/production-concurrency.html) for the lock-owner model.
 
+**Every WebSocket path needs its own location block.** The catch-all `location /` deliberately sets `Connection ""` (no upgrade) and a short 60s read timeout, so any route that falls through to it cannot complete a WebSocket handshake - the browser reports `WebSocket connection failed` with no close code (the upgrade never happens). When you add a new WebSocket endpoint, add a dedicated `location` (exact path or a `~` regex) that forwards `Upgrade $http_upgrade` and `Connection $connection_upgrade` with long timeouts, **above** `location /`. The existing examples are `/devii/ws`, the SEO Diagnostics `location ~ ^/tools/seo/[^/]+/ws$`, and the container ingress `/p/` block. Like Devii, the SEO progress socket is served only by the service-lock owner; a non-owner closes with code 4013 and the client fast-retries until it lands on the owner.
+
 ## Upload size ceiling
 
 `client_max_body_size` is set from `NGINX_MAX_BODY_SIZE` (default `50m`). It must be **>= the admin-configurable `max_upload_size_mb`** (Admin -> Settings). If the app limit exceeds the nginx limit, nginx rejects the upload with **HTTP 413** before it ever reaches the app. After raising `max_upload_size_mb`, raise `NGINX_MAX_BODY_SIZE` to match and restart nginx.
@@ -45,7 +49,8 @@ The server block sets `X-Content-Type-Options`, `X-Frame-Options: DENY`, `X-XSS-
 ## Troubleshooting
 
 - **Devii terminal will not connect** - confirm the `map` block and the `/devii/ws` location are present in the rendered config (`docker compose exec nginx cat /etc/nginx/conf.d/default.conf`); a connection that closes immediately with 1013 is the lock-owner reconnect, not a failure.
+- **A WebSocket reports `connection failed` with no close code** - the route has no dedicated upgrade `location` and fell through to `location /`, which strips the upgrade headers. Add a matching `location` block (see WebSockets above) and reload nginx. This is what breaks the SEO Diagnostics live progress if the `location ~ ^/tools/seo/[^/]+/ws$` block is missing.
 - **Uploads fail with 413** - `NGINX_MAX_BODY_SIZE` is below `max_upload_size_mb`; raise it and restart nginx.
 - **Uploaded file renders inline instead of downloading** - the `/static/uploads/` location lost its `Content-Disposition` rule.
-- **Static assets stale after deploy** - the static bind mount is read-only and live; a hard refresh clears the 7-day browser cache.
+- **Static assets stale after deploy** - every app-owned asset URL carries the boot-version path segment (`/static/v<timestamp>/...`), so a restart busts the cache automatically; if assets still look stale, confirm the app actually restarted (the `<meta name="asset-version">` value in page source changed). See [Static asset caching](/docs/static-caching.html).
 </div>

devplacepy/templates/docs/static-caching.html

@@ -0,0 +1,118 @@
+<div class="docs-content" data-render>
+{% raw %}
+# Static asset caching and versioning
+
+DevPlace serves its CSS and JavaScript with a one-year immutable cache for the best
+possible "Serve static assets with an efficient cache policy" score, while still letting a
+deploy take effect immediately. The two goals are reconciled by stamping a **boot-time
+version** into every app-owned static URL.
+
+See also [nginx and networking](/docs/production-nginx.html) and
+[Deploy and update](/docs/production-deploy.html).
+
+## How it works
+
+Every static URL the app emits carries a version path segment:
+
+```
+/static/css/base.css        ->  /static/v1718040000/css/base.css
+/static/js/Application.js    ->  /static/v1718040000/js/Application.js
+```
+
+`v1718040000` is the unix timestamp captured once when the server process starts
+(`config.STATIC_VERSION`). Because the value is fixed for the life of the process:
+
+- **Heavy caching is safe.** Each versioned URL is unique per deploy, so the browser may
+  cache it for a full year (`Cache-Control: public, immutable, max-age=31536000`).
+- **Deploys are instant.** Restarting the server changes the boot timestamp, so every
+  asset URL changes, so every returning browser refetches on its next page load. No manual
+  cache purge, no hashing build step, no waiting for a TTL to expire.
+
+## Why a path segment, not a query string
+
+DevPlace ships unbundled ES6 modules wired together with **relative imports**
+(`import { Http } from "./Http.js"`). A relative import resolves against the URL of the
+module that contains it. If only the entry `<script>` carried a version (for example a
+`?v=` query string), its transitively imported modules would resolve to **unversioned**
+URLs and stay frozen under the immutable cache - a deploy would not propagate JavaScript
+changes for up to a year.
+
+Putting the version in the path solves this for free: a module loaded from
+`/static/v1718040000/js/Application.js` resolves `./Http.js` to
+`/static/v1718040000/js/Http.js`, so the **entire module graph and every relative CSS
+`url()`** inherits the version automatically with no per-file rewriting.
+
+## The helpers
+
+- **Templates** use the `static_url` Jinja global (registered in `templating.py`):
+
+```
+<link rel="stylesheet" href="{{ static_url('/static/css/base.css') }}">
+<script type="module" src="{{ static_url('/static/js/Application.js') }}"></script>
+```
+
+  It returns the path unchanged for anything outside `/static/` and for `/static/uploads/`
+  (user content, addressed by stable database paths), so it is safe to wrap around dynamic
+  values such as `{{ static_url(og_image or '/static/og-default.png') }}`.
+
+- **JavaScript** that builds an absolute static URL at runtime (component CSS link
+  injection, lazily loaded xterm/md-clippy modules) uses the `assetUrl` helper
+  (`static/js/assetVersion.js`), which reads the version from the
+  `<meta name="asset-version">` tag rendered in `base.html`:
+
+```
+import { assetUrl } from "../assetVersion.js";
+link.href = assetUrl("/static/css/components.css");
+```
+
+Relative module imports never need either helper - they inherit the version from the
+importing module's URL.
+
+## Serving and cache headers
+
+| Layer | Versioned `/static/v<n>/...` | Unversioned `/static/...` |
+|-------|------------------------------|---------------------------|
+| App (dev / `make dev`) | `CachedStaticFiles` mount sets `public, immutable, max-age=31536000` | plain mount, validator-based revalidation |
+| nginx (prod) | regex `location ~ ^/static/v\d+/` sets `public, immutable, max-age=31536000` | `public, max-age=3600` (one hour, not immutable) |
+
+The unversioned `/static/` location stays a short, non-immutable cache because a few
+unversioned URLs still exist (the service worker's precache icons, direct hits); they must
+not be frozen for a year.
+
+**Service worker exception.** `service-worker.js` is served with `Cache-Control: no-cache`
+(by its `/service-worker.js` route and, defensively, by `CachedStaticFiles`) so a new
+worker always deploys. It keeps a stable, unversioned URL so its registration scope does
+not change on every deploy.
+
+**Uploads exclusion.** `/static/uploads/` is user-generated content with its own public
+cache and stable stored paths; it is never boot-versioned.
+
+## Multi-worker consistency
+
+In production the app runs multiple uvicorn workers. Each worker is a separate process, so
+if every worker computed its own timestamp they could disagree by a second and emit
+mismatched asset URLs. The version is therefore fixed **once at launch** and shared through
+the `DEVPLACE_STATIC_VERSION` environment variable:
+
+- `make prod` prefixes the command with `DEVPLACE_STATIC_VERSION=$(date +%s)`.
+- The Docker image launches uvicorn through `sh -c` so a single `date +%s` is captured and
+  exported before the workers fork.
+
+When `DEVPLACE_STATIC_VERSION` is unset (dev with `--reload`), each process boot computes a
+fresh timestamp, so a reload after editing a file naturally produces a new version. You can
+also pin the variable to a build id or git sha in CI for reproducible URLs.
+
+## Verify
+
+```
+curl -sI http://localhost:10500/static/v$(date +%s)/css/base.css | grep -i cache-control
+# -> Cache-Control: public, max-age=31536000, immutable
+
+curl -sI http://localhost:10500/service-worker.js | grep -i cache-control
+# -> Cache-Control: no-cache
+```
+
+In a rendered page, the `<meta name="asset-version">` value and every `/static/v.../`
+URL change after a server restart.
+{% endraw %}
+</div>

devplacepy/templates/docs/styles-consistency.html

@@ -14,7 +14,7 @@ This is the posts page reduced to its structure. A content page looks like this
 {% extends "base.html" %}
 
 {% block extra_head %}
-<link rel="stylesheet" href="/static/css/feed.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/feed.css') }}">
 {% endblock %}
 
 {% block content %}

devplacepy/templates/docs/tools-seo.html

@@ -0,0 +1,69 @@
+<div class="docs-content" data-render>
+# SEO Diagnostics
+
+SEO Diagnostics audits a web page or an entire sitemap against a broad battery of checks, far wider
+than a single Core Web Vitals or rich-results test. It loads each page in a real headless browser,
+inspects the rendered DOM, measures performance, and grades the result. Progress streams live while
+the audit runs, and a full categorised report is produced at the end.
+
+Open it from the **Tools** menu, or go straight to `/tools/seo`. It is public: you do not need an
+account.
+
+## Running an audit
+
+1. Paste a URL (for a single page) or a `sitemap.xml` URL.
+2. Choose **Single URL** or **Sitemap**. In sitemap mode, set how many pages to crawl (up to 50).
+3. Press **Run audit**. Progress appears immediately: each page is loaded, then every check runs.
+4. When the audit finishes, the score, grade, per-category breakdown and every individual check
+   with its recommendation are shown. A link opens the full standalone report.
+
+You can run one audit at a time. Targets that resolve to private or local addresses are refused.
+
+## What gets checked
+
+The auditor groups its findings into categories:
+
+- **Crawlability and indexing** - HTTP status, redirect chains, HTTPS and HSTS, canonical tags,
+  meta-robots and `X-Robots-Tag`, `robots.txt`, XML sitemap, URL hygiene, and mixed content.
+- **On-page meta and content** - title and meta description (presence and length), single H1 and
+  heading hierarchy, content depth, language, character encoding, viewport, and favicon.
+- **Links** - internal and external link profile, descriptive anchor text.
+- **Structured data** - JSON-LD validity, recognised schema types and their required properties,
+  microdata and RDFa detection.
+- **Social cards** - Open Graph and Twitter Card tags for rich link previews.
+- **Performance and Core Web Vitals** - Largest Contentful Paint, Cumulative Layout Shift, First
+  Contentful Paint, Time To First Byte, page weight, request count, DOM size, compression, caching,
+  image optimisation, and console errors.
+- **Mobile and accessibility** - responsive layout (no horizontal overflow), tap-target sizing,
+  image alt coverage, and form labels.
+- **Security** - the common security response headers and TLS.
+- **AI and LLM-search readiness** - whether your primary content is in the initial HTML (server
+  rendered) rather than JavaScript-only, presence of an `llms.txt`, and semantic HTML landmarks.
+
+Each check reports a status (pass, warn, fail or info), its severity, the observed value, and a
+recommendation when something can be improved. The overall score is a severity-weighted pass rate,
+shown as a 0-100 number and an A to F grade, with a subscore per category.
+
+## Scoring
+
+Failing a critical check (for example a non-200 status or HTTPS) costs far more than a low-severity
+warning. Informational checks never affect the score. A score of 90 or above is an A.
+
+## Ask Devii
+
+You can also run an audit in plain language through the Devii assistant:
+
+> Run SEO diagnostics on https://example.com and tell me the score.
+
+Devii queues the audit, polls it, and reports the score, grade and a link to the full report.
+
+## Programmatic access
+
+The same audit is available over the API: `POST /tools/seo/run` to queue, `GET /tools/seo/{uid}`
+to poll, and `GET /tools/seo/{uid}/report` for the full report (HTML or JSON). See the
+[Tools (SEO Diagnostics)](/docs/tools.html) API group for request and response shapes.
+</div>
+
+<div class="devii-doc-cta">
+    <a href="/tools/seo" class="sidebar-link">Open SEO Diagnostics</a>
+</div>

devplacepy/templates/docs_base.html

@@ -1,8 +1,8 @@
 {% extends "base.html" %}
 {% block extra_head %}
-<link rel="stylesheet" href="/static/css/sidebar.css">
-<link rel="stylesheet" href="/static/css/docs.css">
-<link rel="stylesheet" href="/static/css/components-docs.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/sidebar.css') }}">
+<link rel="stylesheet" href="{{ static_url('/static/css/docs.css') }}">
+<link rel="stylesheet" href="{{ static_url('/static/css/components-docs.css') }}">
 {% endblock %}
 {% block content %}
 <script>window.DEVPLACE_DOCS = {{ {
@@ -44,7 +44,11 @@
     </aside>
     <article class="docs-main">
         {% if kind == 'search' %}
-            {% include "docs/_search.html" %}
+            {% if search_mode == 'agent' %}
+                {% include "docs/_chat.html" %}
+            {% else %}
+                {% include "docs/_search.html" %}
+            {% endif %}
         {% elif kind == 'api' %}
             {% include "docs/_api_page.html" %}
         {% elif prose_html %}
@@ -54,8 +58,7 @@
         {% endif %}
     </article>
 </div>
-<span data-devii-autoopen hidden></span>
 {% endblock %}
 {% block extra_js %}
-<script type="module" src="/static/js/ApiDocs.js"></script>
+<script type="module" src="{{ static_url('/static/js/ApiDocs.js') }}"></script>
 {% endblock %}

devplacepy/templates/feed.html

@@ -1,9 +1,9 @@
 {% extends "base.html" %}
 {% from "_macros.html" import modal %}
 {% block extra_head %}
-<link rel="stylesheet" href="/static/css/feed.css">
-<link rel="stylesheet" href="/static/css/sidebar.css">
-<link rel="stylesheet" href="/static/css/post.css">
+<link rel="stylesheet" href="{{ static_url('/static/css/feed.css') }}">
+<link rel="stylesheet" href="{{ static_url('/static/css/sidebar.css') }}">
+<link rel="stylesheet" href="{{ static_url('/static/css/post.css') }}">
 {% endblock %}
 {% block content %}
 <h1 class="sr-only">Feed</h1>
@@ -29,18 +29,25 @@
                 <span class="icon">&#x2753;</span>
                 Question
             </a>
-            <a href="/feed?topic=rant" class="sidebar-link {% if current_topic == 'rant' %}active{% endif %}" data-topic="rant">
-                <span class="icon">&#x1F4A2;</span>
-                Rant
-            </a>
-            <a href="/feed?topic=fun" class="sidebar-link {% if current_topic == 'fun' %}active{% endif %}" data-topic="fun">
-                <span class="icon">&#x1F3AE;</span>
-                Fun
-            </a>
-            <a href="/feed?topic=signals" class="sidebar-link {% if current_topic == 'signals' %}active{% endif %}" data-topic="signals">
-                <span class="icon">&#x1F4E1;</span>
-                Signals
-            </a>
+            <details class="sidebar-more" {% if current_topic in ['rant', 'fun', 'signals'] %}open{% endif %}>
+                <summary class="sidebar-link sidebar-more-toggle">
+                    <span class="icon">&#x2026;</span>
+                    More
+                    <span class="sidebar-more-chevron">&#x203A;</span>
+                </summary>
+                <a href="/feed?topic=rant" class="sidebar-link {% if current_topic == 'rant' %}active{% endif %}" data-topic="rant">
+                    <span class="icon">&#x1F4A2;</span>
+                    Rant
+                </a>
+                <a href="/feed?topic=fun" class="sidebar-link {% if current_topic == 'fun' %}active{% endif %}" data-topic="fun">
+                    <span class="icon">&#x1F3AE;</span>
+                    Fun
+                </a>
+                <a href="/feed?topic=signals" class="sidebar-link {% if current_topic == 'signals' %}active{% endif %}" data-topic="signals">
+                    <span class="icon">&#x1F4E1;</span>
+                    Signals
+                </a>
+            </details>
         </div>
     </aside>
 
@@ -70,6 +77,9 @@
     <aside class="feed-right">
         <div class="daily-topic-card">
             <div class="daily-topic-label">Daily Topic</div>
+            {% if daily_topic.get('image_url', '') %}
+            <img src="{{ daily_topic['image_url'] }}" alt="{{ daily_topic.get('title', '') }}" loading="lazy" class="daily-topic-image">
+            {% endif %}
             <h4>{{ daily_topic.get('title', '') }}</h4>
             <p>{{ daily_topic.get('summary', '') }}</p>
             <div class="topic-links">