Add personal notes, DeepSearch history, backup offload, and gateway auth throttling

Also streamline the top navigation: drop the Tools dropdown, make Quizzes
and Battles icon-only entries, and remove the Workspace, Containers and
Editor entry points from the project detail page.

README.md

@@ -66,15 +66,15 @@ devplacepy/
 | `/topics` | Crawlable per-topic category pages (public): `/topics` hub links every topic with a live post count, `/topics/{topic}` lists that topic's posts with its own canonical URL, title, and breadcrumbs |
 | `/news` | Developer news listing, detail page with comments |
 | `/posts` | Post detail, creation |
-| `/gists` | Code gist listing, detail, creation, and editing; left panel offers language filtering and free-text `search` (title, description, and author username), public read |
+| `/gists` | Code gist listing, detail, creation, and editing; left panel offers language filtering and free-text `search` (title, description, and author username), public read. A plain Markdown gist has a **View rendered / View raw** toggle beside its Copy button, switching between the raw source and the same rendered view a `Markdown Rendered` gist always shows |
-| `/comments` | Comment creation, owner editing (`POST /comments/edit/{comment_uid}`), deletion |
+| `/comments` | Comment creation, owner editing (`POST /comments/edit/{comment_uid}`), deletion. Every comment has a **Copy link** button that copies its permalink (the parent post/gist/project/news URL plus `#comment-{uid}`) to the clipboard |
 | `/projects` | Project listing (left panel offers type filtering and free-text `search` over title, description, and author username), creation, owner editing (`POST /projects/edit/{slug}`), and per-project visibility toggles: `POST /projects/{slug}/private` (owner-only visibility) and `POST /projects/{slug}/readonly` (immutable files). A project hidden by a member stays visible to administrators, but a project hidden by an administrator is visible only to that owner administrator - other administrators cannot see it, its files, or its containers (web UI and REST API alike). The primary administrator (the first Admin account) is the single exception and retains full visibility |
 | `/projects/{slug}` | Dedicated project page: one encompassing card with a cover banner and project logo (owner-uploaded through the standard attachment uploader), the title overlaid on the banner, status/type/platform chips, owner-set Website and Repository links, section tabs (Overview, Devlog, Screenshots, Comments, Files), an About section, the Devlog timeline of every post linked to the project (owners post updates straight from the page via the shared composer preset to the `devlog` topic), a Screenshots gallery built from image attachments (owners add more from the More menu), and a sidebar with links, stats and the author card |
 | `/projects/{slug}/files` | Per-project filesystem: directory and file CRUD, upload, inline editing, and line-range operations (`lines` read, `replace-lines`, `insert-lines`, `delete-lines`, `append`) for surgical edits to large text files (public read, owner write; all writes refused while the project is read-only) |
 | `/zips` | Zip job status (`/zips/{uid}`) and archive download (`/zips/{uid}/download`); archives are queued via `/projects/{slug}/zip` and `/projects/{slug}/files/zip` |
 | `/forks` | Fork job status (`/forks/{uid}`); forks are queued via `/projects/{slug}/fork`. Any signed-in user can fork a project they can view into a new project they own; once the job finishes the response carries the new project URL |
-| `/tools` | Public developer tools. `/tools/seo` is **SEO Diagnostics**: audit any URL or sitemap and stream live progress over a websocket. Queue with `POST /tools/seo/run`, poll `GET /tools/seo/{uid}`, read the full report at `GET /tools/seo/{uid}/report`. `/tools/deepsearch` is **DeepSearch**: a multi-agent deep web researcher that crawls and indexes sources, synthesises a cited report with confidence scoring, and lets you chat over the gathered evidence. Queue with `POST /tools/deepsearch/run`, poll `GET /tools/deepsearch/{uid}`, read the report at `GET /tools/deepsearch/{uid}/session`, export at `/export.{md,json,pdf}`. `/tools/isslop` is the **AI Usage Analyzer**: classify a git repository or website as AI slop, sophisticated AI-assisted work or genuine human work. Queue with `POST /tools/isslop/run`, poll `GET /tools/isslop/{uid}` or the event trail at `GET /tools/isslop/{uid}/events`, read the report at `GET /tools/isslop/{uid}/report` (`.md` to download) and embed the SVG authenticity badge from `GET /tools/isslop/{uid}/badge.svg` |
+| `/tools` | Public developer tools. `/tools/seo` is **SEO Diagnostics**: audit any URL or sitemap and stream live progress over a websocket. Queue with `POST /tools/seo/run`, poll `GET /tools/seo/{uid}`, read the full report at `GET /tools/seo/{uid}/report`. `/tools/deepsearch` is **DeepSearch**: a multi-agent deep web researcher that crawls and indexes sources, synthesises a cited report with confidence scoring, and lets you chat over the gathered evidence. Queue with `POST /tools/deepsearch/run`, poll `GET /tools/deepsearch/{uid}`, read the report at `GET /tools/deepsearch/{uid}/session`, export at `/export.{md,json,pdf}`. Every past run is kept at `GET /tools/deepsearch/history`, which lists your own research sessions with a link back to reopen the report and continue its grounded chat. `/tools/isslop` is the **AI Usage Analyzer**: classify a git repository or website as AI slop, sophisticated AI-assisted work or genuine human work. Queue with `POST /tools/isslop/run`, poll `GET /tools/isslop/{uid}` or the event trail at `GET /tools/isslop/{uid}/events`, read the report at `GET /tools/isslop/{uid}/report` (`.md` to download) and embed the SVG authenticity badge from `GET /tools/isslop/{uid}/badge.svg` |
-| `/projects/{slug}/containers` | Admin per-project container manager: create and control container instances, all running the shared prebuilt `ppy` image (there is no in-app image building). Reachable from the project page via the admin-only **Containers** button |
+| `/projects/{slug}/containers` | Admin per-project container manager: create and control container instances, all running the shared prebuilt `ppy` image (there is no in-app image building). Reachable by direct URL and from the admin index |
 | `/admin/containers` | Admin **Containers** manager: list, create, edit, and control container instances across projects, under strict per-user isolation: the primary administrator sees and manages every instance; every other administrator sees instances on public projects plus their own (instances attached to another user's private project are excluded entirely) and manages only the instances they own (created by them or attached to their own project) - all other rows are view-only. The list (`/admin/containers`) has inline start/stop/restart/terminal/edit/delete on each row and a create form (pick a project, optionally a run-as user, a boot language with a source editor, restart policy, start-on-boot, plus env/ports/limits/ingress). Each instance has a detail page (`/admin/containers/{uid}`) with lifecycle controls, live logs and metrics, an interactive terminal, schedules, ingress, workspace sync, and a status history, and an edit page (`/admin/containers/{uid}/edit`) |
 | `/p/{slug}` | Public ingress proxy (HTTP + WebSocket) to a running container instance's published port, opt-in per instance via `ingress_slug` |
 | `/profile` | Profile view, editing, a public **Media** tab (`?tab=media`) showing every attachment a user uploaded newest first, and a live **online / last-seen** presence indicator |
@@ -87,6 +87,7 @@ devplacepy/
 | `/votes` | Upvote/downvote on posts, comments, projects |
 | `/reactions` | Emoji reactions on posts, comments, gists, projects |
 | `/bookmarks` | Save/unsave content; `/bookmarks/saved` personal list |
+| `/notes` | Private per-user notes attached to a post, gist, project, or news article; `POST /notes/{target_type}/{target_uid}` adds or replaces the note, `POST /notes/{target_type}/{target_uid}/delete` removes it, `/notes/saved` is your personal notes list |
 | `/polls` | Vote on post-attached polls |
 | `/follow` | Follow/unfollow users |
 | `/reports` | Content reporting: `POST /reports/{target_type}/{target_uid}` files a report against any user-generated surface, `GET /reports/mine` lists the reports you filed and their outcome, `GET /reports/reasons` serves the reason registry so every client renders the same dialog |
@@ -108,7 +109,7 @@ devplacepy/
 | `/openai` | OpenAI-compatible LLM gateway service (`/openai/v1/chat/completions`, `/openai/v1/*`) |
 | `/devii` | Devii agentic assistant: WebSocket terminal (`/devii/ws`), standalone page, usage (`/devii/usage`), session bootstrap |
 | `(none)` | `/robots.txt`, `/sitemap.xml` (SEO) |
-| `(none)` | `/push.json` (VAPID key + subscribe), `/service-worker.js`, `/manifest.json` (push + PWA) |
+| `(none)` | `/push.json` (VAPID key + subscribe/unsubscribe), `/service-worker.js`, `/manifest.json` (push + PWA) |
 
 ## Gamification
 
@@ -225,6 +226,7 @@ The farm refreshes live over the pub/sub bus (a watered build appears on the own
 - **Polls** - a post can carry a poll (question plus up to six options); results appear as live bars once the viewer votes, one vote per member. A poll can be attached when the post is created or added later by editing a post that has none.
 - **Paste an image to attach it** - pressing Ctrl+V (Cmd+V) with a screenshot or copied image on the clipboard while writing a post, a comment, a direct message, an issue, a gist, or a project attaches it immediately, with no trip through the file picker. The upload, its limits, and the resulting attachment are identical to picking the file by hand.
 - **Bookmarks** - save posts, gists, projects, and news to a personal list at `/bookmarks/saved`.
+- **Personal notes** - attach a private note to a post, gist, project, or news article; nobody else, not even the content's author, can ever see it. Managed from an "Add note"/"Edit note" button on the item and listed on a personal page at `/notes/saved`.
 - **Private projects** - an owner can mark a project private so it is visible only to them (and administrators) and excluded from listings, profiles, search, the sitemap, and zip access. Set at creation or toggled later from the project page.
 - **Read-only projects** - an owner can mark a project read-only, making its entire virtual filesystem immutable: every write, edit, line-edit, move, delete, and upload is refused from all paths (the web UI, the HTTP API, the Devii agent, and container workspace sync) until read-only is turned off. Devii may toggle read-only only after the user explicitly confirms.
 
@@ -495,10 +497,9 @@ and its full configuration are documented automatically - including future servi
 ### Dev Workspaces and the browser editor
 
 A **workspace** is a member-facing container running the DevPlace browser editor, layered on the
-container runtime above. It is opened from a project's **Workspace** page and reached at
+container runtime above. It is reached at `/projects/{slug}/workspace`; the editor itself is proxied
-`/projects/{slug}/workspace`; the editor itself is proxied at
+at `/projects/{slug}/containers/instances/{uid}/code/`, and the workspace page shows an **Open
-`/projects/{slug}/containers/instances/{uid}/code/`, and an **Editor** button appears on the project
+editor** link whenever the editor is actually reachable.
-page whenever the editor is actually reachable.
 
 **The workspace page reports the editor's real state, live.** A workspace has a *phase* derived on
 the server from its desired state, its container status and a TCP probe of the editor port:
@@ -566,15 +567,15 @@ restart.
 
 `ForkService` copies a project into a new project owned by the forking user. The **Fork** button on the project page (any signed-in user) prompts for a name; the job creates the destination project, duplicates the entire virtual filesystem, and records a directional `project_forks` relation so each fork shows a "Forked from X" link. The frontend `app.projectForker` enqueues, polls `/forks/{uid}`, and redirects to the new project once it is done; on failure the partially created project is rolled back. The forked project is permanent, so retention removes only the job tracking row. CLI: `devplace forks prune` / `devplace forks clear` (job rows only).
 
-`SeoService` powers the public **Tools -> SEO Diagnostics** auditor. It runs a headless-browser (Playwright) crawl of a single URL or a sitemap (capped pages) in a subprocess and runs a broad battery of checks across eleven categories: crawlability and indexing (status, redirects, HTTPS/HSTS, canonical, robots/meta-robots, sitemap, URL hygiene, mixed content), on-page meta and content (title, description, headings, language, charset, viewport, favicon, content depth), links, structured data and rich results (JSON-LD validity and required properties, microdata/RDFa), social cards (Open Graph, Twitter), Core Web Vitals and performance (LCP, CLS, FCP, TTFB, page weight, requests, DOM size, compression, caching, image optimisation, console errors), mobile and accessibility (responsive layout, tap targets, image alt, form labels), security headers, and AI/LLM-search readiness (server-rendered-vs-JS content parity, `llms.txt`, semantic HTML). It produces a weighted score and grade with per-category subscores and a recommendation for every finding. Progress streams live over `WS /tools/seo/{uid}/ws`; the full report is available at `/tools/seo/{uid}/report` (HTML or JSON). CLI: `devplace seo prune` / `devplace seo clear`. Playwright is a core dependency; `make install` fetches the Chromium browser.
+`SeoService` powers the public **SEO Diagnostics** auditor at `/tools/seo`. It runs a headless-browser (Playwright) crawl of a single URL or a sitemap (capped pages) in a subprocess and runs a broad battery of checks across eleven categories: crawlability and indexing (status, redirects, HTTPS/HSTS, canonical, robots/meta-robots, sitemap, URL hygiene, mixed content), on-page meta and content (title, description, headings, language, charset, viewport, favicon, content depth), links, structured data and rich results (JSON-LD validity and required properties, microdata/RDFa), social cards (Open Graph, Twitter), Core Web Vitals and performance (LCP, CLS, FCP, TTFB, page weight, requests, DOM size, compression, caching, image optimisation, console errors), mobile and accessibility (responsive layout, tap targets, image alt, form labels), security headers, and AI/LLM-search readiness (server-rendered-vs-JS content parity, `llms.txt`, semantic HTML). It produces a weighted score and grade with per-category subscores and a recommendation for every finding. Progress streams live over `WS /tools/seo/{uid}/ws`; the full report is available at `/tools/seo/{uid}/report` (HTML or JSON). CLI: `devplace seo prune` / `devplace seo clear`. Playwright is a core dependency; `make install` fetches the Chromium browser.
 
 `SeoMetaService` is a separate AI subservice that generates a clean, search-optimised title, description and short keyword list for every published post, project, gist, news article and issue, entirely off the request path so it never slows the web server. The work is queued whenever content is created, edited or published; until the AI value is ready a plain-content default (built from the markdown-stripped text and clamped to safe lengths) fills the fields, so a page's metadata is **always populated, never empty**. The service uses the built-in internal AI gateway and meters its own AI cost and statistics in a dedicated usage table, surfaced together with its live task pipeline on the **Admin -> Services** page. This release also fixes the on-page metadata: the `<meta name="description">` is now stripped of markdown markup (it previously leaked `#`, `**` and `[](...)` from the raw body), a `<meta name="keywords">` tag is emitted (a short honest list, not stuffed), and social-card image dimensions and alt text are added. CLI: `devplace seo-meta prune` / `devplace seo-meta clear` (job rows only; the generated metadata persists).
 
-`DeepsearchService` powers the public **Tools -> DeepSearch** researcher, an iterative agent modelled on the way Perplexity and OpenAI's own deep-research agents work: search, read, decide what is still missing, search again. Given a single research question it plans a set of diverse web search queries, interleaves their results so every angle contributes sources, and crawls the most relevant pages in a subprocess (concurrent fetches, plain HTTP first with a headless-browser fallback for JavaScript-heavy pages, PDF documents streamed and text-extracted, every URL SSRF-guarded). After the first pass it runs up to two automatic gap-filling refinement rounds - a lightweight planner looks at what has been gathered so far and proposes a few more targeted queries only if real coverage gaps remain, bounded by the page budget, and stops the moment nothing more is needed. Social sites that block bots (X, YouTube, Reddit and similar) contribute the readable text supplied by the search engine, so their content is not lost to a login wall. A readability-grade extractor isolates the main article content of each page (navigation, cookie banners and footers are discarded), and the configured depth follows the most relevant in-page links for deeper coverage. Content is de-duplicated and indexed into a per-session ChromaDB vector collection (embeddings via the AI gateway with a local fallback); retrieval reranks candidate passages with Maximal Marginal Relevance so the report is grounded on genuinely diverse evidence rather than several near-duplicate hits on the same page. A chain of agents (report writer, findings extractor, linker) then synthesises a thorough cited markdown report grounded on the passages retrieved from that index, with key findings, a confidence score, source diversity and a short list of suggested follow-up questions you can ask straight into the chat; if synthesis fails the report is clearly marked as degraded instead of silently shipping raw source material. Progress streams live over `WS /tools/deepsearch/{uid}/ws`; the report is at `/tools/deepsearch/{uid}/session` (HTML or JSON) and can be exported as Markdown, JSON or PDF. A grounded chat over the session's evidence runs at `WS /tools/deepsearch/{uid}/chat` using hybrid retrieval (vector + keyword/BM25). Runs can be paused, resumed or cancelled. A cross-session URL cache avoids re-fetching pages seen by earlier runs. CLI: `devplace deepsearch prune` / `devplace deepsearch clear`. ChromaDB, weasyprint, pypdf and Playwright are core dependencies.
+`DeepsearchService` powers the public **DeepSearch** researcher at `/tools/deepsearch`, an iterative agent modelled on the way Perplexity and OpenAI's own deep-research agents work: search, read, decide what is still missing, search again. Given a single research question it plans a set of diverse web search queries, interleaves their results so every angle contributes sources, and crawls the most relevant pages in a subprocess (concurrent fetches, plain HTTP first with a headless-browser fallback for JavaScript-heavy pages, PDF documents streamed and text-extracted, every URL SSRF-guarded). After the first pass it runs up to two automatic gap-filling refinement rounds - a lightweight planner looks at what has been gathered so far and proposes a few more targeted queries only if real coverage gaps remain, bounded by the page budget, and stops the moment nothing more is needed. Social sites that block bots (X, YouTube, Reddit and similar) contribute the readable text supplied by the search engine, so their content is not lost to a login wall. A readability-grade extractor isolates the main article content of each page (navigation, cookie banners and footers are discarded), and the configured depth follows the most relevant in-page links for deeper coverage. Content is de-duplicated and indexed into a per-session ChromaDB vector collection (embeddings via the AI gateway with a local fallback); retrieval reranks candidate passages with Maximal Marginal Relevance so the report is grounded on genuinely diverse evidence rather than several near-duplicate hits on the same page. A chain of agents (report writer, findings extractor, linker) then synthesises a thorough cited markdown report grounded on the passages retrieved from that index, with key findings, a confidence score, source diversity and a short list of suggested follow-up questions you can ask straight into the chat; if synthesis fails the report is clearly marked as degraded instead of silently shipping raw source material. Progress streams live over `WS /tools/deepsearch/{uid}/ws`; the report is at `/tools/deepsearch/{uid}/session` (HTML or JSON) and can be exported as Markdown, JSON or PDF. A grounded chat over the session's evidence runs at `WS /tools/deepsearch/{uid}/chat` using hybrid retrieval (vector + keyword/BM25). Runs can be paused, resumed or cancelled. Every run you have started is listed at `GET /tools/deepsearch/history`, newest first with its query, status and score, so you can reopen a finished report and pick the chat back up later. A cross-session URL cache avoids re-fetching pages seen by earlier runs. CLI: `devplace deepsearch prune` / `devplace deepsearch clear`. ChromaDB, weasyprint, pypdf and Playwright are core dependencies.
 
-`IsslopService` powers the public **Tools -> AI Usage Analyzer**, which classifies a git repository or website as AI slop, sophisticated AI-assisted work or genuine human work. Sources are acquired in an isolated subprocess (git URLs are probed with `git ls-remote` and shallow-cloned with a 3 GB guard; websites render in a stealth headless browser with an HTTP fallback, bounded by depth, file and byte caps), inventoried with exclusion rules, and scored by a multi-signal static engine (twenty-one detector families, 126 signal types across an origin axis and a quality-deficit axis). For a live website, the analyzer also opens the home page in a headless browser and inspects what actually renders: it fingerprints AI website-builder platforms directly (Lovable, Bolt.new, Framer and others) and checks the page's real computed styles, layout and build artifacts, not just its file contents. Representative files receive an AI review pass and images an AI-generation review through the internal gateway (model `molodetz`, internal key); the static engine stays authoritative when the gateway is unreachable. Every pipeline step is persisted as an ordered event trail and streamed live over the pub/sub topic `public.isslop.{uid}`. The verdict is an A-F authenticity grade with a human/AI split and one of five categories (`ai-slop`, `sophisticated-ai`, `human-clean`, `human-messy`, `uncertain`), published as a persistent report with an embeddable SVG badge. Members keep their analysis history on their account; guest history is session-bound and claimed by the account on the first signed-in visit. Admin settings (private-host allowance, AI/image review toggles, image cap, retention, concurrency) live on `/admin/services`. CLI: `devplace isslop analyze <url>` / `devplace isslop prune` / `devplace isslop clear`. Playwright plus playwright-stealth back the website crawler.
+`IsslopService` powers the public **AI Usage Analyzer** at `/tools/isslop`, which classifies a git repository or website as AI slop, sophisticated AI-assisted work or genuine human work. Sources are acquired in an isolated subprocess (git URLs are probed with `git ls-remote` and shallow-cloned with a 3 GB guard; websites render in a stealth headless browser with an HTTP fallback, bounded by depth, file and byte caps), inventoried with exclusion rules, and scored by a multi-signal static engine (twenty-one detector families, 126 signal types across an origin axis and a quality-deficit axis). For a live website, the analyzer also opens the home page in a headless browser and inspects what actually renders: it fingerprints AI website-builder platforms directly (Lovable, Bolt.new, Framer and others) and checks the page's real computed styles, layout and build artifacts, not just its file contents. Representative files receive an AI review pass and images an AI-generation review through the internal gateway (model `molodetz`, internal key); the static engine stays authoritative when the gateway is unreachable. Every pipeline step is persisted as an ordered event trail and streamed live over the pub/sub topic `public.isslop.{uid}`. The verdict is an A-F authenticity grade with a human/AI split and one of five categories (`ai-slop`, `sophisticated-ai`, `human-clean`, `human-messy`, `uncertain`), published as a persistent report with an embeddable SVG badge. Members keep their analysis history on their account; guest history is session-bound and claimed by the account on the first signed-in visit. Admin settings (private-host allowance, AI/image review toggles, image cap, retention, concurrency) live on `/admin/services`. CLI: `devplace isslop analyze <url>` / `devplace isslop prune` / `devplace isslop clear`. Playwright plus playwright-stealth back the website crawler.
 
-`BackupService` powers the admin **Admin -> Backups** dashboard, an enterprise-grade backup system that runs entirely as asynchronous jobs so it never impacts the running server. An administrator can back up one of four targets: the **database** (a consistent SQLite snapshot of the main database and the Devii task/lesson databases, taken with SQLite's online backup API so it is consistent under WAL), **uploads** (every attachment and project file), **keys and config** (VAPID keys), or the **full data directory** (database snapshot, uploads, and keys in one archive, excluding regenerable staging, locks, caches, and container workspaces). Each backup is compressed to a `tar.gz` in a stdlib subprocess off the request path and recorded with its size, file count, and a SHA-256 checksum. Archives live under `data/backups/` (sharded on the random uuid tail) and are served only through `/admin/backups/{uid}/download`, which is restricted to the **primary administrator** - the first user created with the Admin role. Every other administrator receives a 403 from the endpoint and sees the Download button disabled with the tooltip `Not available`; creating, running, deleting, and scheduling backups remain available to all administrators. The dashboard reports detailed storage usage - the size and file count of every major data area, the total data-directory footprint, the total size and count of stored backups, and disk usage (total, used, free, percent), computed in a worker thread and cached briefly so the page never blocks. Backups can be **scheduled** (CRUD) on an interval or 5-field cron expression with a `keep_last` rotation count that prunes older backups of the same schedule; the service evaluates schedules only on the lock-owning worker so each fires exactly once. Backup archives are permanent operational artifacts: job retention only removes the tracking row, never the archive, which is deleted only by an administrator, by schedule rotation, or via the CLI. CLI: `devplace backups list` / `devplace backups run <target>` / `devplace backups prune` / `devplace backups clear`. Devii tools: `backups_overview`, `backup_run`, `backup_status`, `backup_delete`, `backup_schedule_create`, `backup_schedule_delete` (all admin-only). The service creates and stores backups but does not restore them into a live server; restore is a documented manual procedure (stop the server, unpack the archive over the data directory, verify the checksum, restart).
+`BackupService` powers the admin **Admin -> Backups** dashboard, an enterprise-grade backup system that runs entirely as asynchronous jobs so it never impacts the running server. An administrator can back up one of four targets: the **database** (a consistent SQLite snapshot of the main database and the Devii task/lesson databases, taken with SQLite's online backup API so it is consistent under WAL), **uploads** (every attachment and project file), **keys and config** (VAPID keys), or the **full data directory** (database snapshot, uploads, and keys in one archive, excluding regenerable staging, locks, caches, and container workspaces). Each backup is compressed to a `tar.gz` in a stdlib subprocess off the request path and recorded with its size, file count, and a SHA-256 checksum. Archives live under `data/backups/` (sharded on the random uuid tail) and are served only through `/admin/backups/{uid}/download`, which is restricted to the **primary administrator** - the first user created with the Admin role. Every other administrator receives a 403 from the endpoint and sees the Download button disabled with the tooltip `Not available`; creating, running, deleting, and scheduling backups remain available to all administrators. The dashboard reports detailed storage usage - the size and file count of every major data area, the total data-directory footprint, the total size and count of stored backups, and disk usage (total, used, free, percent). Disk percent is an O(1) volume stat used by the backup service tick; per-directory file counts are a single walk of the data directory, run in a worker thread and cached for minutes, so neither the request path nor the event loop ever walks the tree. Backups can be **scheduled** (CRUD) on an interval or 5-field cron expression with a `keep_last` rotation count that prunes older backups of the same schedule; the service evaluates schedules only on the lock-owning worker so each fires exactly once. Backup archives are permanent operational artifacts: job retention only removes the tracking row, never the archive, which is deleted only by an administrator, by schedule rotation, or via the CLI. CLI: `devplace backups list` / `devplace backups run <target>` / `devplace backups prune` / `devplace backups clear`. Devii tools: `backups_overview`, `backup_run`, `backup_status`, `backup_delete`, `backup_schedule_create`, `backup_schedule_delete` (all admin-only). The service creates and stores backups but does not restore them into a live server; restore is a documented manual procedure (stop the server, unpack the archive over the data directory, verify the checksum, restart).
 
 ### Adding a service
 
@@ -713,6 +714,8 @@ Configuration on the Services tab:
 | `gateway_allow_admins` / `gateway_allow_users` | on / off | Which DevPlace users may call it (any auth scheme) |
 | `gateway_access_key` | empty | A standalone key (sent as `X-API-KEY`/Bearer) that always grants access |
 | `gateway_internal_key` | auto (uuid4) | Auto-generated on boot; DevPlace's own services authenticate with this. Clear and restart to rotate |
+| `gateway_auth_throttle_enabled` | on | Track failed authentication attempts per IP and block further unauthenticated attempts once an IP crosses the failure threshold. Never blocks a request presenting valid credentials |
+| `gateway_auth_throttle_max_failures` / `_window_seconds` | 10 / 60 | Failed-auth attempts allowed per IP within the sliding window before further unauthenticated attempts get `429` |
 | `gateway_price_cache_hit_per_m` / `_cache_miss_per_m` / `_output_per_m` | 0.0028 / 0.14 / 0.28 | Chat cost per 1M tokens, used when the upstream returns no native cost (DeepSeek) |
 | `gateway_vision_price_input_per_m` / `_output_per_m` | 0 / 0 | Vision cost per 1M tokens, used only when the vision upstream returns no native cost |
 | `gateway_embed_price_input_per_m` | 0.01 | Embeddings cost per 1M input tokens, used only when the embeddings upstream returns no native cost |
@@ -979,6 +982,14 @@ accepting registrations; when `apns` is active it includes `environment` (`produ
 `sandbox`). A provider that is disabled or not fully configured accepts no registrations
 and is skipped during delivery, so an unconfigured provider is inert rather than an error.
 
+`DELETE /push.json` unregisters exactly one registration, identified the same way it was
+created (`endpoint` for webpush, `token` or `client_id` for apns). It is idempotent - an
+unknown or already-removed identity still returns `200 {"unregistered": false}`. The web
+frontend calls it automatically before navigating to `/auth/logout` (`PushManager.js`), so a
+browser subscription stops receiving notifications the moment the user signs out; a native
+app integrating `apns` must call it itself at logout, since the server has no way to detect
+a native client closing on its own.
+
 Every provider setting is edited at **`/admin/services/push`**: per provider an `Enabled`
 toggle, the VAPID subject for `webpush`, and team id, key id, `.p8` auth key (stored as a
 masked secret), topic and environment (production or sandbox) for `apns`. The same page

devplacepy/content.py

@@ -19,6 +19,7 @@ from devplacepy.database import (
     get_user_votes,
     get_reactions_by_targets,
     get_user_bookmarks,
+    get_user_notes,
     get_blocked_uids,
     get_poll_for_post,
     update_target_stars,
@@ -70,6 +71,7 @@ CREATE_METADATA_KEYS = ("project_type", "is_private", "language", "topic", "stat
 
 BOOKMARKABLE_TYPES = {"post", "gist", "project", "news", "quiz"}
 REACTABLE_TYPES = {"post", "comment", "gist", "project", "quiz"}
+NOTABLE_TYPES = {"post", "gist", "project", "news"}
 
 logger = logging.getLogger(__name__)
 
@@ -325,10 +327,16 @@ def apply_vote(request, user: dict, target_type: str, target_uid: str, value: in
     existing = votes.find_one(
         user_uid=user["uid"], target_uid=target_uid, target_type=target_type
     )
-    old_value = int(existing["value"]) if existing else 0
+    old_value = int(existing["value"]) if existing and not existing.get("deleted_at") else 0
     did_upvote = False
     new_value = value
-    if existing:
+    if value == 0:
+        if existing and not existing.get("deleted_at"):
+            votes.update(
+                {"id": existing["id"], "deleted_at": _now_iso(), "deleted_by": user["uid"]},
+                ["id"],
+            )
+    elif existing:
         if existing.get("deleted_at"):
             votes.update(
                 {
@@ -648,6 +656,7 @@ def detail_context(
         "attachments": detail["attachments"],
         "reactions": detail.get("reactions", {"counts": {}, "mine": []}),
         "bookmarked": detail.get("bookmarked", False),
+        "note_content": detail.get("note_content"),
         "poll": detail.get("poll"),
         "war": detail.get("war"),
         "project_link": detail.get("project_link"),
@@ -828,6 +837,11 @@ def load_detail(
         and target_type in BOOKMARKABLE_TYPES
         and item["uid"] in get_user_bookmarks(user["uid"], target_type, [item["uid"]])
     )
+    note_content = (
+        get_user_notes(user["uid"], target_type, [item["uid"]]).get(item["uid"])
+        if user and target_type in NOTABLE_TYPES
+        else None
+    )
     return {
         "item": item,
         "author": author,
@@ -841,6 +855,7 @@ def load_detail(
         "time_ago": time_ago(item["created_at"]),
         "reactions": reactions,
         "bookmarked": bookmarked,
+        "note_content": note_content,
         "poll": get_poll_for_post(item["uid"], user) if target_type == "post" else None,
         "war": war_store.get_war_serialized_for_post(item["uid"], user)
         if target_type == "post"

devplacepy/database/CLAUDE.md

@@ -124,7 +124,7 @@ Profile with the real database before and after any change here (`cProfile` arou
 
 Attachments (see "Profile media gallery and soft-deleted attachments" below) are one instance of a platform-wide model: **every removal is a soft delete; only garbage collection is a hard delete.** Every removable row carries two columns, `deleted_at` (ISO timestamp) and `deleted_by` (actor uid, or `system`). A live row has `deleted_at = NULL`, and every list/count read filters `deleted_at IS NULL`.
 
-- **Tables (`database.SOFT_DELETE_TABLES`):** posts, comments, gists, projects, news, news_images, project_files, attachments, votes, reactions, bookmarks, follows, poll_votes, polls, poll_options, sessions, instances, instance_schedules, devii_conversations, devii_tasks, devii_lessons, devii_virtual_tools, user_customizations, project_forks. `init_db` loops `ensure_soft_delete_columns(t)` for each (adds both columns + `idx_<t>_deleted` / the partial trash index described above). The Devii task/lesson/virtual-tool stores run on their own DB handles, so their `_ensure_indexes` adds the columns there too.
+- **Tables (`database.SOFT_DELETE_TABLES`):** posts, comments, gists, projects, news, news_images, project_files, attachments, votes, reactions, bookmarks, notes, follows, poll_votes, polls, poll_options, sessions, instances, instance_schedules, devii_conversations, devii_tasks, devii_lessons, devii_virtual_tools, user_customizations, project_forks. `init_db` loops `ensure_soft_delete_columns(t)` for each (adds both columns + `idx_<t>_deleted` / the partial trash index described above). The Devii task/lesson/virtual-tool stores run on their own DB handles, so their `_ensure_indexes` adds the columns there too.
 - **Every INSERT into a soft-deletable table MUST write `deleted_at: None, deleted_by: None`.** `dataset.find(deleted_at=None)` on a table that lacks the column matches NOTHING (a false predicate), silently hiding all rows - the born-live insert is what creates the column. Add the pair to any new insert.
 - **Central helpers (`database/`):** `soft_delete(table, deleted_by, *, stamp=None, **criteria)` (equality), `soft_delete_in(table, column, uids, deleted_by, *, stamp=None, **extra)` (IN-clause cascade), `restore(table, **criteria)`, `purge(table, **criteria)` (real delete), `list_deleted(table, page)` / `count_deleted(table)` (trash listings), and the event helpers `restore_event(stamp)` / `purge_event(stamp)` that act across ALL tables sharing one `deleted_at` stamp.
 - **Two generic chokepoints are conditionally filtered:** `resolve_by_slug(table, slug, include_deleted=False)` (detail-page lookups; restore passes `include_deleted=True`) and `paginate(table, ...)` (auto-appends `deleted_at IS NULL` when the table has the column and the caller did not pass `deleted_at`). `seo._collect` does the same for the sitemap. Read filters were threaded through every batch helper, analytics/activity/leaderboard UNION, feed/profile/listing route, and store; never re-introduce an unfiltered read of a soft-deletable table.

devplacepy/database/__init__.py

@@ -8,7 +8,7 @@ from .users import get_users_by_uids, _admins_cache, invalidate_admins_cache, ge
 from .relations import _relations_cache, get_user_relations, get_blocked_uids, get_muted_uids, get_silenced_uids, invalidate_user_relations
 from .pagination import PAGE_SIZE, paginate, interleave_by_author, paginate_diverse, get_user_post_count, clear_user_post_count, build_pagination
 from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns, soft_delete, soft_delete_in, restore, purge, list_deleted, count_deleted, restore_event, purge_event
-from .engagement import _comment_count_cache, get_comment_counts_by_post_uids, get_post_counts_by_user_uids, get_vote_counts, get_user_votes, get_reactions_by_targets, get_user_bookmarks, get_polls_by_post_uids, get_poll_for_post
+from .engagement import _comment_count_cache, get_comment_counts_by_post_uids, get_post_counts_by_user_uids, get_vote_counts, get_user_votes, get_reactions_by_targets, get_user_bookmarks, get_user_notes, get_polls_by_post_uids, get_poll_for_post
 from .usage import _add_usage, _get_usage, add_correction_usage, get_correction_usage, add_modifier_usage, get_modifier_usage, NEWS_USAGE_KEY, add_news_usage, get_news_usage, ISSUE_USAGE_KEY, add_issue_usage, get_issue_usage, SEO_USAGE_KEY, add_seo_usage, get_seo_usage, AWARD_USAGE_KEY, add_award_usage, get_award_usage
 from .awards import (
     AWARDS_PER_PAGE,
@@ -34,7 +34,7 @@ from .email import EMAIL_ACCOUNT_DEFAULTS, list_email_accounts, get_email_accoun
 from .notifications import NOTIFICATION_TYPES, NOTIFICATION_CHANNELS, _NOTIFICATION_CHANNEL_COLUMNS, _NOTIFICATION_CHANNEL_DEFAULTS, _NOTIFICATION_TYPE_KEYS, _notification_prefs_cache, _notification_default, get_notification_default, set_notification_default, _notification_overrides, notification_enabled, get_notification_prefs, set_notification_pref, reset_notification_prefs, mark_notifications_read_by_target
 from .forks import record_fork, get_fork_parent, count_forks, soft_delete_fork_relations, delete_fork_relations
 from .follows import get_follow_counts, get_follow_list, get_following_among
-from .deepsearch import _ds_now, create_deepsearch_session, update_deepsearch_session, get_deepsearch_session, add_deepsearch_message, get_deepsearch_messages, get_cached_deepsearch_url, upsert_deepsearch_url_cache
+from .deepsearch import _ds_now, create_deepsearch_session, update_deepsearch_session, get_deepsearch_session, list_deepsearch_sessions, add_deepsearch_message, get_deepsearch_messages, get_cached_deepsearch_url, upsert_deepsearch_url_cache
 from .ranking import VOTABLE_TARGETS, STAR_TARGETS, _authors_cache, _ranked_authors, _rank_map, get_top_authors, get_leaderboard, get_user_rank, get_user_stars, clear_user_stars, update_target_stars, soft_delete_engagement, delete_engagement, get_target_owner_uid
 from .moderation import (
     ACTIONS_TABLE,
@@ -157,6 +157,7 @@ __all__ = [
     "get_user_votes",
     "get_reactions_by_targets",
     "get_user_bookmarks",
+    "get_user_notes",
     "get_polls_by_post_uids",
     "get_poll_for_post",
     "_add_usage",
@@ -236,6 +237,7 @@ __all__ = [
     "create_deepsearch_session",
     "update_deepsearch_session",
     "get_deepsearch_session",
+    "list_deepsearch_sessions",
     "add_deepsearch_message",
     "get_deepsearch_messages",
     "get_cached_deepsearch_url",

devplacepy/database/deepsearch.py

@@ -54,6 +54,20 @@ def get_deepsearch_session(uid: str) -> dict | None:
     return get_table("deepsearch_sessions").find_one(uid=uid, deleted_at=None)
 
 
+def list_deepsearch_sessions(owner_kind: str, owner_id: str, limit: int = 50) -> list[dict]:
+    if "deepsearch_sessions" not in db.tables:
+        return []
+    return list(
+        get_table("deepsearch_sessions").find(
+            owner_kind=owner_kind,
+            owner_id=owner_id,
+            deleted_at=None,
+            order_by=["-created_at"],
+            _limit=limit,
+        )
+    )
+
+
 def add_deepsearch_message(
     uid: str, session_uid: str, role: str, content: str, citations: str = ""
 ) -> None:

devplacepy/database/engagement.py

@@ -116,6 +116,19 @@ def get_user_bookmarks(user_uid, target_type, target_uids):
     return {row["target_uid"] for row in rows}
 
 
+def get_user_notes(user_uid, target_type, target_uids):
+    if not user_uid or not target_uids or "notes" not in db.tables:
+        return {}
+    placeholders, params = _in_clause(target_uids)
+    params["u"] = user_uid
+    params["tt"] = target_type
+    rows = db.query(
+        f"SELECT target_uid, content FROM notes WHERE user_uid=:u AND target_type=:tt AND target_uid IN ({placeholders}) AND deleted_at IS NULL",
+        **params,
+    )
+    return {row["target_uid"]: row["content"] for row in rows}
+
+
 def get_polls_by_post_uids(post_uids, user=None):
     if not post_uids or "polls" not in db.tables:
         return {}

devplacepy/database/moderation.py

@@ -47,6 +47,7 @@ UNREPORTABLE_TABLES: dict[str, str] = {
     "votes": "engagement counters, carry no authored content",
     "reactions": "engagement counters, carry no authored content",
     "bookmarks": "private to the owner",
+    "notes": "private to the owner",
     "follows": "relationship rows, carry no authored content",
     "poll_votes": "private ballots",
     "opinion_war_fighters": "membership and damage counters, carry no authored content",

devplacepy/database/notifications.py

@@ -24,6 +24,7 @@ NOTIFICATION_TYPES = [
     {"key": "workspace", "label": "Dev workspaces", "description": "Idle, quota, retention and moderation notices for your dev workspaces"},
     {"key": "moderation", "label": "Moderation", "description": "Acknowledgements for reports you file and decisions taken on your content"},
     {"key": "system", "label": "System alerts", "description": "Platform infrastructure alerts (e.g. the AI gateway going down)"},
+    {"key": "ai_quota_warning", "label": "AI quota warnings", "description": "You're approaching your daily Devii AI usage limit"},
 ]
 
 

devplacepy/database/schema.py

@@ -179,6 +179,7 @@ def init_db():
         ("read", False),
         ("created_at", ""),
         ("updated_at", ""),
+        ("client_id", ""),
     ):
         if not messages.has_column(column):
             messages.create_column_by_example(column, example)
@@ -194,6 +195,7 @@ def init_db():
         ["receiver_uid", "sender_uid"],
     )
     _index(db, "messages", "idx_messages_updated_at", ["updated_at"])
+    _index(db, "messages", "idx_messages_dedupe", ["sender_uid", "client_id"])
     _index(db, "notifications", "idx_notifications_user", ["user_uid"])
     _index(db, "notifications", "idx_notifications_user_read", ["user_uid", "read"])
     push_registration = get_table("push_registration")
@@ -372,6 +374,8 @@ def init_db():
     )
     _index(db, "bookmarks", "idx_bookmarks_user", ["user_uid"])
     _index(db, "bookmarks", "idx_bookmarks_target", ["target_type", "target_uid"])
+    _index(db, "notes", "idx_notes_user", ["user_uid"])
+    _index(db, "notes", "idx_notes_target", ["target_type", "target_uid"])
     _index(db, "polls", "idx_polls_post", ["post_uid"])
     _index(db, "poll_options", "idx_poll_options_poll", ["poll_uid"])
     _index(db, "poll_votes", "idx_poll_votes_poll", ["poll_uid"])
@@ -1248,6 +1252,12 @@ def init_db():
     _index(db, "deepsearch_sessions", "idx_deepsearch_sessions_owner", ["owner_kind", "owner_id"])
     _index(db, "deepsearch_sessions", "idx_deepsearch_sessions_status", ["status"])
     _index(db, "deepsearch_sessions", "idx_deepsearch_sessions_created", ["created_at"])
+    _index(
+        db,
+        "deepsearch_sessions",
+        "idx_deepsearch_sessions_owner_created",
+        ["owner_kind", "owner_id", "created_at"],
+    )
 
     deepsearch_messages = get_table("deepsearch_messages")
     for column, example in (
@@ -2062,6 +2072,7 @@ def init_db():
     _index(db, "reactions", "idx_reactions_created_at", ["created_at"])
     _index(db, "votes", "idx_votes_created_at", ["created_at"])
     _index(db, "bookmarks", "idx_bookmarks_created_at", ["created_at"])
+    _index(db, "notes", "idx_notes_created_at", ["created_at"])
     _index(db, "badges", "idx_badges_created_at", ["created_at"])
     _index(db, "audit_log", "idx_audit_result_created", ["result", "created_at"])
     _index(db, "jobs", "idx_jobs_created_at", ["created_at"])
@@ -2281,6 +2292,10 @@ def backfill_api_keys() -> int:
         users.create_column_by_example("suspension_reason", "")
     if not users.has_column("deletion_requested_at"):
         users.create_column_by_example("deletion_requested_at", "")
+    if not users.has_column("active_conversation_uid"):
+        users.create_column_by_example("active_conversation_uid", "")
+    if not users.has_column("active_conversation_at"):
+        users.create_column_by_example("active_conversation_at", "")
     with db:
         db.query(
             "UPDATE users SET ai_modifier_enabled = 1 WHERE ai_modifier_enabled IS NULL"

devplacepy/database/soft_delete.py

@@ -16,6 +16,7 @@ SOFT_DELETE_TABLES = [
     "votes",
     "reactions",
     "bookmarks",
+    "notes",
     "follows",
     "poll_votes",
     "polls",

devplacepy/database/users.py

@@ -118,15 +118,18 @@ def search_users_by_username(q, *, exclude_uid=None, limit=10):
         return []
     if exclude_uid is not None:
         rows = db.query(
-            "SELECT uid, username FROM users WHERE username LIKE :q AND uid != :me LIMIT :limit",
+            "SELECT uid, username, avatar_seed FROM users WHERE username LIKE :q AND uid != :me LIMIT :limit",
             q=f"%{q}%",
             me=exclude_uid,
             limit=limit,
         )
     else:
         rows = db.query(
-            "SELECT uid, username FROM users WHERE username LIKE :q LIMIT :limit",
+            "SELECT uid, username, avatar_seed FROM users WHERE username LIKE :q LIMIT :limit",
             q=f"%{q}%",
             limit=limit,
         )
-    return [{"uid": r["uid"], "username": r["username"]} for r in rows]
+    return [
+        {"uid": r["uid"], "username": r["username"], "avatar_seed": r["avatar_seed"]}
+        for r in rows
+    ]

devplacepy/docs_api/_shared.py

@@ -3,6 +3,7 @@
 VOTE_TARGETS = ["post", "comment", "gist", "project", "quiz"]
 REACTION_TARGETS = ["post", "comment", "gist", "project", "quiz"]
 BOOKMARK_TARGETS = ["post", "gist", "project", "news", "quiz"]
+NOTE_TARGETS = ["post", "gist", "project", "news"]
 COMMENT_TARGETS = ["post", "project", "news", "issue", "gist", "quiz"]
 PROJECT_TYPES = ["game", "game_asset", "software", "mobile_app", "website"]
 GIST_LANGUAGES = [
@@ -38,6 +39,7 @@ def field(
     example="",
     description="",
     options=None,
+    nullable=False,
 ):
     spec = {
         "name": name,
@@ -46,6 +48,7 @@ def field(
         "required": required,
         "example": example,
         "description": description,
+        "nullable": nullable,
     }
     if options:
         spec["options"] = list(options)

devplacepy/docs_api/groups/content.py

@@ -62,6 +62,48 @@ four ways to sign requests.
                         "Search post title, content, and author username.",
                     ),
                     field("before", "query", "string", False, "", "Pagination cursor."),
+                    field(
+                        "items[].poll",
+                        "response",
+                        "object",
+                        description="Present only when the post has a poll attached.",
+                        nullable=True,
+                    ),
+                    field(
+                        "items[].war",
+                        "response",
+                        "object",
+                        description="Present only when an Opinion War is running on the post.",
+                        nullable=True,
+                    ),
+                    field(
+                        "items[].project_link",
+                        "response",
+                        "object",
+                        description="Present only when the post is attached to a project.",
+                        nullable=True,
+                    ),
+                    field(
+                        "items[].maturity",
+                        "response",
+                        "string",
+                        description="Content maturity label; absent for unrated posts.",
+                        nullable=True,
+                    ),
+                    field(
+                        "items[].my_vote",
+                        "response",
+                        "integer",
+                        description="Viewer's own vote on the post, defaults to 0 (never null).",
+                        nullable=False,
+                    ),
+                    field(
+                        "items[].comment_count",
+                        "response",
+                        "integer",
+                        description="Total comment count, defaults to 0 (never null).",
+                        nullable=False,
+                    ),
                 ],
             ),
             endpoint(
@@ -179,7 +221,49 @@ four ways to sign requests.
                         True,
                         "POST_SLUG",
                         "Slug or UID of the post.",
-                    )
+                    ),
+                    field(
+                        "post.title",
+                        "response",
+                        "string",
+                        description="Optional title, blank when the author posted without one.",
+                        nullable=True,
+                    ),
+                    field(
+                        "post.image",
+                        "response",
+                        "string",
+                        description="Cover image URL; null unless one was attached.",
+                        nullable=True,
+                    ),
+                    field(
+                        "post.project_uid",
+                        "response",
+                        "string",
+                        description="Attached project uid; null unless the post is linked to a project.",
+                        nullable=True,
+                    ),
+                    field(
+                        "post.updated_at",
+                        "response",
+                        "string",
+                        description="ISO timestamp of the last edit; null until the post is edited for the first time.",
+                        nullable=True,
+                    ),
+                    field(
+                        "post.content",
+                        "response",
+                        "string",
+                        description="Post body, always present (minimum 10 characters at creation).",
+                        nullable=False,
+                    ),
+                    field(
+                        "post.slug",
+                        "response",
+                        "string",
+                        description="Always generated at creation, never null.",
+                        nullable=False,
+                    ),
                 ],
             ),
             endpoint(

devplacepy/docs_api/groups/gateway.py

@@ -130,9 +130,17 @@ X-App-Reference: devplace-devii-v-1-0-0
 Administrators enable and configure this gateway under [Background Services](/docs/services.html)
 (the `openai` service).
 
-The gateway is exempt from rate limiting, but every other endpoint follows the shared
+The gateway is exempt from general rate limiting, but every other endpoint follows the shared
 [Conventions & Errors](/docs/conventions.html); see [Authentication](/docs/authentication.html)
 for signing DevPlace's own requests.
+
+A dedicated failed-authentication throttle protects the gateway from unauthenticated probing:
+an IP that repeatedly presents no credentials at all and fails authentication is answered `429`
+(with a `Retry-After` header) once it crosses a configurable threshold within a rolling window.
+This never affects a request that presents a valid key or session, even from an IP that has
+recently failed - a legitimate caller sharing a network address with a prior bad actor is never
+locked out. A `429` returned to a properly-authenticated call is a separate daily quota limit,
+not this throttle.
 """,
         "endpoints": [
             endpoint(

devplacepy/docs_api/groups/lookups.py

@@ -34,7 +34,13 @@ four ways to sign requests.
                     )
                 ],
                 sample_response={
-                    "results": [{"uid": "8f14e45f-...", "username": "alice_test"}]
+                    "results": [
+                        {
+                            "uid": "8f14e45f-...",
+                            "username": "alice_test",
+                            "avatar_seed": None,
+                        }
+                    ]
                 },
             ),
             endpoint(
@@ -54,7 +60,13 @@ four ways to sign requests.
                     )
                 ],
                 sample_response={
-                    "results": [{"uid": "0cc175b9-...", "username": "bob_test"}]
+                    "results": [
+                        {
+                            "uid": "0cc175b9-...",
+                            "username": "bob_test",
+                            "avatar_seed": None,
+                        }
+                    ]
                 },
             ),
         ],

devplacepy/docs_api/groups/profiles.py

@@ -65,6 +65,55 @@ four ways to sign requests.
                         "Profile tab.",
                         ["posts", "activity", "followers", "following", "media", "awards"],
                     ),
+                    field(
+                        "api_key",
+                        "response",
+                        "string",
+                        description="The profile's API key; null unless the viewer is the owner or an admin (see can_view_api_key).",
+                        nullable=True,
+                    ),
+                    field(
+                        "rank",
+                        "response",
+                        "integer",
+                        description="Leaderboard rank; null for a user with no stars yet.",
+                        nullable=True,
+                    ),
+                    field(
+                        "follow_pagination",
+                        "response",
+                        "object",
+                        description="Pagination metadata for the followers/following tabs; null on every other tab.",
+                        nullable=True,
+                    ),
+                    field(
+                        "profile_user.avatar_seed",
+                        "response",
+                        "string",
+                        description="Nullable users.avatar_seed override; null falls back to the username as the avatar seed.",
+                        nullable=True,
+                    ),
+                    field(
+                        "profile_user.bio",
+                        "response",
+                        "string",
+                        description="Optional profile bio; null when never set.",
+                        nullable=True,
+                    ),
+                    field(
+                        "is_following",
+                        "response",
+                        "boolean",
+                        description="Always a boolean, defaults to false, never null.",
+                        nullable=False,
+                    ),
+                    field(
+                        "profile_user.username",
+                        "response",
+                        "string",
+                        description="Always present, never null.",
+                        nullable=False,
+                    ),
                 ],
             ),
             endpoint(

devplacepy/docs_api/groups/push.py

@@ -20,9 +20,9 @@ its build. A registration body without a `provider` field is a `webpush` registr
 existing clients need no change. An APNs body may include a stable `client_id` so a later
 token rotation updates the same device instead of inserting another row.
 
-There is no server-side unsubscribe endpoint: unsubscription is handled entirely in the
+`DELETE /push.json` removes a single registration by the same identity used to create it
-browser by calling `PushManager.unsubscribe()` on the subscription. The server stops delivering
+(`endpoint` for webpush, `token` or `client_id` for apns). The server also stops delivering to a
-to a subscription once its push endpoint reports it as gone. These mirror the in-app
+subscription once its push endpoint reports it as gone. These mirror the in-app
 [Notifications](/docs/notifications.html) feed.
 
 Every endpoint follows the shared [Conventions & Errors](/docs/conventions.html) (auth, content
@@ -104,5 +104,56 @@ four ways to sign requests.
                 ],
                 sample_response={"registered": True, "delivered": True},
             ),
+            endpoint(
+                id="push-unregister",
+                method="DELETE",
+                path="/push.json",
+                title="Unregister a subscription",
+                summary="Remove one push registration by its identity.",
+                auth="user",
+                encoding="json",
+                interactive=False,
+                params=[
+                    field(
+                        "provider",
+                        "json",
+                        "string",
+                        False,
+                        "webpush",
+                        "Provider the registration belongs to. Omit for webpush.",
+                    ),
+                    field(
+                        "endpoint",
+                        "json",
+                        "string",
+                        False,
+                        "https://fcm.googleapis.com/...",
+                        "Subscription endpoint URL. Identifies a webpush registration.",
+                    ),
+                    field(
+                        "token",
+                        "json",
+                        "string",
+                        False,
+                        "a1b2c3...",
+                        "Hexadecimal device token. Identifies an apns registration.",
+                    ),
+                    field(
+                        "client_id",
+                        "json",
+                        "string",
+                        False,
+                        "vendor-uuid",
+                        "Stable per-device id. Identifies an apns registration when set.",
+                    ),
+                ],
+                notes=[
+                    "Exactly one identity field is required: `endpoint` for webpush, `token` or `client_id` for apns.",
+                    "Matches the same identity priority as registration: `client_id`, then `token`, then `endpoint`.",
+                    "Idempotent: unregistering an unknown or already-removed identity still returns 200 with `unregistered: false`.",
+                    "Call this before logging out to stop delivery to the device that is logging out - the server has no way to know a browser tab or native app closed on its own.",
+                ],
+                sample_response={"unregistered": True},
+            ),
         ],
 }

devplacepy/docs_api/groups/social_actions.py

@@ -1,18 +1,29 @@
 # retoor <retoor@molodetz.nl>
 
-from .._shared import BOOKMARK_TARGETS, REACTION_TARGETS, VOTE_TARGETS, endpoint, field
+from .._shared import (
+    BOOKMARK_TARGETS,
+    NOTE_TARGETS,
+    REACTION_TARGETS,
+    VOTE_TARGETS,
+    endpoint,
+    field,
+)
 from devplacepy.constants import REACTION_EMOJI
 
 GROUP = {
         "slug": "social-actions",
-        "title": "Votes, Reactions, Bookmarks & Polls",
+        "title": "Votes, Reactions, Bookmarks, Notes & Polls",
         "intro": """
-# Votes, Reactions, Bookmarks & Polls
+# Votes, Reactions, Bookmarks, Notes & Polls
 
-Lightweight engagement actions. The POST endpoints here are **toggles** - sending the same
+Lightweight engagement actions. The vote/reaction/bookmark POST endpoints here are **toggles** -
-action again removes it. They return JSON when called with `X-Requested-With: fetch` (sent
+sending the same action again removes it. They return JSON when called with
-automatically by the panels below); the [Conventions & Errors](/docs/conventions.html) page
+`X-Requested-With: fetch` (sent automatically by the panels below); the
-explains that header rule and the response envelope.
+[Conventions & Errors](/docs/conventions.html) page explains that header rule and the response
+envelope.
+
+Personal notes are private, per-user annotations attached to a piece of content - only you can
+ever see your own notes.
 
 Every endpoint follows the shared [Conventions & Errors](/docs/conventions.html) (auth, content
 negotiation, pagination, status codes); see [Authentication](/docs/authentication.html) for the
@@ -24,7 +35,7 @@ four ways to sign requests.
                 method="POST",
                 path="/votes/{target_type}/{target_uid}",
                 title="Cast or toggle a vote",
-                summary="Upvote or downvote a target. Re-sending the same value removes the vote.",
+                summary="Upvote or downvote a target. Re-sending the same value, or sending 0, removes the vote.",
                 auth="user",
                 ajax=True,
                 encoding="form",
@@ -52,8 +63,8 @@ four ways to sign requests.
                         "enum",
                         True,
                         "1",
-                        "1 to upvote, -1 to downvote.",
+                        "1 to upvote, -1 to downvote, 0 to retract your existing vote.",
-                        ["1", "-1"],
+                        ["1", "-1", "0"],
                     ),
                 ],
                 sample_response={"net": 3, "up": 4, "down": 1, "value": 1},
@@ -151,6 +162,97 @@ four ways to sign requests.
                     "Bookmarks target posts, projects, gists, and news; see [Posts, Comments, Projects, Gists & News](/docs/content.html)."
                 ],
             ),
+            endpoint(
+                id="notes-set",
+                method="POST",
+                path="/notes/{target_type}/{target_uid}",
+                title="Add or update a personal note",
+                summary="Save a private note on a target. Only you can ever see it.",
+                auth="user",
+                ajax=True,
+                encoding="form",
+                params=[
+                    field(
+                        "target_type",
+                        "path",
+                        "enum",
+                        True,
+                        "post",
+                        "Type of content to annotate.",
+                        NOTE_TARGETS,
+                    ),
+                    field(
+                        "target_uid",
+                        "path",
+                        "string",
+                        True,
+                        "POST_UID",
+                        "UID of the target.",
+                    ),
+                    field(
+                        "content",
+                        "form",
+                        "string",
+                        True,
+                        "Remember to check this later.",
+                        "Note body, up to 4000 characters. Sending again on the same target replaces the note.",
+                    ),
+                ],
+                sample_response={"uid": "NOTE_UID", "content": "Remember to check this later."},
+            ),
+            endpoint(
+                id="notes-delete",
+                method="POST",
+                path="/notes/{target_type}/{target_uid}/delete",
+                title="Delete a personal note",
+                summary="Remove your private note from a target.",
+                auth="user",
+                ajax=True,
+                encoding="none",
+                params=[
+                    field(
+                        "target_type",
+                        "path",
+                        "enum",
+                        True,
+                        "post",
+                        "Type of content the note is on.",
+                        NOTE_TARGETS,
+                    ),
+                    field(
+                        "target_uid",
+                        "path",
+                        "string",
+                        True,
+                        "POST_UID",
+                        "UID of the target.",
+                    ),
+                ],
+                sample_response={"deleted": True},
+            ),
+            endpoint(
+                id="notes-saved",
+                method="GET",
+                path="/notes/saved",
+                title="View your personal notes",
+                summary="Render your saved notes. Returns an HTML page.",
+                auth="user",
+                interactive=True,
+                params=[
+                    field(
+                        "before",
+                        "query",
+                        "string",
+                        False,
+                        "",
+                        "Pagination cursor (created_at of the last item).",
+                    )
+                ],
+                notes=[
+                    "Notes target posts, projects, gists, and news; see [Posts, Comments, Projects, Gists & News](/docs/content.html).",
+                    "Nobody else can ever see your notes, not even the author of the content you annotated.",
+                ],
+            ),
             endpoint(
                 id="polls-vote",
                 method="POST",

devplacepy/docs_api/groups/tools.py

@@ -233,6 +233,36 @@ status and report.
                     "export_pdf_url": "/tools/deepsearch/DEEPSEARCH_JOB_UID/export.pdf",
                 },
             ),
+            endpoint(
+                id="tools-deepsearch-history",
+                method="GET",
+                path="/tools/deepsearch/history",
+                title="My DeepSearch history",
+                summary="List the caller's own past DeepSearch runs, newest first, with a link back to each report/chat. Member history is account-bound; guest history is scoped to the visitor's address. Negotiates HTML or JSON.",
+                auth="public",
+                params=[
+                    field("limit", "query", "integer", False, "20", "Maximum sessions to return (1-100)."),
+                ],
+                sample_response={
+                    "sessions": [
+                        {
+                            "uid": "DEEPSEARCH_JOB_UID",
+                            "query": "history of the transistor",
+                            "status": "done",
+                            "score": 78,
+                            "confidence": 0.72,
+                            "page_count": 11,
+                            "chunk_count": 240,
+                            "summary": "The transistor was invented at Bell Labs in 1947...",
+                            "reopen_url": "/tools/deepsearch/DEEPSEARCH_JOB_UID/session",
+                            "chat_available": True,
+                            "available": True,
+                            "created_at": "2026-06-14T10:00:00+00:00",
+                            "completed_at": "2026-06-14T10:01:40+00:00",
+                        }
+                    ]
+                },
+            ),
             endpoint(
                 id="tools-isslop-run",
                 method="POST",

devplacepy/docs_api/negotiation.py

@@ -37,6 +37,7 @@ _PAGE_RESPONSES = {
     "issues-detail": schemas.IssueDetailOut,
     "issues-attachments-list": schemas.IssueAttachmentsOut,
     "bookmarks-saved": schemas.SavedOut,
+    "notes-saved": schemas.NotesOut,
     "admin-users": schemas.AdminUsersOut,
     "admin-news-list": schemas.AdminNewsOut,
     "admin-settings-get": schemas.AdminSettingsOut,

devplacepy/docs_export.py

@@ -31,8 +31,8 @@ def _params_table(params: list) -> str:
     if not params:
         return ""
     rows = [
-        "| Name | In | Type | Required | Description |",
+        "| Name | In | Type | Required | Nullable | Description |",
-        "|------|----|------|----------|-------------|",
+        "|------|----|------|----------|----------|-------------|",
     ]
     for p in params:
         desc = (p.get("description", "") or "").replace("|", "\\|")
@@ -43,7 +43,8 @@ def _params_table(params: list) -> str:
             desc = f"{desc} Allowed: {allowed}.".strip()
         rows.append(
             f"| `{p['name']}` | {p['location']} | {p['type']} | "
-            f"{'yes' if p['required'] else 'no'} | {desc} |"
+            f"{'yes' if p['required'] else 'no'} | "
+            f"{'yes' if p.get('nullable') else 'no'} | {desc} |"
         )
     return "\n".join(rows)
 

devplacepy/main.py

@@ -78,6 +78,7 @@ from devplacepy.routers import (
     reactions,
     reports,
     bookmarks,
+    notes,
     polls,
     docs,
     openai_gateway,
@@ -493,6 +494,7 @@ app.include_router(votes.router, prefix="/votes")
 app.include_router(reactions.router, prefix="/reactions")
 app.include_router(reports.router, prefix="/reports")
 app.include_router(bookmarks.router, prefix="/bookmarks")
+app.include_router(notes.router, prefix="/notes")
 app.include_router(polls.router, prefix="/polls")
 app.include_router(avatar.router, prefix="/avatar")
 app.include_router(awards.router, prefix="/awards")

devplacepy/models.py

@@ -258,6 +258,10 @@ class CommentEditForm(BaseModel):
     content: str = Field(min_length=3, max_length=125000)
 
 
+class NoteForm(BaseModel):
+    content: str = Field(min_length=1, max_length=4000)
+
+
 class ProjectForm(BaseModel):
     title: str = Field(min_length=1, max_length=200)
     description: str = Field(min_length=1, max_length=5000)
@@ -547,8 +551,8 @@ class VoteForm(BaseModel):
     @field_validator("value")
     @classmethod
     def valid_value(cls, value):
-        if value not in (1, -1):
+        if value not in (1, -1, 0):
-            raise ValueError("value must be 1 or -1")
+            raise ValueError("value must be 1, -1 or 0")
         return value
 
 

devplacepy/project_files.py

@@ -3,6 +3,7 @@
 import logging
 import os
 import shutil
+import time
 from datetime import datetime, timezone
 from pathlib import Path
 
@@ -18,6 +19,8 @@ MAX_FILES_PER_PROJECT = 5000
 MAX_PATH_LENGTH = 1024
 MAX_SEGMENT_LENGTH = 255
 MAX_DEPTH = 32
+_MISSING_BLOB_LOG_SECONDS = 60
+_missing_blob_logged: dict[str, tuple[int, float]] = {}
 
 TEXT_EXTENSIONS = {
     ".txt",
@@ -597,6 +600,7 @@ def sync_dir_bidirectional(project_uid: str, workspace, user: dict) -> dict:
     manifest = _load_sync_manifest(project_uid)
     new_manifest: dict = {}
     counts = dict(empty)
+    missing_blobs = 0
 
     for path in set(manifest) | set(db_files) | set(fs_files):
         row = db_files.get(path)
@@ -621,6 +625,8 @@ def sync_dir_bidirectional(project_uid: str, workspace, user: dict) -> dict:
                 if recorded:
                     counts["exported"] += 1
                     new_manifest[path] = recorded
+                elif row.get("is_binary"):
+                    missing_blobs += 1
             else:
                 recorded = _record_import(project_uid, user, path, fs_full, fs_epoch)
                 if recorded:
@@ -639,6 +645,8 @@ def sync_dir_bidirectional(project_uid: str, workspace, user: dict) -> dict:
                 if recorded:
                     counts["exported"] += 1
                     new_manifest[path] = recorded
+                elif row.get("is_binary"):
+                    missing_blobs += 1
             else:
                 _delete_db_row_for_sync(row, user["uid"])
                 counts["deleted_in_project"] += 1
@@ -670,6 +678,7 @@ def sync_dir_bidirectional(project_uid: str, workspace, user: dict) -> dict:
             continue
 
     _save_sync_manifest(project_uid, manifest, new_manifest)
+    _warn_missing_blobs(project_uid, missing_blobs, "while syncing")
     return counts
 
 
@@ -692,6 +701,34 @@ def _record_import(project_uid: str, user: dict, path: str, fs_full: Path, fs_ep
     return {"db_epoch": _epoch_of(imported.get("updated_at")), "fs_epoch": fs_epoch}
 
 
+def _copy_blob(src: Path, target: Path) -> bool:
+    try:
+        shutil.copyfile(src, target)
+        return True
+    except (FileNotFoundError, OSError):
+        return False
+
+
+def _warn_missing_blobs(project_uid: str, missing: int, action: str) -> None:
+    if missing <= 0:
+        return
+    now = time.monotonic()
+    previous = _missing_blob_logged.get(project_uid)
+    if (
+        previous is not None
+        and previous[0] == missing
+        and (now - previous[1]) < _MISSING_BLOB_LOG_SECONDS
+    ):
+        return
+    _missing_blob_logged[project_uid] = (missing, now)
+    logger.warning(
+        "Skipped %s missing blob file(s) %s project %s",
+        missing,
+        action,
+        project_uid,
+    )
+
+
 def _export_node(row: dict, dest: Path):
     target = (dest / row["path"]).resolve()
     if target != dest and not target.is_relative_to(dest):
@@ -701,10 +738,7 @@ def _export_node(row: dict, dest: Path):
         target.unlink()
     if row.get("is_binary") and row.get("stored_name") and row.get("directory"):
         src = PROJECT_FILES_DIR / row["directory"] / row["stored_name"]
-        try:
+        if not _copy_blob(src, target):
-            shutil.copyfile(src, target)
-        except (FileNotFoundError, OSError):
-            logger.warning("Blob file missing during export: %s", src)
             return None
     else:
         target.write_text(row.get("content") or "", encoding="utf-8")
@@ -853,6 +887,7 @@ def export_to_dir(project_uid: str, subpath: str, dest_dir) -> int:
         rows = list(_table().find(project_uid=project_uid, deleted_at=None))
         strip = ""
     written = 0
+    missing = 0
     for row in sorted(rows, key=lambda r: r["path"]):
         relative = row["path"][len(strip) :].lstrip("/") if strip else row["path"]
         target = (dest / relative).resolve()
@@ -866,14 +901,13 @@ def export_to_dir(project_uid: str, subpath: str, dest_dir) -> int:
             target.unlink()
         if row.get("is_binary") and row.get("stored_name") and row.get("directory"):
             src = PROJECT_FILES_DIR / row["directory"] / row["stored_name"]
-            try:
+            if not _copy_blob(src, target):
-                shutil.copyfile(src, target)
+                missing += 1
-            except (FileNotFoundError, OSError):
-                logger.warning("Blob file missing: %s", src)
                 continue
         else:
             target.write_text(row.get("content") or "", encoding="utf-8")
         written += 1
+    _warn_missing_blobs(project_uid, missing, "during export of")
     return written
 
 

devplacepy/push/CLAUDE.md

@@ -2,7 +2,7 @@ This file documents `devplacepy/push/` - push notification delivery and its prov
 
 ## What this package is
 
-One delivery library behind one public surface. `devplacepy.push` exports `notify_user`, `notify_registration`, `register`, `ensure_certificates`, `public_key_standard_b64` and the Web Push crypto helpers; every caller in the codebase (`main.py`, `utils/notifications.py`, `routers/push.py`) imports only those names. Everything else is internal to the package.
+One delivery library behind one public surface. `devplacepy.push` exports `notify_user`, `notify_registration`, `register`, `unregister`, `ensure_certificates`, `public_key_standard_b64` and the Web Push crypto helpers; every caller in the codebase (`main.py`, `utils/notifications.py`, `routers/push.py`) imports only those names. Everything else is internal to the package.
 
 | Module | Role |
 |---|---|
@@ -57,6 +57,14 @@ The default `delivery_client` is `stealth_async_client`. Override it only when t
 
 A match **updates** the row (token, client_id, environment) and clears `deleted_at` if it was dead. Token rotation with the same `client_id` therefore replaces the token on one row. Sibling live rows that share the new token, the previous token, or the same `client_id` are marked dead so one device cannot accumulate duplicates. A body without `client_id` still works: the same token revives, a new token inserts. `push.update` is a real update, not a no-op of an identical POST.
 
+## Unregistering
+
+`DELETE /push.json` (`routers/push.py` `push_unregister`) removes exactly the one registration named by the caller - never every registration for the user, mirroring how `POST /push.json` creates or updates exactly one row. `store.unregister(user_uid, provider, fields)` looks the row up with the **same identity priority as `register`**: `client_id`, then `token`, then `endpoint`. A match stamps `deleted_at` (the same bespoke soft-delete `mark_dead` already uses for a dead token - `push_registration` has no `deleted_by` and is not restorable from Trash by design, see "Invariants" above), so a later re-`register` with the same identity revives the row exactly like a dead token does, rather than accumulating a duplicate. No match (unknown identity, already-removed, or wrong owner) is a no-op that still returns `200 {"unregistered": false}` - the call is idempotent so a client can fire it speculatively at logout without first checking whether a subscription exists. The route validates that at least one identity field is a non-empty string before calling the store, refusing `400` on a body carrying none, exactly like `POST /push.json` refuses an unparseable registration body.
+
+**Client wiring.** `static/js/PushManager.js` intercepts every `a[href="/auth/logout"]` click, resolves the current `PushManager.getSubscription()` (only ever a webpush one - browser JS has no access to a native APNs device token), calls `DELETE /push.json` with `{endpoint}`, then `subscription.unsubscribe()` client-side, then navigates to the logout link's `href`. This closes the webpush half of the "token stays registered after logout" gap. **A native (iOS/Android) client integrating APNs must call `DELETE /push.json` with `{provider: "apns", token}` (or `client_id`) itself before or alongside its own logout** - there is no way for server-side `GET /auth/logout` (a plain, bodyless navigation) or this web frontend to know about, let alone unregister, a native app's device token.
+
+**No Devii action.** Unlike most mutating endpoints, `push.json` (register OR unregister) has no Devii catalog entry - a push identity (a webpush subscription endpoint/keys or an APNs device token) is private per-device browser/OS state that Devii cannot obtain, generate, or usefully ask the user to paste into a chat turn. This mirrors `POST /push.json` already having no Devii action for the identical reason.
+
 ## APNs specifics
 
 - **Persistent HTTP/2 connection, not one per notification.** `apns.cached_client(timeout)` lazily creates ONE module-level `httpx.AsyncClient(http2=True, ...)` and reuses it across every `notify_user`/`notify_registration` call for the lifetime of the process, following Apple's explicit guidance to keep the connection open rather than repeatedly opening/closing (`sending-notification-requests-to-apns`). `ApnsProvider.closes_delivery_client()` returns `False` so `delivery.py` never closes it after a batch (Web Push still opens/closes per call via `stealth_async_client`, `closes_delivery_client()` defaulting `True` on the base class). Closed once, gracefully, in `main.py`'s shutdown via `push.shutdown_providers()` -> `ApnsProvider.aclose()` -> `apns.close_client()`, mirroring the identical `services/containers/forward.py` `client()`/`close_client()` pattern. Not stealth, not curl_cffi Chrome impersonation, not the outbound proxy - Apple's provider API is a first-party HTTP/2 service; browser PRIORITY frames, HPACK indexing of `:path`/`authorization`, extra `sec-ch-ua` headers, and a scraping proxy all violate that contract. This is the second documented exception to the stealth-only outbound rule (the other is container reverse-proxy forwarding). If the admin-configured delivery timeout changes, the cached client is rebuilt with the new timeout on next use and the old one is left for GC (not explicitly closed) - a deliberate, rare-path simplification.

devplacepy/push/__init__.py

@@ -13,7 +13,7 @@ from devplacepy.push.providers.webpush import (
     hkdf,
     public_key_standard_b64,
 )
-from devplacepy.push.store import register
+from devplacepy.push.store import register, unregister
 
 __all__ = [
     "browser_base64",
@@ -29,4 +29,5 @@ __all__ = [
     "public_key_standard_b64",
     "register",
     "shutdown_providers",
+    "unregister",
 ]

devplacepy/push/store.py

@@ -179,6 +179,30 @@ def register(
     return RegistrationWrite(record, True, False)
 
 
+def unregister(user_uid: str, provider: str, fields: dict[str, Any]) -> bool:
+    fields = _filled(fields)
+    client_id = fields.get("client_id")
+    token = fields.get("token")
+    endpoint = fields.get("endpoint")
+
+    row = None
+    if client_id:
+        row = _lookup(user_uid, provider, client_id=client_id)
+    if row is None and token:
+        row = _lookup(user_uid, provider, token=token)
+    if row is None and endpoint:
+        row = _lookup(user_uid, provider, endpoint=endpoint)
+    if row is None or row.get("deleted_at"):
+        return False
+
+    table().update(
+        {"id": row["id"], "deleted_at": datetime.now(timezone.utc).isoformat()},
+        ["id"],
+    )
+    logger.info("Unregistered %s push subscription for user %s", provider, user_uid)
+    return True
+
+
 def mark_dead(registration_id: int, dead_before: str | None = None) -> None:
     if dead_before is not None:
         row = table().find_one(id=registration_id)

devplacepy/rendering.py

@@ -70,6 +70,26 @@ _TOKEN_RE = re.compile(
 _SCHEME_RE = re.compile(r"^([a-z][a-z0-9+.\-]*):", re.I)
 _ALLOWED_SCHEMES = {"http", "https", "mailto", "tel"}
 
+_TRAILING_PUNCT = ".,;:!?"
+_TRAILING_CLOSERS = {")": "(", "]": "[", "}": "{"}
+
+
+def _split_trailing_punct(url: str) -> tuple[str, str]:
+    trailing: list[str] = []
+    while url:
+        char = url[-1]
+        if char in _TRAILING_PUNCT:
+            trailing.append(char)
+            url = url[:-1]
+            continue
+        opener = _TRAILING_CLOSERS.get(char)
+        if opener is not None and url.count(char) > url.count(opener):
+            trailing.append(char)
+            url = url[:-1]
+            continue
+        break
+    return url, "".join(reversed(trailing))
+
 _YOUTUBE_ALLOW = (
     "accelerometer; autoplay; clipboard-write; encrypted-media; "
     "gyroscope; picture-in-picture"
@@ -112,6 +132,40 @@ _content_markdown = mistune.create_markdown(
     plugins=["strikethrough", "table"],
 )
 
+_structure_markdown = mistune.create_markdown(
+    renderer=None,
+    plugins=["strikethrough", "table"],
+)
+
+_STRUCTURE_TOKEN_KEYS = {
+    "block_code": "code_fences",
+    "list_item": "list_items",
+    "heading": "headers",
+    "link": "links",
+}
+
+
+def _count_structure_tokens(tokens, signature: dict[str, int]) -> None:
+    for token in tokens or []:
+        key = _STRUCTURE_TOKEN_KEYS.get(token.get("type"))
+        if key:
+            signature[key] += 1
+        children = token.get("children")
+        if children:
+            _count_structure_tokens(children, signature)
+
+
+def markdown_structure_signature(text: str) -> dict[str, int]:
+    signature = {key: 0 for key in _STRUCTURE_TOKEN_KEYS.values()}
+    if not text or not text.strip():
+        return signature
+    try:
+        tokens = _structure_markdown(text)
+    except (TypeError, ValueError):
+        return signature
+    _count_structure_tokens(tokens, signature)
+    return signature
+
 
 def _normalize_dashes(text: str) -> str:
     text = text.replace("\u2014", "-")
@@ -183,7 +237,10 @@ def _transform_text(text: str) -> str:
         if match.start() > pos:
             out.append(html.escape(_mask_emails(text[pos:match.start()])))
         if match.group("url"):
-            out.append(_embed_url(match.group("url")))
+            url, trailing = _split_trailing_punct(match.group("url"))
+            out.append(_embed_url(url))
+            if trailing:
+                out.append(html.escape(trailing))
         else:
             user = match.group("mention")
             out.append(
@@ -335,3 +392,21 @@ def content_preview(text, length: int = 60) -> str:
     if len(plain) <= length:
         return plain
     return plain[:length].rstrip() + "..."
+
+
+def safe_truncate(text, length: int = 300) -> str:
+    if not text:
+        return ""
+    text_str = str(text)
+    if len(text_str) <= length:
+        return text_str
+    cutoff = length
+    for match in _TOKEN_RE.finditer(text_str):
+        if match.start() < cutoff < match.end():
+            cutoff = match.start()
+            break
+    if cutoff > 0 and not text_str[cutoff - 1].isspace() and not text_str[cutoff].isspace():
+        boundary = text_str.rfind(" ", 0, cutoff)
+        if boundary > 0:
+            cutoff = boundary
+    return text_str[:cutoff].rstrip()

devplacepy/routers/CLAUDE.md

@@ -21,6 +21,7 @@ Prefixes are wired in `main.py`:
 | `/votes` | votes.py |
 | `/reactions` | reactions.py - emoji reaction toggle on any target: `POST /reactions/{target_type}/{target_uid}` (optimistic `ReactionBar`) |
 | `/bookmarks` | bookmarks.py - bookmark/favorite toggle: `GET /bookmarks/saved` (the viewer's saved list) and `POST /bookmarks/{target_type}/{target_uid}` (toggle a bookmark) |
+| `/notes` | notes.py - private per-user annotation on a target: `GET /notes/saved` (the viewer's personal notes list), `POST /notes/{target_type}/{target_uid}` (add/replace the note), `POST /notes/{target_type}/{target_uid}/delete` (remove it) |
 | `/polls` | polls.py - poll voting: `POST /polls/{poll_uid}/vote` |
 | `/avatar` | avatar.py |
 | `/follow` | follow.py |
@@ -39,8 +40,8 @@ Prefixes are wired in `main.py`:
 | `/devii` | devii.py - WebSocket terminal (`/devii/ws`), page, `/devii/usage`, `/devii/session` |
 | `/zips` | zips.py - generic zip job status (`/zips/{uid}`) and archive download (`/zips/{uid}/download`); enqueued from `/projects/{slug}/zip` and `/projects/{slug}/files/zip` |
 | `/forks` | forks.py - fork job status (`/forks/{uid}`); enqueued from `/projects/{slug}/fork`. When done its `project_url` points at the new forked project |
-| `/tools` | tools/ package - public developer tools surface. `index.py` (`/tools` landing) plus `seo.py` (**SEO Diagnostics**): `GET /tools/seo` page, `POST /tools/seo/run` (enqueue `seo` job, per-owner one-active-job cap), `GET /tools/seo/{uid}` (`SeoJobOut`), `GET /tools/seo/{uid}/report` (HTML+JSON `SeoReportOut`), `WS /tools/seo/{uid}/ws` (live progress, lock-owner gated, close `4013` retry), `GET /tools/seo/{uid}/screenshot/{n}`. Also `deepsearch.py` (**DeepSearch**): `GET /tools/deepsearch` page, `POST /tools/deepsearch/run` (enqueue `deepsearch` job, per-owner one-active-job cap; resolves the user `api_key` into the payload, guests use the internal key), `GET /tools/deepsearch/{uid}` (`DeepsearchJobOut`), `GET /tools/deepsearch/{uid}/session` (HTML+JSON `DeepsearchSessionOut`), `WS /tools/deepsearch/{uid}/ws` (live progress, lock-owner gated, `4013` retry), `WS /tools/deepsearch/{uid}/chat` (grounded RAG chat over the session collection), `POST /tools/deepsearch/{uid}/{pause|resume|cancel}`, `GET /tools/deepsearch/{uid}/export.{md,json,pdf}`. The shared owner helper is `routers/tools/_shared.py` `owner_for`. Also `isslop.py` (**AI Usage Analyzer**): `GET /tools/isslop` page, `POST /tools/isslop/run` (enqueue `isslop` job, per-owner one-active-job cap; owner = user uid or the shared `DEVII_GUEST_COOKIE` guest identity, minted when absent), `GET /tools/isslop/list` (owner history; a signed-in request first claims any guest-cookie analyses via `store.claim_guest_analyses` - a move, never a copy), `GET /tools/isslop/{uid}` (`IsslopAnalysisOut`), `GET /tools/isslop/{uid}/events` (persisted ordered event trail, `?after=SEQ` incremental poll; live frames also publish on pub/sub `public.isslop.{uid}` - the DB trail is the source of truth, pub/sub the fast path), `GET /tools/isslop/{uid}/report` (HTML+JSON `IsslopReportOut`; live `<dp-isslop-run>` while running, server-rendered report via `render_content` when done), `GET /tools/isslop/{uid}/report.md`, `GET /tools/isslop/{uid}/badge.svg` (embeddable SVG authenticity badge). Analyses/reports/badges are permanent public capability URLs (`IsslopService.cleanup` never deletes them; only the job row is swept). Surfaced by a collapsible **Tools** header dropdown (`base.html`, visible to all) |
+| `/tools` | tools/ package - public developer tools surface. `index.py` (`/tools` landing) plus `seo.py` (**SEO Diagnostics**): `GET /tools/seo` page, `POST /tools/seo/run` (enqueue `seo` job, per-owner one-active-job cap), `GET /tools/seo/{uid}` (`SeoJobOut`), `GET /tools/seo/{uid}/report` (HTML+JSON `SeoReportOut`), `WS /tools/seo/{uid}/ws` (live progress, lock-owner gated, close `4013` retry), `GET /tools/seo/{uid}/screenshot/{n}`. Also `deepsearch.py` (**DeepSearch**): `GET /tools/deepsearch` page, `POST /tools/deepsearch/run` (enqueue `deepsearch` job, per-owner one-active-job cap; resolves the user `api_key` into the payload, guests use the internal key), `GET /tools/deepsearch/history` (HTML+JSON `DeepsearchHistoryOut`, owner's own past sessions newest-first with a `reopen_url` back into the session/chat routes - `database.list_deepsearch_sessions`, the `deepsearch_sessions` table already persists past runs independently of job retention), `GET /tools/deepsearch/{uid}` (`DeepsearchJobOut`), `GET /tools/deepsearch/{uid}/session` (HTML+JSON `DeepsearchSessionOut`), `WS /tools/deepsearch/{uid}/ws` (live progress, lock-owner gated, `4013` retry), `WS /tools/deepsearch/{uid}/chat` (grounded RAG chat over the session collection, reconnectable on any completed session within its job's retention window), `POST /tools/deepsearch/{uid}/{pause|resume|cancel}`, `GET /tools/deepsearch/{uid}/export.{md,json,pdf}`. The shared owner helper is `routers/tools/_shared.py` `owner_for`. Also `isslop.py` (**AI Usage Analyzer**): `GET /tools/isslop` page, `POST /tools/isslop/run` (enqueue `isslop` job, per-owner one-active-job cap; owner = user uid or the shared `DEVII_GUEST_COOKIE` guest identity, minted when absent), `GET /tools/isslop/list` (owner history; a signed-in request first claims any guest-cookie analyses via `store.claim_guest_analyses` - a move, never a copy), `GET /tools/isslop/{uid}` (`IsslopAnalysisOut`), `GET /tools/isslop/{uid}/events` (persisted ordered event trail, `?after=SEQ` incremental poll; live frames also publish on pub/sub `public.isslop.{uid}` - the DB trail is the source of truth, pub/sub the fast path), `GET /tools/isslop/{uid}/report` (HTML+JSON `IsslopReportOut`; live `<dp-isslop-run>` while running, server-rendered report via `render_content` when done), `GET /tools/isslop/{uid}/report.md`, `GET /tools/isslop/{uid}/badge.svg` (embeddable SVG authenticity badge). Analyses/reports/badges are permanent public capability URLs (`IsslopService.cleanup` never deletes them; only the job row is swept). Reachable by direct URL only; the former **Tools** header dropdown in `base.html` was removed |
-| `/projects/{slug}/containers` | projects/containers/ subpackage - admin per-project container manager (`instances.py` for creation/lifecycle/exec/logs/metrics/sync plus the exec websocket, `schedules.py` for cron/interval/once schedules, shared helpers in `_shared.py`). Every instance runs the shared `ppy` image. Discoverable from the project detail page (admin-only **Containers** button) and from the admin index |
+| `/projects/{slug}/containers` | projects/containers/ subpackage - admin per-project container manager (`instances.py` for creation/lifecycle/exec/logs/metrics/sync plus the exec websocket, `schedules.py` for cron/interval/once schedules, shared helpers in `_shared.py`). Every instance runs the shared `ppy` image. Reachable by direct URL and from the admin index; the former project detail page **Containers** button was removed |
 | `/admin/containers` | admin/containers.py - admin **Containers** manager: `/admin/containers` lists every instance across all projects with inline actions (start/stop/restart/terminal/edit/delete) plus a create modal (project search-select, run-as user search-select, boot language + source editor, restart policy, start-on-boot, env/ports/limits/ingress); `/admin/containers/{uid}` is the per-instance detail page (lifecycle, live logs/metrics, PTY terminal, schedules, ingress, sync, status history) and `/admin/containers/{uid}/edit` edits run-as user, boot language/script/command, restart policy, start-on-boot, and limits. `POST /admin/containers/create`, `/{uid}/edit`, `/{uid}/{start,stop,restart,pause,resume}`, `/{uid}/sync`, `/{uid}/delete` call `api.*` directly under `require_admin` (no docker/exec backend duplicated); `GET /admin/containers/projects/search` and `/users/search` back the create/edit search-selects. The lifecycle and detail views stay layered over the per-project `/projects/{slug}/containers/instances/{uid}/...` endpoints (the instance carries its `project_uid`) |
 | `/p/{slug}` | proxy.py - public ingress reverse proxy (HTTP + WebSocket) to a running container instance's published host port, opt-in per instance via `ingress_slug` |
 | `/xmlrpc` | xmlrpc.py - reverse-proxies XML-RPC calls to the forking XML-RPC bridge (`services/xmlrpc/`, supervised by `XmlrpcService` on loopback `config.XMLRPC_PORT`), which generates one XML-RPC method per documented REST endpoint from `docs_api.API_GROUPS` (`posts.create`, `feed.list`, ...). One struct of named params per call; auth via in-band `api_key`, `X-API-KEY`/`Bearer` header, or `http://user:pass@host/xmlrpc` Basic. Full introspection + `system.multicall`; REST errors become XML-RPC faults. Exempt from the rate limiter (enforced on the forwarded internal hop). See `devplacepy/services/xmlrpc/CLAUDE.md` |
@@ -49,7 +50,7 @@ Prefixes are wired in `main.py`:
 | `/quizzes` | quizzes/ package - **Quizzes**: `index.py` (hub, create, import, detail, export, edit, delete, publish, per-quiz leaderboard, cross-quiz scoreboard), `questions.py` (builder page + question CRUD + reorder), `attempts.py` (start/resume, play, answer, finish, results). The hub `GET /quizzes` is the approved three-column `.feed-layout` reused verbatim from `/feed`: filters + search left (`?filter=all|todo|done|mine|drafts`, `?search=`, `?page=`), the quiz list centre with the **New quiz** and **Create quiz with Devii** actions, and the cross-quiz **scoreboard** rail right (score per user, best attempt per quiz, 15s display cache). Publishing is terminal - every write on a published quiz is a 400 and there is no unpublish route. See `devplacepy/services/quiz/CLAUDE.md` |
 | `/battles` | battles.py - **Opinion Wars**, week-long two-faction battles attached to posts: `GET ""` (listing, quizzes-style filters active/ended/mine + search), `GET /{uid}` (JSON-only full state, runs lazy resolution), `GET /{uid}/events?after=` (durable event trail replay), `POST /{uid}/join` (join or switch faction), `POST /{uid}/fight` (spend 25 Code Farm coins, deal level-weighted damage, 24h cooldown). The battle card renders on the parent post (no battle detail HTML page). See `devplacepy/services/opinionwar/CLAUDE.md` |
 | `/game` | game/ package - the **Code Farm** idle game (`index.py` + `farm.py`): `GET /game` (page), `GET /game/state`, `GET /game/leaderboard?board=`, `POST /game/{plant,harvest,buy-plot,upgrade,fertilize,daily,grant,perk,prestige,legacy,mastery,quests/claim}`, `POST /game/{defense/upgrade,defense/downgrade,infrastructure/buy,cosmetics/buy,cosmetics/equip}`, plus social `GET /game/farm/{username}` and `POST /game/farm/{username}/{water,steal}`. See `devplacepy/services/game/CLAUDE.md` |
-| (none) | push.py - push + PWA: `GET /push.json` (VAPID public key + the providers accepting registrations; `apns` includes `environment` when active), `POST /push.json` (register with any active provider; a body without `provider` is a `webpush` body; an APNs body is `token` plus optional `client_id` for device-stable upsert; created/revived rows are probed and the JSON may include `delivered`/`error`; an unknown, disabled or unconfigured provider is a 400), `GET /service-worker.js`, `GET /manifest.json`. Provider protocol and delivery live in `devplacepy/push/` - see `devplacepy/push/CLAUDE.md` |
+| (none) | push.py - push + PWA: `GET /push.json` (VAPID public key + the providers accepting registrations; `apns` includes `environment` when active), `POST /push.json` (register with any active provider; a body without `provider` is a `webpush` body; an APNs body is `token` plus optional `client_id` for device-stable upsert; created/revived rows are probed and the JSON may include `delivered`/`error`; an unknown, disabled or unconfigured provider is a 400), `DELETE /push.json` (unregister exactly the one registration named by `endpoint`/`token`/`client_id`, same identity priority as registration; idempotent, always 200 `{unregistered}`; wired into `PushManager.js`'s logout-link interceptor so a webpush subscription is dropped before the browser navigates to `/auth/logout`), `GET /service-worker.js`, `GET /manifest.json`. Provider protocol and delivery live in `devplacepy/push/` - see `devplacepy/push/CLAUDE.md` |
 | (none) | docs.py (docs/ package) - the documentation site (prose pages + API reference). See `routers/docs/CLAUDE.md` for the deep detail on this tree (`DOCS_PAGES`, audience tiers, prose rendering pipeline) |
 | `/pubsub` | pubsub.py - **database-free** publish/subscribe bus. `WS /pubsub/ws` (subscribe/unsubscribe/publish frames, `foo.*` wildcards), `POST /pubsub/publish` + `GET /pubsub/topics` (admin/internal). Lock-owner-gated WS (`4013` retry) so subscribers converge on one worker; topic authz in `services/pubsub/policy.py` (`user.{uid}.*` private, `public.*` shared, admin/internal anywhere, guests opt-in). In-process `services.pubsub.publish(topic, data)` for backends; frontend `app.pubsub`. See `devplacepy/services/pubsub/CLAUDE.md` |
 | (none) | seo.py - `/robots.txt`, `/sitemap.xml` |
@@ -92,6 +93,10 @@ The `comments` table uses `(target_type, target_uid)` so the same `_comment_sect
 
 Every post card on the feed has an inline comment form (`.feed-comment-form`) beneath the post actions. It posts to `/comments/create` like the detail page comment form. Tests must scope Post button clicks to `#create-post-modal button.btn-primary:has-text('Post')` to avoid matching the inline comment's Post button.
 
+### Comment permalinks
+
+Every comment's action bar (`_comment.html`) includes a "Copy link" button built on the existing `id="comment-{uid}"` anchor (already used by notifications, the profile Activity tab, and `NotificationManager.js`'s scroll-to-highlight) - no new route or URL scheme. It is a plain `data-share="#comment-{{ item.comment['uid'] }}"` button: `DomUtils.initShareButtons` resolves the relative hash against `window.location.href` at click time (so it always copies the exact page the viewer is on, canonical or not), copies it via `navigator.clipboard.writeText`, and flashes "Copied!" on the button (`Toast.flash`) - the same mechanism the gist detail page's Share button already uses.
+
 ### Comment editing
 
 A comment's owner (only the owner, never an admin) sees an inline "Edit" button (`data-action='edit'`) in `_comment.html`. `CommentManager.toggleEditForm` swaps the `.comment-text` for a textarea seeded from its `data-raw` attribute (the raw markdown, since `contentRenderer.applyTo` overwrites `textContent` on first render), posts via `Http.send` to `POST /comments/edit/{comment_uid}`, then re-renders the new body in place with `contentRenderer.applyTo`. The route (`content.edit_comment_record`) is `is_owner`-only, writes `content` + `updated_at`, records the `comment.edit` audit event, and branches on `wants_json`: JSON clients get `CommentEditOut{uid, content, url, updated_at}`, the no-JS form falls back to a redirect to the comment anchor. Edits are NOT soft-delete related (the body is overwritten in place). Devii tool: `edit_comment` (owner-only, no confirm). Scope test Edit clicks to `.comment-action-btn:has-text('Edit')`.
@@ -147,8 +152,9 @@ A dedicated `/gists` page for sharing code snippets. Uses `gists` table (auto-cr
 
 - Source code rendered in `<pre><code class="language-xxx">` block on detail page
 - Syntax highlighting handled by existing `highlight.js` loaded globally in `base.html`
-- Copy button uses `navigator.clipboard.writeText()`
+- Copy button uses `navigator.clipboard.writeText()` via the shared `data-copy` handler (`DomUtils.initClipboardCopy`)
 - Cards in listing show language badge, title, truncated description, author, star count
+- **Raw/rendered toggle for `language == "markdown"`:** unlike `"markdown_rendered"` (always rendered) and every other language (always raw), the plain `"markdown"` gist dual-renders server-side in `gist_detail.html` - the existing raw `<pre>` block plus a `render_content(gist['source_code'], ...)` block, the second one starting `hidden`. A `View rendered`/`View raw` button next to Copy uses the generic `data-view-toggle`/`data-view-toggle-alt` (+ the two `-label`/`-label-alt` pairs) attribute pair (`DomUtils.initViewToggles`, see `static/js/CLAUDE.md`) to swap the `hidden` class between the two blocks and its own label - no fetch, no new endpoint.
 
 ### Sitemap
 
@@ -317,6 +323,11 @@ All three columns are populated by `routers/posts.py` `post_page_context()`, the
 - `POST /bookmarks/{target_type}/{target_uid}` toggles a `bookmarks` row; `GET /bookmarks/saved` renders the personal list (`saved.html`). Target types: `post`, `gist`, `project`, `news`.
 - `_bookmark_button.html` takes `_type`, `_uid`, `_bookmarked`; `BookmarkManager.js` swaps the label/`bookmarked` class from the JSON `{saved}`. Batch state via `get_user_bookmarks(user_uid, target_type, uids)`.
 
+### Personal notes
+- A `notes` row is a private, per-user text annotation on a target (`post`, `gist`, `project`, `news` - the four detail pages, unlike bookmarks' listing-card coverage, since a note is read/written on the full content view, not skimmed from a card). `POST /notes/{target_type}/{target_uid}` (`NoteForm{content}`, max 4000 chars) creates or replaces the caller's own note on that target (revives a soft-deleted row rather than duplicating it, exactly like bookmarks); `POST /notes/{target_type}/{target_uid}/delete` soft-deletes it; `GET /notes/saved` renders the personal notes list (`notes.html`), mirroring `saved.html` but including each note's body.
+- There is no read/edit access for anyone but the author - the route only ever looks up `user_uid=user["uid"]`, so there is no "someone else's note" to view or moderate. `database/moderation.py` lists `notes` in `UNREPORTABLE_TABLES` ("private to the owner") for that reason.
+- `_note_button.html` takes `_type`, `_uid`, `_note` (the current content or `None`) and renders a button that opens a small inline textarea editor (not a modal - the body is short and the surrounding action bar has no room for a full dialog); `NoteManager.js` (`app.notes`) wires open/cancel/save/delete and swaps the button label/`has-note` class from the JSON `{content}` / `{deleted}` response, extending the shared `OptimisticAction` base like the other engagement controllers. Batch/single state via `get_user_notes(user_uid, target_type, uids)` (`database/engagement.py`), wired into `content.load_detail`/`detail_context` (post/gist/project) and `routers/news.py`'s detail route as `note_content` on the page context and the matching `*DetailOut` schema.
+
 ### Polls
 - A poll rides on a post (one `polls` row keyed by `post_uid`, options in `poll_options`, one-per-user votes in `poll_votes`). Created in `posts.py:create_poll` when `poll_question` plus >= 2 non-empty `poll_options` are submitted (capped at 6). Both `create_post` and `edit_post` accept the poll fields; `edit_post` only attaches a poll when the post has **none** yet (it never replaces an existing poll). The builders live in the create-post modal (`feed.html`) and the edit-post modal (`post.html`, rendered only when the post has no poll) using `data-poll-toggle` / `data-poll-add-option`.
 - `poll_options` accepts either repeated form fields (the web builders, which preserve commas inside an option label) **or** a single newline- or comma-separated string (the API/Devii path). `models.py:normalize_poll_options` splits a lone delimited element - applied as a `mode="before"` validator on `PostForm`/`PostEditForm` - so the documented "one per line or comma separated" agent format actually produces a multi-option poll instead of a single dropped option.

devplacepy/routers/admin/backups.py

@@ -1,5 +1,6 @@
 # retoor <retoor@molodetz.nl>
 
+import asyncio
 import logging
 from pathlib import Path
 from typing import Annotated
@@ -61,11 +62,12 @@ def _targets() -> list[dict]:
         for key, meta in store.BACKUP_TARGETS.items()
     ]
 
-def _dashboard(can_download: bool) -> dict:
+async def _dashboard(can_download: bool) -> dict:
     backups = [_backup_payload(row, can_download) for row in store.list_backups()]
     schedules = store.list_schedules()
+    storage = await asyncio.to_thread(store.compute_storage_stats)
     return {
-        "storage": store.compute_storage_stats(),
+        "storage": storage,
         "backups": backups,
         "schedules": schedules,
         "targets": _targets(),
@@ -77,7 +79,7 @@ def _dashboard(can_download: bool) -> dict:
 @router.get("/backups", response_class=HTMLResponse)
 async def admin_backups(request: Request):
     admin = require_admin(request)
-    data = _dashboard(is_primary_admin(admin))
+    data = await _dashboard(is_primary_admin(admin))
     base = site_url(request)
     seo_ctx = base_seo_context(
         request,
@@ -107,7 +109,7 @@ async def admin_backups(request: Request):
 @router.get("/backups/data")
 async def admin_backups_data(request: Request):
     admin = require_admin(request)
-    data = _dashboard(is_primary_admin(admin))
+    data = await _dashboard(is_primary_admin(admin))
     return JSONResponse(BackupDashboardOut.model_validate(data).model_dump(mode="json"))
 
 @router.post("/backups/run")

devplacepy/routers/devii.py

@@ -286,6 +286,7 @@ async def devii_ws(websocket: WebSocket):
                         }
                     )
                     continue
+                svc.maybe_warn_quota_threshold(owner_kind, owner_id, owner_is_admin)
                 session.spawn_turn(text)
             elif kind == "reset":
                 await session.reset()

devplacepy/routers/messages.py

@@ -39,6 +39,7 @@ from devplacepy.services.messaging import (
     persist_message,
     redeem_ticket,
     stamp_content_revision,
+    touch_active_conversation,
 )
 
 logger = logging.getLogger(__name__)
@@ -276,6 +277,7 @@ async def send_message(request: Request, data: Annotated[MessageForm, Depends(js
         data.attachment_uids,
         request=request,
         origin="web",
+        client_id=data.client_id,
     )
     if message is None:
         return action_result(request, "/messages")
@@ -457,6 +459,7 @@ async def messages_ws(websocket: WebSocket):
                         attachment_uids,
                         request=websocket,
                         origin="websocket",
+                        client_id=client_id,
                     )
                 except ContentRefused as exc:
                     await websocket.send_json(
@@ -483,6 +486,10 @@ async def messages_ws(websocket: WebSocket):
                     await message_hub.send_to_user(
                         receiver_uid, {"type": "typing", "from_uid": user_uid}
                     )
+            elif kind == "active":
+                with_uid = str(data.get("with_uid", "")).strip()
+                if with_uid:
+                    touch_active_conversation(user_uid, with_uid)
             elif kind == "read":
                 with_uid = str(data.get("with_uid", "")).strip()
                 if with_uid:

devplacepy/routers/news.py

@@ -13,6 +13,7 @@ from devplacepy.database import (
     get_news_images_by_uids,
     get_recent_comments_by_target_uids,
     get_user_bookmarks,
+    get_user_notes,
     paginate,
     resolve_object_url,
     mark_notifications_read_by_target,
@@ -125,6 +126,11 @@ async def news_detail_page(request: Request, news_slug: str):
     bookmarked = bool(user) and article["uid"] in get_user_bookmarks(
         user["uid"], "news", [article["uid"]]
     )
+    note_content = (
+        get_user_notes(user["uid"], "news", [article["uid"]]).get(article["uid"])
+        if user
+        else None
+    )
 
     base = site_url(request)
     page_url = f"{base}/news/{canonical_slug}"
@@ -157,6 +163,7 @@ async def news_detail_page(request: Request, news_slug: str):
             "time_ago": time_ago(article["synced_at"]),
             "comments": comments,
             "bookmarked": bookmarked,
+            "note_content": note_content,
             "maturity": get_maturity("news", article["uid"])["level"],
         },
         model=NewsDetailOut,

devplacepy/routers/notes.py

@@ -0,0 +1,188 @@
+# retoor <retoor@molodetz.nl>
+import logging
+from datetime import datetime, timezone
+from typing import Annotated
+from fastapi import APIRouter, Form, Request
+from fastapi.responses import RedirectResponse, JSONResponse, HTMLResponse
+from devplacepy.database import get_table, db, paginate, resolve_object_url, _now_iso
+from devplacepy.models import NoteForm
+from devplacepy.utils import generate_uid, require_user, time_ago, redirect_back
+from devplacepy.seo import base_seo_context
+from devplacepy.responses import respond
+from devplacepy.schemas import NotesOut
+from devplacepy.services.audit import record as audit
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+NOTABLE: set[str] = {"post", "gist", "project", "news"}
+
+TABLE_BY_TYPE: dict[str, str] = {
+    "post": "posts",
+    "gist": "gists",
+    "project": "projects",
+    "news": "news",
+}
+
+LABEL_BY_TYPE: dict[str, str] = {
+    "post": "Post",
+    "gist": "Gist",
+    "project": "Project",
+    "news": "Article",
+}
+
+
+@router.get("/saved", response_class=HTMLResponse)
+async def notes_page(request: Request, before: str = None):
+    user = require_user(request)
+    notes = get_table("notes")
+    rows, next_cursor = paginate(notes, before=before, user_uid=user["uid"])
+
+    uids_by_type: dict[str, list] = {}
+    for row in rows:
+        uids_by_type.setdefault(row["target_type"], []).append(row["target_uid"])
+
+    resolved: dict[tuple, dict] = {}
+    for target_type, uids in uids_by_type.items():
+        table_name = TABLE_BY_TYPE.get(target_type)
+        if not table_name or table_name not in db.tables:
+            continue
+        table = get_table(table_name)
+        clauses = [table.table.columns.uid.in_(uids)]
+        if table.has_column("deleted_at"):
+            clauses.append(table.table.columns.deleted_at.is_(None))
+        for obj in table.find(*clauses):
+            resolved[(target_type, obj["uid"])] = obj
+
+    items = []
+    for row in rows:
+        obj = resolved.get((row["target_type"], row["target_uid"]))
+        if not obj:
+            continue
+        title = obj.get("title") or (obj.get("content", "") or "")[:80] or "Untitled"
+        items.append(
+            {
+                "target_type": row["target_type"],
+                "type_label": LABEL_BY_TYPE.get(
+                    row["target_type"], row["target_type"].title()
+                ),
+                "title": title,
+                "url": resolve_object_url(row["target_type"], row["target_uid"]),
+                "content": row["content"],
+                "time_ago": time_ago(row.get("updated_at") or row["created_at"]),
+                "updated_at": row.get("updated_at"),
+            }
+        )
+
+    seo_ctx = base_seo_context(
+        request,
+        title="Notes",
+        description="Your personal notes on DevPlace.",
+        robots="noindex,nofollow",
+    )
+    return respond(
+        request,
+        "notes.html",
+        {
+            **seo_ctx,
+            "request": request,
+            "user": user,
+            "items": items,
+            "next_cursor": next_cursor,
+        },
+        model=NotesOut,
+    )
+
+
+@router.post("/{target_type}/{target_uid}")
+async def set_note(
+    request: Request,
+    target_type: str,
+    target_uid: str,
+    data: Annotated[NoteForm, Form()],
+):
+    user = require_user(request)
+    if target_type not in NOTABLE:
+        return JSONResponse({"error": "Invalid target"}, status_code=400)
+
+    notes = get_table("notes")
+    existing = notes.find_one(
+        user_uid=user["uid"], target_uid=target_uid, target_type=target_type
+    )
+    now = datetime.now(timezone.utc).isoformat()
+    content = data.content.strip()
+    if existing:
+        uid = existing["uid"]
+        notes.update(
+            {
+                "id": existing["id"],
+                "content": content,
+                "updated_at": now,
+                "deleted_at": None,
+                "deleted_by": None,
+            },
+            ["id"],
+        )
+    else:
+        uid = generate_uid()
+        notes.insert(
+            {
+                "uid": uid,
+                "user_uid": user["uid"],
+                "target_uid": target_uid,
+                "target_type": target_type,
+                "content": content,
+                "created_at": now,
+                "updated_at": now,
+                "deleted_at": None,
+                "deleted_by": None,
+            }
+        )
+
+    audit.record(
+        request,
+        "note.set",
+        user=user,
+        target_type=target_type,
+        target_uid=target_uid,
+        summary=f"{user['username']} saved a note on {target_type} {target_uid}",
+        links=[audit.target(target_type, target_uid)],
+    )
+
+    if request.headers.get("x-requested-with") == "fetch":
+        return JSONResponse({"uid": uid, "content": content})
+    return RedirectResponse(url=redirect_back(request), status_code=302)
+
+
+@router.post("/{target_type}/{target_uid}/delete")
+async def delete_note(request: Request, target_type: str, target_uid: str):
+    user = require_user(request)
+    if target_type not in NOTABLE:
+        return JSONResponse({"error": "Invalid target"}, status_code=400)
+
+    notes = get_table("notes")
+    existing = notes.find_one(
+        user_uid=user["uid"], target_uid=target_uid, target_type=target_type
+    )
+    if existing and not existing.get("deleted_at"):
+        notes.update(
+            {
+                "id": existing["id"],
+                "deleted_at": _now_iso(),
+                "deleted_by": user["uid"],
+            },
+            ["id"],
+        )
+        audit.record(
+            request,
+            "note.delete",
+            user=user,
+            target_type=target_type,
+            target_uid=target_uid,
+            summary=f"{user['username']} deleted a note on {target_type} {target_uid}",
+            links=[audit.target(target_type, target_uid)],
+        )
+
+    if request.headers.get("x-requested-with") == "fetch":
+        return JSONResponse({"deleted": True})
+    return RedirectResponse(url=redirect_back(request), status_code=302)

devplacepy/routers/projects/CLAUDE.md

@@ -11,7 +11,7 @@ Each project card links to `/projects/{project_uid}` showing full project detail
 
 **Project overview page.** The detail page is a dedicated project showcase: one encompassing dark card (`.project-shell`, the site `--bg-card` surface with clipped corners) wraps the hero, the section tab bar and the two-column body, and every inner panel (tab bar, sidebar cards, devlog post cards, empty state, comments section) sits one elevation lighter on `--bg-secondary`. The hero's cover banner is the attachment referenced by `projects.cover_attachment_uid`, falling back to the first image attachment (brand-gradient band when neither exists); the title block, type/platform chips and author row render OVERLAID on the banner behind a bottom scrim (dark text-shadow for readability) beside the optional `projects.logo_attachment_uid` tile, with an owner-set **Visit Website** CTA (`projects.website_url`). Cover and logo ride the ONE existing upload pipeline: `dp-upload` widgets (`name="cover_attachment_uid"`/`"logo_attachment_uid"`, `max-files="1"`) in the create/edit modals upload to `/uploads/upload`, the route validates each uid via `database.get_user_attachment` (must exist, belong to the actor, be an image - `_hero_attachment_uid`) and links it to the project through `attachments.link_attachments`; an empty value on edit keeps the current image (no removal control). `website_url`/`repo_url` are normalized by `models.normalize_website_url` (scheme-less input gets `https://`, non-http(s) rejected) and render with `rel="noopener nofollow"`. Below the hero an anchor **section tab bar** (`.project-tabs`, underline style, Overview `.active`) links `#about` / `#devlog` / `#screenshots` (only when gallery images exist) / `#comments` / the Files page - server-rendered anchors, no JS tab state. The main column holds **About** (description + non-image attachments), the **Devlog** (every post whose `project_uid` points at the project via `_post_card.html` - the template loads `feed.css` for the card styles alongside `post.css`, the same rule as `news.html`) with `devlog_count` (`content.count_project_devlog`) and an owner **Post update** button (`.project-devlog-post-btn`) opening the shared composer preset to `topic=devlog` + this project (the form lives ONCE in `templates/_post_composer_form.html`, locals `_composer_topic`/`_composer_project`, included by `feed.html` and `project_detail.html` - never fork a second copy), a **Screenshots** gallery (image attachments minus the cover/logo, thumbnails, `data-lightbox`, capped at 12 rendered), and the comment thread; the sidebar holds Links (website/repository/files/fork source), the Stats card (5 `.project-stat` entries + a last-update line) and the Author card. Owners add gallery images via the More-menu **Add screenshots** modal: `_attachment_form.html` uploads, then `POST /projects/{slug}/screenshots` (`ProjectScreenshotsForm`, owner-only, audit `project.screenshots.add`) links the uids through the same `link_attachments` choke point; Devii action `project_add_screenshots`, docs id `projects-screenshots`. `comment_count`/`devlog_count` ride `ProjectDetailOut`; the new project fields ride `ProjectOut`; the page og:image prefers the cover attachment. **Locator discipline:** the page has several `Files` anchors (action row, tab bar, sidebar) and, for owners, a second hidden `textarea[name='content']`/Post button inside the composer modal - tests MUST scope (`.project-detail-actions a:has-text('Files')`, `.comment-form textarea[name='content']`).
 
-**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Workspace, Containers, Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
+**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
 
 **Owner editing.** Mirrors post editing exactly: an owner-only **Edit** menu item (`data-modal="edit-project-modal"`) opens the `modal()` macro's `edit-project-modal`, a plain `POST` form to `/projects/edit/{slug}` (route `edit_project` in `routers/projects/index.py`, body `ProjectEditForm`, owner-gated through the shared `content.edit_content_item` which returns 403 JSON / redirect for non-owners and stamps `updated_at`). The modal is the create modal pre-filled from the `project` row (title, description, type/status radios pre-checked, dates via `format_date()` back to DD/MM/YYYY). `is_private`/`read_only` are NOT edited here - they stay on their dedicated toggles. The platforms tag widget reuses the create modal's `platforms-input`/`platforms`/`platforms-tags` ids; `ProfileEditor.initPlatformTags` now **seeds existing tags** from the hidden `#platforms` value on load, so both the empty create form and the pre-filled edit form work from the same code. Devii tool `edit_project`; documented in `docs_api.py` (`projects-edit`).
 

devplacepy/routers/push.py

@@ -91,6 +91,49 @@ async def push_register(request: Request) -> JSONResponse:
     return JSONResponse(payload)
 
 
+@router.delete("/push.json")
+async def push_unregister(request: Request) -> JSONResponse:
+    user = require_user_api(request)
+    try:
+        body = await request.json()
+    except ValueError:
+        return JSONResponse({"error": "Invalid JSON"}, status_code=400)
+
+    if not isinstance(body, dict):
+        return JSONResponse({"error": "Invalid request"}, status_code=400)
+
+    provider = providers.get(body.get("provider"))
+    if provider is None:
+        return JSONResponse({"error": "Unknown provider"}, status_code=400)
+
+    identity = {key: body.get(key) for key in ("client_id", "token", "endpoint")}
+    if not any(isinstance(value, str) and value.strip() for value in identity.values()):
+        return JSONResponse({"error": "Invalid request"}, status_code=400)
+
+    removed = push.unregister(user["uid"], provider.name, identity)
+
+    if removed:
+        endpoint = identity.get("endpoint")
+        audit.record(
+            request,
+            "push.unsubscribe",
+            user=user,
+            target_type="user",
+            target_uid=user["uid"],
+            target_label=user.get("username"),
+            metadata={
+                "provider": provider.name,
+                "endpoint_host": urlparse(endpoint).hostname
+                if isinstance(endpoint, str) and endpoint
+                else None,
+                "has_client_id": bool(identity.get("client_id")),
+            },
+            summary=f"{user.get('username')} unregistered a push subscription",
+            links=[audit.target("user", user["uid"], user.get("username"))],
+        )
+    return JSONResponse({"unregistered": removed})
+
+
 @router.get("/service-worker.js")
 async def service_worker() -> FileResponse:
     return FileResponse(

devplacepy/routers/seo.py

@@ -15,8 +15,8 @@ async def robots_txt(request: Request):
     return PlainTextResponse(
         f"""User-agent: *
 Disallow: /auth/
-Disallow: /messages/
+Disallow: /messages
-Disallow: /notifications/
+Disallow: /notifications
 Disallow: /votes/
 Disallow: /avatar/
 Disallow: /follow/
@@ -24,6 +24,7 @@ Disallow: /admin/
 Disallow: /uploads/
 Disallow: /reports/mine
 Disallow: /profile/*/delete
+Disallow: /game
 Disallow: /*?tab=
 Disallow: /*?sort=
 Allow: /static/

devplacepy/routers/tools/deepsearch.py

@@ -11,7 +11,7 @@ from devplacepy import database
 from devplacepy.config import DEEPSEARCH_DIR
 from devplacepy.models import DeepsearchChatForm, DeepsearchRunForm
 from devplacepy.responses import respond
-from devplacepy.schemas import DeepsearchJobOut, DeepsearchSessionOut
+from devplacepy.schemas import DeepsearchHistoryOut, DeepsearchJobOut, DeepsearchSessionOut
 from devplacepy.seo import base_seo_context, site_url, web_application_schema, website_schema
 from devplacepy.services.deepsearch.chat import DeepsearchChat
 from devplacepy.services.deepsearch.export import to_json, to_markdown, to_pdf
@@ -197,6 +197,52 @@ def _enqueue(uid: str, payload: dict, owner_kind: str, owner_id: str, query: str
         }
     )
 
+def _history_item(row: dict) -> dict:
+    uid = row.get("uid", "")
+    status = row.get("status", "")
+    job = queue.get_job(uid)
+    return {
+        "uid": uid,
+        "query": row.get("query"),
+        "status": status,
+        "score": row.get("score"),
+        "confidence": row.get("confidence"),
+        "source_diversity": row.get("source_diversity"),
+        "page_count": int(row.get("page_count") or 0),
+        "chunk_count": int(row.get("chunk_count") or 0),
+        "summary": row.get("summary") or None,
+        "reopen_url": f"/tools/deepsearch/{uid}/session",
+        "chat_available": status == "done" and job is not None,
+        "available": job is not None,
+        "created_at": row.get("created_at"),
+        "completed_at": row.get("completed_at") or None,
+    }
+
+@router.get("/history")
+async def deepsearch_history(request: Request, limit: int = 20):
+    owner_kind, owner_id = owner_for(request)
+    user = get_current_user(request)
+    rows = database.list_deepsearch_sessions(owner_kind, owner_id, min(max(1, limit), 100))
+    sessions = [_history_item(row) for row in rows]
+    seo_ctx = base_seo_context(
+        request,
+        title="DeepSearch History",
+        description="Past DeepSearch research runs, with links back to each report and its grounded chat.",
+        robots="noindex,nofollow",
+        breadcrumbs=[
+            {"name": "Home", "url": "/feed"},
+            {"name": "Tools", "url": "/tools"},
+            {"name": "DeepSearch", "url": "/tools/deepsearch"},
+            {"name": "History", "url": "/tools/deepsearch/history"},
+        ],
+    )
+    return respond(
+        request,
+        "tools/deepsearch_history.html",
+        {**seo_ctx, "request": request, "user": user, "sessions": sessions},
+        model=DeepsearchHistoryOut,
+    )
+
 @router.get("/{uid}")
 async def deepsearch_status(request: Request, uid: str):
     job = queue.get_job(uid)

devplacepy/schemas/__init__.py

@@ -43,6 +43,8 @@ from devplacepy.schemas.listings import (
     NewsDetailOut,
     NewsListItemOut,
     NewsListOut,
+    NoteItemOut,
+    NotesOut,
     NotificationGroupOut,
     NotificationItemOut,
     NotificationsOut,
@@ -88,6 +90,8 @@ from devplacepy.schemas.containers import (
 )
 from devplacepy.schemas.jobs import (
     DbQueryJobOut,
+    DeepsearchHistoryItemOut,
+    DeepsearchHistoryOut,
     DeepsearchJobOut,
     DeepsearchSessionOut,
     ForkJobOut,

devplacepy/schemas/jobs.py

@@ -143,6 +143,27 @@ class DeepsearchSessionOut(_Out):
     completed_at: Optional[str] = None
 
 
+class DeepsearchHistoryItemOut(_Out):
+    uid: str = ""
+    query: Optional[str] = None
+    status: str = ""
+    score: Optional[int] = None
+    confidence: Optional[float] = None
+    source_diversity: Optional[float] = None
+    page_count: int = 0
+    chunk_count: int = 0
+    summary: Optional[str] = None
+    reopen_url: Optional[str] = None
+    chat_available: bool = False
+    available: bool = False
+    created_at: Optional[str] = None
+    completed_at: Optional[str] = None
+
+
+class DeepsearchHistoryOut(_Out):
+    sessions: list = []
+
+
 class DbQueryJobOut(_Out):
     uid: str = ""
     kind: str = ""

devplacepy/schemas/listings.py

@@ -152,6 +152,7 @@ class PostDetailOut(_Out):
     attachments: list[AttachmentOut] = []
     reactions: ReactionsOut = ReactionsOut()
     bookmarked: bool = False
+    note_content: Optional[str] = None
     poll: Optional[PollOut] = None
     war: Optional[WarOut] = None
     comment_count: Optional[int] = None
@@ -187,6 +188,7 @@ class ProjectDetailOut(_Out):
     attachments: list[AttachmentOut] = []
     reactions: ReactionsOut = ReactionsOut()
     bookmarked: bool = False
+    note_content: Optional[str] = None
     platforms: Optional[Any] = None
     is_private: bool = False
     read_only: bool = False
@@ -227,6 +229,7 @@ class GistDetailOut(_Out):
     attachments: list[AttachmentOut] = []
     reactions: ReactionsOut = ReactionsOut()
     bookmarked: bool = False
+    note_content: Optional[str] = None
 
 
 class NewsListOut(_Out):
@@ -243,6 +246,7 @@ class NewsDetailOut(_Out):
     time_ago: Optional[str] = None
     comments: list[CommentItemOut] = []
     bookmarked: bool = False
+    note_content: Optional[str] = None
 
 
 class MessagesOut(_Out):
@@ -275,3 +279,19 @@ class SavedOut(_Out):
     items: list[SavedItemOut] = []
     next_cursor: Optional[str] = None
 
+
+class NoteItemOut(_Out):
+    target_type: Optional[str] = None
+    target_uid: Optional[str] = None
+    type_label: Optional[str] = None
+    title: Optional[str] = None
+    url: Optional[str] = None
+    content: Optional[str] = None
+    time_ago: Optional[str] = None
+    updated_at: Optional[str] = None
+
+
+class NotesOut(_Out):
+    items: list[NoteItemOut] = []
+    next_cursor: Optional[str] = None
+

devplacepy/services/CLAUDE.md

@@ -27,9 +27,10 @@ Generic, lightweight, **fire-and-forget** offload for non-critical side-effects
 - **Choke entrypoint.** `schedule_correction(user, table, uid, request=None)` is the only thing handlers call (every hook passes `request`). It is a no-op unless: a user dict is present, `table` is in the registry, `user["ai_correction_enabled"]` is truthy, and the user has a non-empty `api_key`. In **sync** mode (`user["ai_correction_sync"]`) it calls `_run_inline_awaited`, which - only when a `request.scope` and a running event loop exist - submits `_run_correction` to `loop.run_in_executor(AI_APPLY_EXECUTOR, ...)` (off the loop thread, on the dedicated AI pool) and stashes the future on `request.scope[PENDING_SCOPE_KEY]` for the middleware to await; if there is no request/loop it returns False and falls back to background. In **background** mode (default) it `background.submit`s `_run_correction` (per-worker queue; also runs inline under `DEVPLACE_DISABLE_SERVICES=1`). The same `_run_correction` worker function runs in all cases.
 - **The hooks (covers UI + REST + Devii + devRant in one place):** `content.create_content_item` (posts/projects/gists), `content.edit_content_item`, `content.create_comment_record`, `content.edit_comment_record`, `services/messaging/persist.persist_message` (DMs - sender is the user), `routers/profile/index.update_profile` (bio), and the two devRant direct-update edit paths (`routers/devrant/rants.edit_rant`, `routers/devrant/comments.edit_comment`). devRant create paths route through the shared content cores, so they are already hooked. Code-only and external paths (project files, Gitea) are deliberately not hooked.
 - **The gateway call is fail-soft.** `correct_text(api_key, prompt, text)` is synchronous (runs on the background worker thread), POSTs to `INTERNAL_GATEWAY_URL` with `model=correction_model()` (`get_setting("correction_model", "") or INTERNAL_MODEL` - admin-configurable at `/admin/settings`, blank falls back to the gateway default `molodetz`; the gateway URL itself is never configurable per feature, always `INTERNAL_GATEWAY_URL`) via `stealth.stealth_sync_client`, authenticated with the user's own `Bearer` api_key (per-user attribution). It returns the ORIGINAL text on any error, empty output, or suspiciously large output (`len > len(text) * MAX_GROWTH_FACTOR + 200`, rejecting hallucinated expansion). `_run_correction` reads the row back, corrects each registry field that has non-blank text, writes only changed fields via `table.update(updates, ["uid"])` without touching `updated_at` (auto-correction is not a user edit) or the slug (slugs are permanent), and `clear_user_cache(user_uid)` when `table == "users"` so the corrected bio re-caches.
+- **Structural markdown-preservation guard (issue #84).** The growth-factor check alone never caught a correction that kept a similar length while stripping/rewrapping markdown structure (a flattened list, a dropped code fence, a removed header, a stripped link) - correction is prompt-only ("preserve... markdown") with zero enforcement otherwise. `gateway_complete(..., check_structure=False)` gained an opt-in structural check, wired on ONLY from `correct_text` (`check_structure=True`), that runs right after the growth-factor check: `rendering.markdown_structure_signature(text)` parses the text through a dedicated mistune AST pipeline (`_structure_markdown = mistune.create_markdown(renderer=None, plugins=["strikethrough", "table"])`, matching `_content_markdown`'s plugin set) and counts four structural signals recursively over the token tree - `code_fences` (`block_code`), `list_items` (`list_item`, ordered+unordered together), `headers` (`heading`, all levels together), `links` (`link`). `correction.structure_diverges(original, corrected)` compares the two signatures per key with `STRUCTURE_DROP_TOLERANCE = 1`: a count that drops from >0 to exactly 0 always trips it (total loss of a structural element type is the corruption signature - a single header/link/fence is as real as five), any other decrease bigger than the 1-item tolerance also trips it (catches a 5-item list collapsed to 1, not just to 0), and any increase or a decrease of at most 1 passes (tolerates a single word fixed inside a list item, or two adjacent items reflowed into one, without ever changing the actual count of an unrelated structural family). `gateway_complete` logs and returns the ORIGINAL text on divergence, exactly like the growth-factor rejection - no partial write, no exception. `ai_modifier.py`'s `modify_text` intentionally does NOT set `check_structure` - its whole job is to genuinely restructure content per an explicit `@ai` instruction (e.g. "@ai turn this into a numbered list"), so a structural-divergence check there would reject the very thing the user asked for; distinguishing "restructuring the user asked for" from "restructuring elsewhere in the same field the instruction didn't target" is not reliably automatable from a signature diff alone, so the guard is scoped to `correct_text` only, which never intentionally restructures.
 - **Per-user usage aggregation (only on success).** `correct_text` returns `(text, usage)`; `usage` is parsed from the gateway's `X-Gateway-*` response headers (`_usage_from_headers`) whenever the upstream call returned 200 (cost was incurred, even if the corrected output was rejected), else `None` on any failure. `_usage_from_headers` captures the token and cost headers PLUS the timing headers `X-Gateway-Upstream-Latency-Ms` and `X-Gateway-Total-Latency-Ms` (as `upstream_latency_ms`/`total_latency_ms`), so each call's timing is metered. `_run_correction` accumulates the per-field `usage` into one `totals` dict and, when `totals["calls"] > 0`, makes ONE call to `database.add_correction_usage(user_uid, totals)` (a single `totals` dict, not positional args) - so a 2-field content item is a single aggregated write, and a failed/empty correction records nothing. `add_correction_usage` (and `add_modifier_usage`) delegate to the shared `database._add_usage(usage_table, user_uid, totals)`: a single atomic `INSERT ... ON CONFLICT(user_uid) DO UPDATE SET col = col + excluded.col` upsert against the `correction_usage` table (per-user running SUMS: `calls`/`prompt_tokens`/`completion_tokens`/`total_tokens`/`cost_usd`/`upstream_latency_ms`/`total_latency_ms`/`updated_at`, unique index `idx_correction_usage_user`, the two latency columns REAL default 0.0, all ensured in `init_db`). It is a derived counter table (NOT in `SOFT_DELETE_TABLES`, like `gateway_usage_ledger`) and is deliberately separate from `users` so accumulating never invalidates the auth/user cache. `database.get_correction_usage(user_uid)` (via `_get_usage`) returns the stored sums PLUS computed averages: `avg_tokens` (total_tokens/calls), `avg_upstream_latency_ms`, `avg_total_latency_ms`, `avg_tokens_per_second` (completion_tokens over total upstream seconds), and `avg_cost_usd`.
 - **Profile display:** `routers/profile/usage._correction_usage(uid, include_cost)` shapes it via the shared `_usage_view(data, include_cost)` (mirrors `_ai_quota`); `profile/index.py` builds it only for `is_owner or viewer_is_admin` and passes `include_cost=viewer_is_admin`, exposed as the `correction_usage` dict on the context and `ProfileOut`. The view surfaces the sums plus averages - `avg_tokens` (avg tokens/call), `avg_latency_ms` (avg upstream latency), `avg_total_latency_ms`, `avg_tokens_per_second` (avg speed), and `total_time_s` (total upstream seconds) - rendered as extra tiles on the card. **Financial gating:** tokens/call-count and the performance tiles show to the owner and admins; the dollar `cost_usd` and `avg_cost_usd` keys are present ONLY when `viewer_is_admin`, in BOTH the HTML card (`templates/profile.html`, `.correction-usage-*`) and the `respond(..., model=ProfileOut)` JSON (same rule as `_ai_quota`'s `spent_usd` - hiding it in the template alone would leak it to a member fetching their own profile as JSON). The card renders only when `correction_usage.calls` > 0.
-- **Import-cycle discipline.** `correction.py` imports only `stealth`, `config`, `database.get_table`/`add_correction_usage`, and `services.background.background` at module top; `clear_user_cache` is imported lazily inside `_run_correction`. Never import `content` or `utils` at module top.
+- **Import-cycle discipline.** `correction.py` imports only `stealth`, `config`, `database.get_table`/`add_correction_usage`, `rendering.markdown_structure_signature` (structural guard above - `rendering.py` has zero internal imports, so this adds no cycle risk), and `services.background.background` at module top; `clear_user_cache` is imported lazily inside `_run_correction`. Never import `content` or `utils` at module top.
 - **Settings live on `users`:** three columns `ai_correction_enabled` (0/1), `ai_correction_sync` (0/1, default 0 = background), and `ai_correction_prompt` (text, default `config.DEFAULT_CORRECTION_PROMPT`), ensured in `database.backfill_api_keys()` (the user column-ensure block run by `init_db`) and seeded born-live in `utils._create_account`. The edit route is the owner-or-admin leaf `POST /profile/{username}/ai-correction` (`routers/profile/ai_correction.py`, `AiCorrectionForm{enabled, sync, prompt}`, audit key `profile.ai_correction`). The owner-only values are exposed on the profile page context and `ProfileOut` (`ai_correction_enabled`/`ai_correction_sync`/`ai_correction_prompt`, gated by `is_owner`), the UI block lives in `profile.html` (owner-only: enable checkbox, **Apply mode** select, prompt textarea) wired by `static/js/AiCorrection.js` (`app.aiCorrection`), and Devii drives it via the owner-scoped `ai_correction_get`/`ai_correction_set` tools (`services/devii/ai_correction/`, `handler="ai_correction"`, `requires_auth=True`, not confirm-gated - it is a reversible per-user toggle; `ai_correction_set` accepts `enabled`, optional `sync`, optional `prompt`).
 
 ## AI modifier (`services/ai_modifier.py`, `services/ai_context.py`)

devplacepy/services/ai_modifier.py

@@ -43,7 +43,13 @@ def modify_text(
             + context
         )
     return gateway_complete(
-        api_key, system, text, MODIFIER_TIMEOUT_SECONDS, None, model=modifier_model()
+        api_key,
+        system,
+        text,
+        MODIFIER_TIMEOUT_SECONDS,
+        None,
+        model=modifier_model(),
+        bypass_preamble=True,
     )
 
 

devplacepy/services/backup/CLAUDE.md

@@ -12,14 +12,14 @@ Admin-only, enterprise-grade backups built on the **same async-job pattern as zi
 ## Storage and data model
 
 - **Storage:** archives go under `config.BACKUPS_DIR` (`data/backups/`, in `DATA_PATHS`) sharded with `attachments._directory_for` on the **random uuid tail** (same load-bearing reason as zips/blobs), named `{target}-{YYYYMMDD-HHMMSS}-{tail}.tar.gz`. Staging is `config.BACKUP_STAGING_DIR` (`data/backup_staging/`), removed in `process` `finally`.
-- **Data model** (`store.py`, ensured in `database.init_db` via `backup_store.ensure_tables()`): `backups` (NOT soft-deletable - an archive is a reclaimable operational artifact, hard-deleted like zips) and `backup_schedules` (in `SOFT_DELETE_TABLES`, born-live `deleted_at:None`). `store` holds all CRUD plus `compute_storage_stats()` (du of every major data area + `shutil.disk_usage`, run in `asyncio.to_thread` from the route, 30s in-process TTL cache so the walk never blocks).
+- **Data model** (`store.py`, ensured in `database.init_db` via `backup_store.ensure_tables()`): `backups` (NOT soft-deletable - an archive is a reclaimable operational artifact, hard-deleted like zips) and `backup_schedules` (in `SOFT_DELETE_TABLES`, born-live `deleted_at:None`). `store` holds all CRUD plus two storage helpers that must never be confused: `disk_usage()` is an O(1) `shutil.disk_usage` of `DATA_DIR` (15s TTL) and is the only thing the service tick may call; `compute_storage_stats()` does a **single** `os.walk` of `DATA_DIR` (overlapping path buckets, 600s TTL, in-process lock so callers cannot stampede), and is run in `asyncio.to_thread` from the admin route and the live-view relay. **Never call `compute_storage_stats()` on the event loop.** A 30s cache with a walk that itself takes >=30s never hits, pins one core in `pathlib.rglob` / `is_symlink`, stops `accept()`, and the reverse proxy returns 502. That was a production outage.
 - **Permanent artifact:** `cleanup(job)` only removes leftover staging, NEVER the archive. Job retention prunes the `jobs` row; the archive and `backups` row persist until an admin deletes it, a schedule rotates it out (`keep_last`), or `devplace backups clear`. Deleting a backup is a HARD delete (unlink file + delete row) - correct because backups are GC artifacts, the documented exception to the soft-delete rule.
 
 ## Remote offload
 
 `devplacepy/services/backup/offload.py` ships completed archives to a Hetzner Storage Box over WebDAV via `rclone` (`config.RCLONE_BIN`/`config.RCLONE_CONFIG_FILE`, remote name `config.BACKUP_OFFLOAD_REMOTE`, default `storagebox:devplacepy-backups`) - deliberately **not** the `/backup` davfs2 mount, whose FUSE metadata cache lives on the root filesystem and breaks exactly when disk fills (the original outage cause). `BackupService._run_offload_cycle` (throttled to `backup_offload_interval_seconds`, default 300s, via `ConfigField`s in the `Offload` group) runs each cycle after `_fire_due_schedules`:
 
-1. `upload_pending` - every `done` backup with `remote_uploaded_at` unset and a live `local_path` is `rclone copyto`'d to `<remote>/<target>/<filename>`, then verified by exact byte-size match (`rclone size --json`) against `size_bytes` recorded at finalize time. Only on a verified match does `store.mark_remote_uploaded` set `remote_path`/`remote_uploaded_at`. A failed or unverified upload is silently retried next cycle - `remote_uploaded_at` is the only source of truth for "is this backup actually safe off-box."
+1. `upload_pending` - every `done` backup with `remote_uploaded_at` unset and a live `local_path` is `rclone copyto`'d to `<remote>/<target>/<filename>`, then verified by exact byte-size match (`rclone size --json`) against `size_bytes` recorded at finalize time. Only on a verified match does `store.mark_remote_uploaded` set `remote_path`/`remote_uploaded_at`. A failed or unverified upload is retried next cycle - `remote_uploaded_at` is the only source of truth for "is this backup actually safe off-box." **Auth failures short-circuit the rest of the pending list** (HTTP 401 / "didn't find section"): one stale `rclone.conf` password must not retry every local archive in the same cycle. The davfs2 mount at `/backup` is a credential oracle for the same WebDAV host; if rclone 401s while that mount still works, the obscured `pass` in `rclone.conf` is stale and must be re-obscured from `/etc/davfs2/secrets`. Do not "fix" 401 by writing through the davfs2 mount.
 2. `enforce_local_retention` (`backup_offload_keep_local`, default 1) - per target, keeps the newest N **offloaded** local copies and unlinks the rest (`store.mark_local_purged`: clears `local_path`, sets `local_purged_at`, row and `remote_path` persist). A backup with no confirmed remote copy is never touched, no matter how old.
 3. `enforce_remote_retention` (`backup_offload_keep_remote`, default 30) - per target, `rclone lsjson` the remote dir and `deletefile` anything beyond the newest N, sorted by filename (safe because the `{target}-YYYYMMDD-HHMMSS-*` name is lexicographically chronological, same property `schedule.to_iso` relies on).
 
@@ -29,6 +29,10 @@ Admin-only, enterprise-grade backups built on the **same async-job pattern as zi
 
 **Operational prerequisite (production, not automatic):** the `rclone` binary is installed in the shipped Docker image, but a working WebDAV remote still needs to exist at `config.RCLONE_CONFIG_FILE` (default `$HOME/.config/rclone/rclone.conf` inside the app container, overridable via `DEVPLACE_RCLONE_CONFIG`) with a remote named to match `config.BACKUP_OFFLOAD_REMOTE`'s prefix (default `storagebox`) pointing at the Hetzner Storage Box's WebDAV endpoint and credentials - `rclone config` (interactive) or a hand-written `rclone.conf` generates it. Until that file exists, every `upload_pending` attempt fails fast (`rclone` errors "didn't find section") and is logged and retried next cycle; local retention and rotation both stay disabled the whole time (see above), so backups simply accumulate locally with no data loss, they just never leave the box. **In Docker, `HOME=/app` (the bind-mounted repo root, `docker-compose.yml`), so the default config path resolves to `<repo>/.config/rclone/rclone.conf` on the host - `.gitignore` excludes `/.config/` precisely because this file holds live remote-storage credentials; never force-add it.**
 
+## Disk usage warning
+
+`disk_usage()` is the only source of the used/free percentage. `BackupService._check_disk_usage()` and `collect_metrics()` call it (never `compute_storage_stats()`). `disk_warn_percent_field` (`backup_disk_warn_percent`, default 90, group "Alerts") is compared with simple hysteresis - `self.log(...)` fires once when usage reaches the threshold ("Disk usage critical: ...") and once more when it drops back below ("Disk usage back under threshold: ..."), never repeating every tick while the state is unchanged. The warning is visible in this service's own Logs tab (`/admin/services/backup`), which also gets a live "Disk usage" stat card via `collect_metrics()` (`super().collect_metrics()` from `JobService` plus the disk percentage/free space). The admin dashboard's per-directory file counts still come from `compute_storage_stats()`, off-thread. This is intentionally minimal - a log line and a stat card via the existing `BaseService` mechanisms, no new table, route, or notification channel.
+
 ## Schedules
 
 `backup_schedules` carry `kind` (`interval`|`cron`), `every_seconds`/`cron`, `enabled`, `keep_last`, `next_run_at`, run bookkeeping. `_fire_due_schedules` (lock-owner only, so each fires once) compares `next_run_at <= to_iso(now_utc())` and enqueues a `backup` job + a `backups` record, then advances `next_run_at` via `schedule.next_run`. **Timestamp format is load-bearing:** schedule `next_run_at` uses the devii `schedule.to_iso` format (`%Y-%m-%dT%H:%M:%S`, no tz/micros) on BOTH sides of the comparison so lexicographic compare equals chronological - do not mix it with `datetime.isoformat()`.

devplacepy/services/backup/offload.py

@@ -30,6 +30,16 @@ def _remote_dir(target: str) -> str:
     return f"{config.BACKUP_OFFLOAD_REMOTE}/{target}"
 
 
+def _is_auth_error(err: str) -> bool:
+    text = err.lower()
+    return (
+        "401" in text
+        or "unauthorized" in text
+        or "didn't find section" in text
+        or "did not find section" in text
+    )
+
+
 async def upload_pending(log=lambda message: None) -> int:
     uploaded = 0
     for row in store.list_pending_offload():
@@ -38,6 +48,12 @@ async def upload_pending(log=lambda message: None) -> int:
         code, _, err = await _run_rclone("copyto", str(local_path), remote_path)
         if code != 0:
             log(f"Offload failed for {row['filename']}: {err.strip()[:300]}")
+            if _is_auth_error(err):
+                log(
+                    "Offload halted: remote storage rejected credentials; "
+                    "remaining uploads skipped until the next cycle"
+                )
+                break
             continue
         size_code, size_out, size_err = await _run_rclone("size", remote_path, "--json")
         if size_code != 0:

devplacepy/services/backup/service.py

@@ -27,6 +27,7 @@ WORKER_MODULE = "devplacepy.services.jobs.backup_worker"
 DEFAULT_OFFLOAD_INTERVAL_SECONDS = 300
 DEFAULT_OFFLOAD_KEEP_LOCAL = 1
 DEFAULT_OFFLOAD_KEEP_REMOTE = 30
+DEFAULT_DISK_WARN_PERCENT = 90
 
 
 class BackupService(JobService):
@@ -41,6 +42,21 @@ class BackupService(JobService):
     def __init__(self):
         super().__init__(name="backup", interval_seconds=15)
         self._last_offload_at = 0.0
+        self._disk_warned = False
+        self.disk_warn_percent_field = ConfigField(
+            "backup_disk_warn_percent",
+            "Disk usage warning threshold (%)",
+            type="int",
+            default=DEFAULT_DISK_WARN_PERCENT,
+            minimum=50,
+            maximum=99,
+            help=(
+                "Log a warning (visible in this service's Logs tab) once the data "
+                "volume's used disk percentage reaches this threshold, and again "
+                "when it drops back below it."
+            ),
+            group="Alerts",
+        )
         self.offload_enabled_field = ConfigField(
             "backup_offload_enabled",
             "Offload to remote storage",
@@ -82,6 +98,7 @@ class BackupService(JobService):
             group="Offload",
         )
         self.config_fields += [
+            self.disk_warn_percent_field,
             self.offload_enabled_field,
             self.offload_interval_field,
             self.offload_keep_local_field,
@@ -95,6 +112,36 @@ class BackupService(JobService):
         except Exception as exc:
             self.log(f"Schedule pass failed: {exc}")
         await self._run_offload_cycle()
+        try:
+            self._check_disk_usage()
+        except Exception as exc:
+            self.log(f"Disk usage check failed: {exc}")
+
+    def _check_disk_usage(self) -> None:
+        threshold = int(self.disk_warn_percent_field.read())
+        used_percent = store.disk_usage()["used_percent"]
+        if used_percent >= threshold:
+            if not self._disk_warned:
+                self._disk_warned = True
+                self.log(
+                    f"Disk usage critical: {used_percent}% used on the data "
+                    f"volume (warning threshold {threshold}%)"
+                )
+        elif self._disk_warned:
+            self._disk_warned = False
+            self.log(f"Disk usage back under threshold: {used_percent}% used")
+
+    def collect_metrics(self) -> dict:
+        metrics = super().collect_metrics()
+        disk = store.disk_usage()
+        metrics["stats"] = [
+            *metrics.get("stats", []),
+            {
+                "label": "Disk usage",
+                "value": f"{disk['used_percent']}% ({disk['free_human']} free)",
+            },
+        ]
+        return metrics
 
     async def _run_offload_cycle(self) -> None:
         if not self.offload_enabled_field.read():

devplacepy/services/backup/store.py

@@ -1,6 +1,9 @@
 # retoor <retoor@molodetz.nl>
 
+import os
 import shutil
+import stat
+import threading
 import time
 from datetime import datetime, timezone
 from pathlib import Path
@@ -33,9 +36,12 @@ STATUS_RUNNING = "running"
 STATUS_DONE = "done"
 STATUS_FAILED = "failed"
 
-STORAGE_CACHE_TTL_SECONDS = 30
+STORAGE_CACHE_TTL_SECONDS = 600
+DISK_CACHE_TTL_SECONDS = 15
 
 _storage_cache: dict = {"at": 0.0, "data": None}
+_disk_cache: dict = {"at": 0.0, "data": None}
+_storage_lock = threading.Lock()
 
 
 def now_iso() -> str:
@@ -395,20 +401,57 @@ def delete_schedule(uid: str, deleted_by: str) -> bool:
     return True
 
 
+def clear_storage_stats_cache() -> None:
+    _storage_cache["data"] = None
+    _storage_cache["at"] = 0.0
+    _disk_cache["data"] = None
+    _disk_cache["at"] = 0.0
+
+
+def disk_usage() -> dict:
+    now = time.monotonic()
+    cached = _disk_cache["data"]
+    if cached is not None and (now - _disk_cache["at"]) < DISK_CACHE_TTL_SECONDS:
+        return cached
+    target = config.DATA_DIR
+    probe = target if target.exists() else target.parent
+    usage = shutil.disk_usage(str(probe if probe.exists() else Path("/")))
+    data = {
+        "total_bytes": usage.total,
+        "used_bytes": usage.used,
+        "free_bytes": usage.free,
+        "total_human": human_bytes(usage.total),
+        "used_human": human_bytes(usage.used),
+        "free_human": human_bytes(usage.free),
+        "used_percent": round(usage.used / usage.total * 100, 1) if usage.total else 0.0,
+    }
+    _disk_cache["data"] = data
+    _disk_cache["at"] = now
+    return data
+
+
 def _path_size(path: Path) -> tuple[int, int]:
-    if not path.exists():
+    try:
+        info = os.lstat(path)
+    except OSError:
+        return 0, 0
+    if stat.S_ISLNK(info.st_mode):
+        return 0, 0
+    if stat.S_ISREG(info.st_mode):
+        return info.st_size, 1
+    if not stat.S_ISDIR(info.st_mode):
         return 0, 0
-    if path.is_file():
-        return path.stat().st_size, 1
     total = 0
     files = 0
-    for entry in path.rglob("*"):
+    for root, _dirs, names in os.walk(path, followlinks=False):
-        try:
+        for name in names:
-            if entry.is_file() and not entry.is_symlink():
+            try:
-                total += entry.stat().st_size
+                info = os.lstat(os.path.join(root, name))
+            except OSError:
+                continue
+            if stat.S_ISREG(info.st_mode):
+                total += info.st_size
                 files += 1
-        except OSError:
-            continue
     return total, files
 
 
@@ -429,17 +472,57 @@ def _storage_paths() -> list[tuple[str, str, Path]]:
     ]
 
 
-def compute_storage_stats() -> dict:
+def _inventory() -> tuple[list[dict], int, int]:
-    now = time.monotonic()
+    declared = _storage_paths()
-    if (
+    sizes = {key: [0, 0] for key, _label, _path in declared}
-        _storage_cache["data"] is not None
+    dir_prefixes: list[tuple[str, str]] = []
-        and (now - _storage_cache["at"]) < STORAGE_CACHE_TTL_SECONDS
+    file_exact: dict[str, str] = {}
-    ):
+    data_root = os.path.realpath(config.DATA_DIR) if config.DATA_DIR.exists() else ""
-        return _storage_cache["data"]
+
+    for key, _label, path in declared:
+        try:
+            info = os.lstat(path)
+        except OSError:
+            continue
+        if stat.S_ISREG(info.st_mode):
+            file_exact[os.path.realpath(path)] = key
+            sizes[key] = [info.st_size, 1]
+            continue
+        if stat.S_ISDIR(info.st_mode):
+            real = os.path.realpath(path)
+            dir_prefixes.append((key, real.rstrip(os.sep) + os.sep))
+
+    data_size = 0
+    data_files = 0
+    if data_root and os.path.isdir(data_root):
+        for root, _dirs, names in os.walk(data_root, followlinks=False):
+            for name in names:
+                full = os.path.join(root, name)
+                try:
+                    info = os.lstat(full)
+                except OSError:
+                    continue
+                if not stat.S_ISREG(info.st_mode):
+                    continue
+                size = info.st_size
+                data_size += size
+                data_files += 1
+                matched = file_exact.get(full)
+                if matched is not None:
+                    sizes[matched] = [size, 1]
+                for key, prefix in dir_prefixes:
+                    if full.startswith(prefix):
+                        sizes[key][0] += size
+                        sizes[key][1] += 1
+    else:
+        for key, _label, path in declared:
+            sizes[key] = list(_path_size(path))
+            data_size += sizes[key][0]
+            data_files += sizes[key][1]
 
     paths = []
-    for key, label, path in _storage_paths():
+    for key, label, path in declared:
-        size, files = _path_size(path)
+        size, files = sizes[key]
         paths.append(
             {
                 "key": key,
@@ -451,41 +534,46 @@ def compute_storage_stats() -> dict:
                 "exists": path.exists(),
             }
         )
+    return paths, data_size, data_files
 
-    data_size, data_files = _path_size(config.DATA_DIR)
-    backups_size, backups_files = _path_size(config.BACKUPS_DIR)
-    backup_count = len(
-        [b for b in list_backups(limit=100000) if b.get("status") == STATUS_DONE]
-    )
-    usage = shutil.disk_usage(str(config.DATA_DIR))
 
-    data = {
+def compute_storage_stats() -> dict:
-        "paths": paths,
+    now = time.monotonic()
-        "data_dir": {
+    cached = _storage_cache["data"]
-            "path": str(config.DATA_DIR),
+    if cached is not None and (now - _storage_cache["at"]) < STORAGE_CACHE_TTL_SECONDS:
-            "size_bytes": data_size,
+        return cached
-            "size_human": human_bytes(data_size),
+    with _storage_lock:
-            "file_count": data_files,
+        now = time.monotonic()
-        },
+        cached = _storage_cache["data"]
-        "backups_total": {
+        if (
-            "count": backup_count,
+            cached is not None
-            "size_bytes": backups_size,
+            and (now - _storage_cache["at"]) < STORAGE_CACHE_TTL_SECONDS
-            "size_human": human_bytes(backups_size),
+        ):
-            "file_count": backups_files,
+            return cached
-        },
+        paths, data_size, data_files = _inventory()
-        "disk": {
+        backups_entry = next((row for row in paths if row["key"] == "backups"), None)
-            "total_bytes": usage.total,
+        backups_size = backups_entry["size_bytes"] if backups_entry else 0
-            "used_bytes": usage.used,
+        backups_files = backups_entry["file_count"] if backups_entry else 0
-            "free_bytes": usage.free,
+        backup_count = len(
-            "total_human": human_bytes(usage.total),
+            [b for b in list_backups(limit=100000) if b.get("status") == STATUS_DONE]
-            "used_human": human_bytes(usage.used),
+        )
-            "free_human": human_bytes(usage.free),
+        data = {
-            "used_percent": round(usage.used / usage.total * 100, 1)
+            "paths": paths,
-            if usage.total
+            "data_dir": {
-            else 0.0,
+                "path": str(config.DATA_DIR),
-        },
+                "size_bytes": data_size,
-        "generated_at": now_iso(),
+                "size_human": human_bytes(data_size),
-    }
+                "file_count": data_files,
-    _storage_cache["data"] = data
+            },
-    _storage_cache["at"] = now
+            "backups_total": {
-    return data
+                "count": backup_count,
+                "size_bytes": backups_size,
+                "size_human": human_bytes(backups_size),
+                "file_count": backups_files,
+            },
+            "disk": disk_usage(),
+            "generated_at": now_iso(),
+        }
+        _storage_cache["data"] = data
+        _storage_cache["at"] = time.monotonic()
+        return data

devplacepy/services/bot/CLAUDE.md

@@ -15,7 +15,7 @@ This file documents the Playwright-driven AI persona fleet. Claude Code auto-loa
 | `state.py` | `BotState` dataclass + JSON persistence (incl. the per-bot `identity` card) |
 | `llm.py` | `LLMClient` (parameterized: key/url/model/costs); content generation + quality checks + the `decide`/`generate_identity` decision engine |
 | `news_fetcher.py` | `NewsFetcher` - TTL-cached article source |
-| `browser.py` | `BotBrowser` - Playwright wrapper (human-like typing/clicking/scrolling) |
+| `browser.py` | `BotBrowser` - Playwright wrapper (human-like typing/clicking/scrolling). `fill` types character-by-character for text fields; date/time inputs (`type=date` and the other `NATIVE_FILL_TYPES`) use Playwright's native `locator.fill(value)` because Chromium date widgets ignore keystrokes and a typed `YYYY-MM-DD` never lands, which made every bot signup POST 400. |
 | `registry.py` | `ArticleRegistry` - flock-based cross-bot article dedupe |
 | `bot.py` | `DevPlaceBot` - the orchestrator (sessions, action cycle, `run_forever`) |
 | `service.py` | `BotsService(BaseService)` - fleet manager + metrics |

devplacepy/services/bot/browser.py

@@ -22,6 +22,13 @@ DEFAULT_USER_AGENT = (
     "(KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36"
 )
 DEFAULT_VIEWPORT = {"width": 1280, "height": 900}
+NATIVE_FILL_TYPES = frozenset(
+    {"date", "datetime-local", "month", "time", "week"}
+)
+
+
+def uses_native_fill(input_type: str) -> bool:
+    return (input_type or "").lower() in NATIVE_FILL_TYPES
 
 
 class BotBrowser:
@@ -157,11 +164,15 @@ class BotBrowser:
                 return False
             await el.click(timeout=3000)
             await self._idle(0.05, 0.15)
-            await el.fill("")
+            input_type = (await el.get_attribute("type") or "").lower()
-            for ch in val:
+            if uses_native_fill(input_type):
-                await self._page.keyboard.type(ch, delay=random.randint(20, 60))
+                await el.fill(val)
-                if ch == " ":
+            else:
-                    await asyncio.sleep(random.uniform(0.02, 0.08))
+                await el.fill("")
+                for ch in val:
+                    await self._page.keyboard.type(ch, delay=random.randint(20, 60))
+                    if ch == " ":
+                        await asyncio.sleep(random.uniform(0.02, 0.08))
             await self.capture("field input")
             return True
         except Exception as e:

devplacepy/services/containers/CLAUDE.md

@@ -58,7 +58,7 @@ Every exec passes `-w /app` explicitly (`DockerCliBackend.exec` and the PTY exec
 
 ## HTTP routing surface
 
-- `/projects/{slug}/containers` - the `routers/projects/containers/` subpackage: `instances.py` (creation/lifecycle/exec/logs/metrics/sync plus the exec websocket), `schedules.py` (cron/interval/once schedules), shared helpers in `_shared.py`. Every instance runs the shared `ppy` image. Discoverable from the project detail page's admin-only **Containers** button (gated by `content.can_view_project_containers` via the `viewer_can_containers` context flag) and from the admin index.
+- `/projects/{slug}/containers` - the `routers/projects/containers/` subpackage: `instances.py` (creation/lifecycle/exec/logs/metrics/sync plus the exec websocket), `schedules.py` (cron/interval/once schedules), shared helpers in `_shared.py`. Every instance runs the shared `ppy` image. Reachable by direct URL and from the admin index; the project detail page's admin-only **Containers** button was removed (the `viewer_can_containers` context flag and `content.can_view_project_containers` gate remain).
 - `/admin/containers` - `routers/admin/containers.py`: lists every instance across all projects with inline actions (start/stop/restart/terminal/edit/delete) plus a create modal (project search-select, run-as user search-select, boot language + source editor, restart policy, start-on-boot, env/ports/limits/ingress); `/admin/containers/{uid}` is the per-instance detail page (lifecycle, live logs/metrics, PTY terminal, schedules, ingress, sync, status history) and `/admin/containers/{uid}/edit` edits run-as user, boot language/script/command, restart policy, start-on-boot, and limits.
 
 **Key reuse rule:** the admin Containers section adds NO lifecycle/logs/exec endpoints of its own - the instance carries its `project_uid`, so the admin detail route resolves the project and its frontend targets the existing `/projects/{slug}/containers/instances/{uid}/...` routes. Add any new instance operation to `routers/projects/containers/instances.py` only; the admin detail page picks it up for free.
@@ -67,7 +67,7 @@ Every exec passes `-w /app` explicitly (`DockerCliBackend.exec` and the PTY exec
 
 Both are discoverable (an earlier version of the per-project page had zero links to it - fixed).
 
-1. **Per-project manager**: `templates/containers.html` + `static/js/ContainerManager.js` handles instance creation through an app modal form (the `_macros.html` `modal()` macro + `ModalManager` `.visible` toggle); its instance list links out to the shared detail page. Reached from the project detail page's Containers button, and passes breadcrumbs so content clears the fixed nav.
+1. **Per-project manager**: `templates/containers.html` + `static/js/ContainerManager.js` handles instance creation through an app modal form (the `_macros.html` `modal()` macro + `ModalManager` `.visible` toggle); its instance list links out to the shared detail page. Reachable by direct URL only now (the project detail page's Containers button was removed); it passes breadcrumbs so content clears the fixed nav.
 2. **Admin Containers section**: `routers/admin/containers.py` (mounted `/admin/containers`, sidebar link in `admin_base.html`, `admin_section="containers"`). `GET /admin/containers` lists every instance via `store.all_instances()` (decorated with project title/slug from one `projects` lookup) in an `.admin-table`. `GET /admin/containers/data` is the poll JSON. `GET /admin/containers/{uid}` renders `templates/containers_instance.html` + `static/js/ContainerInstance.js` - a dedicated detail page (lifecycle, poll logs/metrics, schedules add/delete, ingress, sync, interactive exec over a PTY WebSocket gated on the lock owner).
 
 All container CSS (`static/css/containers.css`) uses app design tokens (`--bg-card`, `--text-primary`, `--success`/`--danger`/`--warning`, `--radius`) and the shared `.card` recipe.
@@ -728,19 +728,19 @@ view is the only trigger, there is no `Forward a Port` command in the palette in
   required field makes an anonymous request 422 instead of 401 and `tests/api/auth/matrix.py` fails.
   Give the field a default and validate it inside the handler after `require_user`.
 
-**The editor opens through one shared partial.** The project detail page renders an inline **Editor**
+**The editor opens through one shared partial.** `templates/_editor_open.html` renders the **Open
-button in `.project-detail-actions` via `templates/_editor_open.html` (`target="_blank"`, plus the
+editor** link (`target="_blank"`, plus the `data-editor-*` attributes `EditorLauncher` reads) straight
-`data-editor-*` attributes `EditorLauncher` reads) straight to the code-server proxy
+to the code-server proxy `/projects/{slug}/containers/instances/{uid}/code/`, built by `_editor_url`
-`/projects/{slug}/containers/instances/{uid}/code/`, built by `_editor_url` in
+in `routers/projects/index.py` and carried as `workspace_editor_url` on the context and
-`routers/projects/index.py` and carried as `workspace_editor_url` on the context and
 `ProjectDetailOut`. It is emitted ONLY when the viewer passes `can_open_workspace` AND
 `provision.editor_ready(instance)` holds: the workspace exists, is not suspended, is
 `store.ST_RUNNING`, AND the editor port answers a TCP connect (`api.editor_reachable`, the same
 `tunnel_target` the proxy dials) - the states the `editor_proxy` route itself refuses (403 suspended,
 409 not running, 502 no port) plus the boot window in which the container is up but code-server is
-not yet listening, so the button can never open a dead editor. When there is no ready workspace the
+not yet listening, so the link can never open a dead editor. The link is rendered by the workspace
-button is absent and the Workspace menu item below is the way in (create/start it there). The
+page's **ready** phase; the project detail page's inline **Editor** button and its overflow
-workspace page's own **Open editor** link opens in a new tab too; keep both in step.
+**Workspace** item were removed, so the workspace page is the only surface and is reachable by direct
+URL now.
 
 **The workspace page renders a PHASE, and the phase is computed once, server-side.**
 `provision.phase(instance, ready)` is a pure function of `suspended_at`, `desired_state`, `status`
@@ -772,16 +772,14 @@ the same `{workspace, editor_url}` shape the page JSON carries. Only the start/s
 (`data-workspace-action`) go through the manager; tunnels, editor preferences and delete keep
 their native page-reloading submit, which the e2e tests assert with `wait_for_url`.
 
-**Member entry point** is the project detail page's overflow menu (`project_detail.html`), gated by
+**Member entry point** was the project detail page's overflow menu (`project_detail.html`), gated by
-the `viewer_can_workspace` context flag (`can_open_workspace(project, user)`, set in
+the `viewer_can_workspace` context flag (`can_open_workspace(project, user)`, still set in
-`routers/projects/index.py` and declared on `ProjectDetailOut`) - exactly the pattern the admin-only
+`routers/projects/index.py` and declared on `ProjectDetailOut`). That overflow **Workspace** item
-**Containers** item uses with `viewer_can_containers`. `can_open_workspace` folds in the
+(and the admin-only **Containers** item, gated the same way by `viewer_can_containers`) was removed,
-`workspace_enabled` master switch, so the item disappears for everyone while the feature is off and
+so `/projects/{slug}/workspace` is reached by direct URL until the feature is fully retired.
-the route's own `_guard` stays the authority. **A workspace surface with no context flag is
+`can_open_workspace` folds in the `workspace_enabled` master switch, and the route's own `_guard`
-unreachable**: the whole feature shipped once with routes, Devii tools and docs but no link into
+stays the authority. Re-adding any workspace link to the project page must gate it on the context
-`/projects/{slug}/workspace`, so it was reachable only by typing the URL - and then 404'd anyway
+flag again, with the setting turned on.
-because `workspace_enabled` defaults to `"0"`. Any new workspace surface needs both the flag on the
-page that links to it and the setting turned on.
 
 **Admin console** is `/admin/workspaces` (`routers/admin/workspaces.py`, `admin_workspaces.html`):
 list, start/stop, suspend/unsuspend with a required reason, raise/resolve/dismiss flags, per-user

devplacepy/services/correction.py

@@ -13,7 +13,13 @@ from devplacepy.config import (
     INTERNAL_GATEWAY_URL,
     INTERNAL_MODEL,
 )
-from devplacepy.database import add_correction_usage, get_setting, get_table
+from devplacepy.database import (
+    add_correction_usage,
+    get_setting,
+    get_table,
+    internal_gateway_key,
+)
+from devplacepy.rendering import markdown_structure_signature
 from devplacepy.services.background import background
 from devplacepy.services.openai_gateway.usage import parse_usage_headers
 
@@ -31,6 +37,7 @@ CORRECTABLE_FIELDS: dict[str, tuple[str, ...]] = {
 
 CORRECTION_TIMEOUT_SECONDS = 20.0
 MAX_GROWTH_FACTOR = 3
+STRUCTURE_DROP_TOLERANCE = 1
 PENDING_SCOPE_KEY = "devplace_pending_corrections"
 
 AI_APPLY_EXECUTOR = ThreadPoolExecutor(max_workers=4, thread_name_prefix="ai-apply")
@@ -73,6 +80,18 @@ def _usage_from_headers(response_headers) -> dict:
     }
 
 
+def structure_diverges(original: str, corrected: str) -> bool:
+    baseline = markdown_structure_signature(original)
+    candidate = markdown_structure_signature(corrected)
+    for key, before in baseline.items():
+        after = candidate.get(key, 0)
+        if before > 0 and after == 0:
+            return True
+        if after < before - STRUCTURE_DROP_TOLERANCE:
+            return True
+    return False
+
+
 def gateway_complete(
     api_key: str,
     system: str,
@@ -80,6 +99,8 @@ def gateway_complete(
     timeout: float,
     max_growth_factor: int | None = None,
     model: str = INTERNAL_MODEL,
+    bypass_preamble: bool = False,
+    check_structure: bool = False,
 ) -> tuple[str, dict | None]:
     text = text or ""
     if not text.strip():
@@ -92,12 +113,18 @@ def gateway_complete(
         ],
         "temperature": 0.1,
     }
+    if bypass_preamble:
+        payload["bypass_preamble"] = True
     headers = {
         "Content-Type": "application/json",
         "X-App-Reference": "devplace-correction-v-1-0-0",
     }
     if api_key:
         headers["Authorization"] = f"Bearer {api_key}"
+    if bypass_preamble:
+        internal_key = internal_gateway_key()
+        if internal_key:
+            headers["X-Gateway-Internal-Key"] = internal_key
     try:
         response = _client().post(
             INTERNAL_GATEWAY_URL, json=payload, headers=headers, timeout=timeout
@@ -119,6 +146,9 @@ def gateway_complete(
     if max_growth_factor and len(content) > len(text) * max_growth_factor + 200:
         logger.warning("AI gateway output too large, keeping original")
         return text, usage
+    if check_structure and structure_diverges(text, content):
+        logger.warning("AI gateway output changed markdown structure, keeping original")
+        return text, usage
     return content, usage
 
 
@@ -141,6 +171,8 @@ def correct_text(api_key: str, prompt: str, text: str) -> tuple[str, dict | None
         CORRECTION_TIMEOUT_SECONDS,
         MAX_GROWTH_FACTOR,
         model=correction_model(),
+        bypass_preamble=True,
+        check_structure=True,
     )
 
 

devplacepy/services/devii/CLAUDE.md

@@ -96,6 +96,8 @@ A task created as a reminder carries `notify=1`: when it finishes, `session._del
 
 **Quotas are resettable** (clearing the owner's `devii_usage_ledger` rows): per-user via `POST /admin/users/{uid}/reset-ai-quota` (button on the admin Users page), globally via `POST /admin/ai-quota/reset-guests` and `/reset-all` (buttons on `/admin/ai-usage`), and from the CLI via `devplace devii reset-quota <username> | --guests | --all`.
 
+**Proactive 80% quota warning.** `DeviiService.maybe_warn_quota_threshold(owner_kind, owner_id, is_admin)` is called right after the interactive quota gate passes (`routers/devii.py`'s `/devii/ws` handler and `services/telegram/bridge.py`'s `_run_turn`, the two turn-spawning chokepoints), so a signed-in user is warned before they hit the 100% block instead of only discovering it once blocked. It is a no-op for guests (no notifications inbox to deliver to), for administrators, and for an unlimited (`0`) cap; otherwise it fires the `ai_quota_warning` notification (via the single `create_notification` funnel, `NOTIFICATION_TYPES`, toggleable like any other type) once the owner's rolling `spent_24h` reaches 80% of their `daily_limit_for`. Dedup is a plain lookback query on the `notifications` table (`type="ai_quota_warning" AND created_at >= now-24h`) rather than a new table/column - a fresh warning can fire again only once the prior one ages out of the rolling 24h window. The message states only proximity, never a dollar figure, matching the "financial data is admin-only" rule above.
+
 **Financial data is admin-only:** any monetary figure (USD cost, pricing, spend, limit) is restricted to administrators; members and guests see only the **percentage** of their 24h quota used. The owner's admin status is resolved server-side (`is_admin(user)`) and threaded WS -> `hub.get_or_create(is_admin=)` -> `DeviiSession(is_admin=)` -> `Dispatcher(is_admin=)`. The `Action` dataclass has a `requires_admin` flag (`cost_stats` is admin-only USD; the member-safe `usage_quota` tool returns **only** `{used_pct, turns_today, limit_reached}` with no money and is available to everyone). Gating is double: `Catalog.tool_schemas_for(authenticated, is_admin)` never hands an admin-only tool's schema to a non-admin, and the dispatcher independently raises `AuthRequiredError` for any `requires_admin` action a non-admin attempts. `usage_quota`'s data comes from `DeviiSession._quota_snapshot` (ledger `spent_24h`/`turns_24h` over `settings.daily_limit_usd`); for non-admin owners the system prompt also appends a hard rule forbidding any cost disclosure (defense in depth). `GET /devii/usage` mirrors this: it always returns `used_pct`/`turns_today` and adds `spent_24h`/`limit` only for admins. The standalone `devii` CLI runs with `is_admin=True` (the local operator owns the process). The catalog's cost/analytics HTTP tools `ai_usage` (GET `/admin/ai-usage/data`, USD breakdown) and `site_analytics` (GET `/admin/analytics`) are also `requires_admin=True`, so a non-admin session is never offered them and the dispatcher blocks them even if named - the platform 403 is no longer the only guard. The profile page is correctly split too (`_ai_quota(include_cost=viewer_is_admin)` emits dollars only for admins, in both its HTML and JSON forms, so a member fetching their own profile with `Accept: application/json` never sees dollars either).
 
 ## Aggregate analytics (no pagination)

devplacepy/services/devii/actions/catalog/engagement.py

@@ -22,7 +22,7 @@ ENGAGEMENT_ACTIONS: tuple[Action, ...] = (
             ),
             body(
                 "value",
-                "Vote value: 1 to upvote, -1 to downvote (re-send to remove).",
+                "Vote value: 1 to upvote, -1 to downvote (re-send to remove), 0 to explicitly retract your vote.",
                 required=True,
             ),
         ),
@@ -65,6 +65,45 @@ ENGAGEMENT_ACTIONS: tuple[Action, ...] = (
             ),
         ),
     ),
+    Action(
+        name="list_notes",
+        method="GET",
+        path="/notes/saved",
+        summary="List your personal notes",
+        description="Notes are private annotations only you can ever see.",
+        params=(query("before", "Pagination cursor."),),
+    ),
+    Action(
+        name="set_note",
+        method="POST",
+        path="/notes/{target_type}/{target_uid}",
+        summary="Add or update a private note on a target",
+        description="Overwrites any existing note on the target. Only you can ever see this note.",
+        ajax=True,
+        params=(
+            path("target_type", TARGET_TYPE),
+            path(
+                "target_uid",
+                "Uid of the target, copied from a listing response; do not invent it.",
+            ),
+            body("content", "Note text, up to 4000 characters.", required=True),
+        ),
+    ),
+    Action(
+        name="delete_note",
+        method="POST",
+        path="/notes/{target_type}/{target_uid}/delete",
+        summary="Delete your private note from a target",
+        description="Returns {deleted: true}.",
+        ajax=True,
+        params=(
+            path("target_type", TARGET_TYPE),
+            path(
+                "target_uid",
+                "Uid of the target, copied from a listing response; do not invent it.",
+            ),
+        ),
+    ),
     Action(
         name="vote_poll",
         method="POST",

devplacepy/services/devii/actions/catalog/tools.py

@@ -3,7 +3,7 @@
 from __future__ import annotations
 
 from ..spec import Action
-from ._shared import body, path
+from ._shared import body, path, query
 
 
 TOOLS_ACTIONS: tuple[Action, ...] = (
@@ -108,6 +108,19 @@ TOOLS_ACTIONS: tuple[Action, ...] = (
         params=(path("uid", "DeepSearch job uid returned by deepsearch."),),
         requires_auth=False,
     ),
+    Action(
+        name="deepsearch_history",
+        method="GET",
+        path="/tools/deepsearch/history",
+        summary="List the user's past DeepSearch research runs",
+        description=(
+            "Returns the signed-in user's (or guest's) DeepSearch history, newest first, each "
+            "with its query, status, score and a reopen_url. A completed session can be reopened "
+            "with deepsearch_session and its grounded chat continued exactly as when it finished."
+        ),
+        params=(query("limit", "Maximum sessions to return (1-100, default 20)."),),
+        requires_auth=False,
+    ),
     Action(
         name="deepsearch_pause",
         method="POST",

devplacepy/services/devii/registry.py

@@ -60,7 +60,7 @@ GROUP_LABELS: dict[str, str] = {
     "profile": "Profile",
     "messages": "Direct Messages",
     "notifications": "Notifications (HTTP)",
-    "engagement": "Reactions, Bookmarks, Polls, Follow",
+    "engagement": "Reactions, Bookmarks, Notes, Polls, Follow",
     "social": "Leaderboard and Social",
     "issues": "Issue Tracker",
     "gists": "Gists",

devplacepy/services/devii/service.py

@@ -21,6 +21,9 @@ logger = logging.getLogger("devii.service")
 
 INSTANCE_ORIGIN_DEFAULT = f"http://127.0.0.1:{PORT}"
 
+QUOTA_WARNING_RATIO = 0.8
+QUOTA_WARNING_NOTIFICATION_TYPE = "ai_quota_warning"
+
 
 class DeviiService(BaseService):
     default_enabled = False
@@ -421,6 +424,38 @@ class DeviiService(BaseService):
         limit = self.daily_limit_for(owner_kind, is_admin)
         return limit > 0 and self.spent_24h(owner_kind, owner_id) >= limit
 
+    def maybe_warn_quota_threshold(
+        self, owner_kind: str, owner_id: str, is_admin: bool = False
+    ) -> None:
+        if owner_kind != "user" or is_admin or not owner_id:
+            return
+        limit = self.daily_limit_for(owner_kind, is_admin)
+        if limit <= 0:
+            return
+        spent = self.spent_24h(owner_kind, owner_id)
+        if spent < limit * QUOTA_WARNING_RATIO:
+            return
+        from datetime import datetime, timedelta, timezone
+
+        from devplacepy.database import get_table
+        from devplacepy.utils import create_notification
+
+        cutoff = (datetime.now(timezone.utc) - timedelta(hours=24)).isoformat()
+        already_warned = get_table("notifications").find_one(
+            user_uid=owner_id,
+            type=QUOTA_WARNING_NOTIFICATION_TYPE,
+            created_at={">=": cutoff},
+        )
+        if already_warned:
+            return
+        create_notification(
+            owner_id,
+            QUOTA_WARNING_NOTIFICATION_TYPE,
+            "You are approaching your daily AI usage limit.",
+            owner_id,
+            "/devii",
+        )
+
     def reset_quota(self, owner_kind: str, owner_id: str) -> int:
         return self.hub().ledger.reset(owner_kind, owner_id)
 

devplacepy/services/jobs/CLAUDE.md

@@ -49,9 +49,9 @@ Forking copies a source project into a brand-new project owned by the forking us
 
 ## SEO Diagnostics tool - SeoService (kind `seo`, `services/jobs/seo/`, `routers/tools/`)
 
-The public **Tools -> SEO Diagnostics** auditor crawls a URL or sitemap with a headless browser and runs a broad battery of SEO checks, on the **same async-job pattern as zip/fork** plus a live websocket. It is **public (guests included)**; abuse is bounded by the per-IP POST rate limit, a per-owner one-active-job cap, a page cap, and the shared SSRF guard.
+The public **SEO Diagnostics** auditor crawls a URL or sitemap with a headless browser and runs a broad battery of SEO checks, on the **same async-job pattern as zip/fork** plus a live websocket. It is **public (guests included)**; abuse is bounded by the per-IP POST rate limit, a per-owner one-active-job cap, a page cap, and the shared SSRF guard.
 
-- **Surface:** a collapsible **Tools** dropdown in `base.html` (desktop center nav + a mobile section, visible to everyone) toggled by `MobileNav.initToolsDropdown`. `GET /tools` lists tools; `GET /tools/seo` is the auditor page (`static/js/SeoDiagnostics.js` -> `app.seoDiagnostics`, instantiated page-side in the template, not in `Application.js`).
+- **Surface:** reachable by direct URL (`GET /tools` lists tools; `GET /tools/seo` is the auditor page, `static/js/SeoDiagnostics.js` -> `app.seoDiagnostics`, instantiated page-side in the template, not in `Application.js`). The former topnav **Tools** dropdown was removed, so there is currently no navigation entry point.
 - **Enqueue:** `POST /tools/seo/run` (`routers/tools/seo.py`, body `SeoRunForm{url, mode: url|sitemap, max_pages 1-50}`). Owner is `("user", uid)` or `("guest", X-Real-IP)`. It rejects with `429` if the owner already has a pending/running `seo` job, then enqueues `{url, mode, max_pages, allow_private:False}` and returns `{uid, status_url, ws_url}`.
 - **`process`** writes the payload to `config.SEO_REPORTS_DIR/{uid}/payload.json`, launches `python -m devplacepy.services.jobs.seo.worker <payload_json> <output_dir>` via `create_subprocess_exec` (high `limit=` so big lines never overflow the StreamReader), reads **NDJSON frames from stdout** line by line (stage/target/progress/page/site_checks/report_ready), forwards each into the in-process **`ProgressHub`** (`services/jobs/seo/progress.py`, uid -> set of `asyncio.Queue`), and on completion loads `output_dir/report.json` as the job result. `cleanup()` clears the hub buffer and removes the report dir.
 - **Worker** (`worker.py`, subprocess): `crawler.crawl_target` resolves the target (single URL, or sitemap `<loc>` URLs capped at `max_pages`) and fetches `robots.txt`/`sitemap.xml`/`llms.txt` with `httpx`. The audited target host is guarded once with `net_guard.guard_public_url`; candidate URLs **sharing that host are pre-approved** (no redundant per-URL `getaddrinfo` - a transient DNS failure or a self-hosted server resolving its own domain must not blank the whole crawl), and only cross-host sitemap entries are re-guarded. **In sitemap mode the crawler never falls back to auditing the sitemap document itself**: if no page URLs survive it raises a clear error (a stray `or [target]` fallback previously rendered the sitemap XML as one 56k-node "page" with no title/H1). For each page it launches one Playwright navigation: a single `page.evaluate(EXTRACT_SCRIPT)` returns the whole DOM contract (title/metas/canonical/headings/images/links/jsonld/og/twitter/semantic/mixed-content/word-count), an injected `PerformanceObserver` (`add_init_script(INIT_SCRIPT)`) captures LCP/CLS, navigation timing gives TTFB/FCP/transfer/protocol, a mobile-viewport pass measures overflow/tap-targets, a screenshot is saved, and a raw `httpx` GET supplies the SSR HTML for the rendered-vs-server parity check.
@@ -77,7 +77,7 @@ The public **Tools -> SEO Diagnostics** auditor crawls a URL or sitemap with a h
 
 ## DeepSearch tool - DeepsearchService (kind `deepsearch`, `services/jobs/deepsearch/`, `services/deepsearch/`, `routers/tools/deepsearch.py`)
 
-The public **Tools -> DeepSearch** researcher is a multi-agent deep web researcher built on the **same async-job + ProgressHub + 4013-WS pattern as the SEO tool**, plus a per-session vector store and a grounded RAG chat. It is **public (guests included)**; abuse is bounded by the per-IP POST rate limit, a per-owner one-active-job cap, a page cap (1-30), depth cap (1-4), and the shared SSRF guard. Reuse the SEO tool as the template for any new Tools async job.
+The public **DeepSearch** researcher is a multi-agent deep web researcher built on the **same async-job + ProgressHub + 4013-WS pattern as the SEO tool**, plus a per-session vector store and a grounded RAG chat. It is **public (guests included)**; abuse is bounded by the per-IP POST rate limit, a per-owner one-active-job cap, a page cap (1-30), depth cap (1-4), and the shared SSRF guard. Reuse the SEO tool as the template for any new Tools async job.
 
 - **Owner helper is shared:** `routers/tools/_shared.py` `owner_for(request)` returns `("user", uid)` or `("guest", X-Real-IP)`; both `seo.py` and `deepsearch.py` import it (do not re-inline the owner derivation).
 - **Enqueue:** `POST /tools/deepsearch/run` (body `DeepsearchRunForm{query, depth 1-4, max_pages 1-30}`). It rejects with `429` if the owner already has a pending/running `deepsearch` job. It resolves the **logged-in user's `users.api_key`** (guests use `database.internal_gateway_key()`) into the job payload for per-user embedding/LLM spend attribution, generates the uid up front, writes a `deepsearch_sessions` row (`create_deepsearch_session`), enqueues the job carrying `{query, depth, max_pages, api_key, collection}`, and returns `{uid, status_url, ws_url}`. The enqueue uses a local `_enqueue` (not `queue.enqueue`) so the session uid and the job uid match.
@@ -92,6 +92,7 @@ The public **Tools -> DeepSearch** researcher is a multi-agent deep web research
 - **Vector store (`services/deepsearch/store.py`):** `VectorStore` wraps `chromadb.PersistentClient(path=config.DEEPSEARCH_CHROMA_DIR)`, one collection per session (`ds_<uid>`). `Chunk` is the dataclass. `hybrid_search` blends cosine vector similarity with a BM25 keyword score (weights `HYBRID_VECTOR_WEIGHT`/`HYBRID_KEYWORD_WEIGHT`) over the candidate set, with optional metadata `where` filters. `embeddings.py` `embed_texts` calls the gateway embeddings endpoint and **falls back to a deterministic local hashing vector** on any failure (so the tool degrades, never breaks).
 - **RAG chat (`services/deepsearch/chat.py` + `WS /tools/deepsearch/{uid}/chat`):** a dedicated lightweight loop (NOT the Devii hub), served **only by the service-lock owner** (closes `4013` for fast retry). Answers are grounded ONLY in the session collection via `hybrid_search`, cited inline, rendered client-side via `dp-content`. Turns persist to `deepsearch_messages` and audit `deepsearch.chat`. Frontend component `<dp-deepsearch-chat>` (`static/js/components/AppDeepsearchChat.js`) clones `AppDocsChat`'s framing but uses its own WebSocket to the chat path.
 - **Status/report/export routes:** `GET /tools/deepsearch/{uid}` (`DeepsearchJobOut`), `GET /tools/deepsearch/{uid}/session` (`respond(..., DeepsearchSessionOut)`, HTML or JSON), `GET /tools/deepsearch/{uid}/export.{md,json,pdf}` (`services/deepsearch/export.py`; PDF via weasyprint). All are **capability URLs** scoped by the unguessable uuid7. **Viewer-flag discipline:** the session schema/context use `viewer_is_admin`/`viewer_owns` (never `is_admin`/`owns`) so a `respond()` context key never shadows a Jinja global (the same class of issue as the issues `/{number}` route). `tests/api/tools/deepsearch/session.py` guards the HTML render.
+- **History and reopen (`GET /tools/deepsearch/history`).** `deepsearch_sessions` already persists every run's query/status/score/confidence/summary/timestamps independently of the `jobs` table row (unlike the disposable collection + report dir), so the only genuinely missing piece was a listing surface, not new storage. `database.list_deepsearch_sessions(owner_kind, owner_id, limit)` (index `idx_deepsearch_sessions_owner_created` on `(owner_kind, owner_id, created_at)`) backs the owner-scoped, newest-first history via the same `routers/tools/_shared.py` `owner_for(request)` every other DeepSearch route uses - no separate guest-cookie identity, matching the run/status/control routes' IP-based guest scoping. Each item carries a `reopen_url` (`/tools/deepsearch/{uid}/session`), `chat_available` (session status is `done` AND the job row still exists), and `available` (the job row has not yet been swept by the `deepsearch` job kind's 7-day retention - past that point the collection and report are gone even though the `deepsearch_sessions` row survives, so the history item shows "Expired" instead of a dead link). **Reopening a completed session's chat needed no backend change**: the `WS /tools/deepsearch/{uid}/chat` guard already accepts `job.status == queue.DONE OR session.status == "done"`, so a session reached fresh from the history list (not from the just-finished live-progress flow) renders `chat_ws_url` on `GET .../session` exactly the same way. Schema `DeepsearchHistoryOut`/`DeepsearchHistoryItemOut`; Devii tool `deepsearch_history` (public, mirrors `isslop_list`); docs `tools-deepsearch-history`.
 - **Completion race (load-bearing read-path fix).** The worker writes `report.json` to disk and `service.process` publishes the `done` frame **from inside `process()`**, but the `JobService` framework only commits `jobs.result`/`status=DONE` afterwards, in `_reap()` -> `_finish_done()` on a later tick. The frontend navigates to the session page the instant it receives `done`, so a read that keyed only off `jobs.status == DONE` returned an EMPTY report (`None` score, 0 sources) until a manual refresh. Fix: `_report_for(uid, job)` returns `job.result.report` when the job is `DONE` and non-empty, else falls back to the on-disk `report.json` (`_report_from_disk`, `DEEPSEARCH_DIR/{uid}/report.json`) - which exists before the `done` frame is ever sent - and returns `{}` only for a `FAILED` job or a genuinely still-running job with no report on disk. `_session_context` derives `done`/`status` from `bool(report)` (not raw job status), and the chat WS gate accepts `session.status == "done"` (set inside `process()` before the publish) as ready. `_export_report` reuses the same fallback. Regression: `tests/api/tools/deepsearch/session.py::test_session_reads_disk_report_before_result_commit`. **Any new read of a job result that a client reaches immediately after a `done`/`session_url` frame must use this same on-disk fallback, never bare `jobs.status`.**
 - **Clickable inline citations (`services/deepsearch/citations.py`).** The report/findings carry `[n]` markers (and the model sometimes emits `[3][9][1-2]`); the `link_citations(html, source_count)` template global (registered in `templating.py`) rewrites each `[n]` and each `[a-b]` range into `<a class="ds-cite" href="#ds-source-n">[n]</a>` anchors that jump to the numbered `<li id="ds-source-n">` in the Sources list (source numbering is page order, matching the `[n]` the summarizer was given). It splits out `<a>`/`<code>`/`<pre>` regions first so markers inside links/code are left alone, expands ranges to individual links, and drops out-of-range numbers (no broken anchors). The session template nests it over the server render: `{{ link_citations(render_content(summary), sources|length) }}` and `{{ link_citations(finding.detail|e, sources|length) }}`, plus a per-finding `.ds-finding-cites` chip row from `finding.citations`. `.ds-cite`/`.ds-sources li:target` styling lives in `deepsearch.css`. The report prompt asks for one number per bracket (never a range) so output is consistent, but the linkifier handles ranges regardless. Regression: `tests/unit/services/deepsearch/citations.py`.
 - **Tables (`deepsearch_sessions`, `deepsearch_messages` soft-deletable + in `SOFT_DELETE_TABLES`; `deepsearch_url_cache` GC-only):** columns are ensured in `init_db()` (every queried column) with indexes. Every insert writes `deleted_at:None/deleted_by:None`; every read filters `deleted_at IS NULL`.
@@ -105,7 +106,7 @@ The public **Tools -> DeepSearch** researcher is a multi-agent deep web research
 
 ## AI Usage Analyzer tool - IsslopService (kind `isslop`, `services/jobs/isslop/`, `routers/tools/isslop.py`)
 
-The public **Tools -> AI Usage Analyzer** classifies a git repository or website as AI slop, sophisticated AI-assisted work or genuine human work. It is built on the standard async-job pattern (a `JobService` running a subprocess worker), but its live channel is **pub/sub, not a dedicated WS route**: every worker event is published to `public.isslop.{uid}` AND persisted to `isslop_events`, and the frontend pairs the pub/sub subscription with an incremental `GET /tools/isslop/{uid}/events?after=SEQ` poll, so guests (who cannot subscribe to `public.*` unless `pubsub_allow_guests` is on) and reconnecting tabs replay from the durable trail. **Never rely on pub/sub alone for this tool: the DB event trail is the source of truth, pub/sub is the fast path.**
+The public **AI Usage Analyzer** classifies a git repository or website as AI slop, sophisticated AI-assisted work or genuine human work. It is built on the standard async-job pattern (a `JobService` running a subprocess worker), but its live channel is **pub/sub, not a dedicated WS route**: every worker event is published to `public.isslop.{uid}` AND persisted to `isslop_events`, and the frontend pairs the pub/sub subscription with an incremental `GET /tools/isslop/{uid}/events?after=SEQ` poll, so guests (who cannot subscribe to `public.*` unless `pubsub_allow_guests` is on) and reconnecting tabs replay from the durable trail. **Never rely on pub/sub alone for this tool: the DB event trail is the source of truth, pub/sub is the fast path.**
 
 - **Engine layout:** `services/jobs/isslop/` holds `acquisition/` (git probe via `git ls-remote`, depth-1 clone with size preflight + live 3 GB kill guard, stealth Playwright website crawler with HTTP fallback, path-traversal-safe workspace helpers), `analysis/` (exclusion rules, stylometric metrics, language detection, per-repo baselines, `signals/` with one detector family per file, two-axis scoring), `agent/` (gateway LLM client, per-file classifier, vision reviewer, report writer with deterministic fallback), plus `pipeline.py` (the event-yielding run), `worker.py` (subprocess entry), `events.py` (frame protocol), `persistence.py` (`EventPersister` writes events/file results/image results/report and stamps the analysis row), `store.py` (all DB access), `badge.py` (SVG), `service.py` (`IsslopService`), `config.py` (all constants + `WorkerSettings`).
 - **Worker contract:** `IsslopService.process` writes the worker payload (url + admin toggles + gateway endpoint/model/key) to `config.ISSLOP_RUNS_DIR/{uid}/payload.json`, resolves the workspace under `config.ISSLOP_WORKSPACES_DIR` (`workspace_for` rejects any path escaping the root), launches `python -m devplacepy.services.jobs.isslop.worker <payload> <workspace>`, and relays each NDJSON stdout line through `EventPersister.apply` (SQLite) then `pubsub.publish`. The workspace and run dir are removed in a `finally`; the pipeline also deletes the workspace itself as its final act, so **no acquired source survives an analysis** - only the report and its evidence rows.

devplacepy/services/jobs/seo_meta_service.py

@@ -173,6 +173,7 @@ class SeoMetaService(JobService):
             source_text,
             GENERATION_TIMEOUT_SECONDS,
             model=model,
+            bypass_preamble=True,
         )
         if usage:
             for key in totals:

devplacepy/services/live_view_relay.py

@@ -113,7 +113,7 @@ async def _ai_usage(match: re.Match) -> dict:
 async def _backups(_match: re.Match) -> dict:
     from devplacepy.routers.admin.backups import _dashboard
 
-    return _dashboard(can_download=False)
+    return await _dashboard(can_download=False)
 
 
 async def _workspace_detail(match: re.Match) -> Optional[dict]:

devplacepy/services/messaging/CLAUDE.md

@@ -45,7 +45,13 @@ Because the WS accepts on every worker, a message persisted on worker A must sti
 
 ## DRY persist choke point
 
-`services/messaging/persist.py` `persist_message(sender, receiver_uid, content, attachment_uids, *, request=None, origin)` is the ONE function that inserts the row, links attachments, fires `create_notification` + `clear_messages_cache` + `create_mention_notifications`, logs, and writes the `message.send` audit event. Both the WS `send` handler and the HTTP `POST /messages/send` handler call it, so audit/notification/mention behavior is byte-identical on both paths (the Devii `send_message` action is `handler="http"` -> `POST /messages/send`, so it also flows through here and broadcasts live). Content is capped at 2000 chars server-side; `content` itself is allowed to be empty (`MessageForm.content` is `min_length=0`) as long as at least one attachment is present - `persist_message` is the single source of truth for that rule (`if not content and not attachment_uids: return None`), so the HTTP form model deliberately does not duplicate it. Whenever `request` is not `None` it audits via `audit.record(request, ...)` - this covers BOTH the HTTP `Request` and the WS path, since the WS handler passes `request=websocket` and a `WebSocket` object is just as non-`None` as a `Request` (`audit.record` never reads HTTP-specific attributes off it beyond what's already supplied explicitly via `user=sender`). `audit.record_system(..., actor_kind="user", origin=origin)` is the fallback used ONLY when `persist_message` is called with no request/websocket context at all (e.g. a future internal/system-originated send) - same `message.send` key/category either way, no new event invented; typing/read are ephemeral and NOT audited.
+`services/messaging/persist.py` `persist_message(sender, receiver_uid, content, attachment_uids, *, request=None, origin, client_id=None)` is the ONE function that inserts the row, links attachments, fires `create_notification` + `clear_messages_cache` + `create_mention_notifications`, logs, and writes the `message.send` audit event. Both the WS `send` handler and the HTTP `POST /messages/send` handler call it, so audit/notification/mention behavior is byte-identical on both paths (the Devii `send_message` action is `handler="http"` -> `POST /messages/send`, so it also flows through here and broadcasts live). Content is capped at 2000 chars server-side; `content` itself is allowed to be empty (`MessageForm.content` is `min_length=0`) as long as at least one attachment is present - `persist_message` is the single source of truth for that rule (`if not content and not attachment_uids: return None`), so the HTTP form model deliberately does not duplicate it. Whenever `request` is not `None` it audits via `audit.record(request, ...)` - this covers BOTH the HTTP `Request` and the WS path, since the WS handler passes `request=websocket` and a `WebSocket` object is just as non-`None` as a `Request` (`audit.record` never reads HTTP-specific attributes off it beyond what's already supplied explicitly via `user=sender`). `audit.record_system(..., actor_kind="user", origin=origin)` is the fallback used ONLY when `persist_message` is called with no request/websocket context at all (e.g. a future internal/system-originated send) - same `message.send` key/category either way, no new event invented; typing/read are ephemeral and NOT audited.
+
+### Send dedupe (double-submit safety)
+
+`persist_message` dedupes on `(sender_uid, client_id)` **before** it ever inserts a row: `_find_recent_duplicate(sender_uid, client_id)` looks up the most recent `messages` row with the same `sender_uid` and the same client-supplied `client_id` whose `created_at` falls inside `DEDUPE_WINDOW_SECONDS` (30s). A hit short-circuits `persist_message` and returns that ALREADY-persisted row verbatim (no error, no second insert, no second AI correction/notification/audit) - both the WS `send` handler and `POST /messages/send` now pass their `client_id` straight through, so a network-level retry that resends the identical payload (same `client_id`) collapses onto the original row instead of producing a second, independently-AI-corrected message. `client_id` is attacker-controlled but the dedupe key is always scoped by the server-resolved `sender_uid`, so a client can only dedupe its own sends. The column is nullable (`messages.client_id`, ensured in `init_db()`, indexed by `idx_messages_dedupe (sender_uid, client_id)`); a request with no `client_id` (any future non-`dp-chat` caller) skips the check entirely and always inserts. This is defense-in-depth alongside the client-side send-lock below - the lock stops a human double-click from ever firing two requests, the server dedupe stops a retried *identical* request (same `client_id`) from ever becoming two rows.
+
+**Client-side send-lock (`AppChat.js`).** `_initComposer`'s submit handler and the Enter-to-send path both funnel through one `form submit` listener, which now bails out while `this._sendLocked` is true. `_sendViaSocket` sets the lock (and disables the send button via the existing `_refreshSendButton()`, reusing the `.is-sending` spinner state) for every genuine new send (never for a `.failed`-bubble retry, tracked separately so a retry never blocks a fresh message) and only releases it once that specific `client_id` clears out of `_pendingSends` - on the echoed reconcile (`_reconcilePending` -> `_clearPendingSend`) or the 20s send timeout (`_failPendingSend`). Multiple sends never race the lock: `_lockingSends` is a `Set` of in-flight new-send `client_id`s, and the composer re-enables only when it is empty.
 
 ## AI correction and AI modifier apply to direct messages, with LIVE delivery of the final content
 
@@ -53,7 +59,16 @@ Because the WS accepts on every worker, a message persisted on worker A must sti
 
 ## WS protocol frames
 
-Client -> server: `{type:"send", receiver_uid, content, attachment_uids?, client_id}`, `{type:"typing", receiver_uid}` (throttled client-side), `{type:"read", with_uid}`, `{type:"sync", since, with_uid?}` (replay created-or-revised rows after `since`), `{type:"ping"}` (keepalive, ignored). Server -> client: `{type:"ready", user_uid}` (sent first on connect), `{type:"message", uid, sender_uid, sender_username, sender_role, receiver_uid, content, created_at, time_ago, client_id, attachments, ai_processed, ai_pending}` (broadcast to BOTH sender and receiver sockets so multi-tab and the sender's own optimistic bubble reconcile via the echoed `client_id`; a first frame may set `ai_pending` while a sync job runs, and a later frame for the same `uid` sets `ai_processed` with the rewritten body), `{type:"typing", from_uid}` (to the receiver only), `{type:"read", by_uid}` (read-receipt to the other user), `{type:"error", client_id, text}` (sender only, on a dropped send, e.g. the recipient blocked the sender, or a screened body). There is no `presence` frame on this socket - see below.
+Client -> server: `{type:"send", receiver_uid, content, attachment_uids?, client_id}`, `{type:"typing", receiver_uid}` (throttled client-side), `{type:"read", with_uid}`, `{type:"active", with_uid}` (marks the sender as currently viewing that conversation - see "Active-conversation notification suppression" below), `{type:"sync", since, with_uid?}` (replay created-or-revised rows after `since`), `{type:"ping"}` (keepalive, ignored). Server -> client: `{type:"ready", user_uid}` (sent first on connect), `{type:"message", uid, sender_uid, sender_username, sender_role, receiver_uid, content, created_at, time_ago, client_id, attachments, ai_processed, ai_pending}` (broadcast to BOTH sender and receiver sockets so multi-tab and the sender's own optimistic bubble reconcile via the echoed `client_id`; a first frame may set `ai_pending` while a sync job runs, and a later frame for the same `uid` sets `ai_processed` with the rewritten body), `{type:"typing", from_uid}` (to the receiver only), `{type:"read", by_uid}` (read-receipt to the other user), `{type:"error", client_id, text}` (sender only, on a dropped send, e.g. the recipient blocked the sender, or a screened body). There is no `presence` frame on this socket - see below. `active` has no server -> client echo; it only writes the DB-backed marker consumed server-side by `persist_message`.
+
+## Active-conversation notification suppression
+
+A `message` notification (and its live toast) is suppressed when the receiver is demonstrably looking at that exact conversation right now, so an open DM thread never also produces a redundant bell/toast for the same message. The message itself still delivers live over the WS/relay path exactly as before - only the `create_notification("message", ...)` call in `persist_message` is skipped.
+
+- **Marker: two columns on `users`**, `active_conversation_uid` (the uid of the conversation partner currently being viewed) and `active_conversation_at` (UTC ISO timestamp of the last time it was refreshed), ensured in `database.backfill_api_keys()` like every other non-signup `users` column. This follows the `services/presence.py` `last_seen` pattern deliberately - a WS connection is per-worker (`message_hub` has no cross-worker visibility), but the sender and receiver of a DM can be on different uvicorn workers, so the "is the receiver looking at this" check has to be readable from ANY worker. A DB column is the only cross-worker-correct medium here, exactly as presence is.
+- **Write path**: `services/messaging/active_conversation.py` `touch_active_conversation(viewer_uid, with_uid)`, called from the WS handler's new `{type:"active", with_uid}` frame. Throttled per-worker exactly like `presence.touch` (`WRITE_THROTTLE_SECONDS = 5`, an in-memory `dict[f"{viewer_uid}:{with_uid}" -> monotonic]` so a burst of focus/visibility events costs at most one write per 5s per conversation) before the `UPDATE users SET active_conversation_uid=..., active_conversation_at=...` write.
+- **Client trigger (`AppChat.js`)**: `_sendActiveMarker()` sends the frame whenever `_threadIsActive()` is true (desktop: any open thread; mobile: only when the thread pane, not the list pane, is showing). It fires wherever the existing `_markRead()` already fires (initial WS connect, opening/switching a conversation) plus two new listeners bound in `_bindActiveConversation()`: `window` `focus` and `document` `visibilitychange` (`visibilityState === "visible"`) - so returning to a backgrounded/blurred tab that still has the conversation open re-asserts the marker.
+- **Read path / freshness window**: `is_actively_viewing(viewer_uid, other_uid)` (same module) reads the receiver's user row, checks `active_conversation_uid == other_uid`, and requires `active_conversation_at` to be less than `FRESH_SECONDS = 20` old. `persist_message` calls it as `is_actively_viewing(receiver_uid, sender_uid)` right before the `create_notification` call and skips only that call on a hit; `clear_messages_cache(receiver_uid)` still runs unconditionally, and the WS/relay delivery of the message frame is completely unaffected (it is broadcast from `routers/messages.py`, not from `persist_message`). The freshness window (20s) is intentionally wider than the write throttle (5s) so it never lapses between two consecutive throttled writes, and the marker is a soft "recently confirmed as active" signal, not a hard connection state - if it goes stale (tab closed, thread switched away without a fresh trigger) the very next message just resumes notifying normally, which is the safe failure direction.
 
 ## Presence is NOT part of the messaging WS
 
@@ -65,7 +80,9 @@ Live/echoed message bubbles are NEVER injected as raw HTML. `AppChat._buildBubbl
 
 ## Frontend
 
-The messaging frontend is the single self-booting custom element `<dp-chat>` (`static/js/components/AppChat.js`), which replaced the old page-controller trio `MessagesLayout.js`/`MessagesSocket.js`/`MessageSearch.js` (all deleted) and `messages.css` (superseded by `static/css/chat.css`) - see `devplacepy/static/js/CLAUDE.md` for the component roster entry. `MobileNav.js` does **not** own the mobile pane (that duplicate was removed; `dp-chat` is the only pane controller). `templates/messages.html` renders `<dp-chat mode="page" self-uid="..." with-uid="..." conversations-url="/messages/conversations" search-url="/messages/search" send-url="/messages/send" ws-url="/messages/ws" ai-indicator="true">` wrapping the SAME server-rendered `.messages-list`/`.messages-main`/`.messages-thread`/`.message-bubble` markup as before (no-JS/crawler fallback), which the component *adopts* on `connectedCallback` rather than discarding. It owns a `ChatSocket` (`static/js/chat/ChatSocket.js`, exponential backoff, `4013` fast-path, 25s ping), sends over WS with an optimistic pending bubble keyed by `client_id`, reconciles on the echoed `message` frame, applies later `ai_processed` frames in place (compare `dp-content[data-source]`, never rendered `textContent`), injects the Report control on every live incoming bubble, catches up with `{type:"sync"}` on reconnect, and switches conversations without dropping the socket (`GET /messages?with_uid=` JSON + `history.pushState`). Older history loads via `GET /messages?with_uid=&before=` when the thread is scrolled to the top. Mobile: CSS hides the inactive pane from `with-uid` / `.show-list` before JS runs (no stacked FOUC); the composer does not auto-focus on coarse pointers; keyboard inset uses `visualViewport.offsetTop + height`; Enter-to-send is desktop-only. The send button is disabled only while `dp-upload` is busy, never while a send is in flight. Presence in `mode="page"` stays on the page-global `PresenceManager`. An opt-in `ai-indicator="true"` attribute shows "Adjusting..." while `ai_pending` and "Adjusted by AI" when the revision lands. Styling is in `static/css/chat.css` using `variables.css` tokens only, responsive down to 360px.
+The messaging frontend is the single self-booting custom element `<dp-chat>` (`static/js/components/AppChat.js`), which replaced the old page-controller trio `MessagesLayout.js`/`MessagesSocket.js`/`MessageSearch.js` (all deleted) and `messages.css` (superseded by `static/css/chat.css`) - see `devplacepy/static/js/CLAUDE.md` for the component roster entry. `MobileNav.js` does **not** own the mobile pane (that duplicate was removed; `dp-chat` is the only pane controller). `templates/messages.html` renders `<dp-chat mode="page" self-uid="..." with-uid="..." conversations-url="/messages/conversations" search-url="/messages/search" send-url="/messages/send" ws-url="/messages/ws" ai-indicator="true">` wrapping the SAME server-rendered `.messages-list`/`.messages-main`/`.messages-thread`/`.message-bubble` markup as before (no-JS/crawler fallback), which the component *adopts* on `connectedCallback` rather than discarding. It owns a `ChatSocket` (`static/js/chat/ChatSocket.js`, exponential backoff, `4013` fast-path, 25s ping), sends over WS with an optimistic pending bubble keyed by `client_id`, reconciles on the echoed `message` frame, applies later `ai_processed` frames in place (compare `dp-content[data-source]`, never rendered `textContent`), injects the Report control on every live incoming bubble, catches up with `{type:"sync"}` on reconnect, and switches conversations without dropping the socket (`GET /messages?with_uid=` JSON + `history.pushState`). Older history loads via `GET /messages?with_uid=&before=` when the thread is scrolled to the top. Mobile: CSS hides the inactive pane from `with-uid` / `.show-list` before JS runs (no stacked FOUC); the composer does not auto-focus on coarse pointers; keyboard inset uses `visualViewport.offsetTop + height`; Enter-to-send is desktop-only. **The send button is disabled while `dp-upload` is busy AND while a genuine new send is in flight** (the send-lock, see "Send dedupe" above) - a `.failed`-bubble retry never engages it, so retrying an old message never blocks typing/sending a new one. Presence in `mode="page"` stays on the page-global `PresenceManager`. An opt-in `ai-indicator="true"` attribute shows "Adjusting..." while `ai_pending` and "Adjusted by AI" when the revision lands. Styling is in `static/css/chat.css` using `variables.css` tokens only, responsive down to 360px.
+
+**Auto-scroll never fights a manual scroll-up.** `_startScrollWatcher`/`_stabilizeScroll` force `scrollTop = scrollHeight` on every thread mutation (new bubble, attachment/image finishing loading) ONLY while `_userAtBottom` is true, re-checked every `requestAnimationFrame` up to `STABILIZE_MAX_FRAMES`. The one thing that used to break this: while a multi-frame stabilize loop was in flight (`_stabilizePending`), the real `scroll` event listener (`_bindAutoScroll`) ignored EVERY scroll event outright, including a genuine user wheel/touch scroll-up that happened to land inside that window - so a rapid burst of incoming messages (each restarting the stabilize loop while attachments were still loading) could silently overwrite a user's attempt to scroll up to read history. The fix: every programmatic scroll (`_scrollThreadToEnd`, `_stabilizeScroll`) records the exact `scrollTop` it just set in `_lastForcedScrollTop`; the `scroll` listener only skips its own echo (`_stabilizePending && scrollTop === _lastForcedScrollTop`) and always processes a scroll that lands anywhere else, so a real user scroll-up is detected and `_userAtBottom` flips to `false` immediately, even mid-stabilization. Never gate scroll-intent detection on `_stabilizePending` alone again.
 
 ## Attachments stream live
 

devplacepy/services/messaging/__init__.py

@@ -1,5 +1,9 @@
 # retoor <retoor@molodetz.nl>
 
+from devplacepy.services.messaging.active_conversation import (
+    is_actively_viewing,
+    touch_active_conversation,
+)
 from devplacepy.services.messaging.hub import message_hub
 from devplacepy.services.messaging.persist import (
     message_frame,
@@ -11,6 +15,7 @@ from devplacepy.services.messaging.relay import message_relay
 from devplacepy.services.messaging.tickets import issue_ticket, redeem_ticket
 
 __all__ = [
+    "is_actively_viewing",
     "issue_ticket",
     "message_frame",
     "message_hub",
@@ -19,4 +24,5 @@ __all__ = [
     "push_content_revision",
     "redeem_ticket",
     "stamp_content_revision",
+    "touch_active_conversation",
 ]

devplacepy/services/messaging/active_conversation.py

@@ -0,0 +1,52 @@
+# retoor <retoor@molodetz.nl>
+
+import time
+from datetime import datetime, timezone
+from typing import Optional
+
+from devplacepy.database import get_table
+
+WRITE_THROTTLE_SECONDS = 5
+FRESH_SECONDS = 20
+
+_last_write: dict[str, float] = {}
+
+
+def touch_active_conversation(viewer_uid: str, with_uid: str) -> None:
+    if not viewer_uid or not with_uid:
+        return
+    now = time.monotonic()
+    key = f"{viewer_uid}:{with_uid}"
+    if now - _last_write.get(key, 0.0) < WRITE_THROTTLE_SECONDS:
+        return
+    _last_write[key] = now
+    get_table("users").update(
+        {
+            "uid": viewer_uid,
+            "active_conversation_uid": with_uid,
+            "active_conversation_at": datetime.now(timezone.utc).isoformat(),
+        },
+        ["uid"],
+    )
+
+
+def _seconds_since(stamped: Optional[str]) -> Optional[float]:
+    if not stamped:
+        return None
+    try:
+        seen = datetime.fromisoformat(stamped)
+    except (TypeError, ValueError):
+        return None
+    if seen.tzinfo is None:
+        seen = seen.replace(tzinfo=timezone.utc)
+    return (datetime.now(timezone.utc) - seen).total_seconds()
+
+
+def is_actively_viewing(viewer_uid: str, other_uid: str) -> bool:
+    if not viewer_uid or not other_uid:
+        return False
+    row = get_table("users").find_one(uid=viewer_uid)
+    if not row or row.get("active_conversation_uid") != other_uid:
+        return False
+    elapsed = _seconds_since(row.get("active_conversation_at"))
+    return elapsed is not None and elapsed < FRESH_SECONDS

devplacepy/services/messaging/persist.py

@@ -2,7 +2,7 @@
 
 import asyncio
 import logging
-from datetime import datetime, timezone
+from datetime import datetime, timedelta, timezone
 from typing import Any, Optional
 
 from devplacepy.attachments import get_attachments, link_attachments
@@ -18,6 +18,7 @@ from devplacepy.utils import (
 from devplacepy.services.audit import record as audit
 from devplacepy.services.correction import schedule_correction
 from devplacepy.services.ai_modifier import schedule_modification
+from devplacepy.services.messaging.active_conversation import is_actively_viewing
 from devplacepy.services.moderation.screening import (
     record as record_screening,
     refuse_if_blocked,
@@ -27,6 +28,7 @@ from devplacepy.services.moderation.screening import (
 logger = logging.getLogger("messaging.persist")
 
 MAX_CONTENT_LENGTH = 2000
+DEDUPE_WINDOW_SECONDS = 30
 
 
 def _slim_attachment(attachment: dict[str, Any]) -> dict[str, Any]:
@@ -64,6 +66,18 @@ def message_frame(
     }
 
 
+def _find_recent_duplicate(sender_uid: str, client_id: str) -> Optional[dict[str, Any]]:
+    cutoff = (
+        datetime.now(timezone.utc) - timedelta(seconds=DEDUPE_WINDOW_SECONDS)
+    ).isoformat()
+    row = get_table("messages").find_one(
+        sender_uid=sender_uid,
+        client_id=client_id,
+        created_at={">=": cutoff},
+    )
+    return dict(row) if row else None
+
+
 def persist_message(
     sender: dict[str, Any],
     receiver_uid: str,
@@ -72,23 +86,30 @@ def persist_message(
     *,
     request: Any = None,
     origin: str = "web",
+    client_id: Optional[str] = None,
 ) -> Optional[dict[str, Any]]:
     content = (content or "").strip()[:MAX_CONTENT_LENGTH]
     attachment_uids = attachment_uids or []
     if not content and not attachment_uids:
         return None
 
+    sender_uid = sender["uid"]
+    client_id = (client_id or "").strip()[:64] or None
+    if client_id:
+        duplicate = _find_recent_duplicate(sender_uid, client_id)
+        if duplicate is not None:
+            return duplicate
+
     receiver = get_table("users").find_one(uid=receiver_uid)
     if not receiver:
         return None
 
-    if sender["uid"] in get_blocked_uids(receiver_uid):
+    if sender_uid in get_blocked_uids(receiver_uid):
         return None
 
     screening = screen_fields("messages", {"content": content})
     refuse_if_blocked(screening)
 
-    sender_uid = sender["uid"]
     sender_username = sender.get("username", "")
     messages_table = get_table("messages")
     msg_uid = generate_uid()
@@ -102,6 +123,7 @@ def persist_message(
             "read": False,
             "created_at": created_at,
             "updated_at": None,
+            "client_id": client_id,
         }
     )
 
@@ -117,13 +139,14 @@ def persist_message(
     schedule_modification(sender, "messages", msg_uid, request)
 
     if sender_uid != receiver_uid:
-        create_notification(
+        if not is_actively_viewing(receiver_uid, sender_uid):
-            receiver_uid,
+            create_notification(
-            "message",
+                receiver_uid,
-            f"{sender_username} sent you a message",
+                "message",
-            sender_uid,
+                f"{sender_username} sent you a message",
-            f"/messages?with_uid={sender_uid}",
+                sender_uid,
-        )
+                f"/messages?with_uid={sender_uid}",
+            )
         clear_messages_cache(receiver_uid)
 
     create_mention_notifications(
@@ -182,6 +205,7 @@ def persist_message(
         "read": False,
         "created_at": created_at,
         "updated_at": None,
+        "client_id": client_id,
     }
 
 

devplacepy/services/openai_gateway/CLAUDE.md

@@ -42,6 +42,16 @@ Callers may send an optional `X-App-Reference` header to tag gateway calls by ap
 
 **Per-user attribution:** the gateway also accepts a real user's `api_key` (Bearer / X-API-KEY / session), gated by `gateway_allow_users` (default **on**); `resolve_owner()` records `(owner_kind, owner_id)` per call, so usage is attributed and limitable per user. With `gateway_allow_users` off, only internal/access/admin keys work and per-user attribution never happens. Devii operating a signed-in user authenticates its LLM calls with that user's own `api_key` (set as the session's `ai_key` in `build_settings(..., owner_kind="user")`), so a user's full gateway spend (Devii and direct API calls) rolls up under their uid - surfaced admin-only on the profile page via `build_user_usage()` / `GET /admin/users/{uid}/ai-usage`. Guests keep the internal key.
 
+### Failed-auth throttle (`services/openai_gateway/auth_throttle.py`)
+
+A dedicated per-IP throttle protects `/openai/v1/*` against unauthenticated probing WITHOUT IP-allowlisting the endpoint (which would break legitimate external API consumers using `gateway_access_key`/`gateway_allow_users`). It is scoped ONLY to failed authentication attempts, never to general traffic - the existing global rate limiter already exempts `/openai` entirely (`main.py`), and this throttle does not change that exemption.
+
+- **Config (`gateway_auth_throttle_enabled` default on, `gateway_auth_throttle_max_failures` default 10, `gateway_auth_throttle_window_seconds` default 60, all group "Access").** An admin can retune or disable the throttle with no code change, same as every other gateway setting.
+- **Counter.** `auth_throttle.py` is a small in-process sliding-window store (`dict[ip] -> list[float]` timestamps), the same shape as the global rate limiter's `_rate_limit_store` in `main.py`, but a separate, dedicated store local to the gateway rather than reusing that one (the two throttles key on different events - all mutating traffic vs. failed gateway auth - and `/openai` is deliberately exempt from the general limiter). `record_failure(ip, window)` appends a timestamp and returns the running in-window count; `is_throttled(ip, threshold, window)` checks without recording. Per-worker, like every other in-process cache in this codebase - not divided by worker count, so the effective aggregate threshold across `N` workers is up to `N x max_failures`; this is an accepted tradeoff (it still closes the probe, just at a slightly higher aggregate bound) rather than importing `main.WEB_WORKERS` into a leaf service module.
+- **`GatewayService.authorize()` records a failure on every deny path** (a cheap static-key/internal-key match still short-circuits first and is never counted), so a wrong or garbage key counts exactly like presenting nothing. **The fast 429 short-circuit fires only when the request carries no credential at all** (no `X-API-KEY`, no `Authorization` header of any scheme, no `session` cookie) AND the IP is already over threshold - this is the "skip the rest of `authorize()`" defense against wasted CPU on a blind flood. A request presenting ANY credential - even one that turns out invalid - always runs the full check, so it can never be blocked by another caller's failures sharing its IP (a NAT/proxy caller with its own valid key is never punished for a neighbor's failed probes); only a request presenting nothing is fast-denied once its IP has tripped.
+- **Audit.** Tripping the throttle (the exact request whose failure brings the window count to the configured threshold) records `ai.gateway_auth_throttle.tripped` (category `ai`, `result="denied"`, metadata `ip`/`failed_attempts`/`threshold`/`window_seconds`) via `audit.record(request, ...)`. Every subsequent fast-denied request in the same trip is silent (no audit spam under an actual flood) - the trip itself is the visible signal for admins.
+- **Internal traffic cannot trip this by construction.** Every internal consumer (Devii guests, AI correction/modifier, quiz grading, SEO metadata generation, news, bots) authenticates with `database.internal_gateway_key()`, which is the exact same value `authorize()` compares against as `gateway_internal_key` - that comparison is the FIRST check in `authorize()`, before the throttle is even consulted, so internal self-dial calls always succeed there and never reach the failure-recording branch.
+
 ## Config fields
 
 All config is `config_fields` (upstream url/model/key, force-model, the Prompt group's `gateway_system_preamble` and `gateway_thinking` (default off), timeout, instances, vision url/model/key/cache/toggle, the Embeddings group (`gateway_embed_enabled`/`_url`/`_model`/`_key`), the auth toggles + static key + internal key, plus the Pricing/Reliability/Tracking groups); live metrics (requests/errors/in-flight/vision-calls/embed-calls/latency plus 24h cost/tokens/success rollups) via `collect_metrics`.
@@ -57,6 +67,15 @@ Two behaviors:
 
 Embeddings/passthrough are untouched by this composition step.
 
+### Per-call operator preamble bypass (`bypass_preamble`, internal-only)
+
+A structured/deterministic internal caller (one that expects a strict, machine-parsed output shape - grading JSON, a corrected text field, generated SEO metadata) can ask the gateway to skip `gateway_system_preamble` for that one call, since an operator preamble written for conversational tone can conflict with a task that demands exact output discipline. The date line and the client's own system content are unaffected either way.
+
+- **Signal.** The chat request body carries `"bypass_preamble": true`. `GatewayRuntime.handle_chat` pops it off `body` before it is ever copied into the upstream `payload` (so it never leaks to the upstream provider) and only honors it when the caller also proved internal credentials for that same request (see below); otherwise it is silently discarded, exactly like an ordinary, unsupported field.
+- **Internal-only gate.** `GatewayService.internal_bypass_allowed(request, cfg)` requires the request to carry the `X-Gateway-Internal-Key` header equal to the configured `gateway_internal_key` setting. This is a SEPARATE header from `Authorization`/`X-API-KEY` on purpose: a caller that authenticates with a real user's own `api_key` (correction, the `@ai` modifier, quiz free-text grading - all per-user-attributed on purpose, for correct spend attribution) still proves it is DevPlace's own server code, not the user's browser or a malicious client holding that same api_key, by additionally presenting the internal key on this header. A caller that authenticates AS the internal key directly (SEO metadata generation, via `database.internal_gateway_key()`) can present the same header too - one mechanism covers both shapes of internal caller. `GatewayService.handle()` computes this boolean once per request and passes it into `handle_chat` as `bypass_allowed`; a request with no valid internal credentials gets `bypass_allowed=False` and the body field is ignored even if present - it can never come from an external API consumer or a public-facing client call.
+- **Callers that set it.** The four structured/deterministic internal callers all route through the single shared `services/correction.py::gateway_complete(..., bypass_preamble=True)` helper, which sets both the body field and the `X-Gateway-Internal-Key` header (via `database.internal_gateway_key()`) in one place: `services/correction.py::correct_text` (AI content correction), `services/ai_modifier.py::modify_text` (the `@ai` inline modifier), `services/quiz/grading.py::grade_free_text` (quiz free-text grading), and `services/jobs/seo_meta_service.py`'s generation call (SEO metadata). **Genuinely conversational internal callers are deliberately left unchanged** and keep receiving the operator preamble: Devii chat sessions and the docs chat do not call `gateway_complete` at all, so they are unaffected by construction.
+- **Other internal self-dial callers found but NOT updated** (each builds its own request instead of using `gateway_complete`, and each is a candidate for the same treatment in a follow-up): `services/gitea/enhance.py::enhance_ticket` and `services/gitea/planning.py::generate_plan` (both structured, single-shot ticket text generation), `services/dbapi/nl2sql.py::design_query` (NL-to-SQL, strictly structured), `services/deepsearch/llm.py`/`services/jobs/deepsearch/enhance.py` (planner/synthesis calls), and `services/jobs/isslop/agent/llm.py` (per-file classification). None of these were in scope for this change.
+
 ## Thinking default (fast path)
 
 `thinking.py` (`apply_thinking`) runs in `handle_chat` after `stream_options` is stripped, and in the vision describe call. DeepSeek V4 **enables thinking by default**; leaving the payload alone is the slow path. The gateway therefore always writes an explicit thinking field in the upstream dialect:
@@ -127,7 +146,7 @@ The human-facing **Quota rules** tab of `/admin/gateway` is a third, independent
 
 ## Embeddings
 
-`POST /openai/v1/embeddings` exposes an OpenAI-compatible text-embeddings model. Clients send the generic model `molodetz~embed` (`config.INTERNAL_EMBED_MODEL`), which `handle_embeddings` remaps to `gateway_embed_model` exactly like chat remaps `molodetz` -> `gateway_model` (also remapped when `gateway_force_model` is on or the model is empty). It defaults to OpenRouter's `qwen/qwen3-embedding-8b` at `https://openrouter.ai/api/v1/embeddings` (`config.EMBED_*_DEFAULT`, $0.01 per 1M input tokens). `handle_embeddings` mirrors `handle_chat` but is simpler: no vision augmentation and no streaming - build the payload, forward via `_send`, and record one ledger row through the same `finalize(...)` closure. The config fields are the **Embeddings** group (`gateway_embed_enabled` default on, `gateway_embed_url`, `gateway_embed_model`, `gateway_embed_key`) plus the Pricing-group `gateway_embed_price_input_per_m`. `effective_config()` falls the embed key back to `gateway_vision_key` then `OPENROUTER_API_KEY` (NOT `gateway_api_key`: that is the DeepSeek chat upstream key, whereas embeddings target OpenRouter like vision does). Usage is recorded with **`backend="embed"`**; `usage.compute_cost` adds an `embed` branch (input-only, completion always 0, native OpenRouter `cost` still preferred) and `Pricing` gained `embed_input_per_m`. `analytics.py` groups by `backend` generically, so embed rows roll up automatically; `caching_savings` counts only **non-native** chat rows (native-priced rows did not use the configured cache-hit/miss rates, so folding them in would report a fictional saving). When `gateway_embed_enabled` is off the endpoint returns 503 with no ledger row.
+`POST /openai/v1/embeddings` exposes an OpenAI-compatible text-embeddings model. Clients send the generic model `molodetz~embed` (`config.INTERNAL_EMBED_MODEL`), which `handle_embeddings` remaps to `gateway_embed_model` exactly like chat remaps `molodetz` -> `gateway_model` (also remapped when `gateway_force_model` is on or the model is empty). It defaults to OpenRouter's `qwen/qwen3-embedding-8b` at `https://openrouter.ai/api/v1/embeddings` (`config.EMBED_*_DEFAULT`, $0.01 per 1M input tokens). `handle_embeddings` mirrors `handle_chat` but is simpler: no vision augmentation and no streaming - build the payload, forward via `_send`, and record one ledger row through the same `finalize(...)` closure. The config fields are the **Embeddings** group (`gateway_embed_enabled` default on, `gateway_embed_url`, `gateway_embed_model`, `gateway_embed_key`) plus the Pricing-group `gateway_embed_price_input_per_m`. `effective_config()` falls the embed key back to `gateway_vision_key` then `OPENROUTER_API_KEY` (NOT `gateway_api_key`: that is the DeepSeek chat upstream key, whereas embeddings target OpenRouter like vision does). Usage is recorded with **`backend="embed"`**; `usage.compute_cost` adds an `embed` branch (input-only, completion always 0, native OpenRouter `cost` still preferred) and `Pricing` gained `embed_input_per_m`. `analytics.py` groups by `backend` generically, so embed rows roll up automatically; `caching_savings` counts only **non-native** chat rows (native-priced rows did not use the configured cache-hit/miss rates, so folding them in would report a fictional saving). When `gateway_embed_enabled` is off the endpoint returns 503 with no ledger row. **When the resolved `gateway_embed_key` is still blank after the `effective_config()`/route-overlay merge** (all of `gateway_embed_key`, `gateway_vision_key`, and `OPENROUTER_API_KEY` are unset, or a per-route provider was configured with no `api_key` of its own), `handle_embeddings` returns the same `503` shape (`{"error": {"message": "Embeddings are not configured", "type": "embeddings_not_configured"}}`) with an `ai.gateway.call` audit row (`result="denied"`, `summary="no embeddings key configured"`) instead of forwarding an unauthenticated request upstream - mirrors the `gateway_embed_enabled` disabled-check immediately above it, no ledger row either. `routing.embed_overlay` only overlays `gateway_embed_key` when the matched route's provider actually has a non-blank `api_key`, so a misconfigured route can never downgrade an already-good fallback key to empty.
 
 ## Single point of truth for AI
 

devplacepy/services/openai_gateway/auth_throttle.py

@@ -0,0 +1,56 @@
+# retoor <retoor@molodetz.nl>
+
+import time
+from collections import defaultdict
+
+_failures: dict[str, list[float]] = defaultdict(list)
+_last_sweep = 0.0
+SWEEP_INTERVAL_SECONDS = 60.0
+
+
+def _window_slice(ip: str, window_seconds: int, now: float) -> list[float]:
+    window_start = now - window_seconds
+    timestamps = [t for t in _failures.get(ip, ()) if t > window_start]
+    if timestamps:
+        _failures[ip] = timestamps
+    elif ip in _failures:
+        del _failures[ip]
+    return timestamps
+
+
+def _sweep(window_start: float) -> None:
+    stale = [
+        ip
+        for ip, timestamps in _failures.items()
+        if not timestamps or timestamps[-1] <= window_start
+    ]
+    for ip in stale:
+        del _failures[ip]
+
+
+def is_throttled(ip: str, threshold: int, window_seconds: int) -> bool:
+    now = time.time()
+    return len(_window_slice(ip, window_seconds, now)) >= threshold
+
+
+def record_failure(ip: str, window_seconds: int) -> int:
+    global _last_sweep
+    now = time.time()
+    window_start = now - window_seconds
+    if now - _last_sweep >= SWEEP_INTERVAL_SECONDS:
+        _sweep(window_start)
+        _last_sweep = now
+    timestamps = _window_slice(ip, window_seconds, now)
+    timestamps.append(now)
+    _failures[ip] = timestamps
+    return len(timestamps)
+
+
+def reset(ip: str) -> None:
+    _failures.pop(ip, None)
+
+
+def clear() -> None:
+    _failures.clear()
+    global _last_sweep
+    _last_sweep = 0.0

devplacepy/services/openai_gateway/config.py

@@ -6,6 +6,9 @@ TIMEOUT_DEFAULT = 300
 TIMEOUT_MIN = 300
 INSTANCES_DEFAULT = 4
 
+AUTH_THROTTLE_MAX_FAILURES_DEFAULT = 10
+AUTH_THROTTLE_WINDOW_SECONDS_DEFAULT = 60
+
 SYSTEM_PREAMBLE_DEFAULT = ""
 THINKING_DEFAULT = False
 

devplacepy/services/openai_gateway/gateway.py

@@ -350,9 +350,17 @@ class GatewayRuntime:
         return resp, None, timing
 
     async def handle_chat(
-        self, body: dict, cfg: dict, owner: tuple, user_agent: str, app_reference: str, log=None
+        self,
+        body: dict,
+        cfg: dict,
+        owner: tuple,
+        user_agent: str,
+        app_reference: str,
+        log=None,
+        bypass_allowed: bool = False,
     ):
         log = log or (lambda message: None)
+        bypass_preamble = bool(body.pop("bypass_preamble", False)) and bypass_allowed
         overlay = chat_overlay(body.get("model"), cfg)
         base_cfg = cfg
         if overlay:
@@ -391,7 +399,8 @@ class GatewayRuntime:
             self.vision_calls += augmenter.calls
             vision_cost = augmenter.cost_usd
 
-        messages = apply_system_directives(messages, cfg.get("gateway_system_preamble", ""))
+        preamble = "" if bypass_preamble else cfg.get("gateway_system_preamble", "")
+        messages = apply_system_directives(messages, preamble)
 
         requested = body.get("model")
         allow_client_model = bool(cfg.get("gateway_allow_client_model"))
@@ -842,6 +851,35 @@ class GatewayRuntime:
                     }
                 },
             )
+        if not cfg["gateway_embed_key"]:
+            from devplacepy.services.audit import record as audit
+            from devplacepy.services.openai_gateway.usage import audit_actor_for
+
+            actor_kind, actor_uid, actor_role = audit_actor_for(owner[0], owner[1])
+            audit.record_system(
+                "ai.gateway.call",
+                actor_kind=actor_kind,
+                actor_uid=actor_uid,
+                actor_role=actor_role,
+                origin="api",
+                result="denied",
+                summary="no embeddings key configured",
+                metadata={
+                    "backend": "embed",
+                    "endpoint": "embeddings",
+                    "owner_kind": owner[0],
+                    "owner_id": owner[1],
+                },
+            )
+            return JSONResponse(
+                status_code=503,
+                content={
+                    "error": {
+                        "message": "Embeddings are not configured",
+                        "type": "embeddings_not_configured",
+                    }
+                },
+            )
         client, sem = self._ensure(cfg)
         params = extract_params(body)
         handle_start = time.monotonic()
@@ -866,13 +904,8 @@ class GatewayRuntime:
             "Content-Type": "application/json",
             **_attribution_headers(),
             **_extra_provider_headers(cfg),
+            "Authorization": f"Bearer {cfg['gateway_embed_key']}",
         }
-        if cfg["gateway_embed_key"]:
-            headers["Authorization"] = f"Bearer {cfg['gateway_embed_key']}"
-        else:
-            log(
-                "No upstream embeddings API key configured (gateway_embed_key / gateway_vision_key / OPENROUTER_API_KEY); upstream will likely reject the request"
-            )
 
         resp, exc, timing = await self._send(
             client,

devplacepy/services/openai_gateway/service.py

@@ -11,16 +11,19 @@ from fastapi.responses import JSONResponse
 
 from devplacepy.database import get_int_setting
 from devplacepy.services.base import BaseService, ConfigField
-from devplacepy.services.openai_gateway import config, quota
+from devplacepy.services.openai_gateway import auth_throttle, config, quota
 from devplacepy.services.openai_gateway.analytics import summary_metrics
 from devplacepy.services.openai_gateway.gateway import GatewayRuntime
 from devplacepy.services.openai_gateway.routing import model_store
 from devplacepy.utils import get_current_user
+from devplacepy.utils.auth import _request_has_auth
+from devplacepy.utils.guards import client_ip
 
 logger = logging.getLogger(__name__)
 
 APP_REFERENCE_PATTERN = re.compile(r"^[a-zA-Z0-9_.-]{1,30}$")
 DEFAULT_APP_REFERENCE = "default"
+INTERNAL_KEY_HEADER = "X-Gateway-Internal-Key"
 
 USER_CONTENT_OWNER_KINDS = ("user", "admin")
 
@@ -324,6 +327,36 @@ class GatewayService(BaseService):
             "with this key. Clear it and restart to rotate.",
             group="Access",
         ),
+        ConfigField(
+            "gateway_auth_throttle_enabled",
+            "Failed-auth throttle",
+            type="bool",
+            default=True,
+            help="Track failed authentication attempts per IP and block further "
+            "unauthenticated attempts from an IP once it crosses the failure "
+            "threshold within the window. Never blocks a request that presents "
+            "valid credentials, regardless of what its IP has done.",
+            group="Access",
+        ),
+        ConfigField(
+            "gateway_auth_throttle_max_failures",
+            "Failed-auth threshold",
+            type="int",
+            default=config.AUTH_THROTTLE_MAX_FAILURES_DEFAULT,
+            minimum=1,
+            help="Failed authentication attempts allowed from one IP within the "
+            "window before further unauthenticated attempts are blocked with 429.",
+            group="Access",
+        ),
+        ConfigField(
+            "gateway_auth_throttle_window_seconds",
+            "Failed-auth window (seconds)",
+            type="int",
+            default=config.AUTH_THROTTLE_WINDOW_SECONDS_DEFAULT,
+            minimum=1,
+            help="Sliding window the failed-auth threshold is counted over.",
+            group="Access",
+        ),
         ConfigField(
             quota.FIELD_DEFAULT_USER,
             "Default per-user daily cap ($)",
@@ -537,6 +570,24 @@ class GatewayService(BaseService):
                 )
         return cfg
 
+    def _audit_throttle_tripped(
+        self, request: Request, ip: str, count: int, threshold: int, window: int
+    ) -> None:
+        from devplacepy.services.audit import record as audit
+
+        audit.record(
+            request,
+            "ai.gateway_auth_throttle.tripped",
+            result="denied",
+            summary=f"AI gateway auth throttle tripped for {ip} after {count} failed attempts",
+            metadata={
+                "ip": ip,
+                "failed_attempts": count,
+                "threshold": threshold,
+                "window_seconds": window,
+            },
+        )
+
     def authorize(self, request: Request) -> bool:
         cfg = self.get_config()
         if not cfg["gateway_require_auth"]:
@@ -548,12 +599,35 @@ class GatewayService(BaseService):
             return True
         if presented and internal_key and presented == internal_key:
             return True
+        throttle_enabled = cfg.get("gateway_auth_throttle_enabled", True)
+        threshold = max(1, int(cfg.get("gateway_auth_throttle_max_failures", 10)))
+        window = max(1, int(cfg.get("gateway_auth_throttle_window_seconds", 60)))
+        has_credential = (
+            bool(presented)
+            or bool(request.cookies.get("session"))
+            or _request_has_auth(request)
+        )
+        ip = client_ip(request, default="unknown")
+        if (
+            throttle_enabled
+            and not has_credential
+            and auth_throttle.is_throttled(ip, threshold, window)
+        ):
+            raise HTTPException(
+                status_code=429,
+                detail="Too many failed authentication attempts. Try again later.",
+                headers={"Retry-After": str(window)},
+            )
         user = get_current_user(request)
         if user:
             if user.get("role") == "Admin" and cfg["gateway_allow_admins"]:
                 return True
             if cfg["gateway_allow_users"]:
                 return True
+        if throttle_enabled:
+            count = auth_throttle.record_failure(ip, window)
+            if count == threshold:
+                self._audit_throttle_tripped(request, ip, count, threshold, window)
         return False
 
     def resolve_owner(self, request: Request) -> tuple:
@@ -577,6 +651,13 @@ class GatewayService(BaseService):
             return (kind, user.get("uid") or "unknown")
         return ("anonymous", "anonymous")
 
+    def internal_bypass_allowed(self, request: Request, cfg: dict) -> bool:
+        internal_key = cfg.get("gateway_internal_key")
+        if not internal_key:
+            return False
+        presented = request.headers.get(INTERNAL_KEY_HEADER, "").strip()
+        return bool(presented) and presented == internal_key
+
     def user_content_owner(self, owner: tuple) -> str:
         if owner[0] in USER_CONTENT_OWNER_KINDS and owner[1]:
             return owner[1]
@@ -722,7 +803,10 @@ class GatewayService(BaseService):
             if not isinstance(body, dict):
                 self.log("Rejected chat request: JSON body was not an object")
                 raise HTTPException(status_code=400, detail="Invalid JSON body")
-            return await runtime.handle_chat(body, cfg, owner, user_agent, app_reference, self.log)
+            bypass_allowed = self.internal_bypass_allowed(request, cfg)
+            return await runtime.handle_chat(
+                body, cfg, owner, user_agent, app_reference, self.log, bypass_allowed
+            )
         if subpath == "embeddings" and request.method == "POST":
             try:
                 body = await request.json()

devplacepy/services/quiz/grading.py

@@ -62,6 +62,7 @@ def grade_free_text(api_key: str, question: dict, answer_text: str) -> scoring.G
             build_prompt(question, answer_text),
             QUIZ_GRADING_TIMEOUT_SECONDS,
             model=grading_model(),
+            bypass_preamble=True,
         )
     except Exception as exc:
         logger.warning("Quiz AI grading failed: %s", exc)

devplacepy/services/telegram/bridge.py

@@ -404,6 +404,7 @@ class TelegramBridge:
                 chat_id, "Your daily AI quota is reached (100%). Please try again later."
             )
             return
+        devii.maybe_warn_quota_threshold("user", owner_id, owner_is_admin)
         session = devii.hub().get_or_create(
             "user",
             owner_id,

devplacepy/static/css/CLAUDE.md

@@ -40,3 +40,4 @@ One global rule at the end of `base.css` collapses every animation/transition to
 - Every fluid grid/flex column sets `min-width: 0`, and every fluid grid track that holds content is written `minmax(0, 1fr)`, never a bare `1fr`. A `1fr` track's automatic minimum is the item's min-content size, so one unwrappable line inside a rendered code block (`.rendered-content pre`, `white-space: pre`) widens the track and blows the whole page open sideways. This is why a detail page (a `max-width` block, definite width, the `pre` scrolls inside it) survives content that destroys a listing grid. The rule covers a column that is fixed-width at desktop but becomes the single fluid column at a breakpoint (`.profile-sidebar`), and it is regression-tested by `assert_no_horizontal_overflow` in `tests/conftest.py`.
 - `!important` is allowed only for: the `.hidden`/`[hidden]` display utilities, the global reduced-motion rule, and the devii-avatar third-party-beating override. Anything else is a specificity problem to be fixed structurally.
 - Page-specific CSS lives in its own `static/css/*.css` loaded via `{% block extra_head %}`, never an inline `<style>` block; shared component styles live once (`components.css`, `feed.css` vote buttons) and are never redefined per page.
+- **Never give a sticky column its own CSS `max-height`/`overflow-y`.** A column shorter than the tallest column on the page renders a dead gap below it once capped (see root `CLAUDE.md`, commit `b475e7d6`, reverting exactly that). Independent scroll on a sticky column taller than the viewport is JS-driven: `static/js/StickyScrollPane.js` measures each managed element's real content height against the viewport space actually available and only then toggles `.sticky-scroll-active` (`overflow-y: auto`, defined once in `components.css`) plus a computed inline `max-height`. Add a new sticky column to its selector list in `Application.js` instead of writing CSS for it.

devplacepy/static/css/base.css

@@ -684,24 +684,6 @@ img {
 .topnav-user-dropdown:hover .dropdown-menu,
 .topnav-user-dropdown.open .dropdown-menu { display: block; }
 
-.topnav-tools-dropdown { position: relative; }
-.topnav-tools-toggle { display: inline-flex; align-items: center; }
-.topnav-tools-dropdown.active .topnav-tools-toggle { color: var(--accent); }
-.topnav-tools-dropdown .dropdown-menu {
-    display: none;
-    position: absolute;
-    top: 100%;
-    right: 0;
-    min-width: 180px;
-    background: var(--bg-secondary);
-    border: 1px solid var(--border);
-    border-radius: var(--radius);
-    box-shadow: var(--shadow);
-    padding: 0.25rem 0;
-    z-index: var(--z-nav-drop);
-}
-.topnav-tools-dropdown:hover .dropdown-menu,
-.topnav-tools-dropdown.open .dropdown-menu { display: block; }
 .dropdown-item {
     display: block;
     padding: 0.5rem 0.875rem;
@@ -1213,7 +1195,8 @@ body:has(.page-messages) {
         display: none;
     }
 
-    .topnav-tools-dropdown,
+    .topnav-icon[href="/quizzes"],
+    .topnav-icon[href="/battles"],
     .topnav-icon[href="/leaderboard"],
     .app-store-nav-link {
         display: none;

devplacepy/static/css/components.css

@@ -301,6 +301,11 @@ dp-upload[hidden] {
     border-color: var(--danger);
 }
 
+.sticky-scroll-active {
+    overflow-y: auto;
+    overscroll-behavior: contain;
+}
+
 emoji-picker {
     color-scheme: dark;
     --background: var(--bg-modal);

devplacepy/static/css/deepsearch.css

@@ -393,6 +393,71 @@
     color: var(--accent);
 }
 
+.ds-history-list {
+    display: flex;
+    flex-direction: column;
+    gap: var(--space-lg);
+}
+
+.ds-history-item {
+    margin-bottom: 0;
+}
+
+.ds-history-item-head {
+    display: flex;
+    align-items: center;
+    justify-content: space-between;
+    gap: var(--space-md);
+    margin-bottom: var(--space-sm);
+}
+
+.ds-history-query {
+    font-size: 1.05rem;
+    margin: 0;
+}
+
+.ds-history-status {
+    text-transform: capitalize;
+    font-size: 0.75rem;
+    padding: 2px 10px;
+    border-radius: var(--radius);
+    background: var(--bg-card-hover);
+    color: var(--text-muted);
+    white-space: nowrap;
+}
+
+.ds-history-status.ds-status-done {
+    background: rgba(var(--accent-rgb), 0.15);
+    color: var(--success);
+}
+
+.ds-history-status.ds-status-failed {
+    color: var(--danger);
+}
+
+.ds-history-status.ds-status-running,
+.ds-history-status.ds-status-pending {
+    color: var(--warning);
+}
+
+.ds-history-summary {
+    color: var(--text-secondary);
+    margin: 0 0 var(--space-md);
+}
+
+.ds-history-item-foot {
+    display: flex;
+    align-items: center;
+    justify-content: space-between;
+    gap: var(--space-md);
+    margin-top: var(--space-md);
+}
+
+.ds-history-expired {
+    color: var(--text-muted);
+    font-size: 0.8125rem;
+}
+
 .ds-exports {
     display: flex;
     gap: var(--space-sm);

devplacepy/static/css/docs.css

@@ -218,6 +218,23 @@
     font-weight: 700;
 }
 
+.param-nullable {
+    font-size: 0.625rem;
+    text-transform: uppercase;
+    letter-spacing: 0.05em;
+    color: var(--info);
+    border: 1px solid var(--info);
+    border-radius: 999px;
+    padding: 0.05rem 0.4rem;
+}
+
+.param-static {
+    font-family: "SF Mono", Monaco, "Cascadia Code", monospace;
+    font-size: 0.75rem;
+    color: var(--text-muted);
+    font-style: italic;
+}
+
 .param-loc {
     font-size: 0.625rem;
     text-transform: uppercase;

devplacepy/static/css/engagement.css

@@ -376,3 +376,87 @@
     color: var(--text-muted);
     font-size: 0.8rem;
 }
+
+.note-widget {
+    position: relative;
+    display: inline-flex;
+}
+
+.note-btn.has-note {
+    color: var(--accent);
+    border-color: var(--accent);
+}
+
+.note-editor {
+    position: absolute;
+    top: 100%;
+    left: 0;
+    z-index: var(--z-popover);
+    width: 280px;
+    margin-top: var(--space-xs);
+    padding: var(--space-md);
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    background: var(--bg-card);
+    box-shadow: 0 4px 16px rgba(0, 0, 0, 0.35);
+}
+
+.note-editor-textarea {
+    width: 100%;
+    min-height: 96px;
+    resize: vertical;
+    font-size: 0.85rem;
+}
+
+.note-editor-actions {
+    display: flex;
+    justify-content: flex-end;
+    gap: var(--space-sm);
+    margin-top: var(--space-sm);
+}
+
+.note-item {
+    display: flex;
+    flex-direction: column;
+    gap: var(--space-sm);
+    align-items: stretch;
+}
+
+.note-item-link {
+    display: flex;
+    align-items: center;
+    gap: var(--space-md);
+    text-decoration: none;
+    color: var(--text-primary);
+}
+
+.note-item-content {
+    margin: 0;
+    color: var(--text-muted);
+    font-size: 0.85rem;
+    white-space: pre-wrap;
+}
+
+.note-item-delete-form {
+    align-self: flex-end;
+}
+
+@keyframes engagement-busy-pulse {
+    0%, 100% {
+        opacity: 0.55;
+    }
+    50% {
+        opacity: 0.9;
+    }
+}
+
+.post-action-btn.is-loading,
+.vote-star.is-loading,
+.reaction-chip.is-loading,
+.reaction-add-btn.is-loading,
+.reaction-palette-btn.is-loading,
+.bookmark-btn.is-loading,
+.poll-option.is-loading {
+    pointer-events: none;
+    animation: engagement-busy-pulse 0.9s ease-in-out infinite;
+}

devplacepy/static/css/gists.css

@@ -231,6 +231,18 @@
     color: var(--text-primary);
 }
 
+.gist-code-header-actions {
+    display: flex;
+    align-items: center;
+    gap: var(--space-xs);
+}
+
+.gist-code-block .gist-rendered-content {
+    margin-bottom: 0;
+    padding: 1rem;
+    background: var(--bg-card);
+}
+
 .gist-code-block pre {
     margin: 0;
     padding: 1rem;

devplacepy/static/js/ApiTester.js

@@ -132,13 +132,17 @@ export class ApiTester {
     buildParams() {
         const table = this.el("div", { class: "param-table" });
         for (const param of this.config.params) {
-            const control = this.buildControl(param);
+            const isResponseField = param.location === "response";
-            this.inputs.push({ param, control });
+            const control = isResponseField ? this.el("span", { class: "param-static", text: param.type }) : this.buildControl(param);
-            control.addEventListener("input", () => this.renderSnippets());
+            if (!isResponseField) {
-            control.addEventListener("change", () => this.renderSnippets());
+                this.inputs.push({ param, control });
+                control.addEventListener("input", () => this.renderSnippets());
+                control.addEventListener("change", () => this.renderSnippets());
+            }
             const label = this.el("div", { class: "param-label" }, [
                 this.el("span", { class: "param-name", text: param.name }),
                 param.required ? this.el("span", { class: "param-required", text: "*" }) : null,
+                param.nullable ? this.el("span", { class: "param-nullable", text: "nullable" }) : null,
                 this.el("span", { class: "param-loc param-loc-" + param.location, text: param.location }),
             ]);
             const allowed = param.type === "enum" && param.options && param.options.length

devplacepy/static/js/Application.js

@@ -14,6 +14,7 @@ import { PushManager } from "./PushManager.js";
 import { CounterManager } from "./CounterManager.js";
 import { ReactionBar } from "./ReactionBar.js";
 import { BookmarkManager } from "./BookmarkManager.js";
+import { NoteManager } from "./NoteManager.js";
 import { PollManager } from "./PollManager.js";
 import { WarComposer } from "./WarComposer.js";
 import { ApiKeyManager } from "./ApiKeyManager.js";
@@ -48,6 +49,7 @@ import { ScrollMemory } from "./ScrollMemory.js";
 import { GameFarm } from "./GameFarm.js";
 import { Accessibility } from "./Accessibility.js";
 import { OverflowTabs } from "./OverflowTabs.js";
+import { StickyScrollPane } from "./StickyScrollPane.js";
 import { AiAutoload } from "./autoload/AiAutoload.js";
 import "./components/AiInteraction.js";
 import "./components/AiStatus.js";
@@ -87,6 +89,7 @@ class Application {
         this.counters = new CounterManager(this.pubsub);
         this.reactions = new ReactionBar();
         this.bookmarks = new BookmarkManager();
+        this.notes = new NoteManager();
         this.polls = new PollManager();
         this.warComposer = new WarComposer();
         this.apiKey = new ApiKeyManager();
@@ -118,6 +121,13 @@ class Application {
         this.editorLauncher = new EditorLauncher();
         this.gameFarm = new GameFarm();
         this.overflowTabs = new OverflowTabs();
+        this.stickyScrollPane = new StickyScrollPane([
+            ".feed-right",
+            ".sidebar-card",
+            ".post-page-sidebar",
+            ".profile-sidebar",
+            ".leaderboard-page > aside",
+        ]);
     }
 }
 

devplacepy/static/js/Avatar.js

@@ -1,9 +1,9 @@
 // retoor <retoor@molodetz.nl>
 
 export class Avatar {
-    static imgElement(username, size = 24) {
+    static imgElement(username, size = 24, seed = null) {
         const img = document.createElement("img");
-        img.src = `/avatar/multiavatar/${encodeURIComponent(username)}?size=${size}`;
+        img.src = `/avatar/multiavatar/${encodeURIComponent(seed || username)}?size=${size}`;
         img.className = "avatar-img";
         img.style.width = `${size}px`;
         img.style.height = `${size}px`;

devplacepy/static/js/BookmarkManager.js

@@ -22,6 +22,6 @@ export class BookmarkManager extends OptimisticAction {
             if (label) {
                 label.textContent = result.saved ? "Saved" : "Save";
             }
-        });
+        }, button);
     }
 }

devplacepy/static/js/CLAUDE.md

@@ -38,7 +38,7 @@ Conventions:
 
 ## Shared frontend utilities (do not re-implement these)
 
-A small set of plain ES6 modules under `static/js/` own the cross-cutting patterns so feature code stays tiny. Reach for these instead of hand-rolling a loop, a fetch, or a click handler. `Http` (`static/js/Http.js`) is the single fetch helper (`getJson`, `sendForm`, and `send` which throws `error.message` on non-2xx or a `200 {ok:false}` body); every live-update loop uses `Poller` (`new Poller(fn, intervalMs, {pauseHidden})`); async-job status polls use `JobPoller.run(statusUrl, {onDone, onFailed, onTimeout})` (`ProjectForker`, `ZipDownloader`); click-to-POST engagement controllers (`VoteManager`/`ReactionBar`/`BookmarkManager`/`PollManager`) extend `OptimisticAction` and call `this.submit(url, params, errorTarget, render)`. Floating windows (container terminals and Devii) extend `FloatingWindow`. **Scroll restoration is `ScrollMemory`** (`static/js/ScrollMemory.js`, `app.scrollMemory`): per-tab (sessionStorage) positions keyed by exact `path+search`, restored ONLY on `back_forward`/`reload` navigations or a click on `a.back-link`/`[data-scroll-back]`/breadcrumb/previous-trail-URL links, applied via a layout-stable rAF loop that aborts on user input; it sets `history.scrollRestoration = "manual"` site-wide and upgrades query-less back-link hrefs to the exact previous URL - mark any "back to X" anchor with the `back-link` class and never hand-roll scroll persistence.
+A small set of plain ES6 modules under `static/js/` own the cross-cutting patterns so feature code stays tiny. Reach for these instead of hand-rolling a loop, a fetch, or a click handler. `Http` (`static/js/Http.js`) is the single fetch helper (`getJson`, `sendForm`, and `send` which throws `error.message` on non-2xx or a `200 {ok:false}` body); every live-update loop uses `Poller` (`new Poller(fn, intervalMs, {pauseHidden})`); async-job status polls use `JobPoller.run(statusUrl, {onDone, onFailed, onTimeout})` (`ProjectForker`, `ZipDownloader`); click-to-POST engagement controllers (`VoteManager`/`ReactionBar`/`BookmarkManager`/`PollManager`) extend `OptimisticAction` and call `this.submit(url, params, errorTarget, render)`. Floating windows (container terminals and Devii) extend `FloatingWindow`. **Independent scroll on tall sticky columns is `StickyScrollPane`** (`static/js/StickyScrollPane.js`, `app.stickyScrollPane`): constructed once in `Application.js` with the list of sticky-column selectors (`.feed-right`, `.sidebar-card`, `.post-page-sidebar`, `.profile-sidebar`, `.leaderboard-page > aside`), it measures each element's `scrollHeight` against the viewport space actually available below its computed `top` offset (a one-time probe-element measurement resolves the `--space-lg` gap token to pixels, so no literal is duplicated from CSS) and toggles the `.sticky-scroll-active` class (`overflow-y: auto` in `components.css`) plus an inline `max-height` ONLY when the column's natural content exceeds that space - a column that fits stays untouched `position: sticky` with no cap, so it never re-introduces the dead-space regression a pure CSS `max-height` approach caused (see `CLAUDE.md` root history on commit `b475e7d6`). Recomputes on `window resize` and on a shared `ResizeObserver` watching every managed element (catches content growth). Never add a second max-height/overflow recipe to a sticky column - extend the selector list passed to this one instance instead. **Scroll restoration is `ScrollMemory`** (`static/js/ScrollMemory.js`, `app.scrollMemory`): per-tab (sessionStorage) positions keyed by exact `path+search`, restored ONLY on `back_forward`/`reload` navigations or a click on `a.back-link`/`[data-scroll-back]`/breadcrumb/previous-trail-URL links, applied via a layout-stable rAF loop that aborts on user input; it sets `history.scrollRestoration = "manual"` site-wide and upgrades query-less back-link hrefs to the exact previous URL - mark any "back to X" anchor with the `back-link` class and never hand-roll scroll persistence.
 
 Detail on each utility:
 

devplacepy/static/js/DeviiTerminal.js

@@ -77,6 +77,7 @@ export class DeviiTerminal {
             }
             return;
         }
+        if (this.element && this.element.state !== "closed") return;
         if (this._canOpenOnGesture()) this.open();
     }
 

devplacepy/static/js/DomUtils.js

@@ -7,6 +7,7 @@ export class DomUtils {
         this.initClipboardCopy();
         this.initShareButtons();
         this.initTogglers();
+        this.initViewToggles();
         this.initStopPropagation();
         this.initReload();
         this.initCardNav();
@@ -66,6 +67,18 @@ export class DomUtils {
         });
     }
 
+    initViewToggles() {
+        DomUtils.onDataAttr("view-toggle", "click", (btn) => {
+            const primary = document.getElementById(btn.dataset.viewToggle);
+            const alt = document.getElementById(btn.dataset.viewToggleAlt);
+            if (!primary || !alt) return;
+            const showingAlt = alt.classList.contains("hidden");
+            primary.classList.toggle("hidden", showingAlt);
+            alt.classList.toggle("hidden", !showingAlt);
+            btn.textContent = showingAlt ? btn.dataset.viewToggleLabelAlt : btn.dataset.viewToggleLabel;
+        });
+    }
+
     initStopPropagation() {
         DomUtils.onDataAttr("stop-propagation", "click", (el, e) => e.stopPropagation());
     }

devplacepy/static/js/Http.js

@@ -45,8 +45,11 @@ export class Http {
         return error;
     }
 
-    static async getJson(url) {
+    static async getJson(url, options = {}) {
-        const response = await fetch(url, { headers: { "Accept": "application/json" } });
+        const response = await fetch(url, {
+            headers: { "Accept": "application/json" },
+            signal: options.signal,
+        });
         if (!response.ok) {
             throw Http._error(`request failed with status ${response.status}`, response.status);
         }
@@ -106,10 +109,15 @@ export class Http {
     }
 
     static async sendDelete(url, options = {}) {
-        const response = await fetch(url, {
+        const init = {
             method: "DELETE",
             headers: { "Accept": "application/json" },
-        });
+        };
+        if (options.body !== undefined) {
+            init.headers["Content-Type"] = "application/json";
+            init.body = JSON.stringify(options.body);
+        }
+        const response = await fetch(url, init);
         if (response.redirected && response.url.includes("/auth/login")) {
             Http.toLogin();
             return Http.suspend();

devplacepy/static/js/MentionInput.js

@@ -13,6 +13,7 @@ export class MentionInput {
         this.debounceTimer = null;
         this.lastMatch = null;
         this.nav = null;
+        this.pendingController = null;
         this.build();
     }
 
@@ -69,9 +70,21 @@ export class MentionInput {
         this.debounceTimer = setTimeout(() => this.fetch(query), 200);
     }
 
+    isCurrentQuery(query) {
+        return !!this.lastMatch && this.lastMatch.query === query;
+    }
+
     async fetch(query) {
+        if (this.pendingController) {
+            this.pendingController.abort();
+        }
+        const controller = new AbortController();
+        this.pendingController = controller;
         try {
-            const data = await Http.getJson("/profile/search?q=" + encodeURIComponent(query));
+            const data = await Http.getJson("/profile/search?q=" + encodeURIComponent(query), { signal: controller.signal });
+            if (!this.isCurrentQuery(query)) {
+                return;
+            }
             const results = data.results || [];
             if (results.length === 0) {
                 this.hide();
@@ -79,7 +92,14 @@ export class MentionInput {
             }
             this.render(results);
         } catch (e) {
+            if (e.name === "AbortError") {
+                return;
+            }
             this.hide();
+        } finally {
+            if (this.pendingController === controller) {
+                this.pendingController = null;
+            }
         }
     }
 
@@ -92,7 +112,7 @@ export class MentionInput {
             item.dataset.username = r.username;
             const label = document.createElement("span");
             label.textContent = "@" + r.username;
-            item.append(Avatar.imgElement(r.username), label);
+            item.append(Avatar.imgElement(r.username, 24, r.avatar_seed), label);
             item.addEventListener("mousedown", (e) => {
                 e.preventDefault();
                 this.insert(r.username);

devplacepy/static/js/MobileNav.js

@@ -4,7 +4,6 @@ export class MobileNav {
     constructor() {
         this.initMobileNav();
         this.initProfileDropdown();
-        this.initToolsDropdown();
     }
 
     initMobileNav() {
@@ -80,31 +79,4 @@ export class MobileNav {
             }
         });
     }
-
-    initToolsDropdown() {
-        const dropdown = document.querySelector(".topnav-tools-dropdown");
-        if (!dropdown) return;
-        const btn = dropdown.querySelector(".topnav-tools-toggle");
-        if (!btn) return;
-
-        btn.addEventListener("click", (e) => {
-            e.stopPropagation();
-            const open = dropdown.classList.toggle("open");
-            btn.setAttribute("aria-expanded", open ? "true" : "false");
-        });
-
-        document.addEventListener("click", (e) => {
-            if (!dropdown.contains(e.target)) {
-                dropdown.classList.remove("open");
-                btn.setAttribute("aria-expanded", "false");
-            }
-        });
-
-        document.addEventListener("keydown", (e) => {
-            if (e.key === "Escape") {
-                dropdown.classList.remove("open");
-                btn.setAttribute("aria-expanded", "false");
-            }
-        });
-    }
 }

devplacepy/static/js/ModalManager.js

@@ -29,6 +29,7 @@ export class ModalManager {
                 const modal = document.getElementById(modalId);
                 if (modal) {
                     modal.classList.add("visible");
+                    this.pushModalHistory(modal);
                 }
             });
         });
@@ -37,11 +38,13 @@ export class ModalManager {
             modal.addEventListener("click", (e) => {
                 if (e.target === modal) {
                     modal.classList.remove("visible");
+                    this.popModalHistory(modal);
                 }
             });
             modal.querySelectorAll(".modal-close").forEach((closeBtn) => {
                 closeBtn.addEventListener("click", () => {
                     modal.classList.remove("visible");
+                    this.popModalHistory(modal);
                 });
             });
             this.enhanceModal(modal);
@@ -50,8 +53,30 @@ export class ModalManager {
         document.addEventListener("keydown", (e) => {
             if (e.key !== "Escape") return;
             const open = [...document.querySelectorAll(".modal-overlay.visible")].pop();
-            if (open) open.classList.remove("visible");
+            if (open) {
+                open.classList.remove("visible");
+                this.popModalHistory(open);
+            }
         });
+
+        window.addEventListener("popstate", () => {
+            const open = [...document.querySelectorAll(".modal-overlay.visible")].pop();
+            if (!open) return;
+            open._modalHistoryOpen = false;
+            open.classList.remove("visible");
+        });
+    }
+
+    pushModalHistory(modal) {
+        if (modal._modalHistoryOpen) return;
+        modal._modalHistoryOpen = true;
+        history.pushState({ modalOverlay: true }, "", location.href);
+    }
+
+    popModalHistory(modal) {
+        if (!modal._modalHistoryOpen) return;
+        modal._modalHistoryOpen = false;
+        history.back();
     }
 
     enhanceModal(modal) {

devplacepy/static/js/NoteManager.js

@@ -0,0 +1,135 @@
+// retoor <retoor@molodetz.nl>
+
+import { OptimisticAction } from "./OptimisticAction.js";
+
+export class NoteManager extends OptimisticAction {
+    constructor() {
+        super();
+        document.addEventListener("click", (event) => this.onClick(event));
+    }
+
+    onClick(event) {
+        const toggle = event.target.closest("[data-note-toggle]");
+        if (toggle) {
+            event.preventDefault();
+            this.toggle(toggle);
+            return;
+        }
+        const cancel = event.target.closest(".note-editor-cancel");
+        if (cancel) {
+            event.preventDefault();
+            this.cancel(cancel);
+            return;
+        }
+        const save = event.target.closest(".note-editor-save");
+        if (save) {
+            event.preventDefault();
+            this.save(save);
+            return;
+        }
+        const remove = event.target.closest(".note-editor-delete");
+        if (remove) {
+            event.preventDefault();
+            this.remove(remove);
+        }
+    }
+
+    toggle(button) {
+        const widget = button.closest(".note-widget");
+        const editor = widget ? widget.querySelector(".note-editor") : null;
+        if (!editor) {
+            return;
+        }
+        const opening = editor.hidden;
+        editor.hidden = !opening;
+        button.setAttribute("aria-expanded", opening ? "true" : "false");
+        if (opening) {
+            const textarea = editor.querySelector(".note-editor-textarea");
+            if (textarea) {
+                textarea.focus();
+            }
+        }
+    }
+
+    cancel(button) {
+        const widget = button.closest(".note-widget");
+        const editor = widget ? widget.querySelector(".note-editor") : null;
+        const toggle = widget ? widget.querySelector("[data-note-toggle]") : null;
+        if (!editor) {
+            return;
+        }
+        const textarea = editor.querySelector(".note-editor-textarea");
+        if (textarea) {
+            textarea.value = textarea.dataset.saved || "";
+        }
+        editor.hidden = true;
+        if (toggle) {
+            toggle.setAttribute("aria-expanded", "false");
+        }
+    }
+
+    async save(button) {
+        const widget = button.closest(".note-widget");
+        const editor = widget ? widget.querySelector(".note-editor") : null;
+        const textarea = editor ? editor.querySelector(".note-editor-textarea") : null;
+        if (!widget || !textarea) {
+            return;
+        }
+        const content = textarea.value.trim();
+        if (!content) {
+            return;
+        }
+        const type = widget.dataset.noteType;
+        const uid = widget.dataset.noteUid;
+        await this.submit(
+            `/notes/${type}/${uid}`,
+            { content },
+            widget,
+            (result) => this.applySaved(widget, result.content),
+            button
+        );
+    }
+
+    async remove(button) {
+        const widget = button.closest(".note-widget");
+        if (!widget) {
+            return;
+        }
+        const type = widget.dataset.noteType;
+        const uid = widget.dataset.noteUid;
+        await this.submit(
+            `/notes/${type}/${uid}/delete`,
+            {},
+            widget,
+            () => this.applySaved(widget, ""),
+            button
+        );
+    }
+
+    applySaved(widget, content) {
+        const toggle = widget.querySelector("[data-note-toggle]");
+        const editor = widget.querySelector(".note-editor");
+        const textarea = editor ? editor.querySelector(".note-editor-textarea") : null;
+        const label = toggle ? toggle.querySelector(".note-label") : null;
+        const deleteBtn = editor ? editor.querySelector(".note-editor-delete") : null;
+        if (textarea) {
+            textarea.value = content;
+            textarea.dataset.saved = content;
+        }
+        if (toggle) {
+            toggle.classList.toggle("has-note", Boolean(content));
+        }
+        if (label) {
+            label.textContent = content ? "Edit note" : "Add note";
+        }
+        if (deleteBtn) {
+            deleteBtn.hidden = !content;
+        }
+        if (editor) {
+            editor.hidden = true;
+        }
+        if (toggle) {
+            toggle.setAttribute("aria-expanded", "false");
+        }
+    }
+}

devplacepy/static/js/OptimisticAction.js

@@ -4,7 +4,19 @@ import { Http } from "./Http.js";
 import { Toast } from "./Toast.js";
 
 export class OptimisticAction {
-    async submit(url, params, errorTarget, render) {
+    setBusy(target, busy) {
+        if (!target) return;
+        target.classList.toggle("is-loading", busy);
+        target.disabled = busy;
+        if (busy) {
+            target.setAttribute("aria-busy", "true");
+        } else {
+            target.removeAttribute("aria-busy");
+        }
+    }
+
+    async submit(url, params, errorTarget, render, busyTarget = errorTarget) {
+        this.setBusy(busyTarget, true);
         try {
             const result = await Http.sendForm(url, params, { silent: true });
             if (render) render(result);
@@ -13,11 +25,14 @@ export class OptimisticAction {
             console.error("optimistic action failed", error);
             if (errorTarget) Toast.flash(errorTarget, "Error", 1500);
             return null;
+        } finally {
+            this.setBusy(busyTarget, false);
         }
     }
 
-    async submitOptimistic(url, params, errorTarget, apply, revert, reconcile) {
+    async submitOptimistic(url, params, errorTarget, apply, revert, reconcile, busyTarget = errorTarget) {
         apply();
+        this.setBusy(busyTarget, true);
         try {
             const result = await Http.sendForm(url, params, { silent: true });
             if (reconcile) reconcile(result);
@@ -27,6 +42,8 @@ export class OptimisticAction {
             revert();
             if (errorTarget) Toast.flash(errorTarget, "Error", 1500);
             return null;
+        } finally {
+            this.setBusy(busyTarget, false);
         }
     }
 }

devplacepy/static/js/PollManager.js

@@ -42,7 +42,7 @@ export class PollManager extends OptimisticAction {
         }
         const pollUid = poll.dataset.pollUid;
         const optionUid = option.dataset.optionUid;
-        await this.submit(`/polls/${pollUid}/vote`, { option_uid: optionUid }, null, (result) => this.render(poll, result));
+        await this.submit(`/polls/${pollUid}/vote`, { option_uid: optionUid }, null, (result) => this.render(poll, result), option);
     }
 
     render(poll, result) {

devplacepy/static/js/PushManager.js

@@ -10,11 +10,37 @@ export class PushManager {
         }
         this.userUid = document.body.dataset.userUid || "";
         this.triggers = Array.from(document.querySelectorAll("[data-push-enable]"));
+        this.logoutLinks = Array.from(document.querySelectorAll("a[href='/auth/logout']"));
         this.bindTriggers();
+        this.bindLogoutLinks();
         this.refreshTriggerVisibility();
         this.register(true).catch((error) => console.error("Push silent register failed:", error));
     }
 
+    bindLogoutLinks() {
+        this.logoutLinks.forEach((link) => {
+            link.addEventListener("click", (event) => {
+                event.preventDefault();
+                this.unregister().finally(() => {
+                    window.location.href = link.href;
+                });
+            });
+        });
+    }
+
+    async unregister() {
+        try {
+            const registration = await navigator.serviceWorker.getRegistration();
+            if (!registration) return;
+            const subscription = await registration.pushManager.getSubscription();
+            if (!subscription) return;
+            await Http.sendDelete("/push.json", { body: { endpoint: subscription.endpoint }, silent: true });
+            await subscription.unsubscribe();
+        } catch (error) {
+            console.error("Error unregistering push notifications:", error);
+        }
+    }
+
     bindTriggers() {
         this.triggers.forEach((trigger) => {
             trigger.addEventListener("click", (event) => {

devplacepy/static/js/ReactionBar.js

@@ -121,13 +121,14 @@ export class ReactionBar extends OptimisticAction {
         if (!bar) {
             return;
         }
-        await this.reactWith(bar, trigger.dataset.reactionEmoji);
+        await this.reactWith(bar, trigger.dataset.reactionEmoji, trigger);
     }
 
-    async reactWith(bar, emoji) {
+    async reactWith(bar, emoji, busyTarget = null) {
         const type = bar.dataset.reactionType;
         const uid = bar.dataset.reactionUid;
-        await this.submit(`/reactions/${type}/${uid}`, { emoji }, null, (result) => this.render(bar, result));
+        const busy = busyTarget || bar.querySelector(".reaction-add-btn");
+        await this.submit(`/reactions/${type}/${uid}`, { emoji }, null, (result) => this.render(bar, result), busy);
         this.closeAll();
     }
 

devplacepy/static/js/StickyScrollPane.js

@@ -0,0 +1,81 @@
+// retoor <retoor@molodetz.nl>
+
+export class StickyScrollPane {
+    constructor(selectors, options = {}) {
+        this.selectors = Array.isArray(selectors) ? selectors : [selectors];
+        this.activeClass = options.activeClass || "sticky-scroll-active";
+        this.gapVar = options.gapVar || "--space-lg";
+        this.elements = new Set();
+        this.scheduled = false;
+        this.gapPx = this.resolveGapPx(this.gapVar);
+        this.resizeObserver = typeof ResizeObserver !== "undefined"
+            ? new ResizeObserver(() => this.scheduleRecompute())
+            : null;
+        this.collect();
+        window.addEventListener("resize", () => this.scheduleRecompute(), { passive: true });
+        this.scheduleRecompute();
+    }
+
+    resolveGapPx(varName) {
+        const probe = document.createElement("div");
+        probe.style.position = "absolute";
+        probe.style.visibility = "hidden";
+        probe.style.pointerEvents = "none";
+        probe.style.height = `var(${varName})`;
+        document.body.appendChild(probe);
+        const px = probe.getBoundingClientRect().height;
+        probe.remove();
+        return px;
+    }
+
+    collect() {
+        this.selectors.forEach((selector) => {
+            document.querySelectorAll(selector).forEach((el) => {
+                if (this.elements.has(el)) return;
+                this.elements.add(el);
+                if (this.resizeObserver) this.resizeObserver.observe(el);
+            });
+        });
+    }
+
+    scheduleRecompute() {
+        if (this.scheduled) return;
+        this.scheduled = true;
+        requestAnimationFrame(() => {
+            this.scheduled = false;
+            this.recomputeAll();
+        });
+    }
+
+    recomputeAll() {
+        this.elements.forEach((el) => this.recompute(el));
+    }
+
+    recompute(el) {
+        const style = getComputedStyle(el);
+        if (style.position !== "sticky" || style.display === "none") {
+            this.disengage(el);
+            return;
+        }
+        const top = parseFloat(style.top) || 0;
+        const available = window.innerHeight - top - this.gapPx;
+        const natural = el.scrollHeight;
+        if (available > 0 && natural > available) {
+            this.engage(el, available);
+        } else {
+            this.disengage(el);
+        }
+    }
+
+    engage(el, availablePx) {
+        const height = `${Math.round(availablePx)}px`;
+        if (el.style.maxHeight !== height) el.style.maxHeight = height;
+        el.classList.add(this.activeClass);
+    }
+
+    disengage(el) {
+        if (!el.classList.contains(this.activeClass) && !el.style.maxHeight) return;
+        el.classList.remove(this.activeClass);
+        el.style.maxHeight = "";
+    }
+}

devplacepy/static/js/VoteManager.js

@@ -5,18 +5,18 @@ import { OptimisticAction } from "./OptimisticAction.js";
 export class VoteManager extends OptimisticAction {
     constructor() {
         super();
-        this.initVoteButtons();
+        document.addEventListener("click", (event) => this.onClick(event));
     }
 
-    initVoteButtons() {
+    onClick(event) {
-        document.querySelectorAll('form[action^="/votes/"] button[type="submit"]').forEach((button) => {
+        const button = event.target.closest('form[action^="/votes/"] button[type="submit"]');
-            const form = button.closest("form");
+        if (!button) {
-            button.addEventListener("click", (event) => {
+            return;
-                event.preventDefault();
+        }
-                event.stopPropagation();
+        const form = button.closest("form");
-                this.cast(form, button);
+        event.preventDefault();
-            });
+        event.stopPropagation();
-        });
+        this.cast(form, button);
     }
 
     cast(form, button) {

devplacepy/static/js/components/AppChat.js

@@ -46,7 +46,10 @@ export class AppChat extends Component {
         this._pendingSends = new Map();
         this._failedSends = new Map();
         this._uploading = false;
+        this._sendLocked = false;
+        this._lockingSends = new Set();
         this._userAtBottom = true;
+        this._lastForcedScrollTop = null;
         this._stabilizeFrames = 0;
         this._stabilizePending = false;
         this._lastTypingSent = 0;
@@ -95,6 +98,7 @@ export class AppChat extends Component {
         this._bindAutoScroll();
         this._startScrollWatcher();
         this._ensureJumpButton();
+        this._bindActiveConversation();
         this._connect();
         this._initPresence();
         this._scrollThreadToEnd();
@@ -111,12 +115,16 @@ export class AppChat extends Component {
         if (this._mobileQuery && this._onMobileChange) {
             this._mobileQuery.removeEventListener("change", this._onMobileChange);
         }
+        if (this._onWindowFocus) window.removeEventListener("focus", this._onWindowFocus);
+        if (this._onVisibilityChange) document.removeEventListener("visibilitychange", this._onVisibilityChange);
         clearTimeout(this._typingHideTimer);
         clearTimeout(this._disconnectTimer);
         cancelAnimationFrame(this._autoGrowFrame);
         for (const entry of this._pendingSends.values()) clearTimeout(entry.timeoutId);
         this._pendingSends.clear();
         this._failedSends.clear();
+        this._lockingSends.clear();
+        this._sendLocked = false;
         if (this._presence) this._presence.stop();
         if (this._onPopState) window.removeEventListener("popstate", this._onPopState);
     }
@@ -271,6 +279,7 @@ export class AppChat extends Component {
         if (!this.form || !this.input) return;
         this.form.addEventListener("submit", (event) => {
             event.preventDefault();
+            if (this._sendLocked) return;
             this._sendViaSocket();
         });
         if (this.upload) {
@@ -336,11 +345,17 @@ export class AppChat extends Component {
 
     _refreshSendButton() {
         if (!this.sendBtn) return;
-        const busy = this._uploading;
+        const busy = this._uploading || this._sendLocked;
         this.sendBtn.disabled = busy;
         this.sendBtn.classList.toggle("is-sending", busy);
     }
 
+    _releaseSendLock(clientId) {
+        if (!this._lockingSends.has(clientId)) return;
+        this._lockingSends.delete(clientId);
+        if (this._lockingSends.size === 0) this._sendLocked = false;
+    }
+
     _collectAttachments() {
         const hidden = this.form.querySelector('input[name="attachment_uids"]');
         if (!hidden || !hidden.value) return [];
@@ -376,6 +391,10 @@ export class AppChat extends Component {
 
         const timeoutId = window.setTimeout(() => this._failPendingSend(clientId), SEND_TIMEOUT_MS);
         this._pendingSends.set(clientId, { content, attachmentUids, timeoutId });
+        if (!retryClientId) {
+            this._lockingSends.add(clientId);
+            this._sendLocked = true;
+        }
         this._refreshSendButton();
 
         if (!retryClientId) {
@@ -414,6 +433,7 @@ export class AppChat extends Component {
         if (!entry) return;
         clearTimeout(entry.timeoutId);
         this._pendingSends.delete(clientId);
+        this._releaseSendLock(clientId);
         this._refreshSendButton();
         const bubble = this.thread ? this.thread.querySelector(`.message-bubble[data-client-id="${clientId}"]`) : null;
         if (bubble && !bubble.dataset.msgUid) {
@@ -434,6 +454,7 @@ export class AppChat extends Component {
         if (!entry) return;
         clearTimeout(entry.timeoutId);
         this._pendingSends.delete(clientId);
+        this._releaseSendLock(clientId);
         this._refreshSendButton();
     }
 
@@ -881,9 +902,25 @@ export class AppChat extends Component {
     _markRead() {
         if (this._socketReady && this.withUid && this.socket && this._threadIsActive()) {
             this.socket.send({ type: "read", with_uid: this.withUid });
+            this._sendActiveMarker();
         }
     }
 
+    _sendActiveMarker() {
+        if (this._socketReady && this.withUid && this.socket && this._threadIsActive()) {
+            this.socket.send({ type: "active", with_uid: this.withUid });
+        }
+    }
+
+    _bindActiveConversation() {
+        this._onWindowFocus = () => this._sendActiveMarker();
+        this._onVisibilityChange = () => {
+            if (document.visibilityState === "visible") this._sendActiveMarker();
+        };
+        window.addEventListener("focus", this._onWindowFocus);
+        document.addEventListener("visibilitychange", this._onVisibilityChange);
+    }
+
     _showTyping() {
         if (!this.typingEl) return;
         this.typingEl.hidden = false;
@@ -989,6 +1026,7 @@ export class AppChat extends Component {
     _scrollThreadToEnd() {
         if (!this.thread || !this._userAtBottom) return;
         this.thread.scrollTop = this.thread.scrollHeight;
+        this._lastForcedScrollTop = this.thread.scrollTop;
     }
 
     _startScrollWatcher() {
@@ -1020,6 +1058,7 @@ export class AppChat extends Component {
         }
         this._stabilizeFrames++;
         this.thread.scrollTop = this.thread.scrollHeight;
+        this._lastForcedScrollTop = this.thread.scrollTop;
         const atBottom = this.thread.scrollHeight - this.thread.scrollTop - this.thread.clientHeight < 10;
         if (!atBottom) {
             this._stabilizePending = true;
@@ -1035,13 +1074,16 @@ export class AppChat extends Component {
     _bindAutoScroll() {
         if (!this.thread) return;
         this.thread.addEventListener("scroll", () => {
-            if (this._stabilizePending) return;
+            const isOwnForcedScroll = this._stabilizePending && this.thread.scrollTop === this._lastForcedScrollTop;
+            if (isOwnForcedScroll) return;
             const atBottom = this.thread.scrollHeight - this.thread.scrollTop - this.thread.clientHeight < AUTO_SCROLL_MARGIN_PX;
             this._userAtBottom = atBottom;
             if (atBottom) {
                 this._newCount = 0;
                 this._updateJumpButton();
                 this._stabilizeScroll();
+            } else {
+                this._stabilizeFrames = 0;
             }
             if (this.thread.scrollTop < 48) this._loadOlder();
         }, { passive: true });
@@ -1552,7 +1594,7 @@ export class AppChat extends Component {
                         item.href = `/messages?with_uid=${r.uid}`;
                         const label = document.createElement("span");
                         label.textContent = r.username;
-                        item.append(Avatar.imgElement(r.username), label);
+                        item.append(Avatar.imgElement(r.username, 24, r.avatar_seed), label);
                         dropdown.appendChild(item);
                     }
                     DomUtils.show(dropdown);

devplacepy/templates/_comment.html

@@ -26,6 +26,7 @@
         {% endif %}
         <div class="comment-actions">
             <button type="button" class="comment-action-btn" data-action="reply"{{ guest_disabled(user) }}><span class="icon">💬</span><span class="label"> Reply</span></button>
+            <button type="button" class="comment-action-btn" data-share="#comment-{{ item.comment['uid'] }}"><span class="icon">🔗</span><span class="label"> Copy link</span></button>
             {% if owns(item.comment, user) %}
             <button type="button" class="comment-action-btn" data-action="edit" data-edit-url="/comments/edit/{{ item.comment['uid'] }}"><span class="icon">✏️</span><span class="label"> Edit</span></button>
             {% endif %}

devplacepy/templates/_note_button.html

@@ -0,0 +1,13 @@
+<div class="note-widget" data-note-type="{{ _type }}" data-note-uid="{{ _uid }}">
+    <button type="button" class="post-action-btn note-btn{% if _note %} has-note{% endif %}" data-note-toggle title="Personal note" aria-label="Personal note" aria-expanded="false"{{ guest_disabled(user) }}>
+        <span class="note-icon">&#x1F4DD;</span> <span class="note-label">{% if _note %}Edit note{% else %}Add note{% endif %}</span>
+    </button>
+    <div class="note-editor" hidden>
+        <textarea class="note-editor-textarea" maxlength="4000" placeholder="Write a private note only you can see..." data-saved="{{ _note or '' }}">{{ _note or "" }}</textarea>
+        <div class="note-editor-actions">
+            <button type="button" class="btn btn-secondary btn-sm note-editor-cancel">Cancel</button>
+            <button type="button" class="btn btn-danger btn-sm note-editor-delete"{% if not _note %} hidden{% endif %}>Delete</button>
+            <button type="button" class="btn btn-primary btn-sm note-editor-save">Save</button>
+        </div>
+    </div>
+</div>

devplacepy/templates/_post_card.html

@@ -18,7 +18,7 @@
     {% if maturity_hidden(item.maturity, user) %}
     {% set _level = item.maturity %}{% include "_maturity_gate.html" %}
     {% else %}
-    <div class="post-content rendered-content">{{ render_content(item.post['content'][:300] ~ ('...' if item.post['content']|length > 300 else ''), author_is_admin=is_admin(item.author)) }}</div>
+    <div class="post-content rendered-content">{{ render_content(safe_truncate(item.post['content'], 300) ~ ('...' if item.post['content']|length > 300 else ''), author_is_admin=is_admin(item.author)) }}</div>
     {% endif %}
 
     {% if item.project_link %}

devplacepy/templates/base.html

@@ -74,22 +74,18 @@
                 <a href="/news" class="topnav-link {{ nav_active(request, '/news') }}"><span class="icon">📰</span> News</a>
                 <a href="/gists" class="topnav-link {{ nav_active(request, '/gists') }}"><span class="icon">📄</span> Gists</a>
                 <a href="/projects" class="topnav-link {{ nav_active(request, '/projects') }}"><span class="icon">🚀</span> Projects</a>
-                <a href="/quizzes" class="topnav-link {{ nav_active(request, '/quizzes') }}"><span class="icon">🧩</span> Quizzes</a>
-                <a href="/battles" class="topnav-link {{ nav_active(request, '/battles') }}"><span class="icon">&#x2694;&#xFE0E;</span> Battles</a>
                 {% if user %}<a href="/game" class="topnav-link {{ nav_active(request, '/game') }}"><span class="icon">🌱</span> Farm</a>{% endif %}
             </div>
             <div class="topnav-right">
+                <a href="/quizzes" class="topnav-icon {{ nav_active(request, '/quizzes') }}" title="Quizzes" aria-label="Quizzes">
+                    <span class="nav-bell" aria-hidden="true">🧩</span>
+                </a>
+                <a href="/battles" class="topnav-icon {{ nav_active(request, '/battles') }}" title="Battles" aria-label="Battles">
+                    <span class="nav-bell" aria-hidden="true">&#x2694;&#xFE0E;</span>
+                </a>
                 <a href="/leaderboard" class="topnav-icon {{ nav_active(request, '/leaderboard') }}" title="Leaderboard" aria-label="Leaderboard">
                     <span class="nav-bell" aria-hidden="true">🏆</span>
                 </a>
-                <div class="topnav-tools-dropdown {{ nav_active(request, '/tools') }}">
-                    <button type="button" class="topnav-icon topnav-tools-toggle" aria-expanded="false" aria-controls="tools-menu" title="Tools" aria-label="Tools"><span class="nav-bell" aria-hidden="true">🧰</span></button>
-                    <div class="dropdown-menu" id="tools-menu">
-                        <a href="/tools/seo" class="dropdown-item"><span class="icon">🔍</span> SEO Diagnostics</a>
-                        <a href="/tools/deepsearch" class="dropdown-item"><span class="icon">🧠</span> DeepSearch</a>
-                        <a href="/tools/isslop" class="dropdown-item"><span class="icon">🧪</span> AI Usage Analyzer</a>
-                    </div>
-                </div>
                 {% if user %}
                 {% set msg_unread = get_unread_messages(user["uid"]) %}
                 <a href="/messages" class="topnav-icon {{ nav_active(request, '/messages') }}" data-counter="messages" data-counter-noun="messages" title="Messages" aria-label="{% if msg_unread %}{{ msg_unread }} unread messages{% else %}Messages, no unread{% endif %}">
@@ -119,6 +115,7 @@
                         <a href="/profile/{{ user['username'] }}" class="dropdown-item"><span class="icon">👤</span> Profile</a>
                         <a href="/devii/" class="dropdown-item" data-devii-open><span class="icon">🤖</span> Devii</a>
                         <a href="/bookmarks/saved" class="dropdown-item"><span class="icon">🔖</span> Saved</a>
+                        <a href="/notes/saved" class="dropdown-item"><span class="icon">📝</span> Notes</a>
                         <a href="/auth/logout" class="dropdown-item"><span class="icon">🚪</span> Logout</a>
                     </div>
                 </div>
@@ -149,11 +146,6 @@
             <a href="/battles" class="topnav-mobile-link {{ nav_active(request, '/battles') }}"><span class="icon">&#x2694;&#xFE0E;</span> Battles</a>
             <a href="/leaderboard" class="topnav-mobile-link {{ nav_active(request, '/leaderboard') }}"><span class="icon">🏆</span> Leaderboard</a>
             {% if user %}<a href="/game" class="topnav-mobile-link {{ nav_active(request, '/game') }}"><span class="icon">🌱</span> Farm</a>{% endif %}
-            <div class="topnav-mobile-divider"></div>
-            <div class="topnav-mobile-section-label">Tools</div>
-            <a href="/tools/seo" class="topnav-mobile-link {{ nav_active(request, '/tools/seo') }}"><span class="icon">🔍</span> SEO Diagnostics</a>
-            <a href="/tools/deepsearch" class="topnav-mobile-link {{ nav_active(request, '/tools/deepsearch') }}"><span class="icon">🧠</span> DeepSearch</a>
-            <a href="/tools/isslop" class="topnav-mobile-link {{ nav_active(request, '/tools/isslop') }}"><span class="icon">🧪</span> AI Usage Analyzer</a>
             {% if user %}
             <a href="/messages" class="topnav-mobile-link {{ nav_active(request, '/messages') }}" data-counter="messages" data-counter-noun="messages" aria-label="{% if get_unread_messages(user['uid']) %}Messages, {{ get_unread_messages(user['uid']) }} unread{% else %}Messages, no unread{% endif %}"><span class="icon">✉️</span> Messages<span class="nav-badge nav-badge-inline" data-counter-badge {% if get_unread_messages(user["uid"]) == 0 %}hidden{% endif %} aria-hidden="true">{{ get_unread_messages(user["uid"]) }}</span></a>
             <a href="/notifications" class="topnav-mobile-link {{ nav_active(request, '/notifications') }}" data-counter="notifications" data-counter-noun="notifications" aria-label="{% if get_unread_count(user['uid']) %}Notifications, {{ get_unread_count(user['uid']) }} unread{% else %}Notifications, no unread{% endif %}"><span class="icon">🔔</span> Notifications<span class="nav-badge nav-badge-inline" data-counter-badge {% if get_unread_count(user["uid"]) == 0 %}hidden{% endif %} aria-hidden="true">{{ get_unread_count(user["uid"]) }}</span></a>
@@ -184,6 +176,7 @@
             <a href="/profile/{{ user['username'] }}" class="topnav-mobile-link"><span class="icon">👤</span> Profile</a>
             <a href="/devii/" class="topnav-mobile-link" data-devii-open><span class="icon">🤖</span> Devii</a>
             <a href="/bookmarks/saved" class="topnav-mobile-link {{ nav_active(request, '/bookmarks') }}"><span class="icon">🔖</span> Saved</a>
+            <a href="/notes/saved" class="topnav-mobile-link {{ nav_active(request, '/notes') }}"><span class="icon">📝</span> Notes</a>
             <a href="/auth/logout" class="topnav-mobile-link"><span class="icon">🚪</span> Logout</a>
             {% else %}
             <div class="topnav-mobile-divider"></div>