The workspace editor hung for 60s and then 504'd. Three independent faults were
stacked behind that one symptom.
Reachability: editor_target delegated to proxy_target, which returns
CONTAINER_PROXY_HOST plus the published host port and never falls back to the
container. From inside the app container that address crosses docker0 into the
host INPUT chain, whose policy is DROP with an allow-list that does not include
the published port range, so the packet was dropped and the request hung rather
than being refused. Measured from the app container: container_ip:8443 answers
302, gateway:20006 is dropped. One shared reachable_target now prefers the
direct container leg and falls back to the published port, and editor_target
uses tunnel_target as services/containers/CLAUDE.md already required. The same
defect affected /p/{slug} ingress and every tunnel, since all three resolved
through proxy_target.
The recorded measurement that motivated the old order (container_ip times out,
gateway connects) no longer holds: make docker-attach puts the app on the
instances' bridge network, which is what makes the direct leg work.
Duplicate response headers: the forwarding core relayed the upstream Date and
Server alongside the ones the serving layer generates, so every proxied
response carried two of each. Both are singleton headers and duplicating them
is malformed HTTP.
Serialization: WorkspaceViewOut declared flag_reason and three sibling strings
as str, so a NULL column made the workspace page 500 for JSON clients.
Documents the two public hostnames and the devplace.net SSH tunnel, so a future
session does not conclude the site is down after pointing curl --resolve at an
address the hostname does not resolve to, and adds the layered procedure for
diagnosing a production failure.
Verified on production with Playwright over both hostnames: the code-server
login renders and the workbench loads. Suite: 3345 passed.
An instance kept production-identical for extended manual testing is
otherwise taxed forever by its own safety controls: five consents, a
versioned terms gate on every mutating request, and every account
predating the trust and safety commit reading terms_version NULL because
init_db deliberately never backfills it. AcceptanceService grants each
agreement to each account that has not declined it, so the instance stays
production byte for byte while nobody clicks the same dialog again. It is
opt-in, dry run by default, and one switch per agreement type.
The application is not allowed to know it exists. One registration line
in main.py is the only import anywhere, there is no route, schema,
template, Devii tool or environment flag, and a unit test greps the tree
and fails the suite if a second importer appears. The decline register
needs no storage: the ledger is append-only in effect, the service only
ever grants, so any withdrawn row was written by a human and that pair is
never touched again. No provenance column, nothing to observe.
Satisfaction is the gate's own expression, never a proxy, which is why
the ordering is created_at then id exactly as consent_state selects, and
why the live-account clauses are built with has_column: init_db ensures
terms_version and deletion_requested_at but not is_active, so a hardcoded
reference raises no such column on an instance where nobody was ever
suspended. Every write is one conditional statement decided on the real
rowcount, proven with sixteen processes racing one account to exactly one
ledger row and one audit row. The two existing audit keys carry it, with
actor kind service, because a service that silently mutated consent state
would be the worst possible exception to the append-only rule.
lensfl.md is the source brief accept.md records the design against.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
code-server runs authenticateOrigin on every websocket and resolves the
request host as Forwarded, then X-Forwarded-Host, then Host. The forward
core put the public host into additional_headers, but the websockets
client already writes its own Host for the real TCP target and Headers
appends, so the handshake carried two Host lines; Node keeps the first
(the internal gateway:port), the origin check failed, and code-server
answered 403. Because the browser socket was accepted before the upstream
was dialled, that surfaced as a 101 followed by 1011 and the editor died
on "the workbench failed to connect to the server". Dialling first and
carrying the public host in the connect URI fixes both planes.
The two header builders that had drifted apart are now one core, so a
websocket carries the same client and forwarded headers as an HTTP
request. Responses stream instead of buffering whole, which is what makes
a large tunnel download cost constant memory and lets SSE work; byte
accounting moved onto the completion callback. Subprotocols negotiate,
the upstream client is reused across requests, and the path and query are
forwarded byte-exactly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Implements the moderation and consent obligations a social platform carries,
so the web version and any client that speaks to it enforce the same rules.
Moderation core (services/moderation/, database/moderation.py): a reportable
target registry, the content filter and its choke points, the report queue with
atomic resolution, enforcement actions, consent tracking, maturity gating, and
account deletion with a grace window.
Surfaces: POST /reports plus the member report list, /admin/moderation and the
per-report admin view, /workspaces, terms acceptance at /auth/terms, consent and
account deletion under /profile, the report button and dialog partials, the
maturity gate, and the moderation stylesheet and ReportDialog client.
Every user-generated surface stays reportable by construction: new content tables
are registered in REPORTABLE_TARGETS or listed in UNREPORTABLE_TABLES with a
reason, and the registry test fails the suite on anything left unclassified.
Docs: community guidelines, content moderation, intellectual property, privacy,
terms, contact, and the admin-only moderation operations page, plus the
moderation API group and the Devii moderation actions.
Compliance record: applecomp.md is the requirement register, applechanges.md the
gap analysis against this codebase, and appleimpl.md the implementation design
they resolve to.
Tests cover the report flow, admin moderation, consent, account deletion, terms
acceptance, workspaces, and the registry invariant across the unit, api, and e2e
tiers.
Split the push delivery library into a provider architecture. devplacepy/push
becomes a package: a PushProvider protocol with a registry, the existing Web
Push implementation moved unchanged behind it, a new APNs provider, a store
owning every push_registration access, and a delivery loop that groups a user's
subscriptions by provider, prepares each provider's payload once and sends over
a single shared client.
APNs delivers over HTTP/2 with an ES256 provider token cached per credential
fingerprint, so a worker signs at most one token per 45 minutes. Registrations
carry a hexadecimal device token; 410 and the Unregistered class of reasons soft
delete the subscription exactly like a gone Web Push endpoint.
All provider configuration is edited at /admin/services/push through the same
ConfigField surface every other subsystem uses, assembled from the registry so a
future provider needs no edit to the service. A provider that is disabled,
unconfigured or holding an unusable credential accepts no registrations and is
skipped during delivery, never failing the other providers.
POST /push.json accepts a registration for any active provider; a body without a
provider field is a Web Push body, so existing clients are unchanged. GET
/push.json keeps publicKey at the top level and adds the active providers.
push_registration gains provider and token columns, ensured in init_db with a
converging backfill; existing rows are never rewritten.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Add `cmd_isslop_prune`, `cmd_isslop_clear`, `cmd_isslop_analyze` CLI commands for AI usage analysis job management
- Register `/game` router with `index` and `farm` endpoints for Code Farm idle game
- Add `politics` to allowed TOPICS constant replacing `signals`
- Introduce `ISSLOP_DIR`, `ISSLOP_WORKSPACES_DIR`, `ISSLOP_RUNS_DIR`, `ISSLOP_MEDIA_DIR` config paths
- Add `clear_user_stars` and `clear_user_projects_cache` calls on vote and project create/delete
- Update `make prod` to use `nproc` workers via `DEVPLACE_WEB_WORKERS` env var
- Convert `database.py` and `utils.py` to packages for modular structure
- Add `devplace apikey` and `devplace token` CLI subcommands for API key and access token management
Add `last_seen` column to users table with index, implement `set_last_seen` and `get_online_users` database functions, expose presence config env vars (`PRESENCE_TIMEOUT_SECONDS`, `PRESENCE_ONLINE_LIMIT`, `PRESENCE_ONLINE_MARGIN_SECONDS`), include `last_seen` in follow list responses, and update profile docs to mention online indicator.
Consolidate scattered `X-Real-IP` / `request.client.host` fallback logic into the single `client_ip()` helper from `utils`, reducing duplication in the rate limiter middleware, project file zip routes, tools shared module, and audit record builder. Also refactor three admin endpoints (`/ai-usage/data`, `/analytics`, `/users/{uid}/ai-usage`) to use the existing `require_admin()` guard instead of repeating manual auth checks, and remove now-unused `get_current_user`/`is_admin` imports from those modules.
Extend the notification system with a third channel (telegram) alongside existing in_app and push channels. Add `telegram_enabled` column to `notification_preferences` table, update `NOTIFICATION_CHANNELS` and `_NOTIFICATION_CHANNEL_COLUMNS` mappings, and set telegram default to off (`_NOTIFICATION_CHANNEL_DEFAULTS`). Create `telegram_outbox` table with columns for uid, user_uid, chat_id, text, status, attempts, created_at, and sent_at, plus an index on status/id for efficient polling. Register `TelegramOutboxService` in the service manager lifecycle. Update API documentation strings to describe the new channel and its pairing requirement. Extend admin notification defaults view to include telegram column. Pass `notif_telegram_paired` flag to profile template based on `telegram_store.is_paired()` check. Update `NotificationPrefForm` and `NotificationDefaultForm` model literals to accept "telegram" as a valid channel value.
Implement a Farmville-style cooperative idle game mounted at `/game` with member-only play and public farm viewing. The data layer is pure and timestamp-driven with no background tick: plot states are derived from `ready_at` vs current time, never stored. Key features include plantable software projects (shell script through kernel) that build over real time, harvest for coins and XP, CI tier upgrades for faster builds, plot purchasing with doubling cost, watering mechanics for friends' builds, daily quests, and prestige system. All game endpoints negotiate HTML or JSON and return full farm state for single-request client refresh. Pub/sub notifications broadcast farm updates on `public.game.farm.{username}` topics. Database schema adds `game_farms`, `game_plots`, and `game_quests` tables with appropriate indexes.
- Fix audit event names in CLI prune/clear commands from `cli.seo.*` to `cli.seo_meta.*`
- Refactor `_delete_attachment_file` to accept full attachment dict instead of storage_path string, using directory and stored_name fields with ATTACHMENTS_DIR
- Add `safe_next` validation for referer header in validation error redirect and media redirect
- Add `is_active` check in login router to reject deactivated accounts with "Account is deactivated" error
- Replace raw `request.headers.get("Referer")` with `redirect_back()` utility in bookmarks, polls, reactions, and votes routers
- Move `mark_conversation_read` call from `get_conversation_messages` to `messages_page` to avoid side effects during message retrieval
- Fix poll audit link to use `option.get("label")` instead of `option.get("text")`
- Add `VOTABLE` set validation in votes router to reject invalid target types with 400 response
- Strip control characters (0x00-0x20) from URLs in `_safe_url` instead of simple strip
- Add `__getattr__` fallback in services `__init__.py` for dynamic attribute access
Extend `database.text_search_clause` with an `author_field` parameter that resolves username matches to user UIDs, enabling author-username search across the three public listings (`/feed`, `/gists`, `/projects`). Update the corresponding API docs and README route descriptions to reflect the new search scope. Add six partial indexes (`idx_comments_target_live`, `idx_votes_target_live`, `idx_reactions_target_live`, `idx_gists_live_created`, `idx_projects_live_created`, `idx_attachments_user_created_live`) to optimize filtered queries on non-deleted rows. Introduce a hot-settings cache (`_hot_settings`) with a 2-second TTL for `maintenance_mode`, `rate_limit_per_minute`, and `rate_limit_window_seconds`, plus a periodic rate-limit store sweep (`_sweep_rate_limit_store`) to evict stale IP entries every 60 seconds. Add `GZipMiddleware` to the FastAPI app for response compression.
Implement a new `SeoMetaService` subservice that generates clean SEO title/description/keywords for published content items, distinct from the existing SEO diagnostics auditor. Add `seo_metadata` polymorphic table with soft-delete support, batch query methods, and usage tracking. Extend the CLI with `seo-meta prune` and `seo-meta clear` commands for job row lifecycle management. Wire `schedule_seo_meta_for_table` into content creation and editing flows in `content.py`. Document the new service in `AGENTS.md` and `README.md`, including the `extra_head` site setting for custom `<head>` injection.
- Add `/block`, `/mute` endpoints with block/unblock and mute/unmute functionality in `routers/relations.py`, hiding blocked users' content everywhere except their own profile while muting only suppresses notifications
- Introduce `devplace emoji-sync` CLI command to regenerate `static/js/emoji-shortcodes.js` from the emoji library, documented in `CLAUDE.md` and wired in `cli.py`
- Create `get_blocked_uids()` database helper and apply it in `content.py` `load_detail()` to filter blocked users' posts from detail views
- Implement `_uid_index()` and `_drop_index()` helpers in `database.py` for unique uid indexes across tables, with `user_relations` added to `SOFT_DELETE_TABLES`
- Document new routes in `AGENTS.md` and `README.md`, including emoji shortcodes rendering behavior distinct from the emoji picker
- Add comprehensive documentation for backend content rendering in AGENTS.md, detailing the new `render_content` and `render_title` Jinja globals built on mistune with media processing, emoji shortcodes, and XSS protection
- Document the `X-Response-Time` header and bottom-left render time indicator in README.md
- Update bot token pricing documentation to clarify fallback vs gateway cost headers
- Add `email_accounts` to soft-delete tables and `idx_users_role` composite index in database schema
- Implement `telegram_pairings` and `telegram_links` table creation with column migration and indexes
- Add `/profile/{username}/telegram` endpoint to docs API with request/unpair actions
- Register `TelegramService` in main.py lifespan and add `response_timing` middleware emitting `X-Response-Time` header
- Introduce `TelegramPairForm` model and `guard_public_host_sync` synchronous host validation function
- Add `ai_grade`, `featured`, `featured_locked`, `landing_locked`, `author`, `article_published`, `image_url`, `has_unique_image` columns to news table
- Extend `news_images` schema with `alt_text`, `phash`, `width`, `height`, `is_placeholder` columns
- Create `idx_news_featured` index for efficient featured queries
- Update admin toggle endpoints to set `featured_locked`/`landing_locked` when manually toggling
- Propagate `featured` and `image_url` fields through landing page, news list, and detail page rendering
- Update `get_featured_news` to return `featured` and `image_url` fields
- Document in AGENTS.md the full zero-maintenance pipeline: image perceptual hashing, placeholder detection, AI grading on cleaned text, reliability gate, effective score computation, and post-loop landing rotation
- Update README.md service description to reflect automatic image comparison and landing rotation capabilities
- Clarify docs_api.py summaries that toggling featured/landing now locks articles from auto-rotation
- Add `devplace backups` CLI subcommands (list, run, prune, clear) with job enqueueing and orphan cleanup
- Introduce `BACKUPS_DIR` and `BACKUP_STAGING_DIR` config paths for backup storage
- Implement `schedule_correction` and `schedule_modification` calls in content creation, comment creation, and comment editing flows
- Add `DEFAULT_CORRECTION_PROMPT` and `DEFAULT_MODIFIER_PROMPT` config constants for AI content processing
- Document timezone-aware date display using `local_dt`/`dt_ago` Jinja globals with client-side `Intl` localization
- Update README with AI content correction/modifier support in direct messages via `@ai` inline instructions
- Add `track_action(user["uid"], "vote")` call on upvote in `apply_vote`
Update the feature-builder agent prompt, test-maintainer agent, and all four workflow JS files (devii-tool, endpoint, feature, job-service) to codify the DevPlace test standard as a non-optional project requirement: one test file per endpoint, directory tree mirroring the URL/source path, split into three tiers (unit, api, e2e). Add explicit Test phases to devii-tool, endpoint, feature, and job-service workflows, and embed tier-specific test instructions (path mapping, fixture choice, coverage scope) directly in each workflow's meta description and TESTS constant.
The lifespan context manager replaces the deprecated `@app.on_event("startup")` pattern, centralizing initialization logic for data directories, database, certificates, and all service registrations. It also conditionally starts background services based on the `DEVPLACE_DISABLE_SERVICES` environment variable and acquires a service lock for worker coordination. Additionally, the `pyproject.toml` now suppresses three specific deprecation and syntax warnings during test runs.
Add a new `/dbapi` router package providing a generic database API over `dataset`, restricted to admin sessions, admin API keys, and the internal gateway key. Includes:
- `tables.py`: list all tables and inspect table schemas
- `crud.py`: full CRUD operations (GET, POST, PATCH, DELETE) with soft-delete awareness, born-live inserts, `?include_deleted`, `.../restore`, and `?hard=true` purge
- `query.py`: validated read-only SELECT execution via sqlglot parsing, classification, and EXPLAIN dry-run; async query jobs with WebSocket streaming via `DbApiJobService`
- `nl.py`: natural-language-to-SQL conversion using the platform AI gateway with re-prompting until validation passes
Also register `DbApiJobService` and `PubSubService` in the service manager, add `DBAPI_DIR` to config data paths, and force cleartext `http://` connections to HTTP/1.1 in `curl_transport` to fix large request failures against uvicorn's HTTP/1.1-only internal gateway.
The old `diversify_by_author` capped each author to 2 posts per page, silently dropping excess rows. The new `interleave_by_author` reorders the full result set so no two consecutive posts share an author, preserving every post while breaking up same-author runs. `paginate_diverse` loses its `max_per_author` and `pool` parameters; the home route now fetches exactly 6 rows and interleaves them instead of over-fetching 50 and capping. Tests are rewritten to verify interleaving behavior and per-author chronological order preservation.
Implement a multi-agent deep web research subsystem including CLI commands for pruning expired jobs and clearing all artifacts, database tables for sessions/messages/URL cache with indexes, config paths for chroma storage, and internal embed URL for vector operations.
Add SEO_REPORTS_DIR to config, STATIC_VERSION for cache-busting, seo prune/clear CLI subcommands, tools router with seo job endpoints, and boot-versioned static URLs in Dockerfile and Makefile
Migrate all runtime artifacts (database, uploads, VAPID keys, locks, bot state, container workspaces, zip staging) from scattered locations (`var/`, `devplacepy/static/uploads/`) into a unified `data/` directory. Update `config.py` as single source of truth with `DATA_PATHS` registry and `ensure_data_dirs()`, add `devplace migrate-data` CLI command with CRC verification and idempotent relocation, adjust `.dockerignore`, `.env.example`, `.gitignore`, `Dockerfile`, `Makefile`, `README.md`, `AGENTS.md`, `CLAUDE.md`, and all import paths in `attachments.py` to reference `config.UPLOADS_DIR`/`ATTACHMENTS_DIR` instead of computing from `STATIC_DIR`.
Delete four stale SEO and style agent report JSON/MD files from agents/reports/ that had zero findings or were superseded. Export PYTHONDONTWRITEBYTECODE=1 in the Makefile so all make targets suppress bytecode generation, and add a `tree` target that lists the repository file tree via git ls-tree. Update AGENTS.md and CLAUDE.md documentation to reflect that routers now form a directory tree mirroring URL paths and that bytecode writing is disabled project-wide.
Add a RATE_LIMIT_DISABLED flag read from DEVPLACE_DISABLE_RATE_LIMIT env var that short-circuits the rate_limit_middleware when set, allowing test suites to disable per-IP throttling globally in conftest while individual rate-limit tests re-enable it via monkeypatch.
- Add `agents/reports/` to `.gitignore` to prevent generated agent report files from being tracked
- Implement `_TimestampStream` class and `install_timestamps()` function in `agents/agent.py` for prefixing stdout/stderr with timestamps and elapsed time
- Export `install_timestamps`, `set_write_budget`, `clear_write_budget`, and `set_shell_restricted` from `agents/base.py`; add `_RUN_LOCK`, `AGENT_ICONS` dictionary, and `WRITE_BUDGET = 20` constant
- Add `report_codename()` function and `CODENAME_ADJECTIVES`/`CODENAME_ANIMALS` tuples to `agents/core/__init__.py` for generating random agent report codenames
- Expand `WRITE_TOOLS` tuple in `agents/core/__init__.py` to include `replace_lines`, `insert_lines`, and `delete_lines`; remove `SWARM_TOOLS` from `payloads_for` exclusion list
- Update `CLAUDE.md` and `AGENTS.md` documentation with dataset column initialization rules and new agent infrastructure details
- Reorder route table in `README.md` to list `/uploads` before `/messages` and document Swagger/ReDoc/OpenAPI schema endpoints
The diff adds the mandatory `retoor <retoor@molodetz.nl>` header comment to every `.py` file under `devplacepy/` (including `__init__.py` and `routers/__init__.py`), and updates the `StyleAgent` in `agents/style.py` to instruct agents to only add the header on created or already-edited files rather than sweeping the entire repo. The `agents/base.py` operating protocol is also revised to reorder and clarify tool discipline rules.
- Add new `/projects/{slug}/files` endpoint group for per-project filesystem operations including directory and file CRUD, upload, and inline editing with public read and owner write access
- Extend attachment system to support video formats (webm, ogv, mov, m4v) with proper file icons and MIME types
- Implement configurable allowed file types via `allowed_file_types` site setting, replacing hardcoded `ALLOWED_UPLOAD_TYPES` with dynamic `allowed_extensions()` and `is_extension_allowed()` functions
- Add `delete_all_project_files()` call in `delete_content_item()` to clean up project files when a project is deleted
- Create database indexes on `project_files` table for `(project_uid, path)` and `(project_uid, parent_path)` to optimize file lookups
- Introduce `docs_prose.py` module with `render_prose()` function that renders Markdown content inside `data-render` divs using mistune, enabling dynamic prose rendering in documentation pages
- Enhance docs search with Markdown-aware text stripping (`_demarkdown()`) and improved HTML/script/style sanitization for better search indexing
- Update documentation API samples to reflect new attachment response fields (`is_image`, `is_video`, `mime_type`) and note video format support
- Update README to document the new project files endpoint and clarify AI gateway attribution for guest Devii sessions
This commit introduces a comprehensive set of new features including API key authentication with CLI management commands (get, reset, backfill), a Devii agentic assistant with WebSocket terminal and session bootstrap, an OpenAI-compatible LLM gateway service, and an admin service management panel. It also adds Playwright browser automation for bot support, configures internal gateway URLs, refactors content editing/deletion to support JSON API responses, and updates documentation across AGENTS.md, README.md, and the developer docs site.
Add three new feature modules (reactions, bookmarks, polls) with corresponding database tables, indexes, and router registrations. Introduce `SESSION_MAX_AGE_REMEMBER` config for extended session duration, add `get_unread_messages` template global, and include operational defaults for rate limiting and session settings in `site_settings`.
Introduce a new `/leaderboard` endpoint ranking top 50 members by total stars, wire it into the app router and documentation. Implement `_backfill_gamification()` in `database.py` to compute XP/levels for existing accounts from prior posts, comments, votes, and follows. Extract `is_owner()`, `create_content_item()`, and `detail_context()` into `content.py` to centralize content creation, reward awarding, mention notifications, and attachment linking. Update comment creation/deletion in `comments.py` to use `award_rewards()` with `XP_COMMENT` and the new `is_owner()` helper. Add shared template partials documentation for vote bars, star buttons, post headers, and topic selectors to `AGENTS.md`.
- Introduce push notification module with VAPID certificate generation, Web Push encryption, and subscription management
- Add push registration table index and database schema for storing user subscriptions
- Integrate push notification dispatch into existing notification creation flow via async background tasks
- Include PWA manifest, service worker, and install prompt UI elements in base template
- Register new push router and ensure certificate initialization on application startup
- Add configuration variables for VAPID private/public key file paths and subscription contact email
- Update JavaScript application entry point with PushManager and PwaInstaller modules
- Extend top navigation bar with hidden install and push enable buttons for authenticated users
Introduce `update_target_stars`, `get_target_owner_uid`, and `VOTABLE_TARGETS`/`STAR_TARGETS` in database.py; replace inline badge insertion with `award_badge()` and inline notification creation with `create_notification()` across auth, comments, and follow routers; add `enrich_items()` in new content module to centralize post/gist list assembly and remove duplicated enrichment logic from feed and gists routers.