Add `issue_usage` table with columns for user_uid, tokens, cost, and latency metrics, including a unique index on user_uid. Wire `accumulate_usage` into `enhance_ticket` and `generate_plan` to capture per-request AI usage, persist totals via `add_issue_usage` in both `IssueCreateService` and `PlanningReportService`, and expose aggregated usage as metric cards through `IssueTrackerService.collect_metrics`. Update planning API docs summary to reflect the new phased implementation document format.
Implement full attachment CRUD for issues and comments, restricted to open issues only. Attachments are stored locally and mirrored to the Gitea tracker via new `mirror_attachment_to_gitea` and `set_gitea_asset_id` functions. Add `gitea_asset_id` column to the attachments table, extend `IssueForm` and `IssueCommentForm` with `attachment_uids` field, and expose new API endpoints for listing, adding, and deleting issue attachments. Update agent configuration to include web research tools, and document the new capability in README, AGENTS.md, and CLAUDE.md.
- Add `/block`, `/mute` endpoints with block/unblock and mute/unmute functionality in `routers/relations.py`, hiding blocked users' content everywhere except their own profile while muting only suppresses notifications
- Introduce `devplace emoji-sync` CLI command to regenerate `static/js/emoji-shortcodes.js` from the emoji library, documented in `CLAUDE.md` and wired in `cli.py`
- Create `get_blocked_uids()` database helper and apply it in `content.py` `load_detail()` to filter blocked users' posts from detail views
- Implement `_uid_index()` and `_drop_index()` helpers in `database.py` for unique uid indexes across tables, with `user_relations` added to `SOFT_DELETE_TABLES`
- Document new routes in `AGENTS.md` and `README.md`, including emoji shortcodes rendering behavior distinct from the emoji picker
Add AI-powered body reformatting for valid articles in the news pipeline, converting raw text walls into clean Markdown with paragraphs, headings, and lists. Introduce `news_usage` database table and `add_news_usage`/`get_news_usage` helpers to track per-cycle gateway costs (calls, tokens, latency, USD) from response headers, reported on the admin Services page. Remove unused `.sidebar-more` CSS and apply `render_title()` to poll question/label fields.
- Add comprehensive documentation for backend content rendering in AGENTS.md, detailing the new `render_content` and `render_title` Jinja globals built on mistune with media processing, emoji shortcodes, and XSS protection
- Document the `X-Response-Time` header and bottom-left render time indicator in README.md
- Update bot token pricing documentation to clarify fallback vs gateway cost headers
- Add `email_accounts` to soft-delete tables and `idx_users_role` composite index in database schema
- Implement `telegram_pairings` and `telegram_links` table creation with column migration and indexes
- Add `/profile/{username}/telegram` endpoint to docs API with request/unpair actions
- Register `TelegramService` in main.py lifespan and add `response_timing` middleware emitting `X-Response-Time` header
- Introduce `TelegramPairForm` model and `guard_public_host_sync` synchronous host validation function
Add DevPlace agent configuration with dynamic OpenAPI schema fetching, implement access token issuance/resolution/revocation with configurable expiry, and create generic FastAPI dependency for JSON or form-encoded data validation against Pydantic models. Include comprehensive unit tests for form-data parsing and token lifecycle operations.
Introduce WILDCARD_TOKENS set in attachments.py to treat "*", ".*", "*.*" as wildcards that fall back to ALLOWED_UPLOAD_TYPES. Add cmd_token_issue and cmd_token_list CLI commands for issuing and listing DevPlace access tokens. Register access_tokens table in SOFT_DELETE_TABLES and initialize its columns and indexes in init_db. Replace Form() with Depends(json_or_form(...)) in admin backup, container, notification, settings, and user routers to support both JSON and form data.
- Add `ai_grade`, `featured`, `featured_locked`, `landing_locked`, `author`, `article_published`, `image_url`, `has_unique_image` columns to news table
- Extend `news_images` schema with `alt_text`, `phash`, `width`, `height`, `is_placeholder` columns
- Create `idx_news_featured` index for efficient featured queries
- Update admin toggle endpoints to set `featured_locked`/`landing_locked` when manually toggling
- Propagate `featured` and `image_url` fields through landing page, news list, and detail page rendering
- Update `get_featured_news` to return `featured` and `image_url` fields
- Document in AGENTS.md the full zero-maintenance pipeline: image perceptual hashing, placeholder detection, AI grading on cleaned text, reliability gate, effective score computation, and post-loop landing rotation
- Update README.md service description to reflect automatic image comparison and landing rotation capabilities
- Clarify docs_api.py summaries that toggling featured/landing now locks articles from auto-rotation
- Add `get_admin_uids()` and `get_primary_admin_uid()` to database.py for resolving the earliest-created admin
- Modify `can_view_project()` in content.py so a project hidden by an admin is invisible to other admins (both web UI and REST API)
- Update `_download_url()` and `_backup_payload()` in admin/backups.py to accept a `can_download` flag, gating the download endpoint with `is_primary_admin()`
- Remove `role` from `_user_facts()` in docs_live.py to avoid leaking admin status in live docs
- Update doc summaries in docs_api.py to reflect the new admin-visibility and backup-download semantics
Add name field as third fallback in get_instance lookup chain to support
container retrieval by display name. Introduce new terminal.py module
with async PTY session handling, pubsub-based I/O streaming, resize
control, and audit logging for container exec sessions.
Implement the multi-provider routing system for the OpenAI gateway, including two new database tables (`gateway_providers`, `gateway_models`) ensured at init, a new admin page at `/admin/gateway` with full CRUD for providers and model routes, and a `"gateway"` audit category mapped to `"ai"`. The routing layer sits transparently on top of the existing single-provider default path: unmatched model names fall through unchanged, while matched routes forward to the configured provider with their own pricing economy, vision model, and context window. Cross-worker cache invalidation uses a shared `_ROUTING_CACHE` bumped via `"gateway_routing"` cache version.
Copy the `dpc` binary into the container at `/usr/bin/dpc` and set its permissions to 0755 alongside other executables, enabling the DevPlaceCode service within the containerized environment.
- Record audit events in `admin_trash_restore` and `admin_trash_purge` endpoints with target metadata
- Log `notification.read.all` event when user clears devRant notification feed
- Include `devrant` as a valid origin in audit categories
- Add `admin_section` and `pagination_query` fields to audit and backup schemas for UI consistency
- Add `devplace backups` CLI subcommands (list, run, prune, clear) with job enqueueing and orphan cleanup
- Introduce `BACKUPS_DIR` and `BACKUP_STAGING_DIR` config paths for backup storage
- Implement `schedule_correction` and `schedule_modification` calls in content creation, comment creation, and comment editing flows
- Add `DEFAULT_CORRECTION_PROMPT` and `DEFAULT_MODIFIER_PROMPT` config constants for AI content processing
- Document timezone-aware date display using `local_dt`/`dt_ago` Jinja globals with client-side `Intl` localization
- Update README with AI content correction/modifier support in direct messages via `@ai` inline instructions
- Add `track_action(user["uid"], "vote")` call on upvote in `apply_vote`
- Add `account_api_key` field to `BotState` and `_adopt_account_api_key()` method that fetches the bot's own `users.api_key` from its profile JSON after auth, switching `LLMClient` to use it instead of the shared bootstrap key, routing gateway spend through the bot's user identity
- Replace `interleave_by_author` with a simpler greedy algorithm that picks the first row whose owner differs from the last picked, removing the multi-queue priority logic
- Extend `_ai_quota` response with `unlimited`, `requests`, and `last_used` fields sourced from new `user_spend_24h()` analytics helper
- Pass `recent_post_titles` from `BotState` into `LLMClient.generate_post()` and add a `distinct_rule` prompt instructing the model to avoid repeating framing, examples, or opinions from the bot's last six posts
- Remove early-return dedup check in `ArticleRegistry.admit` that skipped articles already held by the same bot owner, allowing re-admission under a different category
- Render bot username as a clickable link with avatar in the admin bots table
Update the feature-builder agent prompt, test-maintainer agent, and all four workflow JS files (devii-tool, endpoint, feature, job-service) to codify the DevPlace test standard as a non-optional project requirement: one test file per endpoint, directory tree mirroring the URL/source path, split into three tiers (unit, api, e2e). Add explicit Test phases to devii-tool, endpoint, feature, and job-service workflows, and embed tier-specific test instructions (path mapping, fixture choice, coverage scope) directly in each workflow's meta description and TESTS constant.
Implement `_split_sentences`, `_hard_chunks`, and `_pack` helpers to break long bot messages into parts respecting sentence boundaries and character limits. Introduce `COMMENT_CHAR_LIMIT`, `MESSAGE_CHAR_LIMIT`, `PART_SUFFIX_RESERVE`, and `PART_DELIVERY_DELAY` constants to control chunk size, suffix overhead, and inter-part delay. This enables the bot to deliver responses exceeding single-message limits across multiple sequential messages.
Add a new `/dbapi` router package providing a generic database API over `dataset`, restricted to admin sessions, admin API keys, and the internal gateway key. Includes:
- `tables.py`: list all tables and inspect table schemas
- `crud.py`: full CRUD operations (GET, POST, PATCH, DELETE) with soft-delete awareness, born-live inserts, `?include_deleted`, `.../restore`, and `?hard=true` purge
- `query.py`: validated read-only SELECT execution via sqlglot parsing, classification, and EXPLAIN dry-run; async query jobs with WebSocket streaming via `DbApiJobService`
- `nl.py`: natural-language-to-SQL conversion using the platform AI gateway with re-prompting until validation passes
Also register `DbApiJobService` and `PubSubService` in the service manager, add `DBAPI_DIR` to config data paths, and force cleartext `http://` connections to HTTP/1.1 in `curl_transport` to fix large request failures against uvicorn's HTTP/1.1-only internal gateway.
Implement a multi-agent deep web research subsystem including CLI commands for pruning expired jobs and clearing all artifacts, database tables for sessions/messages/URL cache with indexes, config paths for chroma storage, and internal embed URL for vector operations.
Add POST /openai/v1/embeddings route in openai_gateway router, new config fields for embedding upstream URL/model/key/enabled toggle with defaults pointing to OpenRouter Qwen3 8B, INTERNAL_EMBED_MODEL constant in config.py, documentation in docs_api.py and README.md describing the molodetz~embed model mapping, and embed-call tracking in gateway metrics alongside existing chat/vision counters.
Add SEO_REPORTS_DIR to config, STATIC_VERSION for cache-busting, seo prune/clear CLI subcommands, tools router with seo job endpoints, and boot-versioned static URLs in Dockerfile and Makefile
Migrate all runtime artifacts (database, uploads, VAPID keys, locks, bot state, container workspaces, zip staging) from scattered locations (`var/`, `devplacepy/static/uploads/`) into a unified `data/` directory. Update `config.py` as single source of truth with `DATA_PATHS` registry and `ensure_data_dirs()`, add `devplace migrate-data` CLI command with CRC verification and idempotent relocation, adjust `.dockerignore`, `.env.example`, `.gitignore`, `Dockerfile`, `Makefile`, `README.md`, `AGENTS.md`, `CLAUDE.md`, and all import paths in `attachments.py` to reference `config.UPLOADS_DIR`/`ATTACHMENTS_DIR` instead of computing from `STATIC_DIR`.
Add a reusable `text_search_clause` helper in `database.py` that builds a SQLAlchemy `or_` of `ilike` clauses over specified columns, returning `None` when search is blank or the table lacks the columns. Wire it into `get_feed_posts`, `get_gists_list`, and the projects listing so all three public index pages support free-text search via a `search` query parameter. Introduce `_sidebar_search.html` as the single search-box partial, included at the top of each listing's left filter panel with `_action`, `_placeholder`, and `_hidden` locals to preserve active category/tab filters on submit. Expose the `search` field on `FeedOut`, `GistsOut`, and `ProjectsOut` API schemas with corresponding OpenAPI documentation. Update `CLAUDE.md` to note the rate-limiter exemption for `GET`/`HEAD` and the `/openai` gateway, and refresh `README.md` route tables to mention the new search capability on `/feed`, `/gists`, and `/projects`.
Introduce `startup_jitter_seconds` config field so each bot delays its first session by a random amount up to the configured maximum, spreading fleet activity on service start. Persist a randomly chosen user-agent and viewport per bot in `BotState` and pass them to `BotBrowser` constructor, replacing the hardcoded defaults with a varied pool defined in `config.py`. Add `HANDLE_SUFFIX_WORDS` list and remove numeric suffixes from handle generation to avoid bot-farm appearance. Implement `pick_comment_style` and `is_short_comment_style` static methods in `LLMClient`, extend `generate_comment` with a `style` parameter that produces one-liner or question variants using `MIN_SHORT_COMMENT_LEN`, and wire session comment caps (`SESSION_COMMENT_CAP_MIN/MAX`) with cooldown intervals (`COMMENT_COOLDOWN_MIN/MAX_SECONDS`) into the bot loop. Update `BotRuntimeConfig`, `BotsService` config field registration, and architecture docs to reflect the new identity and pacing logic.
Implement configurable notification preferences across in-app and push channels, including a new `notification_preferences` table with soft-delete support, per-user toggle endpoints, admin defaults management, and canonical type definitions. The change introduces `NOTIFICATION_TYPES` and `NOTIFICATION_CHANNELS` constants, `NotificationPrefForm`/`NotificationDefaultForm` models, `notification_enabled` resolution logic, and UI integration via the profile notifications tab and admin panel.
The validation error check was placed before the config save operation, causing early returns that skipped persisting valid fields. This moves the error check after saving valid values and adds a "saved" key to the response listing which fields were successfully stored. Also adds autocomplete and password manager attributes to the password input field in the service config form to prevent browser autofill interference.
Introduce `mark_orchestration_tools`, `set_tool_scope`, and `reset_tool_scope` in `agents/agent.py` to allow runtime filtering of tool payloads by scope and orchestration flag. Refactor `get_tool_payloads` to respect the active scope and exclude orchestration-only tools. Update `agents/core/__init__.py` with `worker_tool_names` helper that returns read/reasoning tools plus conditional write/verify tools per mode, replacing the old `payloads_for` exclusion logic. Modify `agents/maestro.py` to call `mark_orchestration_tools` and wire per-agent result recording via `on_result` callback in `run_fleet`. Extend `agents/orchestrator.py` `run_fleet` signature to accept an optional `on_result` callable and invoke it after each agent run. Revise `agents/style.py` em-dash rule to distinguish prose from data occurrences, and add repository layout guidance to `agents/base.py` MAINT_HEADER.
- Add `agents/reports/` to `.gitignore` to prevent generated agent report files from being tracked
- Implement `_TimestampStream` class and `install_timestamps()` function in `agents/agent.py` for prefixing stdout/stderr with timestamps and elapsed time
- Export `install_timestamps`, `set_write_budget`, `clear_write_budget`, and `set_shell_restricted` from `agents/base.py`; add `_RUN_LOCK`, `AGENT_ICONS` dictionary, and `WRITE_BUDGET = 20` constant
- Add `report_codename()` function and `CODENAME_ADJECTIVES`/`CODENAME_ANIMALS` tuples to `agents/core/__init__.py` for generating random agent report codenames
- Expand `WRITE_TOOLS` tuple in `agents/core/__init__.py` to include `replace_lines`, `insert_lines`, and `delete_lines`; remove `SWARM_TOOLS` from `payloads_for` exclusion list
- Update `CLAUDE.md` and `AGENTS.md` documentation with dataset column initialization rules and new agent infrastructure details
- Reorder route table in `README.md` to list `/uploads` before `/messages` and document Swagger/ReDoc/OpenAPI schema endpoints
The diff adds the mandatory `retoor <retoor@molodetz.nl>` header comment to every `.py` file under `devplacepy/` (including `__init__.py` and `routers/__init__.py`), and updates the `StyleAgent` in `agents/style.py` to instruct agents to only add the header on created or already-edited files rather than sweeping the entire repo. The `agents/base.py` operating protocol is also revised to reorder and clarify tool discipline rules.
Add two new chat commands (`/stop` and `/reset`) to the Devii WebSocket handler, enabling users to stop or reset a session via text input. Introduce a new "Bots internals" documentation section with six prose pages covering architecture, personas, content generation, engagement, realism, and configuration for the autonomous bot fleet. Extend the bot service with article scoring, category picking, configurable pause/break timing, and a `gist_min_lines` parameter for LLM client initialization.
Refactor proxy target resolution to connect straight to the container's bridge IP and container port, bypassing the host port-publishing layer entirely. Introduce `container_ip`/`container_gateway` fields captured from Docker inspect on each running instance, stored alongside status updates. Update the fake backend to emit realistic `NetworkSettings` with per-container IPs. Remove the now-unused `CONTAINER_PROXY_HOST` config default and the old `ports_json`-based host port lookup in the proxy router.
