Commit Graph
19 Commits
Author SHA1 Message Date
retoorandClaude Sonnet 5 569f1dcc64 Add OpenCode Zen support, model health/stats dashboard, and gateway fallback fixes
AI gateway:
- Add a generic, admin-selectable `client_profile` field on gateway_providers
  (e.g. "opencode") so a provider needing special request headers (OpenCode
  Zen's client-identity spoofing) is configured like any other provider, not
  hardcoded by name.
- Track per-(provider, model) reliability/speed/latency health in memory,
  seeded from the existing gateway_usage_ledger at startup - purely
  observational, never influences routing.
- New Stats tab on /admin/gateway: request volume, latency, per-model
  breakdowns, and reliability weight, charted with a vendored Chart.js and
  devplace's own theme tokens.
- Record which model a failed request actually fell back to
  (fallback_used_route), surfaced in the Recent Failures table.
- Stop excluding context_length errors from fallback, and skip a primary
  attempt outright when its known context window is already too small for
  the estimated request size, going straight to the fallback.
- gateway_usage_ledger's provider/fallback_used_route columns and indexes
  are ensured centrally in database/schema.py's init_db(), the single point
  of truth for this table's schema.
- Non-OpenAI upstream routing and client-model passthrough; trust only the
  upstream's own X-Gateway-Model header for served-model attribution.

Devii agent:
- Fix a real lockup: plan/verify tools could be individually disabled via
  the admin tool toggles while still being required by the protocol gate,
  permanently bricking any task that needed tools. They can no longer be
  disabled, and the gate now also checks the tool is actually offered.
- Fix compaction being silently calibrated for a 1M-token model while
  running a much smaller one: context budget is now percentage-based and
  the summarizer's own request is sized to fit the real model.
- Give a specific, actionable retry message when plan()'s own arguments get
  cut off by the output limit, and tighten its schema to discourage
  overlong plans.

Other:
- Backup service: offload completed backups to a remote Hetzner Storage Box.
- Container manager: fix orphan blob leaks from sync races, add a two-phase
  plan/execute `system prune` CLI command.
- Admin gateway UI: replace the JS-rendered model/provider tables with
  server-rendered forms and pages.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DhmEkvutuwtzFVcLbTrhdo
2026-09-09 07:38:24 +02:00
retoor 8e9d3fad98 Add the trust and safety subsystem and the App Store compliance work
Implements the moderation and consent obligations a social platform carries,
so the web version and any client that speaks to it enforce the same rules.

Moderation core (services/moderation/, database/moderation.py): a reportable
target registry, the content filter and its choke points, the report queue with
atomic resolution, enforcement actions, consent tracking, maturity gating, and
account deletion with a grace window.

Surfaces: POST /reports plus the member report list, /admin/moderation and the
per-report admin view, /workspaces, terms acceptance at /auth/terms, consent and
account deletion under /profile, the report button and dialog partials, the
maturity gate, and the moderation stylesheet and ReportDialog client.

Every user-generated surface stays reportable by construction: new content tables
are registered in REPORTABLE_TARGETS or listed in UNREPORTABLE_TABLES with a
reason, and the registry test fails the suite on anything left unclassified.

Docs: community guidelines, content moderation, intellectual property, privacy,
terms, contact, and the admin-only moderation operations page, plus the
moderation API group and the Devii moderation actions.

Compliance record: applecomp.md is the requirement register, applechanges.md the
gap analysis against this codebase, and appleimpl.md the implementation design
they resolve to.

Tests cover the report flow, admin moderation, consent, account deletion, terms
acceptance, workspaces, and the registry invariant across the unit, api, and e2e
tiers.
2026-08-09 00:18:20 +02:00
retoor 9cfaddfc40 Enforce the Devii task quotas with atomic reservations
The creation and run quotas were checked and then acted on, so two concurrent
create_task calls or two schedulers could both pass the check and overshoot the
limit. Both are now a single conditional INSERT decided on the driver rowcount:
reserve_run takes a run slot after the claim and releases the claim by deferring
when the quota is spent, and insert_task_within_quota does the same for the task
row itself. Racing twelve and sixteen processes now yields exactly the limit.

The atomic insert names its columns, and dataset skips a None valued key when it
creates a table lazily, so the store declares the full task column set up front.
Both the column and index ensures now tolerate a concurrent duplicate, since
several processes build a store at once and SQLite DDL is not idempotent.

Adds the quota, task-run context, guard, store and scheduler test suites, and
documents the chokepoints and the unhackable task-run flag.
2026-07-26 19:58:42 +02:00
retoor ca6c527e32 Resolve the primary administrator to an account that can authenticate
The primary administrator was the earliest Admin by created_at with no further
condition, so a soft-deleted or deactivated account could hold the role and then
be refused by the api-key path, leaving nobody able to use the database API, the
backup download or cross-owner container management. Rows with no recorded signup
time also sorted ahead of every real account. Scan the earliest admins instead and
take the first that is neither deleted nor deactivated, with missing timestamps
sorted last.

Also stop the projects listing returning 500 when project_type or description is
NULL (the dict default never applies to an existing NULL column), align the issue
test fixture with its unit twin so a combined run cannot collide on a fixed uid,
read the settings value from the database rather than a stale per-process cache,
and give the seeded and fixture admins the is_active and created_at fields that
every real signup writes.
2026-07-26 19:58:18 +02:00
retoor a3963611f0 ipdate 2026-07-26 17:24:49 +02:00
retoor 4780016980 Quiiz system 2026-07-26 16:46:41 +02:00
retoor c53e2a3319 Update 2026-07-19 18:57:43 +02:00
retoor 32c8bbe0a9 Update 2026-07-07 15:28:28 +02:00
retoor 3fbac87723 feat: replace inline IP resolution with centralized client_ip utility across routers and audit
Consolidate scattered `X-Real-IP` / `request.client.host` fallback logic into the single `client_ip()` helper from `utils`, reducing duplication in the rate limiter middleware, project file zip routes, tools shared module, and audit record builder. Also refactor three admin endpoints (`/ai-usage/data`, `/analytics`, `/users/{uid}/ai-usage`) to use the existing `require_admin()` guard instead of repeating manual auth checks, and remove now-unused `get_current_user`/`is_admin` imports from those modules.
2026-07-04 18:07:33 +00:00
retoor ebf6bf7f82 feat: add Code Farm cooperative idle game with plot-based crop system and pub/sub notifications
Implement a Farmville-style cooperative idle game mounted at `/game` with member-only play and public farm viewing. The data layer is pure and timestamp-driven with no background tick: plot states are derived from `ready_at` vs current time, never stored. Key features include plantable software projects (shell script through kernel) that build over real time, harvest for coins and XP, CI tier upgrades for faster builds, plot purchasing with doubling cost, watering mechanics for friends' builds, daily quests, and prestige system. All game endpoints negotiate HTML or JSON and return full farm state for single-request client refresh. Pub/sub notifications broadcast farm updates on `public.game.farm.{username}` topics. Database schema adds `game_farms`, `game_plots`, and `game_quests` tables with appropriate indexes.
2026-06-22 16:41:53 +00:00
retoor 741d7aade6 docs: add block/mute user relations, emoji-sync CLI, and uid indexes
- Add `/block`, `/mute` endpoints with block/unblock and mute/unmute functionality in `routers/relations.py`, hiding blocked users' content everywhere except their own profile while muting only suppresses notifications
- Introduce `devplace emoji-sync` CLI command to regenerate `static/js/emoji-shortcodes.js` from the emoji library, documented in `CLAUDE.md` and wired in `cli.py`
- Create `get_blocked_uids()` database helper and apply it in `content.py` `load_detail()` to filter blocked users' posts from detail views
- Implement `_uid_index()` and `_drop_index()` helpers in `database.py` for unique uid indexes across tables, with `user_relations` added to `SOFT_DELETE_TABLES`
- Document new routes in `AGENTS.md` and `README.md`, including emoji shortcodes rendering behavior distinct from the emoji picker
2026-06-19 08:06:09 +00:00
retoor 6ceca3d0d4 docs: document server-side rendering pipeline, response timing middleware, and Telegram pairing API
- Add comprehensive documentation for backend content rendering in AGENTS.md, detailing the new `render_content` and `render_title` Jinja globals built on mistune with media processing, emoji shortcodes, and XSS protection
- Document the `X-Response-Time` header and bottom-left render time indicator in README.md
- Update bot token pricing documentation to clarify fallback vs gateway cost headers
- Add `email_accounts` to soft-delete tables and `idx_users_role` composite index in database schema
- Implement `telegram_pairings` and `telegram_links` table creation with column migration and indexes
- Add `/profile/{username}/telegram` endpoint to docs API with request/unpair actions
- Register `TelegramService` in main.py lifespan and add `response_timing` middleware emitting `X-Response-Time` header
- Introduce `TelegramPairForm` model and `guard_public_host_sync` synchronous host validation function
2026-06-18 22:09:34 +00:00
retoor 0a554ebc32 feat: restrict backup archive download to primary admin and hide admin-hidden projects from other admins
- Add `get_admin_uids()` and `get_primary_admin_uid()` to database.py for resolving the earliest-created admin
- Modify `can_view_project()` in content.py so a project hidden by an admin is invisible to other admins (both web UI and REST API)
- Update `_download_url()` and `_backup_payload()` in admin/backups.py to accept a `can_download` flag, gating the download endpoint with `is_primary_admin()`
- Remove `role` from `_user_facts()` in docs_live.py to avoid leaking admin status in live docs
- Update doc summaries in docs_api.py to reflect the new admin-visibility and backup-download semantics
2026-06-17 14:08:28 +00:00
retoor 99ed5c4f15 feat: add audit logging for admin trash restore/purge and notification clear actions
- Record audit events in `admin_trash_restore` and `admin_trash_purge` endpoints with target metadata
- Log `notification.read.all` event when user clears devRant notification feed
- Include `devrant` as a valid origin in audit categories
- Add `admin_section` and `pagination_query` fields to audit and backup schemas for UI consistency
2026-06-16 05:08:58 +00:00
retoor e59bc2d34e feat: adopt per-bot account api key for gateway attribution and improve post distinctness
- Add `account_api_key` field to `BotState` and `_adopt_account_api_key()` method that fetches the bot's own `users.api_key` from its profile JSON after auth, switching `LLMClient` to use it instead of the shared bootstrap key, routing gateway spend through the bot's user identity
- Replace `interleave_by_author` with a simpler greedy algorithm that picks the first row whose owner differs from the last picked, removing the multi-queue priority logic
- Extend `_ai_quota` response with `unlimited`, `requests`, and `last_used` fields sourced from new `user_spend_24h()` analytics helper
- Pass `recent_post_titles` from `BotState` into `LLMClient.generate_post()` and add a `distinct_rule` prompt instructing the model to avoid repeating framing, examples, or opinions from the bot's last six posts
- Remove early-return dedup check in `ArticleRegistry.admit` that skipped articles already held by the same bot owner, allowing re-admission under a different category
- Render bot username as a clickable link with avatar in the admin bots table
2026-06-15 22:44:12 +00:00
retoor d31ccba6c1 feat: add admin/internal database API with CRUD, read-only query, and natural-language SQL endpoints
Add a new `/dbapi` router package providing a generic database API over `dataset`, restricted to admin sessions, admin API keys, and the internal gateway key. Includes:

- `tables.py`: list all tables and inspect table schemas
- `crud.py`: full CRUD operations (GET, POST, PATCH, DELETE) with soft-delete awareness, born-live inserts, `?include_deleted`, `.../restore`, and `?hard=true` purge
- `query.py`: validated read-only SELECT execution via sqlglot parsing, classification, and EXPLAIN dry-run; async query jobs with WebSocket streaming via `DbApiJobService`
- `nl.py`: natural-language-to-SQL conversion using the platform AI gateway with re-prompting until validation passes

Also register `DbApiJobService` and `PubSubService` in the service manager, add `DBAPI_DIR` to config data paths, and force cleartext `http://` connections to HTTP/1.1 in `curl_transport` to fix large request failures against uvicorn's HTTP/1.1-only internal gateway.
2026-06-14 23:00:30 +00:00
retoor f2b910ea75 fix: correct "bugs" to "issues" in routing table and README references across multiple documentation files 2026-06-14 07:48:10 +00:00
retoor 61c1ae8c5d feat: add multi-channel Devii sessions with docs search mode and channel-aware conversation persistence 2026-06-13 21:37:13 +00:00
retoor 43f4011005 chore: reorganize test files into domain-specific subdirectories under tests/
Split the monolithic test directory into three tiers (unit, api, e2e) with a path-mirroring directory structure. Added corresponding Makefile targets (test-unit, test-api, test-e2e) and updated all documentation references (CLAUDE.md, README.md, testing-cicd.html, testing-framework.html, testing-make.html) to reflect the new layout and naming conventions.
2026-06-13 14:32:33 +00:00