blindxfish/devplacepy
1
0
Fork 0
forked from retoor/devplacepy
Code Pull Requests Activity

docs: document server-side rendering pipeline, response timing middleware, and Telegram pairing API

Browse Source 
- Add comprehensive documentation for backend content rendering in AGENTS.md, detailing the new `render_content` and `render_title` Jinja globals built on mistune with media processing, emoji shortcodes, and XSS protection
- Document the `X-Response-Time` header and bottom-left render time indicator in README.md
- Update bot token pricing documentation to clarify fallback vs gateway cost headers
- Add `email_accounts` to soft-delete tables and `idx_users_role` composite index in database schema
- Implement `telegram_pairings` and `telegram_links` table creation with column migration and indexes
- Add `/profile/{username}/telegram` endpoint to docs API with request/unpair actions
- Register `TelegramService` in main.py lifespan and add `response_timing` middleware emitting `X-Response-Time` header
- Introduce `TelegramPairForm` model and `guard_public_host_sync` synchronous host validation function
This commit is contained in:
retoor
2026-06-18 22:09:34 +00:00
parent 95dca73291
commit 6ceca3d0d4
146 changed files with 6079 additions and 392 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+66 -5
View File
@@ -200,6 +200,10 @@ uniform envelope `{ "ok": true, "redirect": "…", "data": {…} }`; errors retu
JSON → `401`, non-admin → `403`). The core lives in `devplacepy/responses.py`
(`wants_json`, `respond`, `action_result`). Full details: `/docs/conventions.html`.
Every response carries an `X-Response-Time: <ms>ms` header (set by the outermost `response_timing`
middleware, the full request total), and every rendered HTML page shows that server render time as a
small fixed indicator in the bottom-left corner.
```bash
curl -H "Accept: application/json" https://your-host/feed
curl -H "Accept: application/json" -X POST -d "content=hi&title=T&topic=devlog" https://your-host/posts/create
@@ -407,7 +411,7 @@ Configuration on the Services tab:
| `bot_news_api` | `https://news.app.molodetz.nl/api` | Article source |
| `bot_model` | `molodetz` | Generic model name; the gateway maps it to the real model |
| `bot_api_key` | internal key | LLM key (defaults to the auto-generated gateway internal key) |
| `bot_input_cost_per_1m` / `bot_output_cost_per_1m` | `0.27` / `1.10` | Token pricing for live cost tracking |
| `bot_input_cost_per_1m` / `bot_output_cost_per_1m` | `0.14` / `0.28` | Fallback token pricing; used only when the LLM endpoint returns no gateway cost headers. Live cost is read from the gateway's authoritative `X-Gateway-Cost-USD` per-call header |
| `bot_max_per_article` | `2` | How many bots may post about one article, each from a different angle |
| `bot_article_ttl_days` | `7` | How long an article stays covered before it can be posted again |
| `bot_gist_min_lines` | `6` | Reject generated snippets shorter than this many non-empty lines |
@@ -530,6 +534,19 @@ Spend is capped per owner over a rolling 24 hours. Every turn appends a row to
`devii_usage_ledger` (the authoritative source for the cap - the in-memory cost tracker is
display-only) and an audit row to `devii_turns`. The cap is checked before each turn.
**Reminders and scheduled tasks.** Ask Devii to remind you of something ("remind me to go
upstairs in 40 seconds", "every weekday at 9am post the news"), and it schedules the work
instead of doing it immediately. A scheduled task stores a self-contained prompt that a fresh
agent runs when it fires - once after a delay or at an absolute time, on a repeating interval,
or on a cron expression. Reminders are **timezone-aware**: your browser's timezone is sent to
Devii and used to interpret wall-clock times you give ("3pm" means 3pm where you are),
converting them to UTC for storage. They are **persistent**: tasks live in `devii_tasks` and
are run by the background service, so a queued reminder survives a server restart and fires
even if you have closed the Devii terminal. When a reminder fires you receive an in-app
notification and a live toast carrying its message (the **Reminders** notification type, which
you can toggle like any other on your profile), in addition to the result appearing in the
terminal. Manage your reminders conversationally (list, change, run now, or delete them).
Configuration on the Services tab:
| Parameter | Default | Purpose |
@@ -550,6 +567,8 @@ Configuration on the Services tab:
| `devii_rsearch_enabled` | on | Enable the external web search tools (`rsearch_*`) |
| `devii_rsearch_url` | `https://rsearch.app.molodetz.nl` | Base URL of the web search service those tools call |
| `devii_rsearch_timeout` | `300` | Read timeout (seconds) for `rsearch_*` calls; web-grounded answers can take minutes; minimum five minutes |
| `devii_email_enabled` | on | Enable the email tools (`email_*`) for signed-in users |
| `devii_email_timeout` | `30` | Connection/read timeout (seconds) for IMAP and SMTP calls |
Beyond the platform tools, Devii has external **web** tools. `fetch_url` reads a web page;
`http_request` makes an arbitrary HTTP call (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS) to any
@@ -566,6 +585,18 @@ preferred; Devii uses them only when the user explicitly asks to search the web
source. They are gated by `devii_rsearch_enabled` and the service URL is configurable via
`devii_rsearch_url`.
A signed-in user's Devii can also connect to their **own email** over IMAP and SMTP. The user
configures one or more accounts conversationally (`email_account_set` stores a connection under a
label - host, port, username, password, with sensible defaults of IMAP 993 over SSL and SMTP 587
with STARTTLS), and Devii can then list folders, list and search messages, read a full message with
its attachments, mark messages read or flagged, move messages between folders, delete messages, and
send mail. Credentials are stored per user and never shown back (the password is only reported as
set or not), the mail server is SSRF-guarded against private and loopback addresses, sending is an
explicit action Devii confirms with the user first, and deleting a message or removing a saved
account is confirmation-gated. The tools are gated by `devii_email_enabled` and time out per
`devii_email_timeout`. Email is configured only through Devii (no separate settings page) and is
available to signed-in users, not guests.
A `devii` console script ships the same agent as an interactive terminal:
```bash
@@ -574,6 +605,36 @@ devii --api-key <your DevPlace api_key> --base-url https://your-host
devii -p "List my unread notifications as a bullet list." # one-shot
```
### Devii on Telegram
`TelegramService` (`devplacepy/services/telegram/`) puts Devii on Telegram. A user pairs
their Telegram account by requesting a four digit code from their profile (valid one hour by
default), then sends that code to the bot; once paired, chatting with the bot talks to their
own Devii exactly like the web terminal, with markdown replies, a typing indicator, live
message editing instead of message spam, and image understanding (a sent photo is read by the
gateway vision model). The Telegram thread is an isolated conversation but shares the user's
Devii memory, tools, and the same rolling 24 hour spend cap.
The service is **off by default** (not every deployment has a bot token) and is started,
stopped, configured, and monitored from `/admin/services` like any other background service.
Its operational log streams live on the service detail page and is never written to the
database. The Telegram long-poller runs as a supervised subprocess so it stays isolated from
the web workers; because only the background-service lock owner runs the service, there is
always exactly one poller (Telegram rejects concurrent polling per token).
Devii also gains a `telegram_send` tool (only usable by a signed-in, paired user) so it can
push a message to the user's Telegram from a turn or a scheduled task - the basis for future
Telegram notifications.
Configuration on the Services tab:
| Parameter | Default | Purpose |
|-----------|---------|---------|
| `telegram_bot_token` | (secret) | Bot token from @BotFather; required to start |
| `telegram_poll_timeout` | `25` | getUpdates long-poll hold time (seconds) |
| `telegram_code_ttl_minutes` | `60` | Pairing code lifetime |
| `telegram_max_concurrent_turns` | `8` | Upper bound on Devii turns across all chats |
### Site customization (per-user CSS/JS)
Each user can reshape the site to taste by injecting their own **CSS** (look) and
@@ -721,21 +782,21 @@ Removing a record is a **soft delete**, not a physical one: it stamps `deleted_a
A member may delete only their own content, but an **administrator may delete any member's** post, comment, gist, project, project file, or attachment - the owner-or-admin check lives on each delete endpoint, so it applies equally to the web UI and to the Devii assistant (which acts purely through the platform API as the signed-in user). When an admin's Devii is asked to delete something it requires explicit confirmation before each deletion, and the result is the same soft delete, restorable from Trash.
### Database API (`/dbapi`, admin or internal only)
### Database API (`/dbapi`, primary administrator only)
An administrator (session or admin API key) or an internal service (the gateway internal key) can **read** any table through a single, safe API; members and guests get `403`. The database API is **strictly read-only** - it can never insert, update, replace, delete, or restore data in any way.
The **primary administrator** (the oldest Admin account, the same identity that may download backups) authenticated by session or their API key can **read** any table through a single, safe API; members, guests, every other administrator, and internal/service callers (the gateway internal key is not accepted) all get `403`. The database API is **strictly read-only** - it can never insert, update, replace, delete, or restore data in any way.
- **Read per table:** `GET /dbapi/{table}` (filtered, searchable, keyset pagination) and `GET /dbapi/{table}/{key}/{value}`. There are no write endpoints; deny-listed tables (sessions, password resets) are never exposed.
- **`query()` is read-only:** `POST /dbapi/query` runs a single validated SELECT and returns rows. Every query is parsed (sqlglot), classified, and dry-run with `EXPLAIN` on a read-only connection before execution; non-SELECT statements (INSERT/UPDATE/DELETE/DDL) are refused, and a SELECT with no WHERE/JOIN/LIMIT is flagged as suspicious.
- **Ask in plain language:** `POST /dbapi/nl` turns a question such as *"all users registered longer than three days"* into a validated SELECT (auto-adding `deleted_at IS NULL` for soft-delete tables), using the platform AI gateway and re-prompting until the SQL validates; pass `execute=true` to also run it read-only.
- **Async:** `POST /dbapi/query/async` runs a heavy read query off the request path and streams progress over `WS /dbapi/query/{uid}/ws`.
- The Devii assistant exposes the same read-only capability to administrators (list, get, query, and natural-language SELECT); it cannot change data through the database API.
- The Devii assistant exposes the same read-only capability to the **primary administrator only** (list, get, query, and natural-language SELECT); the database tools are added to the tool list only for that user, so every other administrator's Devii does not see them and is unaware the database API exists. It cannot change data through the database API.
### Pub/Sub bus (`/pubsub`)
A database-free publish/subscribe bus for live updates without polling. Clients connect to `WS /pubsub/ws` to subscribe to topics (with `foo.*` wildcards) and publish messages; backends and administrators can also publish over `POST /pubsub/publish`. Users may use their own `user.{uid}.*` namespace and subscribe to shared `public.*` topics; administrators and internal services may use any topic. The browser client is available as `app.pubsub.subscribe(topic, cb)` / `app.pubsub.publish(topic, data)`. The bus is in-memory and best-effort by design.
Two background services bridge persisted state onto the bus so the interface updates without per-client polling: the **Notification relay** pushes new in-app notifications as live toasts and refreshes each viewer's unread notification and message badges instantly, and the **Live view relay** pushes the admin live views (container list and instances, bot fleet, background services, AI usage) to whichever administrators are watching, computing a snapshot only for views that currently have subscribers. Both run on the single service-lock owner and degrade to a low-frequency HTTP poll if the bus is unavailable.
Two background services bridge persisted state onto the bus so the interface updates without per-client polling: the **Notification relay** pushes new in-app notifications as live toasts and refreshes each viewer's unread notification and message badges instantly, and the **Live view relay** pushes the admin live views (container list and instances, bot fleet, background services, AI usage, backups) to whichever administrators are watching, computing a snapshot only for views that currently have subscribers. Both run on the single service-lock owner and degrade to a low-frequency HTTP poll if the bus is unavailable.
## Testing