forked from retoor/devplacepy
feat: add api key auth, devii agent, openai gateway, and admin service management
This commit introduces a comprehensive set of new features including API key authentication with CLI management commands (get, reset, backfill), a Devii agentic assistant with WebSocket terminal and session bootstrap, an OpenAI-compatible LLM gateway service, and an admin service management panel. It also adds Playwright browser automation for bot support, configures internal gateway URLs, refactors content editing/deletion to support JSON API responses, and updates documentation across AGENTS.md, README.md, and the developer docs site.
This commit is contained in:
@@ -23,7 +23,7 @@ Open `http://localhost:10500`.
|
||||
| Templates | Jinja2 (server-side rendered) |
|
||||
| Frontend | Pure ES6 JavaScript, one class per file |
|
||||
| Database | SQLite via `dataset` (auto-sync schema, `uid` PKs, WAL mode, 30s busy timeout) |
|
||||
| Auth | Session cookies, PBKDF2-SHA256 via passlib |
|
||||
| Auth | Session cookie, API key (`X-API-KEY`/Bearer), or HTTP Basic; PBKDF2-SHA256 via passlib |
|
||||
| Avatars | Multiavatar (local SVG generation, no external API, <5ms) |
|
||||
| Validation | `hawk` (Python/JS/CSS/HTML) |
|
||||
| Load testing | Locust (locustfile.py) |
|
||||
@@ -68,8 +68,11 @@ devplacepy/
|
||||
| `/leaderboard` | Contributor ranking by total stars earned |
|
||||
| `/avatar` | Multiavatar proxy with in-memory cache |
|
||||
| `/bugs` | Bug reports listing, creation |
|
||||
| `/services` | Background service monitoring (status, logs) |
|
||||
| `/admin/services` | Background service management (start/stop, config, status, logs) |
|
||||
| `/admin` | Admin panel (user management, news curation, settings) |
|
||||
| `/docs` | Developer documentation site with a complete, interactive HTTP API reference |
|
||||
| `/openai` | OpenAI-compatible LLM gateway service (`/openai/v1/chat/completions`, `/openai/v1/*`) |
|
||||
| `/devii` | Devii agentic assistant: WebSocket terminal (`/devii/ws`), standalone page, usage (`/devii/usage`), session bootstrap |
|
||||
| `(none)` | `/robots.txt`, `/sitemap.xml` (SEO) |
|
||||
| `(none)` | `/push.json` (VAPID key + subscribe), `/service-worker.js`, `/manifest.json` (push + PWA) |
|
||||
|
||||
@@ -82,13 +85,14 @@ Member progression is driven by activity and peer recognition.
|
||||
- **Badges** are awarded once per milestone: first post/comment/project/gist, 10 posts (Prolific), 25 stars (Rising Star), 100 stars (Star Author), 10 followers (Popular), a 7-day activity streak (On Fire), and reaching levels 5 and 10. Badges render with an icon and description on the profile.
|
||||
- **Leaderboard** (`/leaderboard`) ranks the top 50 members by total stars (single page, no pagination); a member's own rank is shown on their profile.
|
||||
- **Contribution heatmap and streaks.** Each profile shows a 12-month activity heatmap and the current/longest daily streak, derived from post/comment/gist/project timestamps (no extra storage).
|
||||
- **Social graph listings.** Each profile has Followers and Following tabs that paginate the follow graph (25 per page) and show a follow/unfollow control for each person. The same data is available as JSON at `GET /profile/{username}/followers` and `GET /profile/{username}/following`.
|
||||
- **Reward notifications** fire when a member levels up or earns a badge.
|
||||
|
||||
## Engagement
|
||||
|
||||
- **Emoji reactions** — a fixed palette of reactions on posts, comments, gists, and projects, separate from voting and carrying no ranking weight.
|
||||
- **Polls** — a post can carry a poll (question plus up to six options); results appear as live bars once the viewer votes, one vote per member.
|
||||
- **Bookmarks** — save posts, gists, projects, and news to a personal list at `/bookmarks/saved`.
|
||||
- **Emoji reactions** - a fixed palette of reactions on posts, comments, gists, and projects, separate from voting and carrying no ranking weight.
|
||||
- **Polls** - a post can carry a poll (question plus up to six options); results appear as live bars once the viewer votes, one vote per member. A poll can be attached when the post is created or added later by editing a post that has none.
|
||||
- **Bookmarks** - save posts, gists, projects, and news to a personal list at `/bookmarks/saved`.
|
||||
|
||||
XP awards are wired at the existing content-creation, vote, and follow hook points in the routers and centralized in `award_xp()` / `check_milestone_badges()` (`devplacepy/utils.py`). Existing accounts have their XP and levels backfilled once from prior activity at startup (`init_db()`).
|
||||
|
||||
@@ -99,6 +103,8 @@ XP awards are wired at the existing content-creation, vote, and follow hook poin
|
||||
| `DEVPLACE_DATABASE_URL` | `sqlite:///devplace.db` | Database connection string |
|
||||
| `SECRET_KEY` | hardcoded fallback | Session signing key |
|
||||
| `DEVPLACE_VAPID_SUB` | `mailto:retoor@molodetz.nl` | Contact address in the VAPID JWT `sub` claim |
|
||||
| `DEVPLACE_INTERNAL_BASE_URL` | `http://localhost:10500` | Base URL the platform's own services dial for the AI gateway |
|
||||
| `DEEPSEEK_API_KEY` / `OPENROUTER_API_KEY` | unset | Upstream provider keys; migrated into the gateway settings on first boot |
|
||||
|
||||
### Runtime settings
|
||||
|
||||
@@ -108,7 +114,6 @@ Operational behavior is tunable live from `/admin/settings` (stored in `site_set
|
||||
|---------|---------|--------|
|
||||
| `rate_limit_per_minute` | `60` | Mutating requests allowed per IP per window |
|
||||
| `rate_limit_window_seconds` | `60` | Rolling window for the request limit |
|
||||
| `news_service_interval` | `3600` | Seconds between news fetch cycles (min 60) |
|
||||
| `session_max_age_days` | `7` | Standard session cookie lifetime |
|
||||
| `session_remember_days` | `30` | Remember-me session cookie lifetime |
|
||||
| `registration_open` | `1` | When `0`, new sign-ups are rejected |
|
||||
@@ -117,51 +122,272 @@ Operational behavior is tunable live from `/admin/settings` (stored in `site_set
|
||||
|
||||
Numeric values are floored to safe minimums so an invalid entry cannot lock out writes or stall services. Consumers read via `get_setting`/`get_int_setting`, which fall back to these defaults when a row is absent.
|
||||
|
||||
## Authentication
|
||||
|
||||
The website uses a `session` cookie. For automation, every page and action also
|
||||
accepts three header-based methods, resolved centrally in `get_current_user`
|
||||
(`utils.py`) so they work everywhere with no per-route changes:
|
||||
|
||||
- **API key** - `X-API-KEY: <key>`
|
||||
- **Bearer** - `Authorization: Bearer <key>`
|
||||
- **HTTP Basic** - `Authorization: Basic base64(username-or-email:password)`
|
||||
|
||||
Every user has an `api_key` (a uuid7), set at signup and backfilled for existing
|
||||
users on startup (and via `devplace apikey backfill`). The key is shown on the
|
||||
profile page to its owner (and to admins for any user) with a live **Regenerate**
|
||||
button; regenerating invalidates the old key immediately. Invalid credentials
|
||||
return `401`; requests with no credentials behave as anonymous (browser redirect to
|
||||
login). Full details and copy-paste `curl` examples live at `/docs/authentication.html`.
|
||||
|
||||
```bash
|
||||
curl -H "X-API-KEY: <your-key>" https://your-host/notifications
|
||||
curl -u <username>:<password> -X POST https://your-host/follow/<username>
|
||||
```
|
||||
|
||||
CLI: `devplace apikey get <username>` / `reset <username>` / `backfill`.
|
||||
|
||||
## Content negotiation (HTML or JSON)
|
||||
|
||||
Every endpoint that renders a page or returns a redirect also speaks JSON, so anything the
|
||||
website does is automatable from the same URLs. A request gets JSON when it sends
|
||||
`Accept: application/json` or `Content-Type: application/json`; a normal browser navigation
|
||||
(`Accept: text/html`) always gets HTML, so existing behaviour is unchanged (the legacy
|
||||
`X-Requested-With: fetch` AJAX header still drives the four engagement endpoints only). JSON responses are defined by Pydantic models in `devplacepy/schemas.py` and built
|
||||
from the same context the templates use (sensitive user fields like `email`/`api_key`/
|
||||
`password_hash` are never exposed). Page GETs return the page payload; form actions return a
|
||||
uniform envelope `{ "ok": true, "redirect": "…", "data": {…} }`; errors return
|
||||
`{ "error": { "status", "message" } }` (validation → `422` with a `fields` map, unauthenticated
|
||||
JSON → `401`, non-admin → `403`). The core lives in `devplacepy/responses.py`
|
||||
(`wants_json`, `respond`, `action_result`). Full details: `/docs/conventions.html`.
|
||||
|
||||
```bash
|
||||
curl -H "Accept: application/json" https://your-host/feed
|
||||
curl -H "Accept: application/json" -X POST -d "content=hi&title=T&topic=devlog" https://your-host/posts/create
|
||||
```
|
||||
|
||||
## Documentation site
|
||||
|
||||
`/docs` serves a server-rendered docs site with a feed-style left sidebar
|
||||
(`routers/docs.py`). `/docs/index.html` is the start page and `/docs/authentication.html`
|
||||
documents the auth methods; both are curated markdown pages rendered through the shared
|
||||
content renderer (markdown + highlight.js) with the current host and, when logged in, your
|
||||
own API key and username filled in.
|
||||
|
||||
Every other endpoint is documented from a single source of truth, the `API_GROUPS` registry
|
||||
in `devplacepy/docs_api.py`. Each group becomes a reference page listing its endpoints with
|
||||
copy-paste cURL, JavaScript, and Python examples and an interactive "try it" panel
|
||||
(`static/js/ApiTester.js`) that executes the real call from the browser with your API key
|
||||
pre-filled. Each panel has a response-format selector (defaulting to JSON) that drives the
|
||||
`Accept` header for both the live request and the generated snippets, an always-visible
|
||||
**Expected** tab showing the modeled response, and a **Live response** tab for the real
|
||||
result. To document a new endpoint, add an entry to `docs_api.py`; the page, sidebar link,
|
||||
examples, runner, and expected-response sample are generated automatically. FastAPI's
|
||||
built-in Swagger is disabled so `/docs` belongs to this site.
|
||||
|
||||
Operator pages are admin-only: `/docs/admin.html` (user/news/settings administration) and
|
||||
`/docs/services.html` (Background Services) are hidden from the sidebar and return 404 for
|
||||
non-admins. The Background Services page is generated live from the service registry
|
||||
(`build_services_group()` over `service_manager.describe_all()`), so every registered service
|
||||
and its full configuration are documented automatically - including future services.
|
||||
|
||||
## Background Services
|
||||
|
||||
`devplacepy/services/` provides a generic async service framework. Services start on server boot, run at configurable intervals, and are gracefully cancelled on shutdown.
|
||||
`devplacepy/services/` provides a generic async service framework. `/admin/services` is a slim index listing each service with its description and live status; clicking a service opens its detail page where everything for that service lives - start/stop, enable-on-boot, run-now, clear logs, adjustable log size, and live status/metrics - organized into **Overview / Configuration / Logs** tabs (configuration grouped into sections). State is stored in the database (desired state in `site_settings`, observed state in the `service_state` table), so control and status are correct even with multiple workers where only one runs the services.
|
||||
|
||||
### Service framework
|
||||
|
||||
- **`BaseService`** - abstract class with async run loop, `deque(maxlen=20)` log buffer, `name`, `interval_seconds`
|
||||
- **`ServiceManager`** - singleton that registers, starts, and stops all services
|
||||
- **`ConfigField`** - declarative parameter spec (type, default, validation, secret) a service uses to declare its editable settings
|
||||
- **`BaseService`** - abstract class with a reconciling run loop that honors the persisted `enabled`/command/interval state, plus `config_fields`, `get_config()`, `describe()`, and a log buffer
|
||||
- **`ServiceManager`** - singleton: `register`, `describe_all`, `set_enabled`, `send_command`, `save_config`, `supervise`, `shutdown_all`
|
||||
- **`NewsService`** - fetches news from `news.app.molodetz.nl/api`, grades with AI, stores articles >= threshold
|
||||
- **`BotsService`** - Playwright fleet of AI personas that browse and interact with a DevPlace instance, with live cost/usage metrics (opt-in; install the `bots` extra)
|
||||
- **`GatewayService`** - OpenAI-compatible LLM gateway at `/openai/v1/*`, forwarding to DeepSeek (default) with optional vision augmentation; the single point of truth for AI that every other service routes through (enabled by default)
|
||||
|
||||
### Adding a service
|
||||
|
||||
Create a class extending `BaseService`, override `run_once()`, register in `main.py`:
|
||||
Create a class extending `BaseService`, declare its `config_fields`, override `run_once()` (read parameters via `self.get_config()`), and register in `main.py`:
|
||||
```python
|
||||
service_manager.register(YourService())
|
||||
```
|
||||
The service appears at `/services` with live status and log tail.
|
||||
The service appears at `/admin/services` with controls, a generated config form, live status, and the log tail - no template, JS, or router changes.
|
||||
|
||||
### News service
|
||||
|
||||
Fetches news from a configurable API, grades each article via AI, stores ALL articles in the `news` table (nothing is silently skipped). Articles with grade >= the configurable threshold are auto-published; the rest go to draft. Admin can manually publish/draft, toggle for landing page appearance, and delete. Images are extracted from article URLs.
|
||||
|
||||
Configuration via admin site settings:
|
||||
Configuration on the Services tab (`/admin/services`):
|
||||
|
||||
| Setting key | Default | Purpose |
|
||||
|-------------|---------|---------|
|
||||
| Parameter | Default | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| `news_grade_threshold` | `7` | Minimum AI grade for auto-publish |
|
||||
| `news_api_url` | `https://news.app.molodetz.nl/api` | News source |
|
||||
| `news_ai_url` | `https://openai.app.molodetz.nl/v1/chat/completions` | AI grading endpoint |
|
||||
| `news_ai_model` | `molodetz` | AI model |
|
||||
| `news_ai_url` | `http://localhost:10500/openai/v1/chat/completions` | AI grading endpoint (the internal gateway) |
|
||||
| `news_ai_model` | `molodetz` | Generic model name; the gateway maps it to the real model |
|
||||
| `news_ai_key` | internal key | AI API key (`NEWS_AI_KEY` env, then the auto-generated gateway internal key) |
|
||||
| `news_service_interval` | `3600` | Seconds between fetch cycles (min 60) |
|
||||
|
||||
News articles have detail pages at `/news/{slug}` with full comment support (same component as posts/projects). The landing page can display curated articles toggled from admin.
|
||||
|
||||
CLI: `devplace news clear` - delete all news from local database.
|
||||
|
||||
### Bots service
|
||||
|
||||
A Playwright-driven fleet of AI personas (`devplacepy/services/bot/`) that browse and interact with a DevPlace instance: posting, commenting, voting, reacting, creating gists/projects, filing bugs, following, and messaging. It is the former standalone `dpbot.py`, refactored into a package and managed entirely from the Services tab. Disabled by default.
|
||||
|
||||
Install the extra and the browser binary, then enable it on `/admin/services`:
|
||||
|
||||
```bash
|
||||
pip install -e ".[bots]"
|
||||
playwright install chromium
|
||||
```
|
||||
|
||||
Configuration on the Services tab:
|
||||
|
||||
| Parameter | Default | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| `bot_fleet_size` | `1` | Number of concurrent bot accounts |
|
||||
| `bot_headless` | enabled | Run Chromium headless |
|
||||
| `bot_max_actions` | `0` | Stop a bot after N actions (0 = forever) |
|
||||
| `bot_base_url` | `https://pravda.education` | Target DevPlace instance |
|
||||
| `bot_api_url` | `http://localhost:10500/openai/v1/chat/completions` | LLM endpoint (the internal gateway) |
|
||||
| `bot_news_api` | `https://news.app.molodetz.nl/api` | Article source |
|
||||
| `bot_model` | `molodetz` | Generic model name; the gateway maps it to the real model |
|
||||
| `bot_api_key` | internal key | LLM key (defaults to the auto-generated gateway internal key) |
|
||||
| `bot_input_cost_per_1m` / `bot_output_cost_per_1m` | `0.27` / `1.10` | Token pricing for live cost tracking |
|
||||
|
||||
The Services tab shows live fleet metrics (bots running, total cost, LLM calls, tokens, posts/comments/votes) and a per-bot breakdown that update on the page poll. Each bot keeps a stable identity and cumulative cost in `~/.devplace_bots/state_slot{N}.json`.
|
||||
|
||||
### LLM gateway service
|
||||
|
||||
`GatewayService` (`devplacepy/services/openai_gateway/`) exposes an OpenAI-compatible
|
||||
endpoint at `/openai/v1/chat/completions` (plus a transparent `/openai/v1/*`
|
||||
passthrough). It forwards to the configured upstream (DeepSeek by default) and, when a
|
||||
request carries image content, first describes the image(s) via a vision model
|
||||
(OpenRouter/Gemma by default) and inlines the description as text - so a vision-less
|
||||
upstream can still answer. Enabled by default and configurable on `/admin/services`.
|
||||
|
||||
It is the **single point of truth for AI**: the news, bots, and Devii services call this
|
||||
gateway by default instead of an external provider, send the generic model name `molodetz`,
|
||||
and authenticate with an auto-generated internal key. The real provider URLs, models, and keys
|
||||
live only here, so providers and backends are switched in one place. `DEEPSEEK_API_KEY` and
|
||||
`OPENROUTER_API_KEY` are migrated into the editable settings on first boot, and the value in use
|
||||
is shown. The internal base URL the other services dial is `DEVPLACE_INTERNAL_BASE_URL`
|
||||
(default `http://localhost:10500`).
|
||||
|
||||
Configuration on the Services tab:
|
||||
|
||||
| Parameter | Default | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| `gateway_upstream_url` | `https://api.deepseek.com/chat/completions` | Where requests are forwarded |
|
||||
| `gateway_model` | `deepseek-chat` | Real model sent upstream (what `molodetz` maps to) |
|
||||
| `gateway_force_model` | on | Override the client-requested model (and the `molodetz` alias) |
|
||||
| `gateway_api_key` | migrated from env | Upstream key, shown and editable (auto-migrated from `DEEPSEEK_API_KEY`/`OPENROUTER_API_KEY`) |
|
||||
| `gateway_instances` | `4` | Max concurrent upstream forwards per worker (pool + semaphore) |
|
||||
| `gateway_timeout` | `180` | Upstream timeout (seconds) |
|
||||
| `gateway_vision_enabled` / `_url` / `_model` / `_key` / `_cache_size` | on / OpenRouter / Gemma / env / 256 | Image-description augmentation |
|
||||
| `gateway_require_auth` | on | When off, the gateway is open |
|
||||
| `gateway_allow_admins` / `gateway_allow_users` | on / off | Which DevPlace users may call it (any auth scheme) |
|
||||
| `gateway_access_key` | empty | A standalone key (sent as `X-API-KEY`/Bearer) that always grants access |
|
||||
| `gateway_internal_key` | auto (uuid4) | Auto-generated on boot; DevPlace's own services authenticate with this. Clear and restart to rotate |
|
||||
| `gateway_price_cache_hit_per_m` / `_cache_miss_per_m` / `_output_per_m` | 0.0028 / 0.14 / 0.28 | Chat cost per 1M tokens, used when the upstream returns no native cost (DeepSeek) |
|
||||
| `gateway_vision_price_input_per_m` / `_output_per_m` | 0 / 0 | Vision cost per 1M tokens, used only when the vision upstream returns no native cost |
|
||||
| `gateway_max_retries` / `gateway_retry_backoff_ms` | 2 / 250 | Retry attempts and linear backoff on timeout, connection error, or upstream 5xx |
|
||||
| `gateway_circuit_threshold` / `gateway_circuit_cooldown_seconds` | 5 / 30 | Consecutive failures before the circuit breaker opens, and its cooldown |
|
||||
| `gateway_usage_retention_hours` | 720 | How long per-call usage rows are kept before pruning (30 days) |
|
||||
| `gateway_model_context_map` | JSON map | Model to max-context-tokens map for context-window utilization tracking |
|
||||
|
||||
Callers authenticate with the static `gateway_access_key`, the auto-generated
|
||||
`gateway_internal_key` (used by the platform's own services), or - when allowed - their
|
||||
DevPlace credentials via any scheme (`X-API-KEY`, Bearer, Basic). The endpoint is
|
||||
exempt from the per-IP rate limit and the maintenance gate, and its concurrency is
|
||||
bounded by `gateway_instances`; scale further with uvicorn workers. The Services tab
|
||||
shows live request/error/latency/vision-call counters.
|
||||
|
||||
#### Usage and cost metrics
|
||||
|
||||
Every upstream call (chat, vision, and passthrough) is recorded to `gateway_usage_ledger`
|
||||
with its tokens, cost, latency breakdown, status, and caller. Cost is taken from the
|
||||
upstream native `cost` field when present (OpenRouter) and computed from the configured
|
||||
per-million pricing otherwise (DeepSeek, which reports no cost). The admin **AI usage**
|
||||
page (`/admin/ai-usage`) reports per-hour and 24h metrics - request volume and throughput,
|
||||
token usage with averages and percentiles (p50/p90/p95/p99), latency (upstream round-trip,
|
||||
gateway overhead, semaphore queue wait, connection establishment), error rates by category,
|
||||
cost (per model, per caller, input vs output, projected monthly burn, caching savings),
|
||||
caller behavior, and an hourly breakdown - divided and combined. The gateway also retries
|
||||
transient upstream failures and opens a circuit breaker after repeated failures. Time to
|
||||
first token and inter-token latency are not reported because the gateway forwards
|
||||
non-streaming to the upstream.
|
||||
|
||||
```bash
|
||||
curl -H "X-API-KEY: <gateway_access_key>" \
|
||||
-d '{"messages":[{"role":"user","content":"hi"}]}' \
|
||||
https://your-host/openai/v1/chat/completions
|
||||
```
|
||||
|
||||
### Devii assistant service
|
||||
|
||||
`DeviiService` (`devplacepy/services/devii/`) is an in-platform agentic assistant exposed
|
||||
as a WebSocket terminal at `/devii/ws`, reachable from the **Devii** item in the user
|
||||
dropdown menu and embedded on the documentation page for guests. It runs a ReAct loop
|
||||
(planning, reflection, a verification gate, self-learning memory, autonomous tasks) over a
|
||||
declarative tool catalog. A signed-in user's Devii operates their **own** DevPlace account -
|
||||
the agent authenticates to this instance with the user's `api_key` - while guests get a
|
||||
sandbox session. Disabled by default; enable and configure it on `/admin/services`.
|
||||
|
||||
Sessions are durable. A logged-in user's conversation is keyed by user uid, shared live
|
||||
across every tab, window, and device (each turn is broadcast to all connected sockets), and
|
||||
persisted to `devii_conversations` so it survives a server restart - on reconnect the prior
|
||||
conversation is rehydrated. Guests get a non-persistent session keyed by a `devii_guest`
|
||||
cookie. To keep one logical session and one cost/broadcast source per user, the `/devii/ws`
|
||||
endpoint is served only by the worker that holds the background-service lock; other workers
|
||||
close the socket so the client reconnects to the owning worker.
|
||||
|
||||
Spend is capped per owner over a rolling 24 hours. Every turn appends a row to
|
||||
`devii_usage_ledger` (the authoritative source for the cap - the in-memory cost tracker is
|
||||
display-only) and an audit row to `devii_turns`. The cap is checked before each turn.
|
||||
|
||||
Configuration on the Services tab:
|
||||
|
||||
| Parameter | Default | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| `devii_ai_url` | `https://openai.app.molodetz.nl/v1/chat/completions` | OpenAI-compatible reasoning endpoint |
|
||||
| `devii_ai_model` | `molodetz` | Model name |
|
||||
| `devii_ai_key` | env fallback (`DEVII_AI_KEY`) | AI API key |
|
||||
| `devii_base_url` | this instance's origin | Platform Devii drives via each user's API key |
|
||||
| `devii_plan_required` / `devii_verify_required` | on / on | Enforce plan-first and verify-after-mutation |
|
||||
| `devii_max_iterations` | `40` | Tool-loop iterations per turn |
|
||||
| `devii_user_daily_usd` | `1.0` | Rolling 24h spend cap per signed-in user |
|
||||
| `devii_guest_daily_usd` | `0.05` | Rolling 24h spend cap per guest |
|
||||
| `devii_guests_enabled` | on | Allow anonymous guests |
|
||||
| `devii_price_cache_hit` / `_cache_miss` / `_output` | `0.0028` / `0.14` / `0.28` | USD per 1M tokens (drives the cap) |
|
||||
| `devii_rsearch_enabled` | on | Enable the external web search tools (`rsearch_*`) |
|
||||
| `devii_rsearch_url` | `https://rsearch.app.molodetz.nl` | Base URL of the web search service those tools call |
|
||||
|
||||
Beyond the platform tools, Devii has external **web search** tools - `rsearch` (web/image
|
||||
search), `rsearch_answer` (a web-grounded AI answer with sources), `rsearch_chat` (direct AI
|
||||
chat), and `rsearch_describe_image` (vision). These reach an external public service rather
|
||||
than this platform, so platform tools are always preferred; Devii uses them only when the user
|
||||
explicitly asks to search the web or an outside source. They are gated by `devii_rsearch_enabled`
|
||||
and the service URL is configurable via `devii_rsearch_url`.
|
||||
|
||||
A `devii` console script ships the same agent as an interactive terminal:
|
||||
|
||||
```bash
|
||||
pip install -e .
|
||||
devii --api-key <your DevPlace api_key> --base-url https://your-host
|
||||
devii -p "List my unread notifications as a bullet list." # one-shot
|
||||
```
|
||||
|
||||
## Push notifications & PWA
|
||||
|
||||
Authenticated users can receive native web push notifications, and the site is an
|
||||
installable Progressive Web App. Push uses only standard libraries (`cryptography`,
|
||||
`PyJWT`, `httpx`) against the Web Push Protocol — no third-party push wrapper.
|
||||
`PyJWT`, `httpx`) against the Web Push Protocol - no third-party push wrapper.
|
||||
|
||||
### Events
|
||||
|
||||
Every event that already produces an in-app notification also sends a web push,
|
||||
because both share a single funnel — `create_notification()` in `utils.py`:
|
||||
because both share a single funnel - `create_notification()` in `utils.py`:
|
||||
|
||||
| Event | Recipient |
|
||||
|-------|-----------|
|
||||
@@ -249,6 +475,51 @@ Uses [Multiavatar](https://github.com/multiavatar/multiavatar-python) - generate
|
||||
|
||||
Avatar URL format: `/avatar/multiavatar/{username}?size={size}`
|
||||
|
||||
## Deployment
|
||||
|
||||
Production runs on a single host via Docker Compose: an **app** container (Uvicorn, 2 workers) behind an **nginx** container that terminates HTTP, serves static assets, and proxies the rest.
|
||||
|
||||
### Shared database and uploads
|
||||
|
||||
The app container bind-mounts the host project directory (`./` to `/app`) and runs as the host user (`DEVPLACE_UID`/`DEVPLACE_GID`, default `1000`). Because `config.py` resolves the database to an absolute path under the project root, the container reads and writes the **same `devplace.db`** as `make dev`, plus the same `static/uploads/`, `devii_lessons.db`, `devii_tasks.db`, VAPID keys, and `devplace-services.lock`. SQLite WAL mode allows the concurrent access, and the file lock (`fcntl.flock` on `devplace-services.lock`) guarantees only one process - dev or prod - runs the background services (news, bots, Devii hub), so they never double-fire.
|
||||
|
||||
This requires prod and dev to be on the **same host/filesystem**; SQLite is a local-file database and cannot be shared across machines. No `DEVPLACE_DATABASE_URL` override is set, so nothing diverges.
|
||||
|
||||
### First deploy
|
||||
|
||||
```bash
|
||||
cp .env.example .env # set SECRET_KEY; adjust PORT, DEVPLACE_SITE_URL
|
||||
make docker-up # docker compose up -d (builds on first run)
|
||||
```
|
||||
|
||||
Open `http://<host>:${PORT}` (default 10500). `make docker-logs` tails output; `make docker-down` stops.
|
||||
|
||||
### Updating
|
||||
|
||||
```bash
|
||||
git pull
|
||||
docker compose up -d # restart with new code (bind-mounted, no rebuild)
|
||||
docker compose build && \
|
||||
docker compose up -d # only when dependencies in pyproject.toml change
|
||||
```
|
||||
|
||||
The `make deploy` target fast-forwards the `production` branch (`git checkout production && git merge master && git push origin production`); pull that branch on the server.
|
||||
|
||||
### nginx specifics
|
||||
|
||||
- **WebSockets:** `/devii/ws` (the Devii terminal) is proxied with `Upgrade`/`Connection` headers and a 1h read timeout.
|
||||
- **Uploads:** `/static/uploads/` is served with `Content-Disposition: attachment` and `X-Content-Type-Options: nosniff`, matching the app's `UploadStaticFiles` control so user uploads download instead of rendering inline (stored-XSS defense).
|
||||
- **Upload size:** `NGINX_MAX_BODY_SIZE` (default `50m`) must be **>= the admin-configurable `max_upload_size_mb`** (Admin -> Settings), or large uploads are rejected with HTTP 413 before reaching the app.
|
||||
- **Micro-cache:** off by default; enable with `NGINX_CACHE_ENABLED=true`.
|
||||
|
||||
### Bare-metal alternative
|
||||
|
||||
`make prod` runs the same app without containers (`uvicorn ... --workers 2 --proxy-headers`) from the project root, sharing the identical database and files. Note it binds port 10500, so it conflicts with the Docker front door on the same port - run one, or set a different `PORT`.
|
||||
|
||||
### Multi-worker safety
|
||||
|
||||
The app runs correctly under multiple Uvicorn workers (independent processes sharing only the DB and lock files). Cross-worker invalidation of the auth and settings caches is coordinated through a `cache_state` version table (propagation bounded to ~2s); the rate limiter enforces `ceil(limit / DEVPLACE_WEB_WORKERS)` per process so the aggregate honours the configured value - **set `DEVPLACE_WEB_WORKERS` to match `--workers`** (done in `make prod` and the Dockerfile). First-boot DB init and VAPID key generation are `flock`-serialized. Full detail and the design rules are in the admin docs at `/docs/production-concurrency.html`.
|
||||
|
||||
## CI/CD
|
||||
|
||||
Gitea Actions workflow at `.gitea/workflows/test.yaml`:
|
||||
|
||||
Reference in New Issue
Block a user