feat: add project file system with CRUD, upload, inline editing, and video attachment support

- Add new `/projects/{slug}/files` endpoint group for per-project filesystem operations including directory and file CRUD, upload, and inline editing with public read and owner write access
- Extend attachment system to support video formats (webm, ogv, mov, m4v) with proper file icons and MIME types
- Implement configurable allowed file types via `allowed_file_types` site setting, replacing hardcoded `ALLOWED_UPLOAD_TYPES` with dynamic `allowed_extensions()` and `is_extension_allowed()` functions
- Add `delete_all_project_files()` call in `delete_content_item()` to clean up project files when a project is deleted
- Create database indexes on `project_files` table for `(project_uid, path)` and `(project_uid, parent_path)` to optimize file lookups
- Introduce `docs_prose.py` module with `render_prose()` function that renders Markdown content inside `data-render` divs using mistune, enabling dynamic prose rendering in documentation pages
- Enhance docs search with Markdown-aware text stripping (`_demarkdown()`) and improved HTML/script/style sanitization for better search indexing
- Update documentation API samples to reflect new attachment response fields (`is_image`, `is_video`, `mime_type`) and note video format support
- Update README to document the new project files endpoint and clarify AI gateway attribution for guest Devii sessions
This commit is contained in:
2026-06-08 20:51:09 +00:00
parent e0535bb7c5
commit 97eb58fc19
110 changed files with 5534 additions and 603 deletions
+23 -4
View File
@@ -57,6 +57,7 @@ devplacepy/
| `/posts` | Post detail, creation |
| `/comments` | Comment creation, deletion |
| `/projects` | Project listing, creation |
| `/projects/{slug}/files` | Per-project filesystem: directory and file CRUD, upload, inline editing (public read, owner write) |
| `/profile` | Profile view, editing |
| `/messages` | Direct messaging |
| `/notifications` | Notification list, mark read, live unread counts (`/notifications/counts`) |
@@ -266,7 +267,7 @@ request carries image content, first describes the image(s) via a vision model
(OpenRouter/Gemma by default) and inlines the description as text - so a vision-less
upstream can still answer. Enabled by default and configurable on `/admin/services`.
It is the **single point of truth for AI**: the news, bots, and Devii services call this
It is the **single point of truth for AI**: the news, bots, and guest Devii sessions call this
gateway by default instead of an external provider, send the generic model name `molodetz`,
and authenticate with an auto-generated internal key. The real provider URLs, models, and keys
live only here, so providers and backends are switched in one place. `DEEPSEEK_API_KEY` and
@@ -274,16 +275,29 @@ live only here, so providers and backends are switched in one place. `DEEPSEEK_A
is shown. The internal base URL the other services dial is `DEVPLACE_INTERNAL_BASE_URL`
(default `http://localhost:10500`).
Authenticated users may call the gateway with their own API key (the `Allow users` setting,
on by default), and Devii operating a signed-in user authenticates its LLM calls with that
user's key. Every call is therefore attributed to the user, so their full AI spend (through
Devii or direct API calls) is tracked and limitable per user. Administrators see a per-user
"AI usage (last 24h)" panel on each profile page - request volume, success and error rates,
token totals, cost, average latency and throughput, a per-model breakdown, a 30-day cost
projection, and a quota bar - served from `GET /admin/users/{uid}/ai-usage`. A non-admin user
sees only one figure on their own profile: the percentage of their daily AI quota used so far.
The same rule governs the Devii assistant: monetary figures (cost, pricing, spend, limit) are
disclosed only to administrators, while members and guests can see only the percentage of their
24-hour quota used. The `/devii/usage` endpoint returns that percentage and the day's turn count to
everyone, and includes dollar figures only for administrators.
Configuration on the Services tab:
| Parameter | Default | Purpose |
|-----------|---------|---------|
| `gateway_upstream_url` | `https://api.deepseek.com/chat/completions` | Where requests are forwarded |
| `gateway_model` | `deepseek-chat` | Real model sent upstream (what `molodetz` maps to) |
| `gateway_model` | `deepseek-v4-flash` | Real model sent upstream (what `molodetz` maps to); 1M-token context, 384K max output |
| `gateway_force_model` | on | Override the client-requested model (and the `molodetz` alias) |
| `gateway_api_key` | migrated from env | Upstream key, shown and editable (auto-migrated from `DEEPSEEK_API_KEY`/`OPENROUTER_API_KEY`) |
| `gateway_instances` | `4` | Max concurrent upstream forwards per worker (pool + semaphore) |
| `gateway_timeout` | `180` | Upstream timeout (seconds) |
| `gateway_timeout` | `300` | Upstream timeout (seconds); minimum five minutes; also bounds the vision describe-image call |
| `gateway_vision_enabled` / `_url` / `_model` / `_key` / `_cache_size` | on / OpenRouter / Gemma / env / 256 | Image-description augmentation |
| `gateway_require_auth` | on | When off, the gateway is open |
| `gateway_allow_admins` / `gateway_allow_users` | on / off | Which DevPlace users may call it (any auth scheme) |
@@ -356,12 +370,15 @@ Configuration on the Services tab:
| `devii_base_url` | this instance's origin | Platform Devii drives via each user's API key |
| `devii_plan_required` / `devii_verify_required` | on / on | Enforce plan-first and verify-after-mutation |
| `devii_max_iterations` | `40` | Tool-loop iterations per turn |
| `devii_timeout` | `300` | Read timeout (seconds) for each AI chat-completions call; minimum five minutes |
| `devii_fetch_timeout` | `300` | Read timeout (seconds) for the fetch tool; minimum five minutes |
| `devii_user_daily_usd` | `1.0` | Rolling 24h spend cap per signed-in user |
| `devii_guest_daily_usd` | `0.05` | Rolling 24h spend cap per guest |
| `devii_guests_enabled` | on | Allow anonymous guests |
| `devii_price_cache_hit` / `_cache_miss` / `_output` | `0.0028` / `0.14` / `0.28` | USD per 1M tokens (drives the cap) |
| `devii_rsearch_enabled` | on | Enable the external web search tools (`rsearch_*`) |
| `devii_rsearch_url` | `https://rsearch.app.molodetz.nl` | Base URL of the web search service those tools call |
| `devii_rsearch_timeout` | `300` | Read timeout (seconds) for `rsearch_*` calls; web-grounded answers can take minutes; minimum five minutes |
Beyond the platform tools, Devii has external **web search** tools - `rsearch` (web/image
search), `rsearch_answer` (a web-grounded AI answer with sources), `rsearch_chat` (direct AI
@@ -453,6 +470,8 @@ PRAGMA mmap_size=268435456; -- 256MB memory map for reads
All indexes are created via `CREATE INDEX IF NOT EXISTS` wrapped in try/except - safe to run on every startup regardless of table state.
SQLite is synchronous by design and will never be made async. It is more than fast enough for this platform: the database is a local file with WAL, a 30s busy timeout, and a 256MB memory map, so queries are sub-millisecond. The synchronous database layer is a deliberate architectural decision, not a limitation, and is not subject to change.
## Testing
- **419 tests** across 34 files: Playwright integration + in-process unit tests
@@ -508,7 +527,7 @@ The `make deploy` target fast-forwards the `production` branch (`git checkout pr
### nginx specifics
- **WebSockets:** `/devii/ws` (the Devii terminal) is proxied with `Upgrade`/`Connection` headers and a 1h read timeout.
- **Uploads:** `/static/uploads/` is served with `Content-Disposition: attachment` and `X-Content-Type-Options: nosniff`, matching the app's `UploadStaticFiles` control so user uploads download instead of rendering inline (stored-XSS defense).
- **Uploads:** `/static/uploads/` is served with `X-Content-Type-Options: nosniff` and a `Content-Disposition` that mirrors the app's `UploadStaticFiles` control: known-safe media (images, video, audio) is served `inline` so it embeds and plays in the page, while every other type is forced to `attachment` so it downloads instead of rendering (stored-XSS defense). SVG is deliberately excluded from the inline set.
- **Upload size:** `NGINX_MAX_BODY_SIZE` (default `50m`) must be **>= the admin-configurable `max_upload_size_mb`** (Admin -> Settings), or large uploads are rejected with HTTP 413 before reaching the app.
- **Micro-cache:** off by default; enable with `NGINX_CACHE_ENABLED=true`.