forked from retoor/devplacepy
Compare commits
26
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
  retoorandClaude Sonnet 5.5
|
6a5ed75072 | ||
|
retoorandClaude Sonnet 5.5
|
585fab74a6 | ||
|
retoorandClaude Sonnet 5.5
|
2452f9e85b | ||
|
retoorandClaude Sonnet 5.5
|
d815fbe9f8 | ||
|
retoorandClaude Sonnet 5.5
|
376c6ba99d | ||
|
retoorandClaude Sonnet 5.5
|
8b45a515f2 | ||
|
retoorandClaude Sonnet 5.5
|
695be8ba9a | ||
|
retoorandClaude Sonnet 5.5
|
1a9201cb91 | ||
|
retoorandClaude Sonnet 5.5
|
8d4d06de61 | ||
|
retoorandClaude Sonnet 5.5
|
8e36807926 | ||
|
retoorandClaude Sonnet 5.5
|
151476125f | ||
|
retoorandClaude Sonnet 5.5
|
3f6e83c753 | ||
|
retoor
|
8a48e3ef85 | ||
|
retoor
|
dbdd0d29dd | ||
|
retoor
|
33eb090e27 | ||
|
retoor
|
078d0657ff | ||
 blindxfishandClaude Opus 5.5
|
17da9cb30c | ||
|
retoorandClaude Sonnet 5.5
|
920a679dcf | ||
|
retoorandClaude Sonnet 5.5
|
05d6633204 | ||
|
retoorandClaude Sonnet 5.5
|
bc11475f29 | ||
|
retoorandClaude Sonnet 5.5
|
ec30ed408c | ||
|
retoorandClaude Sonnet 5.5
|
f705099928 | ||
|
retoor
|
762c06886c | ||
|
retoor
|
3bfde87fdb | ||
|
retoor
|
407673bf57 | ||
|
retoor
|
52a4ebbe58 |
@@ -1,3 +1,4 @@
|
||||
.*
|
||||
.docs
|
||||
.dpc
|
||||
.devplace
|
||||
|
||||
@@ -2,19 +2,22 @@
|
||||
|
||||
This file provides guidance to Claude Code when working with code in this repository. It holds only what applies regardless of which part of the codebase is being touched. Deep, subsystem-specific detail lives in nested `CLAUDE.md` files placed inside the relevant directory - Claude Code auto-loads a nested file only when a file under that directory is read or edited, so the always-loaded cost of this repository stays proportional to this file alone. See "Subsystem map" below for the full list.
|
||||
|
||||
It is a big project, whatever you are implementing, it is probably done before. You should look it up and match the implementation structurely and visually. For inconsistency there is zero tolerance policy. Develop dry, kiss, re-usable code, consistent with existing implementation. Literally always try to find relatable examples before making a modification. If no-example exists, explain to user what is the case and let user decide what to do and how to continue.
|
||||
It is a big project, whatever you are implementing, it is probably done before. You should look it up and match the implementation structurely and visually. For inconsistency there is zero tolerance policy. Develop dry, kiss, re-usable code, consistent with existing implementation. Literally always try to find relatable examples before making a modification. If no-example exists, explain to user what is the case and let user decide what to do and how to continue.
|
||||
|
||||
The completion contract for every change is [DEFINITION_OF_DONE.md](./DEFINITION_OF_DONE.md): universal gates, test obligations, the four-layer rigor procedure for spendable resources and races, per-subsystem rules, and the final checklist. A change is done only when every applicable gate in it passes.
|
||||
|
||||
|
||||
## Project
|
||||
|
||||
DevPlace is a server-rendered social network for developers. FastAPI backend serves Jinja2 templates with pure ES6 module JavaScript on the frontend. SQLite via the `dataset` library (auto-syncs schema). No JS framework, no NPM, no JWT.
|
||||
|
||||
- **Database:** `dataset` (auto-syncs schema, uses `uid` for PKs). SQLite.
|
||||
- **Auth:** `session` cookie, plus `X-API-KEY` / `Authorization: Bearer <api_key>` / HTTP Basic (username-or-email:password) - all resolved in `get_current_user`. PBKDF2-SHA256 via passlib. No JWT. Every user has an `api_key` (uuid7), set at signup and backfilled in `init_db`/`devplace apikey backfill`. Docs site at `/docs` (`routers/docs/` package, `DOCS_PAGES` registry; FastAPI's Swagger is moved to `/swagger` so `/docs` is free).
|
||||
- **Auth:** three account credentials: username+password, email+password, personal `api_key`. All password exchanges (`POST /auth/login` form+JSON, `POST /auth/token`, HTTP Basic, devRant `POST /api/users/auth-token`) take username-or-email as identifier (username case-sensitive, email lowered); the `api_key` rides `X-API-KEY` / `Authorization: Bearer` (client-side `DEVPLACE_API_KEY` env var holds its value, injected into containers) plus `session` cookie - all resolved in `get_current_user` (order: session, X-API-KEY, Bearer, Basic; headers also try DevRant/access tokens). Every JSON return of the personal key also returns `username`+`email` (login/signup `data`, gated `ProfileOut.email` beside `api_key`, regenerate), so `GET /profile` with an API key is the whoami/account fetch; never add `email` to public `UserOut`. PBKDF2-SHA256 via passlib. No JWT. Every user has an `api_key` (uuid7), set at signup and backfilled in `init_db`/`devplace apikey backfill`. Docs site at `/docs` (`routers/docs/` package, `DOCS_PAGES` registry; FastAPI's Swagger is moved to `/swagger` so `/docs` is free).
|
||||
- **Static:** `devplacepy/static/` mounted at `/static`; URLs are boot-versioned (`/static/v<ts>/...`) via `static_url`/`assetUrl` and served immutable for a year.
|
||||
- **Templates:** `devplacepy/templates/`. Shared `templates` instance from `devplacepy.templating` - all routers import from there, never instantiate their own.
|
||||
- **Ports:** 10500 (dev), 10501 (tests; the serial suite uses a single uvicorn subprocess).
|
||||
- **Username:** letters, numbers, hyphens, underscores only, 3-32 chars. **Password:** minimum 6 chars.
|
||||
- **Avatars:** Multiavatar SVG generated locally from a seed in <5ms, in-memory cache cleared on restart, fallback to initial-based SVG on error. URL `/avatar/multiavatar/{seed}?size={size}`. The seed is per-user: the nullable `users.avatar_seed` column overrides the username. Always resolve it through the single null-safe choke point `avatar.avatar_seed(user)` (a Jinja global) - `user.get("avatar_seed") or user.get("username")` - never read `avatar_seed` or pass `username` to `avatar_url(...)` directly; every render site (the `_avatar_link.html` partial, `og_image`, the devRant avatar payload, etc.) goes through it. Regenerate is owner-or-admin at `POST /profile/{username}/regenerate-avatar` (writes a fresh `generate_uid()`, invalidates the user cache, audits `profile.avatar.regenerate`); the old seed is never stored, so the old avatar cannot return. Devii tool `regenerate_avatar` (`CONFIRM_REQUIRED`).
|
||||
- **Avatars:** Multiavatar SVG generated locally at `/avatar/multiavatar/{seed}?size={size}`. Seed resolution, regenerate flow, and the owner-or-admin rule live in `devplacepy/templates/CLAUDE.md` ("Avatar seed choke point").
|
||||
|
||||
## Commands
|
||||
|
||||
@@ -55,59 +58,12 @@ devplace token list <username> # list a user's active access tokens
|
||||
devplace token revoke <token_uid> # revoke a single access token by uid
|
||||
devplace token revoke-all <username> # revoke all access tokens for a user
|
||||
devplace token prune # soft-delete all expired access tokens
|
||||
devplace news clear # delete all news rows
|
||||
devplace news sanitize # strip HTML from news descriptions/content
|
||||
devplace attachments prune # remove orphan attachment records/files
|
||||
devplace system prune [--dry-run] # safe but aggressive: purges soft-deleted attachment/project-file blobs, sweeps blob files with zero DB reference at all (e.g. left by an interrupted/racing sync), and GCs orphaned container workspace dirs
|
||||
devplace devii reset-quota <username> # reset one user's rolling 24h AI quota
|
||||
devplace devii reset-quota --guests # reset every guest quota
|
||||
devplace devii reset-quota --all # reset every quota (users and guests)
|
||||
devplace devii tasks list [--all] # list scheduled Devii tasks and their owners
|
||||
devplace devii tasks disable <uid> # disable one scheduled task
|
||||
devplace devii tasks prune # disable every task whose owner may not schedule
|
||||
devplace gateway quota list # list AI gateway quota rules and current 24h spend
|
||||
devplace gateway quota set --limit-usd N [--owner-kind K] [--owner-id ID] [--app-reference APP] [--label L] [--uid UID]
|
||||
devplace gateway quota delete <uid> # delete a quota rule
|
||||
devplace gateway quota reset [--owner-kind K] [--owner-id ID] [--app-reference APP] # clear the counted 24h spend (keeps the usage history)
|
||||
devplace zips prune # delete expired zip archives + job rows
|
||||
devplace zips clear # delete every zip archive + job row
|
||||
devplace forks prune # delete expired completed fork job rows (forked projects persist)
|
||||
devplace forks clear # delete every fork job row (forked projects persist)
|
||||
devplace community discover <query> [--run] [--relearn] [--dry-run] # discover, learn and generate collectors for an external community
|
||||
devplace community run <slug> [--dry-run] # run a community's collectors through the sync API
|
||||
devplace community login <slug> [--login-user U] [--login-email E] [--login-password P | --clear] # store or remove the account used to log in
|
||||
devplace community list # list communities
|
||||
devplace messaging prune-tickets # delete expired WebSocket auth tickets (ws_tickets)
|
||||
devplace seo prune # delete expired SEO audit reports + job rows
|
||||
devplace seo clear # delete every SEO audit report + job row
|
||||
devplace seo-meta prune # delete expired SEO metadata job rows (generated metadata persists)
|
||||
devplace seo-meta clear # delete every SEO metadata job row (generated metadata persists)
|
||||
devplace deepsearch prune # delete expired DeepSearch sessions + job rows + collections
|
||||
devplace deepsearch clear # delete every DeepSearch session + job row + collection
|
||||
devplace isslop analyze <url> # run a AI usage analysis from the terminal (report persists)
|
||||
devplace isslop prune # delete expired AI usage analysis job rows (analyses + reports persist)
|
||||
devplace isslop clear # delete every AI usage analysis, its report and job rows
|
||||
devplace quiz prune # delete abandoned/expired quiz attempts older than the retention window
|
||||
devplace game market prune # delete Code Farm market tick buckets older than the tracking window
|
||||
devplace game steals prune # delete Code Farm raid records older than the raid-efficiency window
|
||||
devplace game era status # show the current Code Farm Era
|
||||
devplace game era start <name> [--days N] # start a Code Farm Era (default 28 days)
|
||||
devplace game era end # end the running Code Farm Era (ranks, awards Stars, records results)
|
||||
devplace accounts pending # list deleted accounts awaiting their purge
|
||||
devplace accounts prune # permanently purge accounts past the deletion grace window (--dry-run to preview)
|
||||
devplace backups list # list recorded backups
|
||||
devplace backups run <database|uploads|keys|full> # enqueue a backup (processed by the running server)
|
||||
devplace backups prune # remove backup records whose archive file is missing
|
||||
devplace backups clear # delete every backup archive + record
|
||||
devplace containers list # list container instances
|
||||
devplace containers reconcile # run one reconcile pass (desired vs docker ps)
|
||||
devplace containers prune # reap orphan containers + dangling images
|
||||
devplace containers prune-builds # remove legacy per-project images + clear dockerfiles/builds tables (one-time)
|
||||
devplace containers gc-workspaces # remove workspace dirs with no instances
|
||||
devplace emoji-sync # regenerate static/js/emoji-shortcodes.js from the emoji library (run after an emoji dep bump)
|
||||
devplace migrate-data # relocate legacy runtime files into data/ (idempotent; --dry-run to preview)
|
||||
```
|
||||
|
||||
Per-subsystem commands live in the nested file: `devplace devii ...` (Devii), `devplace gateway quota ...` (AI gateway), `devplace backups ...` (backup service), `devplace containers ...` (container manager), `devplace news ...` (news service), `devplace community ...` (community discovery), `devplace messaging prune-tickets` (messaging), `devplace {zips|forks|seo|seo-meta|deepsearch|isslop} ...` (async jobs), `devplace quiz prune` (quizzes), `devplace game ...` (Code Farm), `devplace accounts ...` (moderation), `devplace apikey backfill`, `devplace attachments prune`, `devplace system prune` (database), `devplace emoji-sync` (frontend JS), `devplace embeds-sync` (templates).devplace migrate-data # relocate legacy runtime files into data/ (idempotent; --dry-run to preview)
|
||||
```
|
||||
|
||||
### Runtime data layout (single source of truth)
|
||||
|
||||
Every runtime/user-generated artifact lives under one root, `config.DATA_DIR` (`DEVPLACE_DATA_DIR`, default `<repo>/data`). `config.py` derives every runtime path from it and lists them in the `DATA_PATHS` registry; `ensure_data_dirs()` creates the whole tree before anything is written. **No module computes a runtime path from scratch** - import the constants (`UPLOADS_DIR`, `ATTACHMENTS_DIR`, `PROJECT_FILES_DIR`, `ZIPS_DIR`, `ZIP_STAGING_DIR`, `FORK_STAGING_DIR`, `KEYS_DIR`, `BOT_DIR`, `LOCKS_DIR`, `CONTAINER_WORKSPACES_DIR`, `DEVII_TASKS_DB`, `DEVII_LESSONS_DB`, `VAPID_*_FILE`, `SERVICE_LOCK_FILE`, `INIT_LOCK_FILE`). Uploads are physically under `data/uploads/` but still served at the unchanged `/static/uploads/` URL. Stored DB values for attachments/project files are relative (`directory` + `stored_name`), so no DB rewrite is ever needed for a layout change.
|
||||
@@ -141,6 +97,7 @@ Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file i
|
||||
| Path | Covers |
|
||||
|------|--------|
|
||||
| `devplacepy/routers/CLAUDE.md` | Full URL prefix map, route aggregation rules, HTML/JSON negotiation, FastAPI patterns, gists, feed/listing features, engagement (reactions/bookmarks/polls/heatmap/follow/block-mute), SEO implementation, polymorphic comments/votes |
|
||||
| `devplacepy/routers/admin/CLAUDE.md` | The admin page specification (A1-A11): rendering, response contract, page anatomy, class vocabulary, time, caps, remote work, live refresh, assets, the nav registry, responsiveness |
|
||||
| `devplacepy/routers/projects/CLAUDE.md` | Project detail page, virtual filesystem routes, visibility/read-only UI, deletion confirmation |
|
||||
| `devplacepy/routers/docs/CLAUDE.md` | The `/docs` documentation site: `DOCS_PAGES`, audience tiers, prose rendering, API tester |
|
||||
| `devplacepy/routers/devrant/CLAUDE.md` | devRant-compatible REST API (`/api`) |
|
||||
@@ -165,6 +122,7 @@ Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file i
|
||||
| `devplacepy/services/game/CLAUDE.md` | Code Farm idle game (economy invariants, raids, the one-pure-function rule) |
|
||||
| `devplacepy/services/opinionwar/CLAUDE.md` | Opinion Wars (week-long faction battles on posts: atomic fight/resolve transitions, cooldown-before-coins compensation, event seq allocation, relay-on-lock-owner) |
|
||||
| `devplacepy/services/quiz/CLAUDE.md` | Quizzes (the terminal publish lock, attempt atomicity, answer-key withholding, AI free-text grading, the best-attempt scoreboard) |
|
||||
| `devplacepy/services/giphy/CLAUDE.md` | GIF picker: GIPHY search through the server, admin-held API key, GIFs stored as ordinary attachments, cache and per-user caps |
|
||||
| `devplacepy/services/skills/CLAUDE.md` | Agent Skills: strict `SKILL.md` parser, SSRF-guarded fetcher, the bounded prompt index, per-user privacy, soft delete and account-deletion cascade, why a skill is data and never code |
|
||||
| `devplacepy/services/nodes/CLAUDE.md` | Community node network: the protocol specification (single source of truth), the no-open-proxy rule, the two transports and their trust levels, the decisions taken, and the gaps per phase |
|
||||
| `devplacepy/services/community/CLAUDE.md` | Community discovery: rsearch + feed detection, one-time structure learning, generated collectors, dry run, the failure floor, admin-only visibility |
|
||||
@@ -189,72 +147,23 @@ The AI usage analyzer (`devplace isslop analyze`, the `/tools/isslop` job servic
|
||||
|
||||
### Routing layout
|
||||
|
||||
Routers in `devplacepy/routers/` are organised as a **directory tree that mirrors the endpoint (URL) path**, exactly like `tests/`. A domain with a single resource stays one flat file (`feed.py`, `posts.py`, ...); a domain with several sub-resources is a **package directory** split one file per sub-resource. Each leaf declares its own `router = APIRouter()`; the package `__init__.py` aggregates with `router.include_router(...)`. A leaf owning the domain's collection-root (`""`) route must be the package's base router (FastAPI rejects an empty path under an empty prefix). Full elaboration, per-domain detail, and the complete deep-dive live in `devplacepy/routers/CLAUDE.md` - read it before touching routing. Compact map:
|
||||
|
||||
| Prefix | Router |
|
||||
|--------|--------|
|
||||
| `/auth` | auth/ package |
|
||||
| `/feed`, `/posts`, `/comments` | flat files |
|
||||
| `/topics` | topics.py - crawlable per-topic category index pages (`/topics` hub, `/topics/{topic}` listing) |
|
||||
| `/projects`, `/projects/{slug}/files`, `/projects/{slug}/containers` | projects/ package - see `routers/projects/CLAUDE.md` |
|
||||
| `/profile` | profile/ package (customization, notifications, ai-correction, ai-modifier, interactions, telegram, usage) |
|
||||
| `/messages` | messages.py - see `services/messaging/CLAUDE.md` |
|
||||
| `/notifications`, `/votes`, `/reactions`, `/bookmarks`, `/polls`, `/avatar`, `/follow`, `/leaderboard` | flat files |
|
||||
| (none) | relations.py - block/mute |
|
||||
| `/` | landing.py - home route (guests get the splash, users the dashboard) |
|
||||
| `/admin`, `/admin/services`, `/admin/containers` | admin/ package |
|
||||
| `/issues` | issues/ package - see `services/gitea/CLAUDE.md` |
|
||||
| `/gists`, `/news`, `/uploads`, `/media` | flat files |
|
||||
| `/openai` | openai_gateway.py - see `services/openai_gateway/CLAUDE.md` |
|
||||
| `/devii` | devii.py - see `services/devii/CLAUDE.md` |
|
||||
| `/zips`, `/forks` | see `services/jobs/CLAUDE.md` |
|
||||
| `/tools` | tools/ package (SEO diagnostics, DeepSearch, AI Usage Analyzer) - see `services/jobs/CLAUDE.md` |
|
||||
| `/p/{slug}` | proxy.py - container ingress reverse proxy |
|
||||
| `/xmlrpc` | xmlrpc.py - see `services/xmlrpc/CLAUDE.md` |
|
||||
| `/api` | devrant/ package - see `routers/devrant/CLAUDE.md` |
|
||||
| `/dbapi` | dbapi/ package, **primary-administrator-only, strictly READ-ONLY** - see `services/dbapi/CLAUDE.md` |
|
||||
| `/sync` | sync/ package, **administrator only**, off by default - see `services/sync/CLAUDE.md` |
|
||||
| `/game` | game/ package - see `services/game/CLAUDE.md` |
|
||||
| `/reports`, `/admin/moderation`, `/workspaces` | reports.py, admin/moderation.py, workspaces.py - see `services/moderation/CLAUDE.md` |
|
||||
| `/quizzes` | quizzes/ package - see `services/quiz/CLAUDE.md` |
|
||||
| `/battles` | battles.py - see `services/opinionwar/CLAUDE.md` |
|
||||
| (none) | push.py (web push/PWA), docs.py (see `routers/docs/CLAUDE.md`) |
|
||||
| `/pubsub` | pubsub.py - see `services/pubsub/CLAUDE.md` |
|
||||
| (none) | seo.py - `/robots.txt`, `/sitemap.xml` |
|
||||
Routers in `devplacepy/routers/` are organised as a **directory tree that mirrors the endpoint (URL) path**, exactly like `tests/` (one segment per directory, last segment is the file; packages aggregate with `include_router`). A leaf owning the domain's collection-root (`""`) route must be the package's base router. Full URL prefix map, aggregation rules, HTML/JSON negotiation, FastAPI patterns, and per-domain detail live in `devplacepy/routers/CLAUDE.md` - read it before touching routing. Sub-domain pages: `routers/projects/`, `routers/docs/`, `routers/devrant/` (each with its own nested file, see Subsystem map).
|
||||
|
||||
### Templates and frontend
|
||||
|
||||
- All routers MUST import the shared `templates` from `devplacepy.templating`. Do NOT instantiate `Jinja2Templates` per router.
|
||||
- Templates extend `base.html`; page CSS via `{% block extra_head %}`, page JS via `{% block extra_js %}`.
|
||||
- **Nav active state is the `nav_active` Jinja global**, never an inline path check.
|
||||
- **Static asset URLs are boot-versioned:** never hardcode a bare `/static/...` href/src - use `static_url(path)` (Jinja) / `assetUrl(path)` (JS).
|
||||
- All JS is ES6 modules, one class per file, instantiated as `app`. Custom web components (`dp-*` prefix) and shared frontend utilities (`Http`, `Poller`, `JobPoller`, `OptimisticAction`, `FloatingWindow`, `ScrollMemory`) are documented in `devplacepy/static/js/CLAUDE.md` - reuse them, never re-implement.
|
||||
- CDN scripts in `base.html` MUST use `defer` or `type="module"`.
|
||||
- Modal system, shared template partials (`_avatar_link.html`, `_user_link.html`, `_sidebar_search.html`) are documented in `devplacepy/templates/CLAUDE.md`.
|
||||
All routers import the shared `templates` from `devplacepy.templating` (never instantiate per router); templates extend `base.html` (`extra_head`/`extra_js` blocks, `nav_active` global, boot-versioned `static_url`/`assetUrl`, deferred CDN scripts). JS is ES6 modules, one class per file, `dp-*` web components plus shared utilities (see `devplacepy/static/js/CLAUDE.md`). Modal system and shared partials: see `devplacepy/templates/CLAUDE.md`.
|
||||
|
||||
### Content rendering pipeline
|
||||
|
||||
**Server-rendered content and titles are rendered on the BACKEND for SEO** (`devplacepy/rendering.py`, Jinja globals `render_content(text)`/`render_title(text)`). This is the default for all content that exists at request time (post bodies, comments, news, project/gist descriptions, DM history, every content title). It mirrors the client pipeline using **mistune**: em-dash normalization, emoji shortcodes, GFM markdown (`escape=True` - the server XSS control), then a media pass turning bare URLs into embeds and `@mentions` into links. Both are `@lru_cache`d. Server-rendered code blocks get syntax highlighting + Copy button client-side via `ContentEnhancer`.
|
||||
|
||||
The CLIENT pipeline (`ContentRenderer.js`, `dp-content`/`dp-title`) is retained ONLY for genuinely live/dynamic content that does not exist at request time (live comments, DM bubbles, Devii/DeepSearch/Docs chat, planning report). Runs `marked` -> **`DOMPurify.sanitize`** (the client XSS control, fail-closed) -> highlight.js -> media/autolink. **Do NOT add `data-render` to server-rendered content - call `render_content`/`render_title` instead.**
|
||||
|
||||
**Emoji shortcodes are the full GitHub/Discord `:name:` set** (~4869 names), generated once from the `emoji` library by `rendering.py` `build_emoji_shortcodes()`; the frontend gets the identical map via the generated `static/js/emoji-shortcodes.js` (regenerate with `devplace emoji-sync` after bumping the dependency, never hand-edit).
|
||||
|
||||
**Email anonymization:** both `_render_content` and `_render_title` mask email addresses to prevent doxing. `_mask_emails` in `rendering.py` runs on rendered text nodes only (inside `_transform_text`, `_MediaProcessor`, and `_InlineFilter` - never before mistune, or the `*` mask characters would be parsed as emphasis) and stars ~80% of the local part (keeps a leading 20%, minimum one visible char). Addresses on `molodetz.nl` (and its subdomains) are exempt and render verbatim.
|
||||
|
||||
**Em-dash normalization:** `_normalize_dashes` in `rendering.py` replaces all forms (em dash `\u2014`, en dash `\u2013`, and their HTML entities) with a hyphen BEFORE mistune processes the text. Runs inside `_render_content`/`_render_title` which are `@lru_cache`d, so each unique text is normalized once. No per-template overhead.
|
||||
Server content renders on the backend for SEO (`render_content`/`render_title` in `devplacepy/rendering.py`); the client pipeline is only for live content (`DOMPurify` fail-closed). Full pipeline rules (emoji map, YouTube spec, email masking, dash normalization) live in `devplacepy/templates/CLAUDE.md` ("Content rendering pipeline").
|
||||
|
||||
### Auth
|
||||
|
||||
Session cookies named `session`, value a 64-char hex token. Passwords hashed with `pbkdf2_sha256` via passlib. `get_current_user(request)` returns user dict or `None`, per-process TTL cache (300s) keyed by token. `require_user(request)` raises a 303 redirect to `/` for guests; `require_admin(request)` redirects to `/feed` for non-admins. Public pages use `get_current_user` so guests can browse - POSTs are all guarded by `require_user`.
|
||||
Session cookies named `session`, value a 64-char hex token. Passwords hashed with `pbkdf2_sha256` via passlib. `get_current_user(request)` returns user dict or `None`, per-process TTL cache (300s) keyed by token. `require_user(request)` raises a 303 redirect to `/` for guests; `require_admin(request)` redirects to `/feed` for non-admins. Public pages use `get_current_user` so guests can browse - POSTs are all guarded by `require_user`. Password exchanges (`routers/auth/login.py`, `routers/auth/token.py`) reuse the `LoginForm.email` field as username-or-email: look up `email=identifier.lower()` first, then `username=identifier` un-lowered (mirroring `_user_from_basic` semantics); never lower the identifier before the username lookup.
|
||||
|
||||
### Database
|
||||
|
||||
SQLite via `dataset.connect` with WAL + 30s busy timeout + 256MB mmap. **SQLite is synchronous by design and will never be made async (hard rule).** The `dataset`/SQLite layer is called directly inside async route handlers - the database is a local file tuned with WAL/`synchronous=NORMAL`/8MB page cache/256MB mmap, so queries are sub-millisecond and never meaningfully stall the event loop. Do NOT convert the database layer to async, wrap calls in a threadpool/`run_in_executor`/`to_thread`, or raise synchronous DB access as a blocking-I/O concern - this is a settled design decision, not open for revisiting.
|
||||
|
||||
`init_db()` is idempotent and **ensures the full column set of any table code filters on** before creating indexes - `dataset` gives a lazily-created table only the columns of its first insert, so a partial insert elsewhere would leave a reduced schema that later 500s with `no such column`. Full dataset rules, indexing conventions (the soft-delete planner trap), site settings, and the complete table list live in `devplacepy/database/CLAUDE.md`.
|
||||
|
||||
Runtime config lives in `site_settings`, read via `get_setting(key, default)`/`get_int_setting(key, default)` (60s TTL cache, cross-worker invalidated via `cache_state`). Batch helpers (`get_users_by_uids`, `get_comment_counts_by_post_uids`, `get_vote_counts`, `load_comments`) exist specifically to avoid N+1 - use them instead of per-row lookups.
|
||||
**SQLite is synchronous by design and will never be made async (hard rule).** The `dataset`/SQLite layer is called directly inside async route handlers; never wrap it in a threadpool/`run_in_executor`/`to_thread` or raise sync DB access as a blocking-I/O concern. Dataset rules, indexing, soft delete, visibility spec, `site_settings`, batch helpers (use instead of N+1), and the table list live in `devplacepy/database/CLAUDE.md`.
|
||||
|
||||
### Per-user site customization (CSS/JS)
|
||||
|
||||
@@ -269,19 +178,19 @@ Users and guests inject their own CSS and JS, scoped to a page type or globally,
|
||||
|
||||
### Background task queue, AI correction/modifier, background services
|
||||
|
||||
`services/background.py` `background.submit(fn, *args)` is a generic fire-and-forget offload onto one per-worker `asyncio.Queue`; when the consumer isn't running (tests, full queue) it runs the callable **inline**, so audit/notification/XP writes stay deterministic for the suite while production defers them. It is the choke point for every audit-log write, every XP award, and every notification - handlers call `award_rewards`/`create_notification` directly (never wrap them in `background.submit`, that double-queues). AI content correction (opt-in, off by default) and the AI modifier (`@ai <instruction>` inline directive, on by default) rewrite user prose via the internal gateway; sync apply mode never blocks the event loop (`run_in_executor` + the `await_pending_corrections` middleware). `BaseService`/`ServiceManager` provide the async run loop and singleton registry for background services (`NewsService`, `GatewayService`, `DeviiService`, `BotsService`, container/audit/telegram reconcilers). Full detail on all of this, plus presence and the live view relay, is in `devplacepy/services/CLAUDE.md`.
|
||||
`background.submit` offload, AI correction/modifier via the internal gateway, `BaseService`/`ServiceManager`, presence, and the live view relay: see `devplacepy/services/CLAUDE.md`.
|
||||
|
||||
### Container manager, Devii assistant, AI gateway, async jobs, audit log
|
||||
|
||||
DevPlace ships a Docker-backed **Container manager** (admin-only, one shared `ppy:latest` image, security-hardened rootless workflow via a `sudo` superclone and `aptroot`), the **Devii** in-platform agentic assistant (WebSocket terminal, per-owner-channel sessions, scheduler/reminders, self-learning lessons, user-defined virtual tools, self-configured behavior, browser-automation client tools), the **AI gateway** (`/openai/v1/*`, single point of truth for every AI call, per-user cost attribution, provider/model routing overlay), **async job services** (`JobService` base pattern: zip, fork, SEO diagnostics, SEO metadata generation, DeepSearch, AI Usage Analyzer), and an admin-only append-only **audit log** (`record`/`record_system` entrypoints, `events.md` catalogue of every recorded event key, never raises into the caller). These are among the largest subsystems in the codebase - read their dedicated nested `CLAUDE.md` files (see Subsystem map) before working in any of them; do not assume prior knowledge from this summary.
|
||||
Container manager, Devii, AI gateway, async jobs, audit log: see their nested `CLAUDE.md` files (Subsystem map). Read the dedicated file before working in any of them.
|
||||
|
||||
### Telegram bot, email, devRant compatibility API, issue tracker
|
||||
|
||||
Devii is also reachable over **Telegram** (one supervised long-poller subprocess, `channel="telegram"` isolated conversation thread), can drive a user's own **external mailbox** over IMAP/SMTP (stdlib only, credentials in a soft-deletable table, SSRF-guarded), and DevPlace exposes a **devRant-compatible REST API** at `/api` (translates devRant requests onto native posts/comments/votes via reused audited cores, ID mapping is `posts.id`/`comments.id` directly). The **issue tracker** at `/issues` has no local store - it reads/writes Gitea live via one shared bot token, filing is an async AI-enhanced job. Full detail in the respective nested `CLAUDE.md` files.
|
||||
Telegram bridge, email tools, devRant API, Gitea issue tracker: see their nested `CLAUDE.md` files (Subsystem map).
|
||||
|
||||
### SEO
|
||||
|
||||
`devplacepy/seo.py` generates JSON-LD schemas (WebSite, BreadcrumbList, DiscussionForumPosting, ProfilePage, SoftwareApplication). Every router builds context via `base_seo_context(request, ...)`. Auth/messages/notifications are `noindex,nofollow`; profiles with fewer than 2 posts are `noindex,follow`. `/robots.txt` and `/sitemap.xml` are served by `routers/seo.py`. Full implementation map (template layer, heading hierarchy, slugs, related posts, performance, default OG image, SEO tests) is in `devplacepy/routers/CLAUDE.md`.
|
||||
JSON-LD schemas via `devplacepy/seo.py` and `base_seo_context`; index/follow rules per surface. Full implementation map in `devplacepy/routers/CLAUDE.md`.
|
||||
|
||||
## The production database is never touched without explicit confirmation (hard rule)
|
||||
|
||||
@@ -306,78 +215,41 @@ The escape hatch is deliberately two-factor and must never be self-served: after
|
||||
- **Roles are stored capitalized:** `users.role` is exactly `"Admin"` or `"Member"`. Always test admin-ness through the `is_admin(user)` global (case-sensitive `== "Admin"`) - never hand-roll a lowercase compare. **Any write to `users.role` MUST call `database.invalidate_admins_cache()`.**
|
||||
- **Admin seniority: a junior admin cannot manage a more senior admin.** Every per-user mutation in `routers/admin/users.py` and `routers/admin/moderation.py` is gated by `is_senior_admin(actor, target)` (`routers/admin/_shared.py`) - blocks (audits `result="denied"`) when the target is an Admin who registered earlier. Server-side, so it also covers Devii's admin tools.
|
||||
- **Never pass a `respond()` context key that collides with a Jinja global.** `respond(request, template, ctx, model=XOut)` feeds the same `ctx` to both the Pydantic model (JSON) and the template. A key like `is_admin`/`avatar_url`/`is_self` holding a non-callable value shadows the global across the whole inheritance chain, turning `{% if is_admin(user) %}` into `False(user)` -> `TypeError`, a 500 that fires only for the branch that calls the global. Name viewer/permission flags distinctly (`viewer_is_admin`, not `is_admin`) in both schema and context.
|
||||
- **Project visibility (`is_private`) and read-only (`read_only`)** are owner-controlled flags on `projects`. Read access is gated by the single `content.can_view_project(project, user)` predicate at EVERY read surface - never re-implement the check inline. Predicate: `not is_private OR is_owner OR (is_admin AND owner is not an admin)` - a project hidden by a member stays visible to any admin, but one hidden by an admin is visible only to that owner admin. **Containers have their own, stricter isolation predicates** (`owns_instance`, `can_view_project_containers`, `can_view_instance`, `can_manage_instance`) - the primary administrator sees/manages every container; any other admin can VIEW others' containers only on public projects and can MANAGE only instances they own. Read-only is enforced as a single data-layer guard `project_files._guard_writable(project_uid)` at the top of every mutation entrypoint - add it to any NEW file-mutating function. Devii may flip read-only/visibility only after explicit confirmation (`CONFIRM_REQUIRED`). Full UI-level detail in `devplacepy/routers/projects/CLAUDE.md`.
|
||||
- **Project visibility and read-only** are owner-controlled flags gated by single predicates (`can_view_project`, container isolation predicates, `_guard_writable`): see `devplacepy/routers/projects/CLAUDE.md`.
|
||||
- **Content visibility on posts, gists, projects and news is one specification** (`database/visibility.py`, choke points `listed_clause`/`listed_sql`, `can_view_item`/`can_view_target`): see `devplacepy/database/CLAUDE.md`.
|
||||
- **Deletions ALWAYS require confirmation:** `confirmation_error` gates EVERY content delete tool via `CONFIRM_REQUIRED` (`delete_post`, `delete_comment`, `delete_gist`, `delete_project`, `project_delete_file`, `delete_media`, `delete_attachment`, `admin_delete_news`, container delete, and any `container_exec` matching `dispatcher.DESTRUCTIVE_COMMAND`). The first call is refused; the agent must show the exact target then pass `confirm=true`. **Load-bearing: every confirmation-gated tool MUST also declare a `confirm` boolean param in its catalog spec** - schemas set `additionalProperties: false`, so a gated tool without a declared `confirm` param can never receive it and loops forever.
|
||||
|
||||
## Every user-generated surface is reportable by construction (hard rule)
|
||||
|
||||
A new content table added to `SOFT_DELETE_TABLES` that is visible to anyone other than its author MUST be registered in `database/moderation.py` `REPORTABLE_TARGETS`, MUST resolve in `resolve_object_url`, and MUST include `_report_button.html` in its action bar. A table that is genuinely private to its owner goes in `UNREPORTABLE_TABLES` **with its reason** instead. `tests/unit/database/moderation.py` computes the difference and fails the suite on anything unclassified, so report coverage is closed under future additions rather than remembered; the e2e coverage test enforces the third requirement.
|
||||
New user-generated tables are classified in `REPORTABLE_TARGETS` (with `resolve_object_url` and `_report_button.html`) or in `UNREPORTABLE_TABLES` with a reason; the suite enforces it. No social login, payment, purchasable randomness, advertising, or cross-app tracking without their app-store obligations. Full detail in `devplacepy/services/moderation/CLAUDE.md`.
|
||||
|
||||
The same rule keeps the untriggered app-store conditionals untriggered: **no social login, no payment path, no purchasable randomness, no advertising, and no cross-app tracking** may be introduced without also implementing the obligations each of them creates (Sign in with Apple, in-app purchase, odds disclosure, ad reporting, App Tracking Transparency). Full detail in `devplacepy/services/moderation/CLAUDE.md`.
|
||||
## Admin pages follow one specification (hard rule)
|
||||
|
||||
Every `/admin` page is built to `devplacepy/routers/admin/CLAUDE.md`: backend-rendered first paint with no "Loading..." placeholder, one builder shared by the HTML route and the `/data` refresh endpoint, `respond()` with an `*Out` model, `admin-*` classes only, `local_dt` for every instant, every list paginated or capped, no remote call in a GET, `Poller` only, one Chart.js path, and the section list declared once in `devplacepy/admin_sections.py`. `tests/unit/admin_spec.py` enforces the mechanical rules. A new admin page that breaks the spec does not ship.
|
||||
|
||||
## Modal pattern
|
||||
|
||||
`Application.js` `initModals()` toggles a `.visible` CSS class on `.modal-overlay`; the CSS rule `.modal-overlay.visible { display: flex; }` handles visibility. Triggers usually have `href="#"`, so call `e.preventDefault()`. `.modal-close` is wired generically - no inline JS needed. Full modal/partial/CDN detail in `devplacepy/templates/CLAUDE.md`.
|
||||
See `devplacepy/templates/CLAUDE.md` (modal system, partials, CDN).
|
||||
|
||||
## Polymorphic comments and votes
|
||||
|
||||
The `comments` table uses `(target_type, target_uid)` so `_comment_section.html` works for `post`, `project`, `gist`, and `news`. Votes follow the same shape via `/votes/{target_type}/{uid}`. `resolve_target_redirect()` in `comments.py` maps target_type back to the correct detail URL. Full detail (reactions/bookmarks/polls/heatmap/follow/block-mute reuse the same target-type pattern) in `devplacepy/routers/CLAUDE.md`.
|
||||
`(target_type, target_uid)` shapes comments, votes, and the engagement reuse family: see `devplacepy/routers/CLAUDE.md`.
|
||||
|
||||
## Project-wide soft delete (hard rule)
|
||||
|
||||
**Every removal is a soft delete; only garbage collection is a hard delete.** Removable rows carry `deleted_at` (ISO timestamp) + `deleted_by` (actor uid or `system`); a live row has `deleted_at = NULL` and every list/count read filters `deleted_at IS NULL`. The table set is `database.SOFT_DELETE_TABLES`; `init_db` ensures both columns + a partial index per table. Core primitives in `database/`: `soft_delete`, `soft_delete_in`, `restore`, `purge`, `list_deleted`/`count_deleted`, `restore_event`/`purge_event`.
|
||||
|
||||
- **Every INSERT into a soft-deletable table MUST write `deleted_at: None, deleted_by: None`** - `dataset.find(deleted_at=None)` on a table missing the column matches NOTHING, silently hiding all rows on a fresh DB.
|
||||
- **Any new read of a soft-deletable table MUST filter `deleted_at IS NULL`.**
|
||||
- **Toggles revive, never duplicate:** look up the physical row ignoring `deleted_at`, stamp on toggle-off, clear on re-toggle.
|
||||
- **Cascades share one `stamp`** so the event restores/purges atomically.
|
||||
- **Delete authz is owner-OR-admin on the endpoint** - one check covers the UI and Devii.
|
||||
- **GC stays HARD** (job sweep, metrics ring, usage-ledger prune/reset, expired-session cleanup, fork rollback). Logout is soft (auditable); only expiry GC is hard.
|
||||
|
||||
Admin **Trash** at `/admin/trash` restores/purges by event. Full table list, dataset rules, indexing conventions (the soft-delete planner trap), and site settings are in `devplacepy/database/CLAUDE.md`.
|
||||
**Every removal is a soft delete; only garbage collection is a hard delete.** Inserts write `deleted_at: None, deleted_by: None`; reads filter `deleted_at IS NULL`; toggles revive; cascades share one stamp; delete authz is owner-OR-admin. Full table list, primitives, and indexing rules in `devplacepy/database/CLAUDE.md`.
|
||||
|
||||
## Testing
|
||||
|
||||
Playwright (NOT pytest-playwright). Around 2882 tests in `tests/`, split into three category directories by *what they exercise*: `tests/api/` (HTTP integration, no browser), `tests/e2e/` (Playwright browser), `tests/unit/` (pure in-process). **The directory tree mirrors the path** - one segment per directory, last segment is the file. `api`/`e2e` mirror the URL path (`GET /admin/ai-usage` -> `tests/e2e/admin/aiusage.py`); `unit` mirrors the source module path (`devplacepy/utils.py` -> `tests/unit/utils.py`). A test's tier is decided by its fixtures: `page`/`alice`/`bob` = e2e; `app_server`/`seeded_db` or any HTTP call = api; `local_db`-only or no fixture = unit. Run a tier with `make test-unit`/`make test-api`/`make test-e2e`; `make test` runs all three.
|
||||
|
||||
Fixture stack: `app_server` (session-scoped uvicorn subprocess on port 10501), `browser_context` (session-scoped Playwright context), `page` (function-scoped, fresh cookies), `alice`/`bob` (seeded logged-in users, `bob` gets its own context for multi-user tests).
|
||||
|
||||
Required patterns: every `page.goto(...)`/`page.wait_for_url(...)` MUST pass `wait_until="domcontentloaded"`; prefer `page.locator(...).wait_for(state="visible")` over `wait_for_selector`; scope ambiguous selectors (e.g. comment Delete is `.comment-action-btn:has-text('Delete')`). A test that flips a global `site_settings` value MUST restore it in `try/finally`. Full pitfalls/patterns catalogue is in `tests/CLAUDE.md`.
|
||||
|
||||
**Always run the full test suite (`make test` - unit, api, and e2e, every test) as the final validation of every change.** No tier may be skipped and no subset substituted for the whole. The clean import (`python -c "from devplacepy.main import app"`) and per-language manual checks are preliminary gates before the suite, not replacements for it. Any failure is a real signal and blocks completion until fixed.
|
||||
Tiers, fixtures, required patterns (domcontentloaded, try/finally restores), and the triage loop live in `tests/CLAUDE.md`. The mandate: **full `make test` green as the final validation of every change, no tier skipped, no subset substituted.** Import gate and per-language manual checks are preliminary gates, never replacements.
|
||||
|
||||
## Rigorous correctness verification (money, state machines, concurrency)
|
||||
|
||||
The persisted test suite (`tests/unit`/`api`/`e2e`) is example-based: it proves the specific inputs someone wrote down behave correctly. It is structurally blind to three classes of bug - a formula wrong at an input nobody tried, an invariant violated only after a long sequence of actions, and a race that only two nearly-simultaneous requests trigger. **Any feature that touches a spendable resource (coins, credits, quotas), a bounded state machine (levels, tiers, counters with a floor/ceiling), or a read-then-write mutation reachable from more than one request path applies this procedure in addition to, not instead of, the normal persisted tests.** A simple CRUD toggle doesn't need it; an economy, inventory, ledger, or scoring feature does. This was built out fully for the Code Farm economy rebalance (see `devplacepy/services/game/CLAUDE.md`, "Every purchase/upgrade is atomic..." - the worked example, including every bug it actually caught) and is the standing procedure for anything shaped like it going forward.
|
||||
|
||||
Four escalating layers, run in order, as disposable Python scripts that call the real functions directly against a temp DB (`DEVPLACE_DATABASE_URL`/`DEVPLACE_DATA_DIR` pointed at a scratch file) - not pytest files, unless the user separately asks for persisted tests too:
|
||||
|
||||
1. **Property/invariant checks on every pure function, across the full input domain.** For each formula (a reward, a cost curve, a score, a discount), assert the mathematical property it must have - monotonicity, bounds, non-negativity, idempotency, additivity over a partition of its domain - by iterating the real range (hundreds to thousands of values), not a handful of hand-picked spot checks. A monotonicity bug at input 347 is invisible if only 0, 1, and 10 are tried.
|
||||
2. **Stateful fuzzing.** Build N independent entities, fire a long randomized sequence of every mutating action across all of them against a live temp DB, and re-check invariants after every single action: balances never negative, counters that should only increase never decrease, levels/tiers never exceed their declared bounds. Catch the domain's expected exception type and continue - the goal is catching invariant violations and *unexpected* exceptions, not asserting every action succeeds. This proves safety (nothing bad happens) but not liveness (legitimate actions actually succeed) - a bug that wrongly blocks a valid action just raises a well-formed expected exception and sails through a pure fuzz test undetected. Layer 3 exists because of exactly this gap.
|
||||
3. **Concurrency: prove races are closed, never infer it from luck.** Any read-then-write mutation reachable from more than one worker process (`uvicorn --workers N`) is a TOCTOU race until proven otherwise. Test with **real separate OS processes**, not threads in one process - `dataset` gives each thread its own pooled connection, and enough threads exhausts that pool and produces `database is locked` noise that is a test-harness artifact, not a finding about the application. Set up genuinely fresh, production-representative state before racing - **never pre-seed or zero a column that the real code path leaves unset/NULL.** This was the single most expensive mistake made building this procedure: a first race-safety pass "passed" only because its own setup script had artificially pre-zeroed columns that a real fresh row leaves as SQL `NULL` - silently hiding the exact bug the fix was supposed to prevent (`NULL = 0` evaluates to `NULL`, not true, in a SQL `WHERE` clause; any column not written in the row's original `INSERT` needs `COALESCE(column, 0)` in every later precondition and every arithmetic `SET`, not a bare comparison). Fire many concurrent attempts at the same resource, assert the exact right number succeed, and verify the final state matches the hand-computed expected total exactly (currency spent, levels advanced) - not just "the others were blocked." Fix a real race with a single atomic conditional SQL statement at the exact chokepoint (`UPDATE ... SET ... WHERE <precondition>`, checked via the driver's real `rowcount` - `dataset`'s wrapped `db.query()` does not expose it, use `db.executable.execute(sqlalchemy.text(...), params).rowcount` inside `with db:`), not a client-side lock or an optimistic-locking library this codebase doesn't otherwise use.
|
||||
4. **Static analysis beyond `py_compile`.** A clean `python -m py_compile` and a clean `from devplacepy.main import app` only prove syntax and module-load order - neither catches a missing import inside a function body, which is a runtime `NameError` invisible until that exact line executes. Run `pyflakes`/`ruff check` on every touched file before calling a change done; it is nearly free and catches an entire class of bug that layers 1-3 can each individually miss if they don't happen to exercise the broken line.
|
||||
Spendable resources, bounded state machines, and multi-worker read-then-write mutations additionally require the four-layer procedure (property checks, stateful fuzzing, real-process races, `pyflakes`/`ruff`): see `tests/CLAUDE.md` ("Rigorous correctness verification").
|
||||
|
||||
## Feature workflow
|
||||
|
||||
Every feature in DevPlace is **one data source fanning out into several consumers**. DevPlace has no isolated change - internalize this before editing. A single handler in `routers/{area}.py` is simultaneously the **four faces of one route**:
|
||||
|
||||
1. **HTML** - `respond()` returns a rendered template for browsers.
|
||||
2. **JSON** - the SAME `respond(..., model=XOut)` returns JSON when `Accept: application/json`. The `*Out` schema is the gate: a context key not declared on `*Out` is silently dropped from JSON even though the template still sees it.
|
||||
3. **Agent tool** - `services/devii/actions/catalog/` (the relevant module inside the package, e.g. `posts.py`, `admin.py`) exposes the route as an `Action` so Devii can call it on the user's behalf. `requires_auth` mirrors the route guard.
|
||||
4. **Documented endpoint** - `docs_api.py` `endpoint()` describes it (params, auth, `sample_response`) in the right group, rendered at `/docs/{group}.html`.
|
||||
|
||||
A new public read almost always needs all four. The cardinal failure mode is changing one face and forgetting a connected one - a real worked example (followers/following listing) touched nine files across data helpers, output schema, route, view, agent tool, API docs, and the docs trio below for one conceptual feature; that count is the norm, not the exception. If a change touches only one file, confirm none of the four faces were missed. Checklist, ordered by data flow:
|
||||
|
||||
1. **Understand.** Read the router, template, matching tests (per the tier naming rule above), and the matching nested `CLAUDE.md`. Trace input model -> router -> data helper -> response (HTML and JSON).
|
||||
2. **Data layer.** `database/` for query/batch helpers (never inline N+1 loops - use `get_users_by_uids`, `build_pagination`, `_in_clause`; guard raw SQL with `if "table" in db.tables`). `models/` for the Pydantic `Form` input model (add the class to the matching domain module and to the re-export list plus `__all__` in `models/__init__.py`). `schemas/` for the `*Out` JSON response model - every context key a JSON route exposes via `respond(..., model=XOut)` MUST exist on `XOut` or it is silently dropped. Schema auto-syncs via `dataset`; add indexes in `init_db()` with `CREATE INDEX IF NOT EXISTS`.
|
||||
3. **Server layer.** Handler with the right auth guard (`get_current_user` public read, `require_user` member POST, `require_admin` admin). Specific paths (`/{username}/followers`) declared before catch-alls (`/{username}`). Return HTML+JSON via `respond(request, template, ctx, model=XOut)` or pure JSON via `JSONResponse`. Register any NEW router in `main.py`.
|
||||
4. **View layer.** Extend `base.html`; page CSS in `{% block extra_head %}`, page JS in `{% block extra_js %}`. Reuse partials and template globals. JS is ES6 modules, one class per file, instantiated on `app`.
|
||||
5. **Safety layer (skip only when the feature stores nothing and shows nothing).** If the feature adds a surface that carries user-generated content, register it in `database/moderation.py` `REPORTABLE_TARGETS` (or in `UNREPORTABLE_TABLES` with a reason), resolve it in `resolve_object_url`, and include `_report_button.html` in its action bar - the registry test and the e2e coverage test both fail otherwise. If it collects or transmits a new *category* of personal data, or sends anything new to a third party, update `/docs/privacy.html` and the app-store privacy declarations in the same change; that disclosure is part of the feature, not a follow-up.
|
||||
6. **Agent + docs layer (the most-forgotten step - do not skip).** `services/devii/actions/catalog/` (the relevant module inside the package) - add an `Action` tool if a user could ask Devii to do this (`requires_auth=False` for public reads). `docs_api.py` - every public/auth endpoint gets an `endpoint()` entry with params and a `sample_response`. `routers/docs/pages.py` `DOCS_PAGES` for prose pages. SEO (`seo.py`) - `base_seo_context` and any JSON-LD for a new public page; add to `routers/seo.py` sitemap if indexable.
|
||||
7. **Document.** `README.md` (product-facing, routes/config/dependencies/user-visible features), the relevant nested `CLAUDE.md` (any new mechanic, helper, pitfall, or cross-layer wiring), this root `CLAUDE.md` only when a NEW architectural rule or workflow step is introduced (not per feature).
|
||||
8. **Validate.** Confirm a clean import (`python -c "from devplacepy.main import app"`), check each touched language manually, grep for em-dash in touched files (neither the character nor its HTML entity - use a hyphen), and run `pyflakes`/`ruff check` on every touched file. **If the feature touches a spendable resource, a bounded state machine, or a read-then-write mutation reachable from more than one request path, also run the four-layer procedure in "Rigorous correctness verification" above** - it is the default for that shape of feature, not an optional extra. **Finish by running the full persisted test suite (`make test` - every tier, every test) and fixing any failure before the change is done.** Write new tests in the matching tier/path, following the required patterns above.
|
||||
|
||||
Failures at any implementation step block the workflow - never skip a failed step.
|
||||
Every feature is one data source fanning out into HTML, JSON, a Devii action, and a documented endpoint. The ordered checklist (understand, data, server, view, safety, agent+docs, document, validate) lives in `devplacepy/routers/CLAUDE.md` ("Feature checklist"). Failures at any step block the workflow.
|
||||
|
||||
## CI/CD
|
||||
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
# Definition of Done
|
||||
|
||||
This is the completion contract for every change in this repository. A change is done only when every applicable gate below passes. It is linked from the root `CLAUDE.md` and draws on all nested `CLAUDE.md` files; where a nested file states a stricter rule for its subsystem, the stricter rule wins.
|
||||
|
||||
## 1. Universal gates (every change)
|
||||
|
||||
1. **Exact, minimal diffs.** Edit existing files; create a new file only when the change genuinely needs a home. No unrelated edits, no drive-by refactors.
|
||||
2. **DRY, one choke point.** If the behaviour already exists somewhere, reuse it through the existing choke point; never add a second implementation, second focus system, second removal path, second suspension check, second consent gate, second terms check, second proxy, second bulk mechanism, or second query on a different basis. A per-type copy of shared logic is a defect.
|
||||
3. **A specification exists somewhere.** Literally everything has a declared spec: a `spec.py` single source of truth, a registry (`REPORTABLE_TARGETS`, `DOCS_PAGES`, `NOTIFICATION_TYPES`, `GATEWAY_APPS`, `CUSTOMIZATION_PREF_COLUMNS`), a table in a `CLAUDE.md`, or an explicit decision section. The new behaviour is declared in it, not just implemented.
|
||||
4. **Zero tolerance for inconsistency.** New code matches existing implementation structurally and visually. Before modifying, find the relatable example; if none exists, explain and let the user decide how to continue.
|
||||
5. **The four faces.** Every feature is one data source fanning out into HTML, JSON (`respond(..., model=XOut)`, and a context key not on the `*Out` schema is silently dropped from JSON), a Devii agent action, and documentation. State each face explicitly, including "no change" where that is the decision.
|
||||
6. **Import gate and static checks.** `python -c "from devplacepy.main import app"` imports clean; `pyflakes`/`ruff` on every touched file is clean (module-level load order and missing function-level imports are invisible to the other checks). Check each touched language manually (Python compiles/imports, JS parses, CSS braces and HTML tags balance).
|
||||
7. **Full suite green.** `make test` (unit + api + e2e, serial, one pass) passes with no tier skipped and no subset substituted. Triage cheapest-first: `make test-fast`, then `make test-failed` until empty. A failing existing test is a contract to satisfy, never to skip, narrow (`-k`, `--deselect`), or rewrite to fit the change.
|
||||
8. **Verification through the real path.** Real filesystem, real concurrency, real failure injection: bursts, races, and full disks are test cases, not weather. Re-running your own probe is not verification; the oracle must be independent (the repo's own tests, a golden file, a second method).
|
||||
9. **Commit, deploy, verify, in that order, same session.** A change that is not running is not done. After deploy, verify live; green locally means nothing until prod confirms it.
|
||||
10. **Production database untouched without explicit confirmation.** `data/devplace.db` and everything under `data/` is never read or written by an agent-initiated command without the user's own written confirmation plus the `I-HAVE-CONFIRMED-PRODUCTION-DB-ACCESS` token. Disposable verification scripts use `DEVPLACE_DATABASE_URL`/`DEVPLACE_DATA_DIR` on scratch files.
|
||||
11. **No comments, no docstrings.** Code is self-documenting. No em dashes anywhere.
|
||||
12. **Forbidden patterns respected.** No `_new`/`_old`/`_temp`/`_v2`/`better_`/`my_`/`the_` names; `RedirectResponse(url=..., status_code=302)` with an integer literal; no `is_admin`-style context key colliding with a Jinja global (use `viewer_is_admin`-style names); dates shown DD/MM/YYYY via `local_dt`/`dt_ago`.
|
||||
|
||||
## 2. Tests (committed, reusable, in the mirrored tree)
|
||||
|
||||
- HTTP tests under `tests/api/<url path>/`, browser tests under `tests/e2e/<url path>/`, pure-function tests under `tests/unit/<module path>/`. Reuse `alice`/`bob`, `app_server`, `seeded_db`, `local_db`.
|
||||
- Browser rules: every `page.goto`/`wait_for_url` passes `wait_until="domcontentloaded"`; prefer `locator().wait_for(state="visible")`; scope ambiguous selectors; restore global `site_settings` flips in `try/finally`.
|
||||
- A behavior change keeps one focused failing-first test with the change, in the existing layout, using the existing fixtures. No new test framework. Throwaway probes stay out of the repo (and a `/tmp` probe never substitutes for the committed test).
|
||||
- Never weaken a failing test to obtain green; fix the product or justify the expectation change. Observe the authentic failure before the fix when safe and runnable; run the focused test after the final edit; run the broad suite once, at the end.
|
||||
|
||||
## 3. Concurrency and state-machine rigor (the four layers)
|
||||
|
||||
For any spendable resource (coins, credits, quotas), bounded state machine (levels, tiers, counters with a floor/ceiling), or read-then-write mutation reachable from more than one worker, the persisted suite is not enough. Run, in order, as disposable scripts against a temp DB (never pytest files unless also requested):
|
||||
|
||||
1. **Property checks** on every pure function over the full input domain (monotonicity, bounds, non-negativity, idempotency), iterating the real range, not spot checks.
|
||||
2. **Stateful fuzzing**: many entities, long randomized action sequences, invariants re-checked after every action (balances never negative, monotonic counters never decrease, bounds never exceeded).
|
||||
3. **Real multi-process races**: separate OS processes, never threads; production-representative state, never pre-seeded columns (NULL is not 0 in SQL, use `COALESCE`). Prove exact right-wins counts and hand-computed final totals. Fix with one atomic conditional SQL statement (`UPDATE ... WHERE <precondition>`, real driver `rowcount`), never a client-side lock.
|
||||
4. **Static analysis** beyond `py_compile`: `pyflakes`/`ruff` on every touched file.
|
||||
|
||||
Single-process races do not count. Threads do not count. Luck does not count.
|
||||
|
||||
## 4. Data layer
|
||||
|
||||
- Every insert into a soft-deletable table writes `deleted_at: None, deleted_by: None`; every read filters `deleted_at IS NULL`. Toggles revive, never duplicate; cascades share one stamp; delete authz is owner-or-admin on the endpoint; GC stays hard. New user-generated tables are classified in `REPORTABLE_TARGETS` with `resolve_object_url` and `_report_button.html`, or in `UNREPORTABLE_TABLES` with a reason (the suite enforces this).
|
||||
- `init_db` ensures the full column set of any table code filters on before creating indexes; new columns go through the ensure idiom plus the born-live pair, never bare DDL that misses pre-existing tables.
|
||||
- Bounded, indexed reads: window functions over full-table loads, batch helpers over N+1, live-partial indexes built to plan without table scans. Nullable legacy columns need `COALESCE` in every precondition.
|
||||
- Migrations: `init_db` is idempotent, guarded by locks, skipped seeds stay skipped (consumers pass defaults to `get_setting`, never assume the seed ran). Consent backfills are forbidden (never fabricate consent); `terms_version` stays NULL for legacy accounts.
|
||||
- Runtime paths come only from `config.DATA_DIR` constants, never computed from scratch; blobs shard on uuid tail or hash, never the head.
|
||||
|
||||
## 5. Routing, templates, frontend
|
||||
|
||||
- Routers mirror the URL path (`routers/<path>.py`, packages aggregate with `include_router`); route guards mirror auth needs (`require_user`, `require_admin`, `require_user_api`); public pages use `get_current_user`. Mutations are POST with `data-confirm`; deletes additionally allow any admin; seniority gates (`is_senior_admin`) bind Devii too.
|
||||
- One shared `templates` instance; `nav_active` for nav state; `static_url`/`assetUrl` for every asset, never a bare `/static/` href; `dp-*` web components (no duplicates, real-button triggers inside popovers so no-JS stays correct, Escape closes); new JS reuses `Http`/`Poller`/`JobPoller`/`OptimisticAction`/`FloatingWindow`/`ScrollMemory`.
|
||||
- Server pipeline renders content for SEO (`render_content`/`render_title`, with `local_dt`/`dt_ago` for timestamps); the client pipeline is only for live content (`DOMPurify` fail-closed). Notifications use `render_title`. Emoji shortcodes come from the one map; YouTube embeds follow `embeds.py` only; emails are masked; em dashes become hyphens.
|
||||
- CSS: tokens from `variables.css` only (file-scoped `--x-*` palette block per new file, then `--color-*`/`--space-*`); `--z-*` bands; closed breakpoints; `prefers-reduced-motion` disables animation; no literals, no `var()` fallbacks, one radius per element, matching dark theme on first paint.
|
||||
- Admin pages conform to `devplacepy/routers/admin/CLAUDE.md` (A1-A11): server-rendered first paint, one builder for HTML and `/data`, `respond()` with an `*Out` model, `admin-*` classes, `local_dt`, capped lists, `Poller`, the `ADMIN_SECTIONS` registry; `tests/unit/admin_spec.py` passes.
|
||||
|
||||
## 6. Services and jobs
|
||||
|
||||
- Background services run only in the lock owner, supervised with health checks and lock files; a new `JobService` needs a server restart and registers its kind. The DB row is the queue; retention is built in; cleanup decisions are per-kind and deliberate (disposable archive vs permanent artifact).
|
||||
- Services report state through the one live-view relay row, audit through `record`/`record_system` with catalogued `events.md` keys (never raising into the caller), and telegram/push delivery never invents new channels. New websockets get their own nginx upgrade `location` above `location /`.
|
||||
- New dependencies are unpinned only by deliberate exception; outbound HTTP goes through stealth with `net_guard` where untrusted; feature code never blocks the event loop (gateway-adjacent sync calls run in `to_thread`).
|
||||
|
||||
## 7. AI, Devii, and money
|
||||
|
||||
- The gateway is the single point of truth for AI: every consumer points at it, keys fall back to the internal key, spend is attributed and metered, financial figures are admin-only everywhere. Consent (`ai_third_party`) gates user-content calls at exactly one place; structured internal callers may bypass the operator preamble, conversational ones never do.
|
||||
- Devii: every gated tool declares a `confirm` boolean (`additionalProperties: false` makes a gateless confirm fatal); classifier recall beats precision; budget-bounded tools with fail-closed redaction; lessons carry provenance and are never auto-applied from third-party content; channels stay isolated; self-dial uses the real origin URL.
|
||||
- Economy and scoring invariants hold exactly: one pure function per number, atomic transitions only, recomputation beats counters, races proven with real processes (see section 3).
|
||||
|
||||
## 8. Done means
|
||||
|
||||
- [ ] Minimal diff, no unrelated edits; new files only with a home.
|
||||
- [ ] One choke point reused; spec/registry updated; four faces stated.
|
||||
- [ ] Import gate + `pyflakes`/`ruff` clean; per-language manual checks done.
|
||||
- [ ] Committed reusable tests added/updated in the mirrored tree; focused failure observed before, passing after.
|
||||
- [ ] Section 3 layers run where applicable (spendable resource, bounded state machine, multi-worker mutation).
|
||||
- [ ] `make test` fully green; no tier skipped, no test narrowed or rewritten to fit.
|
||||
- [ ] No production data touched; scratch DBs only.
|
||||
- [ ] Docs touched where the behaviour is user-visible (`/docs` prose + `DOCS_PAGES`, `docs_api` groups, `events.md`, nested `CLAUDE.md` notes).
|
||||
- [ ] Committed, deployed, verified live, same session.
|
||||
@@ -89,7 +89,7 @@ devplacepy/
|
||||
| `/admin/trash` | Admin **Trash**: review, restore, and permanently purge soft-deleted content (posts, comments, gists, projects, news, project files, attachments) across the platform |
|
||||
| `/admin/devii-tasks` | Admin **Devii tasks**: every scheduled task across all owners with its schedule, run count, expiry and failure streak, plus per-task disable and delete |
|
||||
| `/notifications` | Notification list, mark read, live unread counts (`/notifications/counts`) |
|
||||
| `/messages` | Real-time direct messaging over WebSocket (`/messages/ws`): live bidirectional delivery, optimistic send, typing indicators, read receipts, reconnect catch-up, and online/last-seen presence. Messages render through the shared content pipeline (emoji shortcodes, image, video and audio embeds, autolink, sanitization). AI content correction and the AI modifier apply to direct messages: the original is delivered immediately, then a second `message` frame with `ai_processed` replaces the bubble for both participants when the rewrite lands (including across workers). An opened conversation loads its 500 most recent messages and can page older rows with `?before=`; switching threads does not drop the socket. `GET /messages/conversations` returns the conversation list as JSON for a live client refresh; the `POST /messages/send` form remains as a no-JavaScript fallback and now accepts an attachment-only, empty-content message. `POST /messages/ws-ticket` exchanges a session/API-key auth to a short-lived (30s), single-use ticket a browser WebSocket can carry as `?ticket=...` on the handshake, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header - this is what makes off-session embedding of the chat possible. CLI: `devplace messaging prune-tickets` removes expired tickets |
|
||||
| `/messages` | Real-time direct messaging over WebSocket (`/messages/ws`): live bidirectional delivery, optimistic send, typing indicators, read receipts, reconnect catch-up, and online/last-seen presence. Messages render through the shared content pipeline (emoji shortcodes, image, video and audio embeds, autolink, sanitization). AI content correction and the AI modifier apply to direct messages: the original is delivered immediately, then a second `message` frame with `ai_processed` replaces the bubble for both participants when the rewrite lands (including across workers). An opened conversation loads its 500 most recent messages and can page older rows with `?before=`; switching threads does not drop the socket. `GET /messages/conversations` returns the conversation list as JSON for a live client refresh; the `POST /messages/send` form remains as a no-JavaScript fallback and now accepts an attachment-only, empty-content message. An administrator can delete the messages they themselves sent (`POST /messages/{uid}/delete`, soft delete, confirmation in the chat UI and in Devii); the bubble disappears live for both participants. `POST /messages/ws-ticket` exchanges a session/API-key auth to a short-lived (30s), single-use ticket a browser WebSocket can carry as `?ticket=...` on the handshake, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header - this is what makes off-session embedding of the chat possible. CLI: `devplace messaging prune-tickets` removes expired tickets. On every other page the topnav Messages icon opens a **Chats** panel (people search, recent conversations, a New message button and a link to the full page), and picking a person opens a small chat window docked at the bottom of the screen, as on Facebook. Up to three windows sit side by side (fewer on narrow screens); a window minimizes to an avatar head that counts unread messages, and open chats survive page navigation within the tab. A profile's Send Message button and a message notification (the live toast and the entry on `/notifications`) open the same docked window instead of leaving the page. Phones keep the full `/messages` page. The message input follows the same layout everywhere: a + button reveals the attachment and GIF buttons, which tuck away again while you type, and the emoji button sits inside the text field |
|
||||
| `/votes` | Upvote/downvote on posts, comments, projects |
|
||||
| `/reactions` | Emoji reactions on posts, comments, gists, projects |
|
||||
| `/bookmarks` | Save/unsave content; `/bookmarks/saved` personal list |
|
||||
@@ -229,6 +229,10 @@ The rack's **control terminal** supports every current game API route. Try `help
|
||||
|
||||
The farm refreshes live over the pub/sub bus (a watered build appears on the owner's screen at once) and every plot countdown ticks client-side. Every endpoint also answers JSON, and Devii can play the game on the member's behalf via the `game_*` tools (`game_state`, `game_plant`, `game_harvest`, `game_plot_stop`, `game_buy_plot`, `game_upgrade_ci`, `game_fertilize`, `game_daily`, `game_claim_quest`, `game_water`, `game_steal`, `game_view_farm`, `game_leaderboard`, `game_upgrade_perk`, `game_upgrade_legacy`, `game_prestige`, `game_claim_grant`, `game_upgrade_mastery`, `game_buy_infrastructure`, `game_upgrade_defense`, `game_buy_cosmetic`, `game_equip_cosmetic`, `game_downgrade_defense`). See the API reference group **Code Farm** and the full player guide at `/docs/code-farm.html`.
|
||||
|
||||
## Content visibility
|
||||
|
||||
Posts, gists, projects and news carry a status: **published** (default), **hidden**, **offline** or **draft**. Hidden items drop out of every list, search, topic page, profile and the sitemap but still open by direct link (marked `noindex`). Offline and draft items are visible only to their owner and administrators; everyone else gets a 404 and cannot comment, vote, react or bookmark them. The field is optional and backwards compatible: omitted on create it defaults to published, omitted on edit the current value is kept, and old rows read as published. Owners set it with the Visibility field in the create and edit forms (and the `visibility` parameter of the API and the Devii create and edit tools); administrators set a news article's status from the admin news page (`POST /admin/news/{uid}/visibility`, Devii `admin_set_news_visibility`). The specification is `devplacepy/database/visibility.py` (a NULL value reads as published, so no data migration), and the user-facing page is `/docs/content-visibility.html`. The website offers published, hidden and offline; draft is accepted by the API and reserved for a future drafting flow.
|
||||
|
||||
## Engagement
|
||||
|
||||
- **Emoji reactions** - react with **any** emoji on posts, comments, gists, and projects, separate from voting and carrying no ranking weight. A short quick-pick palette covers the common reactions, and a `+` button next to it opens the full searchable emoji picker (every standard emoji, including skin tones), so a reaction is never limited to a preset list. Emoji already used on an item are shown as counted chips beside the palette.
|
||||
@@ -256,6 +260,12 @@ DevPlace is open in the sense that it does not editorialise technical opinion. I
|
||||
- **Account deletion.** Self-service at `/profile/{username}/delete`, reauthenticated with the account password. Sessions and tokens are revoked, the profile is anonymised immediately, and all content is removed under one deletion event that stays restorable for `account_deletion_grace_hours` (default 24) before `devplace accounts prune` (and the Moderation housekeeping service) purges it permanently. The devRant `DELETE /api/users/me` routes into the same cascade.
|
||||
- **Legal pages**: [Terms of Service](/docs/terms.html), [Community Guidelines](/docs/community-guidelines.html), [Privacy Policy](/docs/privacy.html), [How moderation works](/docs/content-moderation.html), [Notice and takedown](/docs/intellectual-property.html) and [Contact](/docs/contact.html). All six are indexed in the sitemap and reachable from the docs index; the footer of every page links Terms, Privacy, Community Guidelines and Contact. Contact details come from the `contact_email`, `contact_phone` and `contact_address` settings, so the in-product page and any app-store trader declaration have one source of truth.
|
||||
|
||||
## GIF picker (GIPHY)
|
||||
|
||||
A "GIF" button beside the attachment button in the post composer, every comment form and the message input (behind its + button) opens a searchable GIPHY panel (trending first, "Powered by GIPHY" attribution). A picked GIF is downloaded by the server and stored as an ordinary attachment, so it renders in the attachment gallery, can be reported and deleted like any file, and counts against the normal attachment limits.
|
||||
|
||||
An administrator enters the GIPHY API key under **Admin > Services > Giphy** (also the content rating, default `pg`, and the API base URL); the key is stored in `site_settings`, never reaches a browser, and the button stays hidden until it is set. Searches run on the server, are cached for five minutes, and each member may make 30 per minute. Routes: `GET /giphy/status`, `/giphy/search?q=`, `/giphy/trending`, `POST /giphy/attach` (documented under Uploads at `/docs/uploads.html`); Devii has `search_gifs`, `trending_gifs` and `attach_gif`. Details: `devplacepy/services/giphy/CLAUDE.md`.
|
||||
|
||||
## Skills
|
||||
|
||||
Members can install [Agent Skills](https://agentskills.io) from a URL at `/skills` so Devii can use them. A skill is a `SKILL.md` (YAML frontmatter plus markdown instructions) with optional text files under `scripts/`, `references/` and `assets/`. Skills are private to their owner, count against a per-user cap, and only the name and a one-line description of enabled skills are added to Devii's system prompt; Devii loads the full text on demand. A skill is untrusted third-party text: it can never grant Devii a tool, bypass a confirmation or run a script. Deleting a skill is a soft delete and needs confirmation when Devii does it. Documented in the API docs under *Skills* and in `devplacepy/services/skills/CLAUDE.md`.
|
||||
@@ -341,13 +351,20 @@ accepts three header-based methods, resolved centrally in `get_current_user`
|
||||
Every user has an `api_key` (a uuid7), set at signup and backfilled for existing
|
||||
users on startup (and via `devplace apikey backfill`). The key is shown on the
|
||||
profile page to its owner (and to admins for any user) with a live **Regenerate**
|
||||
button; regenerating invalidates the old key immediately. A successful JSON
|
||||
`POST /auth/login` or `POST /auth/signup` also returns the account's `email`,
|
||||
`api_key`, and `is_admin` flag, and `GET /profile/{username}` as JSON reports
|
||||
`profile_is_admin` plus the key (null unless the viewer owns the profile or is
|
||||
an admin). Invalid credentials return `401`; requests with no credentials behave
|
||||
as anonymous (browser redirect to login). Full details and copy-paste `curl`
|
||||
examples live at `/docs/authentication.html`.
|
||||
button; regenerating invalidates the old key immediately. The three account
|
||||
credentials are username+password, email+password, and the personal API key: the
|
||||
login form, JSON `POST /auth/login`, `POST /auth/token`, and HTTP Basic all accept
|
||||
the username or the email address as the password identifier (usernames are
|
||||
case-sensitive, email addresses are not), and clients conventionally carry the API
|
||||
key in the `DEVPLACE_API_KEY` environment variable. Every JSON response that returns
|
||||
the personal API key returns the account's `username` and `email` alongside it:
|
||||
login/signup (`data`), `POST /profile/regenerate-api-key`, and the profile JSON
|
||||
(`profile_user.username`, `email`, `api_key`; key and email null unless the viewer
|
||||
owns the profile or is an admin). So a client holding only an API key fetches its
|
||||
account in one call: `GET /profile` with `X-API-KEY` and `Accept: application/json`.
|
||||
Invalid credentials return `401`; requests with no credentials behave as anonymous
|
||||
(browser redirect to login). Full details and copy-paste `curl` examples live at
|
||||
`/docs/authentication.html`.
|
||||
|
||||
```bash
|
||||
curl -H "X-API-KEY: <your-key>" https://your-host/notifications
|
||||
@@ -516,7 +533,7 @@ and its full configuration are documented automatically - including future servi
|
||||
- **`ServiceManager`** - singleton: `register`, `describe_all`, `set_enabled`, `send_command`, `save_config`, `supervise`, `shutdown_all`
|
||||
- **`NewsService`** - a fully automatic, zero-maintenance news pipeline: fetches news from `news.app.molodetz.nl/api`, cleans each article, fetches and perceptually compares the images to reject placeholders and detect uniqueness, grades each one against a free, local, deterministic quality model (aquality - no billed LLM call, no API key), optionally reformats articles that clear the publish grade threshold into clean Markdown with a generative AI model (off by default, since the default grading model cannot generate text), and auto-rotates the best articles to Featured and the landing page. Any AI spend it does incur is metered from the gateway response headers and reported on the admin Services page (calls, tokens, total cost, and per-call averages)
|
||||
- **`BotsService`** - Playwright fleet of AI personas that browse and interact with a DevPlace instance, with live cost/usage metrics and a live screenshot monitor at `/admin/bots` (opt-in; install the `bots` extra)
|
||||
- **`GatewayService`** - OpenAI-compatible LLM gateway at `/openai/v1/*`, forwarding to DeepSeek (default) with optional vision augmentation; the single point of truth for AI that every other service routes through (enabled by default). Every chat request is made date-aware by injecting the current date in EU `DD/MM/YYYY` format into the system message when it contains no date already (date only, never time, to keep upstream prompt caching effective), and an admin-configurable system preamble (`gateway_system_preamble`) can be prepended ahead of the client's system message on every call. **Thinking is disabled by default** on every chat and vision call (`thinking.type=disabled` on DeepSeek, `reasoning.effort=none` on OpenRouter, `think=false` on Ollama) so the fast path is the default; a client may re-enable it per request, and an administrator may flip `gateway_thinking` on. **Provider and model routing** (admin **Gateway** page, `/admin/gateway`) maps any number of requested model names onto named upstream providers and target models, each with its own pricing economy (input, output and cache-hit/cache-miss rates per million tokens) and an optional vision model that describes image content before forwarding, so text and vision models are merged transparently. Unmapped requests fall through to the default upstream unchanged, so existing clients are unaffected. Each route may also name a **fallback model** - another already-configured model of the same kind, picked from a select box of the public model names (never a free-text provider model id) - that the gateway retries once, automatically, whenever the primary model fails after its own retries are exhausted, so a struggling model degrades to a working one instead of failing the caller
|
||||
- **`GatewayService`** - OpenAI-compatible LLM gateway at `/openai/v1/*`, forwarding to DeepSeek (default) with optional vision augmentation; the single point of truth for AI that every other service routes through (enabled by default). Every chat request is made date-aware by injecting the current date in EU `DD/MM/YYYY` format into the system message when it contains no date already (date only, never time, to keep upstream prompt caching effective), and an admin-configurable system preamble (`gateway_system_preamble`) can be prepended ahead of the client's system message on every call. **Thinking is disabled by default** on every chat and vision call (`thinking.type=disabled` on DeepSeek, `reasoning.effort=none` on OpenRouter, `think=false` on Ollama) so the fast path is the default; a client may re-enable it per request, and an administrator may flip `gateway_thinking` on. **Provider and model routing** (admin **Gateway** page, `/admin/gateway`) maps any number of requested model names onto named upstream providers and target models, each with its own pricing economy (input, output and cache-hit/cache-miss rates per million tokens) and an optional vision model that describes image content before forwarding, so text and vision models are merged transparently. Unmapped requests fall through to the default upstream unchanged, so existing clients are unaffected. Each route may also name a **fallback model** - another already-configured model of the same kind, picked from a select box of the public model names (never a free-text provider model id) - that the gateway retries once, automatically, whenever the primary model fails after its own retries are exhausted, so a struggling model degrades to a working one instead of failing the caller. When an upstream rejects a request key with a parsable unrecognized-key 400 (for example a strict-schema provider refusing the `thinking` field), the gateway automatically tries other request shapes, remembers the one that works per provider and target model, and applies it from then on; administrators see it as a `shape:` badge on the model grid with a Reset learned action
|
||||
- **`JobService` / `ZipService` / `ForkService`** - generic async job framework (`services/jobs/`) for heavy, blocking work run off the request path; `ZipService` builds project zip archives in a subprocess, `ForkService` copies a project into a new project owned by the forking user
|
||||
- **`ContainerService`** - the admin container manager (`services/containers/`): a reconciling supervisor for container instances, all running one shared prebuilt image
|
||||
- **`AcceptanceService`** - grants every policy agreement (Terms of Service, Privacy Policy, third-party AI processing, activity recording, container credentials) to every account that has not declined it, so an instance kept production-identical for extended manual testing never interrupts with an acceptance dialog. Administrator-only, **off by default**, with a separate switch per agreement type and a dry run that reports what it would do without writing. An account that withdrew a consent is never granted it again, with no further action: the consent ledger itself is the decline register. It is not appropriate on a real production host
|
||||
@@ -757,6 +774,7 @@ Configuration on the Services tab:
|
||||
| `gateway_rsearch_cost_per_call` | 0.0 | Flat cost attributed to each external `rsearch` call (web search / AI answer / chat / image describe), recorded under backend `rsearch` so external AI spend appears in AI usage |
|
||||
| `gateway_max_retries` / `gateway_retry_backoff_ms` | 2 / 250 | Retry attempts and linear backoff on timeout, connection error, or upstream 5xx |
|
||||
| `gateway_circuit_threshold` / `gateway_circuit_cooldown_seconds` | 5 / 30 | Consecutive failures before the circuit breaker opens, and its cooldown |
|
||||
| `gateway_target_circuit_enabled` | on | Per-model circuit breaker: one upstream error opens it for the cooldown and notifies every admin with the error details |
|
||||
| `gateway_usage_retention_hours` | 720 | How long per-call usage rows are kept before pruning (30 days) |
|
||||
| `gateway_model_context_map` | JSON map | Model to max-context-tokens map for context-window utilization tracking |
|
||||
|
||||
@@ -781,7 +799,7 @@ token usage with averages and percentiles (p50/p90/p95/p99), latency (upstream r
|
||||
gateway overhead, semaphore queue wait, connection establishment), error rates by category,
|
||||
cost (per model, per caller, input vs output, projected monthly burn, caching savings),
|
||||
caller behavior, and an hourly breakdown - divided and combined. The gateway also retries
|
||||
transient upstream failures and opens a circuit breaker after repeated failures. Time to
|
||||
transient upstream failures, opens a circuit breaker after repeated failures, and opens a per-model breaker on the first error of a model and notifies every admin. Time to
|
||||
first token and inter-token latency are not reported because the gateway forwards
|
||||
non-streaming to the upstream.
|
||||
|
||||
@@ -1057,7 +1075,9 @@ in real time, bridged onto the in-process pub/sub bus by a lock-owner relay:
|
||||
| Issue-tracker update | reporter / admins |
|
||||
|
||||
`create_notification` schedules delivery as a fire-and-forget async task, so a dead
|
||||
subscription or push-service error never blocks the triggering request. Delivery
|
||||
subscription or push-service error never blocks the triggering request. The payload carries the recipient's unread count (`unread`, the same number as
|
||||
`/notifications/counts`), which APNs presents as `aps.badge` so the app icon updates even
|
||||
while the app is closed. Delivery
|
||||
(`push.notify_user`) reads a user's subscriptions once, groups them by provider, builds
|
||||
each provider's payload once, and sends over that provider's own HTTP client (stealth for
|
||||
Web Push, a dedicated HTTP/2 client for APNs). A subscription the push service reports as
|
||||
@@ -1116,8 +1136,8 @@ every page load. `PushManager.js` owns registration, subscription, and the opt-i
|
||||
|
||||
`manifest.json` (192/512 and maskable icons) and `service-worker.js` make the app
|
||||
installable via the browser's native install affordance. The service worker uses a
|
||||
network-first strategy for navigations and falls back to `static/offline.html` when
|
||||
offline. Installation requires a secure origin (HTTPS, or `localhost` for development).
|
||||
network-first strategy for navigations, retries a failed navigation once, and falls
|
||||
back to `static/offline.html` when both attempts fail. Installation requires a secure origin (HTTPS, or `localhost` for development).
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
@@ -1188,6 +1208,19 @@ Two background services bridge persisted state onto the bus so the interface upd
|
||||
- Session-scoped browser context with per-test cookie clearing
|
||||
- `page.locator(...).wait_for(state="visible")` preferred over bare selectors
|
||||
|
||||
## Media embeds (YouTube and YouTube Music)
|
||||
|
||||
A bare YouTube or YouTube Music link in a post, comment, message or description renders as an inline player. YouTube Music has no player of its own: its track and playlist ids are YouTube's, so the embed is YouTube's own player, shown with an "Open in YouTube Music" caption that links back to the original.
|
||||
|
||||
| Link | Result |
|
||||
|------|--------|
|
||||
| `youtube.com/watch?v=ID`, `youtu.be/ID`, `/embed/ID`, `/v/ID`, `/shorts/ID`, `m.` and `-nocookie` hosts | Video player; `t=` / `start=` seconds are kept |
|
||||
| `music.youtube.com/watch?v=ID[&list=ID]` | Track player with the caption, queue kept when `list=` is a valid playlist id |
|
||||
| `music.youtube.com/playlist?list=ID`, `youtube.com/playlist?list=ID` | Playlist or album player (`OLAK5uy...` album ids work) |
|
||||
| `music.youtube.com/browse/...`, `/channel/...`, `/search`, `/library` | Plain link: these URLs carry no playlist id |
|
||||
|
||||
Parsing is by exact host allowlist and query parameter, in any order, so look-alike hosts such as `evilyoutube.com` never embed. A single post renders at most 3 music embeds; further music links stay plain links. The URL rules live in one place, `devplacepy/embeds.py`; the browser gets them through the generated `static/js/embed-spec.js` (`devplace embeds-sync` after changing the table, never hand-edit) and `static/js/EmbedParser.js`. A unit test fails when the generated file drifts and an e2e test runs one shared case table through both parsers.
|
||||
|
||||
## Avatars
|
||||
|
||||
Uses [Multiavatar](https://github.com/multiavatar/multiavatar-python) - generates deterministic SVG avatars locally from a seed string (the username). No external API calls, no network dependency. Generation takes <5ms. Results are cached in-memory (cleared on server restart).
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AdminSection:
|
||||
key: str
|
||||
label: str
|
||||
href: str
|
||||
icon: str
|
||||
|
||||
|
||||
ADMIN_SECTIONS: tuple[AdminSection, ...] = (
|
||||
AdminSection("users", "Users", "/admin/users", "\U0001f465"),
|
||||
AdminSection("news", "News", "/admin/news", "\U0001f4f0"),
|
||||
AdminSection("media", "Media", "/admin/media", "\U0001f5bc️"),
|
||||
AdminSection("moderation", "Moderation", "/admin/moderation", "\U0001f6a9"),
|
||||
AdminSection("trash", "Trash", "/admin/trash", "\U0001f5d1️"),
|
||||
AdminSection("services", "Services", "/admin/services", "⚙️"),
|
||||
AdminSection("gateway", "Gateway", "/admin/gateway", "\U0001f500"),
|
||||
AdminSection("containers", "Containers", "/admin/containers", "\U0001f4e6"),
|
||||
AdminSection("workspaces", "Workspaces", "/admin/workspaces", "\U0001f4bb"),
|
||||
AdminSection("devii-tasks", "Devii tasks", "/admin/devii-tasks", "⏰"),
|
||||
AdminSection("communities", "Communities", "/admin/communities", "\U0001f310"),
|
||||
AdminSection("nodes", "Community nodes", "/admin/nodes", "\U0001f4e1"),
|
||||
AdminSection("bots", "Bot Monitor", "/admin/bots", "\U0001f916"),
|
||||
AdminSection("game", "Code Farm", "/admin/game", "\U0001f69c"),
|
||||
AdminSection("ai-usage", "AI usage", "/admin/ai-usage", "\U0001f4ca"),
|
||||
AdminSection("statistics", "Statistics", "/admin/statistics", "\U0001f4c8"),
|
||||
AdminSection("audit-log", "Audit Log", "/admin/audit-log", "\U0001f4cb"),
|
||||
AdminSection("backups", "Backups", "/admin/backups", "\U0001f4be"),
|
||||
AdminSection("issues", "Ticket Planning", "/admin/issues/planning", "\U0001f5c2️"),
|
||||
AdminSection("notifications", "Notifications", "/admin/notifications", "\U0001f514"),
|
||||
AdminSection("settings", "Settings", "/admin/settings", "\U0001f527"),
|
||||
)
|
||||
|
||||
ADMIN_SECTION_KEYS: frozenset[str] = frozenset(section.key for section in ADMIN_SECTIONS)
|
||||
@@ -44,7 +44,7 @@ from devplacepy.cli.containers import (
|
||||
cmd_containers_gc_workspaces,
|
||||
)
|
||||
from devplacepy.cli.quiz import cmd_quiz_prune
|
||||
from devplacepy.cli.migrate import cmd_emoji_sync, cmd_migrate_data
|
||||
from devplacepy.cli.migrate import cmd_embeds_sync, cmd_emoji_sync, cmd_migrate_data
|
||||
from devplacepy.cli.system import cmd_system_prune
|
||||
|
||||
__all__ = [
|
||||
@@ -90,6 +90,7 @@ __all__ = [
|
||||
"cmd_containers_prune_builds",
|
||||
"cmd_containers_gc_workspaces",
|
||||
"cmd_quiz_prune",
|
||||
"cmd_embeds_sync",
|
||||
"cmd_emoji_sync",
|
||||
"cmd_migrate_data",
|
||||
"cmd_system_prune",
|
||||
|
||||
@@ -131,6 +131,14 @@ def cmd_emoji_sync(args):
|
||||
print(f"Wrote {count} emoji shortcodes to {EMOJI_JS_PATH}")
|
||||
|
||||
|
||||
def cmd_embeds_sync(args):
|
||||
from devplacepy.embeds import EMBED_JS_PATH, EmbedParser
|
||||
|
||||
count = EmbedParser.write_module()
|
||||
_audit_cli("cli.embeds.sync", f"CLI regenerated the embed spec ({count} keys)", metadata={"count": count})
|
||||
print(f"Wrote {count} embed spec keys to {EMBED_JS_PATH}")
|
||||
|
||||
|
||||
def cmd_migrate_data(args):
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -221,6 +229,11 @@ def register_migrate(subparsers):
|
||||
help="Regenerate static/js/emoji-shortcodes.js from the emoji library",
|
||||
).set_defaults(func=cmd_emoji_sync)
|
||||
|
||||
subparsers.add_parser(
|
||||
"embeds-sync",
|
||||
help="Regenerate static/js/embed-spec.js from devplacepy/embeds.py",
|
||||
).set_defaults(func=cmd_embeds_sync)
|
||||
|
||||
migrate = subparsers.add_parser(
|
||||
"migrate-data",
|
||||
help="Relocate legacy runtime files into the consolidated data/ directory",
|
||||
|
||||
@@ -92,6 +92,10 @@ INTERNAL_IMAGE_MODEL = "molodetz-img-small"
|
||||
|
||||
AQUALITY_NEWS_GRADING_URL = "https://aquality.cloud.pravda.education/v1/chat/completions"
|
||||
AQUALITY_NEWS_GRADING_MODEL = "aquality"
|
||||
# The aquality scoring service above was retired (no backend serves that
|
||||
# hostname anymore), so news grading defaults back to the internal gateway.
|
||||
# Kept so the settings migration can recognize and heal stale values.
|
||||
NEWS_GRADING_MODEL_DEFAULT = "free"
|
||||
|
||||
AWARD_GIVE_COOLDOWN_HOURS_DEFAULT = 24
|
||||
AWARD_RECEIVE_COOLDOWN_HOURS_DEFAULT = 24
|
||||
|
||||
+140
-5
@@ -41,6 +41,13 @@ from devplacepy.database import (
|
||||
db,
|
||||
get_user_recent_items,
|
||||
invalidate_user_recent_cache,
|
||||
VISIBILITY_COLUMNS,
|
||||
TARGET_TABLES,
|
||||
item_visibility,
|
||||
is_listed,
|
||||
is_open,
|
||||
listed_clause,
|
||||
normalize_visibility,
|
||||
)
|
||||
from devplacepy.utils import (
|
||||
time_ago,
|
||||
@@ -67,7 +74,14 @@ from devplacepy.services.moderation.screening import (
|
||||
screen_fields,
|
||||
)
|
||||
|
||||
CREATE_METADATA_KEYS = ("project_type", "is_private", "language", "topic", "status")
|
||||
CREATE_METADATA_KEYS = (
|
||||
"project_type",
|
||||
"is_private",
|
||||
"language",
|
||||
"topic",
|
||||
"status",
|
||||
"visibility",
|
||||
)
|
||||
|
||||
BOOKMARKABLE_TYPES = {"post", "gist", "project", "news", "quiz"}
|
||||
REACTABLE_TYPES = {"post", "comment", "gist", "project", "quiz"}
|
||||
@@ -95,6 +109,26 @@ def is_owner(item: dict | None, user: dict | None) -> bool:
|
||||
return bool(item and user and item["user_uid"] == user["uid"])
|
||||
|
||||
|
||||
def can_view_item(table_name: str, item: dict | None, user: dict | None) -> bool:
|
||||
if not item:
|
||||
return False
|
||||
if is_open(table_name, item):
|
||||
return True
|
||||
if not user:
|
||||
return False
|
||||
return bool(item.get("user_uid") == user.get("uid") or is_admin(user))
|
||||
|
||||
|
||||
def can_view_target(target_type: str, target_uid: str, user: dict | None) -> bool:
|
||||
table_name = TARGET_TABLES.get(target_type)
|
||||
if not table_name:
|
||||
return True
|
||||
item = resolve_by_slug(get_table(table_name), target_uid)
|
||||
if not item:
|
||||
return True
|
||||
return can_view_item(table_name, item, user)
|
||||
|
||||
|
||||
def mature_hidden_by_default() -> bool:
|
||||
return get_int_setting("moderation_mature_default_hidden", 1) != 0
|
||||
|
||||
@@ -128,6 +162,8 @@ def _owner_is_admin(project: dict) -> bool:
|
||||
def can_view_project(project: dict | None, user: dict | None) -> bool:
|
||||
if not project:
|
||||
return False
|
||||
if not can_view_item("projects", project, user):
|
||||
return False
|
||||
if not project.get("is_private"):
|
||||
return True
|
||||
if is_owner(project, user):
|
||||
@@ -138,14 +174,17 @@ def can_view_project(project: dict | None, user: dict | None) -> bool:
|
||||
|
||||
|
||||
def get_user_sidebar_gists(user_uid: str, limit: int = 5) -> list[dict]:
|
||||
return get_user_recent_items("gists", user_uid)[:limit]
|
||||
rows = get_user_recent_items("gists", user_uid)
|
||||
return [g for g in rows if is_listed("gists", g)][:limit]
|
||||
|
||||
|
||||
def get_user_sidebar_projects(
|
||||
user_uid: str, viewer: dict | None, limit: int = 5
|
||||
) -> list[dict]:
|
||||
rows = get_user_recent_items("projects", user_uid)
|
||||
visible = [p for p in rows if can_view_project(p, viewer)]
|
||||
visible = [
|
||||
p for p in rows if is_listed("projects", p) and can_view_project(p, viewer)
|
||||
]
|
||||
return visible[:limit]
|
||||
|
||||
|
||||
@@ -246,6 +285,13 @@ def first_image_url(item: dict, attachments: list | None) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _with_visibility(table_name: str, fields: dict) -> dict:
|
||||
column = VISIBILITY_COLUMNS.get(table_name)
|
||||
if not column:
|
||||
return fields
|
||||
return {**fields, column: normalize_visibility(fields.get(column))}
|
||||
|
||||
|
||||
def create_content_item(
|
||||
table_name: str,
|
||||
target_type: str,
|
||||
@@ -258,6 +304,7 @@ def create_content_item(
|
||||
attachment_uids: list | None,
|
||||
request=None,
|
||||
) -> tuple[str, str]:
|
||||
fields = _with_visibility(table_name, fields)
|
||||
screening = screen_fields(table_name, fields)
|
||||
refuse_if_blocked(screening)
|
||||
uid = generate_uid()
|
||||
@@ -284,7 +331,10 @@ def create_content_item(
|
||||
award_rewards(user["uid"], xp, badge)
|
||||
if attachment_uids:
|
||||
link_attachments(attachment_uids, target_type, uid)
|
||||
create_mention_notifications(mention_text, user["uid"], f"/{table_name}/{slug}")
|
||||
if is_open(table_name, fields):
|
||||
create_mention_notifications(
|
||||
mention_text, user["uid"], f"/{table_name}/{slug}"
|
||||
)
|
||||
logger.info(f"{target_type} {uid} created by {user['username']}")
|
||||
label = fields.get("title") or slug
|
||||
links = [audit.target(target_type, uid, label)]
|
||||
@@ -667,6 +717,81 @@ def detail_context(
|
||||
return context
|
||||
|
||||
|
||||
def apply_visibility(
|
||||
request,
|
||||
table_name: str,
|
||||
target_type: str,
|
||||
user: dict,
|
||||
item: dict,
|
||||
value: str,
|
||||
) -> tuple[str, str]:
|
||||
column = VISIBILITY_COLUMNS[table_name]
|
||||
old_value = item_visibility(table_name, item)
|
||||
new_value = normalize_visibility(value)
|
||||
get_table(table_name).update({"uid": item["uid"], column: new_value}, ["uid"])
|
||||
owner_uid = item.get("user_uid")
|
||||
if owner_uid:
|
||||
invalidate_user_recent_cache(table_name, owner_uid)
|
||||
if table_name == "posts":
|
||||
clear_user_post_count(owner_uid)
|
||||
if table_name == "projects":
|
||||
from devplacepy.templating import clear_user_projects_cache
|
||||
|
||||
clear_user_projects_cache(owner_uid)
|
||||
label = item.get("title") or item["uid"]
|
||||
logger.info(
|
||||
f"{target_type} {item['uid']} visibility {old_value} -> {new_value} by {user['username']}"
|
||||
)
|
||||
audit.record(
|
||||
request,
|
||||
f"{target_type}.visibility.set",
|
||||
user=user,
|
||||
target_type=target_type,
|
||||
target_uid=item["uid"],
|
||||
target_label=label,
|
||||
old_value=old_value,
|
||||
new_value=new_value,
|
||||
summary=f"{user['username']} set {target_type} {label} to {new_value}",
|
||||
links=[audit.target(target_type, item["uid"], label)],
|
||||
)
|
||||
return old_value, new_value
|
||||
|
||||
|
||||
def set_visibility_item(
|
||||
request,
|
||||
table_name: str,
|
||||
target_type: str,
|
||||
user: dict,
|
||||
slug: str,
|
||||
value: str,
|
||||
redirect_fail: str,
|
||||
):
|
||||
from devplacepy.responses import action_result, wants_json, json_error
|
||||
|
||||
item = resolve_by_slug(get_table(table_name), slug)
|
||||
if not item or not is_admin(user):
|
||||
audit.record(
|
||||
request,
|
||||
f"{target_type}.visibility.set",
|
||||
user=user,
|
||||
result="denied",
|
||||
target_type=target_type,
|
||||
target_uid=item["uid"] if item else slug,
|
||||
target_label=item.get("title") if item else slug,
|
||||
summary=f"{user['username']} denied changing visibility of {target_type} {slug}",
|
||||
)
|
||||
if wants_json(request):
|
||||
return json_error(403, "Not allowed")
|
||||
return RedirectResponse(url=redirect_fail, status_code=302)
|
||||
_, new_value = apply_visibility(request, table_name, target_type, user, item, value)
|
||||
url = f"/{table_name}/{item.get('slug') or item['uid']}"
|
||||
return action_result(
|
||||
request,
|
||||
url,
|
||||
data={"uid": item["uid"], "visibility": new_value, "url": url},
|
||||
)
|
||||
|
||||
|
||||
def edit_content_item(
|
||||
request,
|
||||
table_name: str,
|
||||
@@ -681,6 +806,7 @@ def edit_content_item(
|
||||
kind = target_type or table_name.rstrip("s")
|
||||
table = get_table(table_name)
|
||||
item = resolve_by_slug(table, slug)
|
||||
visibility_value = update_fields.pop("visibility", None)
|
||||
if not is_owner(item, user):
|
||||
audit.record(
|
||||
request,
|
||||
@@ -703,6 +829,10 @@ def edit_content_item(
|
||||
}
|
||||
table.update({"uid": item["uid"], **update_fields}, ["uid"])
|
||||
invalidate_user_recent_cache(table_name, item["user_uid"])
|
||||
if visibility_value is not None and normalize_visibility(
|
||||
visibility_value
|
||||
) != item_visibility(table_name, item):
|
||||
apply_visibility(request, table_name, kind, user, item, visibility_value)
|
||||
record_screening(
|
||||
screening,
|
||||
target_type=kind,
|
||||
@@ -817,6 +947,8 @@ def load_detail(
|
||||
item = resolve_by_slug(get_table(table_name), slug)
|
||||
if not item:
|
||||
return None
|
||||
if not can_view_item(table_name, item, user):
|
||||
return None
|
||||
if user and item["user_uid"] in get_blocked_uids(user["uid"]):
|
||||
return None
|
||||
author = get_users_by_uids([item["user_uid"]]).get(item["user_uid"])
|
||||
@@ -898,7 +1030,10 @@ def enrich_items(
|
||||
|
||||
|
||||
def count_project_devlog(project_uid: str) -> int:
|
||||
return get_table("posts").count(project_uid=project_uid, deleted_at=None)
|
||||
posts = get_table("posts")
|
||||
listed = listed_clause(posts)
|
||||
clauses = [listed] if listed is not None else []
|
||||
return posts.count(*clauses, project_uid=project_uid, deleted_at=None)
|
||||
|
||||
|
||||
def get_project_devlog(
|
||||
|
||||
@@ -198,8 +198,8 @@ Site settings are seeded on startup (`site_settings` table):
|
||||
| `site_name` / `site_description` / `site_tagline` | DevPlace branding | General site metadata |
|
||||
| `news_grade_threshold` | `"7"` | Minimum AI grade for auto-publish |
|
||||
| `news_api_url` | `"https://news.app.molodetz.nl/api"` | News source API |
|
||||
| `news_ai_url` | `"https://aquality.cloud.pravda.education/v1/chat/completions"` | AI grading endpoint - the free, local aquality quality model by default (see `devplacepy/services/news/CLAUDE.md`) |
|
||||
| `news_ai_model` | `"aquality"` | AI model identifier |
|
||||
| `news_ai_url` | internal AI gateway (`INTERNAL_GATEWAY_URL`) | AI grading endpoint - the gateway's free model pool by default (see `devplacepy/services/news/CLAUDE.md`) |
|
||||
| `news_ai_model` | `"free"` | AI model identifier |
|
||||
| `max_upload_size_mb` / `allowed_file_types` / `max_attachments_per_resource` | `"10"` / `""` / `"10"` | Upload limits |
|
||||
| `rate_limit_per_minute` | `"60"` | Mutating requests per IP per window (`main.py` middleware); a `429` carries a `Retry-After: <window>` header |
|
||||
| `rate_limit_window_seconds` | `"60"` | Rolling window for the rate limit (also the `Retry-After` value) |
|
||||
@@ -246,3 +246,30 @@ Operational settings - read sites and rules:
|
||||
**Booleans are `<select>`, never checkboxes.** The settings save handler (`admin.py`) skips empty form values so empty fields don't clobber existing rows. An unchecked checkbox submits nothing, so it could never be turned off - `registration_open` and `maintenance_mode` use `<option value="1">`/`<option value="0">` so a value is always submitted.
|
||||
|
||||
|
||||
|
||||
## Content visibility (`database/visibility.py`, the single specification)
|
||||
|
||||
Posts, gists, projects and news carry one status from `VISIBILITY_VALUES`: `published` (default), `hidden`, `draft`, `offline`. The module is the specification; routers, forms, schemas, templates, Devii and the docs import it and never repeat a literal.
|
||||
|
||||
| Status | Listed (feed, search, topics, profiles, sitemap, counters) | Opens by link | Who |
|
||||
|---|---|---|---|
|
||||
| `published` | yes | yes | everyone |
|
||||
| `hidden` | no | yes, `noindex,nofollow` | everyone with the link |
|
||||
| `offline` | no | no (404) | owner and administrators |
|
||||
| `draft` | no | no (404) | owner and administrators; accepted by every API but not offered by the UI (`VISIBILITY_CHOICES` is the three user-facing values) |
|
||||
|
||||
- **Column.** `VISIBILITY_COLUMNS = {posts, gists, projects: "visibility", news: "status"}`. News reuses its existing `status` column (it already held `published`/`draft`), so no news migration exists. `ensure_visibility_columns()` (called from `init_db`) adds the text column and `idx_<table>_visibility` ONLY to tables that already exist; it must never create the table (an empty `posts` table breaks every `"posts" in db.tables` guard and the first-signup schema). A NULL or unknown value reads as `published` everywhere (`normalize_visibility`, `item_visibility`), so legacy rows needed no backfill and `init_db` does no per-row work.
|
||||
- **Every read goes through one of three choke points, never an inline compare.** `paginate`/`paginate_diverse` auto-append `listed_clause(table)` for any table in `VISIBILITY_COLUMNS` (`include_unlisted_of=<uid>` ORs in that owner's own rows, used for an owner's own profile/listing); `listed_clause(table, own_uid)` for `count()`/`find()` call sites; `listed_sql(table, alias)` for raw SQL (it returns `1 = 1` when the table or column does not exist yet, so a fresh database never errors). The predicates `is_listed`/`is_open` work on a row dict. Single-item access is `content.can_view_item(table, item, user)` (open, or owner, or admin) and `content.can_view_target(target_type, uid, user)` for polymorphic targets; `load_detail` applies the first, so every detail page 404s without per-route code. `can_view_project` composes visibility first, then the `is_private` rule.
|
||||
- **Writes.** `content.apply_visibility` is the one writer (column update, cache invalidation, `<kind>.visibility.set` audit with old/new value). `edit_content_item` pops the optional `visibility` form field and calls `apply_visibility` when it changed; `content.set_visibility_item` exists only for the administrator news route (news has no edit form). `create_content_item` stamps the column (`_with_visibility`) and skips mention notifications for non-open items.
|
||||
- **Public counters** (`get_user_post_count`, `get_site_stats`, `get_post_counts_by_user_uids`, trending topics, topic hub counts, sitemap author counts, happy-404 pool) count listed rows only, so the owner's own profile count excludes their hidden and offline posts.
|
||||
- **Backwards compatibility (hard rule).** `visibility` is optional on every API: `VisibilityCreateMixin` defaults to `published`, `VisibilityEditMixin` defaults to `None` meaning keep the current value (never reset to published), NULL rows read as published, and the docs/Devii parameter descriptions say so (`tests/unit/docs_api/visibility.py`, `tests/api/content/visibility.py` legacy-client tests). Never make the field required.
|
||||
- **Outputs.** `VisibleOut` (`schemas/content.py`) gives `PostOut`/`GistOut`/`ProjectOut` a `visibility` that reads NULL as `published`. Forms use `VisibilityCreateMixin` (default `published`) and `VisibilityEditMixin` (`None` keeps the current value); `VisibilityForm{value}` backs the admin news route.
|
||||
- **Adding a table.** Put it in `VISIBILITY_COLUMNS` and `TARGET_TABLES`, add its form mixin and Out base, apply `listed_clause`/`listed_sql` to each new read, and extend `tests/api/content/visibility.py`. `tests/unit/database/visibility.py` asserts the spec is self-consistent.
|
||||
|
||||
## Notification grouping, throttling and keyboard settings
|
||||
|
||||
`users` carries `enter_to_submit`, `notif_grouping_enabled` and `notif_throttle_enabled` (all default 1, ensured in `schema.py`, owner-or-admin toggles on the profile page).
|
||||
|
||||
## Data CLI
|
||||
|
||||
`devplace apikey backfill` assigns keys to users that lack one. `devplace attachments prune` removes orphan attachment records/files. `devplace system prune` (with `--dry-run`) purges soft-deleted attachment/project-file blobs, sweeps blob files with zero DB reference, and GCs orphaned container workspace dirs. `database/notifications.py` owns the single grouping specification: `NOTIFICATION_GROUP_KEYS` (actor / target / single per type, unlisted types are single), `notification_group_key(row)`, and `count_unread_groups(user_uid)` (groups when enabled, rows when off). Every badge producer calls `count_unread_groups`; none counts rows itself. The push payload (`unread`, presented as `aps.badge`) is one of them. The `notifications` column set is ensured in `init_db` like `messages` (listed in `tests/unit/database/schema.py` `FILTERED_COLUMNS`), and the unread read `find(user_uid=..., read=False)` is served by `idx_notifications_user_read (user_uid, read)`. Delivery throttling (item 5) reuses the same key.
|
||||
|
||||
@@ -7,6 +7,7 @@ from .settings import _settings_cache, get_setting, get_int_setting, set_setting
|
||||
from .users import get_users_by_uids, _admins_cache, invalidate_admins_cache, get_admin_uids, set_user_timezone, set_last_seen, get_online_users, get_primary_admin_uid, is_account_active, search_users_by_username
|
||||
from .relations import _relations_cache, get_user_relations, get_blocked_uids, get_muted_uids, get_silenced_uids, invalidate_user_relations
|
||||
from .pagination import PAGE_SIZE, paginate, interleave_by_author, paginate_diverse, get_user_post_count, clear_user_post_count, build_pagination
|
||||
from .visibility import PUBLISHED, HIDDEN, DRAFT, OFFLINE, VISIBILITY_VALUES, VISIBILITY_CHOICES, DEFAULT_VISIBILITY, LISTED_VALUES, OPEN_VALUES, VISIBILITY_COLUMNS, TARGET_TABLES, VISIBILITY_LABELS, VISIBILITY_HINTS, normalize_visibility, visibility_column, item_visibility, is_listed, is_open, listed_clause, listed_sql, ensure_visibility_columns, visibility_options, visibility_label
|
||||
from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns, soft_delete, soft_delete_in, restore, purge, list_deleted, count_deleted, restore_event, purge_event
|
||||
from .engagement import _comment_count_cache, get_comment_counts_by_post_uids, get_post_counts_by_user_uids, get_vote_counts, get_user_votes, get_reactions_by_targets, get_user_bookmarks, get_user_notes, get_polls_by_post_uids, get_poll_for_post
|
||||
from .usage import _add_usage, _get_usage, add_correction_usage, get_correction_usage, add_modifier_usage, get_modifier_usage, NEWS_USAGE_KEY, add_news_usage, get_news_usage, ISSUE_USAGE_KEY, add_issue_usage, get_issue_usage, SEO_USAGE_KEY, add_seo_usage, get_seo_usage, AWARD_USAGE_KEY, add_award_usage, get_award_usage
|
||||
@@ -31,7 +32,8 @@ from .seo_meta import SEO_META_TYPES, get_seo_metadata, get_seo_metadata_batch,
|
||||
from .activity import record_activity, record_unique_activity, get_user_activity, _activity_cache, _ACTIVITY_TABLES, get_activity_calendar, _activity_level, get_first_activity_date, HEATMAP_WEEKS, get_activity_heatmap, get_activity_months, get_streaks
|
||||
from .customization import CUSTOMIZATION_GLOBAL_SCOPE, CUSTOMIZATION_LANGS, _customizations_cache, _customization_key, CUSTOMIZATION_PREF_COLUMNS, get_customization_prefs, set_customization_pref, get_custom_overrides, get_custom_override, list_custom_overrides, set_custom_override, delete_custom_override
|
||||
from .email import EMAIL_ACCOUNT_DEFAULTS, list_email_accounts, get_email_account, set_email_account, delete_email_account
|
||||
from .notifications import NOTIFICATION_TYPES, NOTIFICATION_CHANNELS, _NOTIFICATION_CHANNEL_COLUMNS, _NOTIFICATION_CHANNEL_DEFAULTS, _NOTIFICATION_TYPE_KEYS, _notification_prefs_cache, _notification_default, get_notification_default, set_notification_default, _notification_overrides, notification_enabled, get_notification_prefs, set_notification_pref, reset_notification_prefs, mark_notifications_read_by_target
|
||||
from .notifications import NOTIFICATION_TYPES, NOTIFICATION_CHANNELS, _NOTIFICATION_CHANNEL_COLUMNS, _NOTIFICATION_CHANNEL_DEFAULTS, _NOTIFICATION_TYPE_KEYS, _notification_prefs_cache, _notification_default, get_notification_default, set_notification_default, _notification_overrides, notification_enabled, get_notification_prefs, set_notification_pref, reset_notification_prefs, mark_notifications_read_by_target, NOTIFICATION_GROUP_KEYS, notification_group_key, notification_throttle_key, get_notification_grouping_enabled, set_notification_grouping_enabled, get_notification_throttle_enabled, set_notification_throttle_enabled, count_unread_groups
|
||||
from .enter_submit import get_enter_to_submit, set_enter_to_submit
|
||||
from .forks import record_fork, get_fork_parent, count_forks, soft_delete_fork_relations, delete_fork_relations
|
||||
from .follows import get_follow_counts, get_follow_list, get_following_among
|
||||
from .deepsearch import _ds_now, create_deepsearch_session, update_deepsearch_session, get_deepsearch_session, list_deepsearch_sessions, add_deepsearch_message, get_deepsearch_messages, get_cached_deepsearch_url, upsert_deepsearch_url_cache
|
||||
@@ -143,6 +145,29 @@ __all__ = [
|
||||
"clear_user_post_count",
|
||||
"build_pagination",
|
||||
"SOFT_DELETE_TABLES",
|
||||
"PUBLISHED",
|
||||
"HIDDEN",
|
||||
"DRAFT",
|
||||
"OFFLINE",
|
||||
"VISIBILITY_VALUES",
|
||||
"VISIBILITY_CHOICES",
|
||||
"DEFAULT_VISIBILITY",
|
||||
"LISTED_VALUES",
|
||||
"OPEN_VALUES",
|
||||
"VISIBILITY_COLUMNS",
|
||||
"TARGET_TABLES",
|
||||
"VISIBILITY_LABELS",
|
||||
"VISIBILITY_HINTS",
|
||||
"normalize_visibility",
|
||||
"visibility_column",
|
||||
"item_visibility",
|
||||
"is_listed",
|
||||
"is_open",
|
||||
"listed_clause",
|
||||
"listed_sql",
|
||||
"ensure_visibility_columns",
|
||||
"visibility_options",
|
||||
"visibility_label",
|
||||
"ensure_soft_delete_columns",
|
||||
"soft_delete",
|
||||
"soft_delete_in",
|
||||
@@ -227,6 +252,16 @@ __all__ = [
|
||||
"set_notification_pref",
|
||||
"reset_notification_prefs",
|
||||
"mark_notifications_read_by_target",
|
||||
"NOTIFICATION_GROUP_KEYS",
|
||||
"notification_group_key",
|
||||
"notification_throttle_key",
|
||||
"get_notification_grouping_enabled",
|
||||
"set_notification_grouping_enabled",
|
||||
"get_notification_throttle_enabled",
|
||||
"set_notification_throttle_enabled",
|
||||
"count_unread_groups",
|
||||
"get_enter_to_submit",
|
||||
"set_enter_to_submit",
|
||||
"record_fork",
|
||||
"get_fork_parent",
|
||||
"count_forks",
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .core import TTLCache, datetime, db, timedelta, timezone
|
||||
from .visibility import listed_sql
|
||||
|
||||
|
||||
def record_activity(user_uid: str, action: str) -> int:
|
||||
@@ -79,7 +80,7 @@ def get_activity_calendar(user_uid: str) -> dict:
|
||||
if sources:
|
||||
cutoff = (datetime.now(timezone.utc) - timedelta(days=364)).date().isoformat()
|
||||
union = " UNION ALL ".join(
|
||||
f"SELECT created_at FROM {table} WHERE user_uid = :u AND deleted_at IS NULL"
|
||||
f"SELECT created_at FROM {table} WHERE user_uid = :u AND deleted_at IS NULL AND {listed_sql(table)}"
|
||||
for table in sources
|
||||
)
|
||||
rows = db.query(
|
||||
|
||||
@@ -7,6 +7,7 @@ from collections import Counter
|
||||
from devplacepy.cache import TTLCache
|
||||
|
||||
from .core import db, get_table, or_
|
||||
from .visibility import listed_sql
|
||||
|
||||
_daily_topic_cache = TTLCache(ttl=60, max_size=1)
|
||||
_trending_cache = TTLCache(ttl=15, max_size=1)
|
||||
@@ -284,6 +285,7 @@ def get_trending_topics(limit: int = 6) -> list[dict]:
|
||||
return []
|
||||
rows = db.query(
|
||||
"SELECT topic FROM posts WHERE deleted_at IS NULL "
|
||||
f"AND {listed_sql('posts')} "
|
||||
"AND topic IS NOT NULL AND topic != '' "
|
||||
"ORDER BY created_at DESC LIMIT 200"
|
||||
)
|
||||
|
||||
@@ -16,6 +16,7 @@ from devplacepy.config import (
|
||||
DEFAULT_CORRECTION_PROMPT,
|
||||
DEFAULT_MODIFIER_PROMPT,
|
||||
INTERNAL_GATEWAY_URL,
|
||||
NEWS_GRADING_MODEL_DEFAULT,
|
||||
ensure_data_dirs,
|
||||
)
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .core import TTLCache, _in_clause, db, defaultdict
|
||||
from .visibility import listed_sql
|
||||
|
||||
|
||||
_comment_count_cache = TTLCache(ttl=15, max_size=10000)
|
||||
@@ -36,7 +37,7 @@ def get_post_counts_by_user_uids(user_uids):
|
||||
return {}
|
||||
placeholders, params = _in_clause(user_uids)
|
||||
rows = db.query(
|
||||
f"SELECT user_uid, COUNT(*) as c FROM posts WHERE user_uid IN ({placeholders}) AND deleted_at IS NULL GROUP BY user_uid",
|
||||
f"SELECT user_uid, COUNT(*) as c FROM posts WHERE user_uid IN ({placeholders}) AND deleted_at IS NULL AND {listed_sql('posts')} GROUP BY user_uid",
|
||||
**params,
|
||||
)
|
||||
return {r["user_uid"]: r["c"] for r in rows}
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .core import db, get_table
|
||||
|
||||
|
||||
def get_enter_to_submit(owner_id: str) -> bool:
|
||||
if "users" not in db.tables:
|
||||
return True
|
||||
user = db["users"].find_one(uid=owner_id)
|
||||
if user is None:
|
||||
return True
|
||||
value = user.get("enter_to_submit")
|
||||
return True if value is None else bool(value)
|
||||
|
||||
|
||||
def set_enter_to_submit(owner_id: str, enabled: bool) -> None:
|
||||
from devplacepy.utils import clear_user_cache
|
||||
|
||||
get_table("users").update(
|
||||
{"uid": owner_id, "enter_to_submit": 1 if enabled else 0}, ["uid"]
|
||||
)
|
||||
clear_user_cache(owner_id)
|
||||
@@ -592,14 +592,23 @@ def raise_flag(kind: str, *, device_uid: str = "", user_uid: str = "", detail: s
|
||||
return table.find_one(uid=uid)
|
||||
|
||||
|
||||
def list_flags(status: str = FLAG_OPEN) -> list[dict]:
|
||||
def list_flags(status: str = FLAG_OPEN, limit: int | None = None) -> list[dict]:
|
||||
window = {} if limit is None else {"_limit": limit}
|
||||
return list(
|
||||
get_table("node_flags").find(
|
||||
status=status, deleted_at=None, order_by=["-created_at"]
|
||||
status=status, deleted_at=None, order_by=["-created_at"], **window
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def count_flags(status: str = FLAG_OPEN) -> int:
|
||||
return get_table("node_flags").count(status=status, deleted_at=None)
|
||||
|
||||
|
||||
def count_all_devices() -> int:
|
||||
return get_table("node_devices").count(deleted_at=None)
|
||||
|
||||
|
||||
def resolve_flag(flag_uid: str, actor_uid: str) -> bool:
|
||||
return (
|
||||
_execute(
|
||||
|
||||
@@ -204,3 +204,87 @@ def mark_notifications_read_by_target(user_uid: str, target_url: str) -> int:
|
||||
|
||||
clear_unread_cache(user_uid)
|
||||
return len(ids)
|
||||
|
||||
|
||||
NOTIFICATION_GROUP_KEYS = {
|
||||
"comment": "actor",
|
||||
"reply": "actor",
|
||||
"thread": "actor",
|
||||
"mention": "actor",
|
||||
"message": "actor",
|
||||
"harvest_stolen": "actor",
|
||||
"award": "actor",
|
||||
"quiz_attempt": "actor",
|
||||
"vote": "target",
|
||||
"follow": "target",
|
||||
"badge": "single",
|
||||
"level": "single",
|
||||
"issue": "single",
|
||||
"reminder": "single",
|
||||
"battle": "single",
|
||||
"workspace": "single",
|
||||
"moderation": "single",
|
||||
"system": "single",
|
||||
"ai_quota_warning": "single",
|
||||
"game_upkeep": "single",
|
||||
}
|
||||
|
||||
|
||||
def notification_group_key(row) -> tuple:
|
||||
ntype = (row.get("type") or "")
|
||||
shape = NOTIFICATION_GROUP_KEYS.get(ntype, "single")
|
||||
if shape == "actor":
|
||||
return (ntype, row.get("related_uid") or "", row.get("target_url") or "")
|
||||
if shape == "target":
|
||||
if ntype == "follow":
|
||||
return (ntype,)
|
||||
return (ntype, row.get("target_url") or "")
|
||||
return (ntype, row.get("uid") or "")
|
||||
|
||||
|
||||
def notification_throttle_key(user_uid: str, notification_type: str, related_uid: str, target_url) -> tuple:
|
||||
row = {"type": notification_type, "related_uid": related_uid or "", "target_url": target_url or "", "uid": ""}
|
||||
return (user_uid,) + notification_group_key(row)
|
||||
|
||||
|
||||
def _user_bool_pref(user_uid: str, column: str) -> bool:
|
||||
if "users" not in db.tables:
|
||||
return True
|
||||
user = db["users"].find_one(uid=user_uid)
|
||||
if user is None:
|
||||
return True
|
||||
value = user.get(column)
|
||||
return True if value is None else bool(value)
|
||||
|
||||
|
||||
def get_notification_grouping_enabled(user_uid: str) -> bool:
|
||||
return _user_bool_pref(user_uid, "notif_grouping_enabled")
|
||||
|
||||
|
||||
def get_notification_throttle_enabled(user_uid: str) -> bool:
|
||||
return _user_bool_pref(user_uid, "notif_throttle_enabled")
|
||||
|
||||
|
||||
def _set_user_bool_pref(user_uid: str, column: str, enabled: bool) -> None:
|
||||
from devplacepy.utils import clear_user_cache
|
||||
|
||||
get_table("users").update({"uid": user_uid, column: 1 if enabled else 0}, ["uid"])
|
||||
clear_user_cache(user_uid)
|
||||
bump_cache_version("notif_prefs")
|
||||
|
||||
|
||||
def set_notification_grouping_enabled(user_uid: str, enabled: bool) -> None:
|
||||
_set_user_bool_pref(user_uid, "notif_grouping_enabled", enabled)
|
||||
|
||||
|
||||
def set_notification_throttle_enabled(user_uid: str, enabled: bool) -> None:
|
||||
_set_user_bool_pref(user_uid, "notif_throttle_enabled", enabled)
|
||||
|
||||
|
||||
def count_unread_groups(user_uid: str) -> int:
|
||||
if "notifications" not in db.tables:
|
||||
return 0
|
||||
rows = list(get_table("notifications").find(user_uid=user_uid, read=False))
|
||||
if not get_notification_grouping_enabled(user_uid):
|
||||
return len(rows)
|
||||
return len({notification_group_key(row) for row in rows})
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from devplacepy.cache import TTLCache
|
||||
from .core import db, get_table
|
||||
from .relations import get_blocked_uids
|
||||
from .visibility import listed_clause
|
||||
|
||||
|
||||
PAGE_SIZE = 25
|
||||
@@ -19,10 +20,14 @@ def paginate(
|
||||
cursor_field="created_at",
|
||||
viewer_uid=None,
|
||||
limit=PAGE_SIZE,
|
||||
include_unlisted_of=None,
|
||||
**filters,
|
||||
):
|
||||
order = order or ["-" + cursor_field]
|
||||
clauses = list(clauses)
|
||||
listed = listed_clause(table, own_uid=include_unlisted_of)
|
||||
if listed is not None:
|
||||
clauses.append(listed)
|
||||
if table.has_column("deleted_at") and "deleted_at" not in filters:
|
||||
clauses.append(table.table.columns.deleted_at.is_(None))
|
||||
if viewer_uid and table.has_column("user_uid"):
|
||||
@@ -66,6 +71,7 @@ def paginate_diverse(
|
||||
uid_key="user_uid",
|
||||
viewer_uid=None,
|
||||
limit=PAGE_SIZE,
|
||||
include_unlisted_of=None,
|
||||
**filters,
|
||||
):
|
||||
rows, next_cursor = paginate(
|
||||
@@ -76,6 +82,7 @@ def paginate_diverse(
|
||||
cursor_field=cursor_field,
|
||||
viewer_uid=viewer_uid,
|
||||
limit=limit,
|
||||
include_unlisted_of=include_unlisted_of,
|
||||
**filters,
|
||||
)
|
||||
return interleave_by_author(rows, uid_key=uid_key), next_cursor
|
||||
@@ -91,7 +98,9 @@ def get_user_post_count(user_uid: str) -> int:
|
||||
return cached
|
||||
if "posts" not in db.tables:
|
||||
return 0
|
||||
count = get_table("posts").count(user_uid=user_uid, deleted_at=None)
|
||||
listed = listed_clause(get_table("posts"))
|
||||
clauses = [listed] if listed is not None else []
|
||||
count = get_table("posts").count(*clauses, user_uid=user_uid, deleted_at=None)
|
||||
_user_post_count_cache.set(user_uid, count)
|
||||
return count
|
||||
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from devplacepy.services.nodes.spec import SPEC as NODES_SPEC
|
||||
from .core import AQUALITY_NEWS_GRADING_MODEL, AQUALITY_NEWS_GRADING_URL, DEFAULT_CORRECTION_PROMPT, DEFAULT_MODIFIER_PROMPT, INTERNAL_GATEWAY_URL, _drop_index, _index, _uid_index, db, defaultdict, get_table, logger
|
||||
from .core import AQUALITY_NEWS_GRADING_MODEL, AQUALITY_NEWS_GRADING_URL, DEFAULT_CORRECTION_PROMPT, DEFAULT_MODIFIER_PROMPT, INTERNAL_GATEWAY_URL, NEWS_GRADING_MODEL_DEFAULT, _drop_index, _index, _uid_index, db, defaultdict, get_table, logger
|
||||
from .settings import get_setting, set_setting
|
||||
from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns
|
||||
from .visibility import ensure_visibility_columns
|
||||
from .ranking import _authors_cache
|
||||
|
||||
|
||||
@@ -181,6 +182,8 @@ def init_db():
|
||||
("created_at", ""),
|
||||
("updated_at", ""),
|
||||
("client_id", ""),
|
||||
("deleted_at", ""),
|
||||
("deleted_by", ""),
|
||||
):
|
||||
if not messages.has_column(column):
|
||||
messages.create_column_by_example(column, example)
|
||||
@@ -197,6 +200,19 @@ def init_db():
|
||||
)
|
||||
_index(db, "messages", "idx_messages_updated_at", ["updated_at"])
|
||||
_index(db, "messages", "idx_messages_dedupe", ["sender_uid", "client_id"])
|
||||
notifications = get_table("notifications")
|
||||
for column, example in (
|
||||
("uid", ""),
|
||||
("user_uid", ""),
|
||||
("type", ""),
|
||||
("message", ""),
|
||||
("related_uid", ""),
|
||||
("target_url", ""),
|
||||
("read", False),
|
||||
("created_at", ""),
|
||||
):
|
||||
if not notifications.has_column(column):
|
||||
notifications.create_column_by_example(column, example)
|
||||
_index(db, "notifications", "idx_notifications_user", ["user_uid"])
|
||||
_index(db, "notifications", "idx_notifications_user_read", ["user_uid", "read"])
|
||||
push_registration = get_table("push_registration")
|
||||
@@ -584,6 +600,41 @@ def init_db():
|
||||
_index(
|
||||
db, "devii_tasks", "idx_devii_tasks_due", ["enabled", "status", "next_run_at"]
|
||||
)
|
||||
agent_runs = get_table("devii_agent_runs")
|
||||
for column, example in (
|
||||
("uid", ""),
|
||||
("owner_kind", ""),
|
||||
("owner_id", ""),
|
||||
("channel", ""),
|
||||
("category", ""),
|
||||
("label", ""),
|
||||
("task", ""),
|
||||
("output_format", ""),
|
||||
("tools_extra", ""),
|
||||
("effort", ""),
|
||||
("task_scope", 0),
|
||||
("status", ""),
|
||||
("progress", ""),
|
||||
("result", ""),
|
||||
("error", ""),
|
||||
("checks", 0),
|
||||
("nudges_used", 0),
|
||||
("retries_used", 0),
|
||||
("error_streak", 0),
|
||||
("iterations", 0),
|
||||
("tokens", 0),
|
||||
("cost_usd", 0.0),
|
||||
("needs_question", ""),
|
||||
("pending_ask_id", ""),
|
||||
("check_at", ""),
|
||||
("last_progress_at", ""),
|
||||
("created_at", ""),
|
||||
("updated_at", ""),
|
||||
):
|
||||
if not agent_runs.has_column(column):
|
||||
agent_runs.create_column_by_example(column, example)
|
||||
_index(db, "devii_agent_runs", "idx_devii_agents_owner", ["owner_kind", "owner_id"])
|
||||
_index(db, "devii_agent_runs", "idx_devii_agents_due", ["status", "check_at"])
|
||||
_index(db, "devii_lessons", "idx_devii_lessons_owner", ["owner_kind", "owner_id"])
|
||||
_index(db, "devii_lessons", "idx_devii_lessons_owner_created", ["owner_kind", "owner_id", "created_at"])
|
||||
_index(
|
||||
@@ -647,6 +698,7 @@ def init_db():
|
||||
("inter_token_ms", 0.0),
|
||||
("provider", ""),
|
||||
("fallback_used_route", ""),
|
||||
("resolved_method", ""),
|
||||
):
|
||||
if not gateway_usage_ledger.has_column(column):
|
||||
gateway_usage_ledger.create_column_by_example(column, example)
|
||||
@@ -912,6 +964,10 @@ def init_db():
|
||||
|
||||
gateway_routing.ensure_tables()
|
||||
|
||||
from devplacepy.services.openai_gateway import learned as gateway_learned
|
||||
|
||||
gateway_learned.ensure_table()
|
||||
|
||||
from devplacepy.services.openai_gateway import quota as gateway_quota
|
||||
|
||||
gateway_quota.ensure_tables()
|
||||
@@ -1864,6 +1920,8 @@ def init_db():
|
||||
for table in SOFT_DELETE_TABLES:
|
||||
ensure_soft_delete_columns(table)
|
||||
|
||||
ensure_visibility_columns()
|
||||
|
||||
from .sync import ensure_export_indexes
|
||||
|
||||
ensure_export_indexes()
|
||||
@@ -1961,8 +2019,8 @@ def init_db():
|
||||
news_defaults = {
|
||||
"news_grade_threshold": "7",
|
||||
"news_api_url": "https://news.app.molodetz.nl/api",
|
||||
"news_ai_url": AQUALITY_NEWS_GRADING_URL,
|
||||
"news_ai_model": AQUALITY_NEWS_GRADING_MODEL,
|
||||
"news_ai_url": INTERNAL_GATEWAY_URL,
|
||||
"news_ai_model": NEWS_GRADING_MODEL_DEFAULT,
|
||||
}
|
||||
for key, value in news_defaults.items():
|
||||
existing = db["site_settings"].find_one(key=key)
|
||||
@@ -2662,12 +2720,18 @@ def migrate_ai_gateway_settings() -> None:
|
||||
if get_setting(key, "") == OLD_GATEWAY_URL:
|
||||
set_setting(key, INTERNAL_GATEWAY_URL)
|
||||
logger.info(f"Migrated {key} to the internal gateway")
|
||||
if get_setting("news_ai_url", "") == INTERNAL_GATEWAY_URL:
|
||||
set_setting("news_ai_url", AQUALITY_NEWS_GRADING_URL)
|
||||
logger.info("Migrated news_ai_url to the free aquality grading model")
|
||||
if get_setting("news_ai_model", "") in ("molodetz", ""):
|
||||
set_setting("news_ai_model", AQUALITY_NEWS_GRADING_MODEL)
|
||||
logger.info("Migrated news_ai_model to aquality")
|
||||
if get_setting("news_ai_url", "") == AQUALITY_NEWS_GRADING_URL:
|
||||
set_setting("news_ai_url", INTERNAL_GATEWAY_URL)
|
||||
logger.info(
|
||||
"Migrated news_ai_url from the retired aquality endpoint "
|
||||
"to the internal gateway"
|
||||
)
|
||||
if get_setting("news_ai_model", "") == AQUALITY_NEWS_GRADING_MODEL:
|
||||
set_setting("news_ai_model", NEWS_GRADING_MODEL_DEFAULT)
|
||||
logger.info(
|
||||
"Migrated news_ai_model from the retired aquality scorer "
|
||||
"to the free gateway model"
|
||||
)
|
||||
if get_setting("bot_model", "") == "deepseek-chat":
|
||||
set_setting("bot_model", "molodetz")
|
||||
logger.info("Migrated bot_model to molodetz")
|
||||
@@ -2712,6 +2776,12 @@ def backfill_api_keys() -> int:
|
||||
users.create_column_by_example("cust_disable_global", 0)
|
||||
if not users.has_column("cust_disable_pagetype"):
|
||||
users.create_column_by_example("cust_disable_pagetype", 0)
|
||||
if not users.has_column("enter_to_submit"):
|
||||
users.create_column_by_example("enter_to_submit", 1)
|
||||
if not users.has_column("notif_grouping_enabled"):
|
||||
users.create_column_by_example("notif_grouping_enabled", 1)
|
||||
if not users.has_column("notif_throttle_enabled"):
|
||||
users.create_column_by_example("notif_throttle_enabled", 1)
|
||||
if not users.has_column("ai_correction_enabled"):
|
||||
users.create_column_by_example("ai_correction_enabled", 0)
|
||||
if not users.has_column("ai_correction_prompt"):
|
||||
|
||||
@@ -17,6 +17,7 @@ SOFT_DELETE_TABLES = [
|
||||
"reactions",
|
||||
"bookmarks",
|
||||
"notes",
|
||||
"messages",
|
||||
"skills",
|
||||
"communities",
|
||||
"node_devices",
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from .core import TTLCache, datetime, db, timedelta, timezone
|
||||
from .ranking import get_top_authors
|
||||
from .visibility import listed_clause
|
||||
|
||||
|
||||
_stats_cache = TTLCache(ttl=30, max_size=50)
|
||||
@@ -16,19 +17,20 @@ def get_site_stats() -> dict:
|
||||
.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
.isoformat()
|
||||
)
|
||||
def listed_count(name, **filters):
|
||||
if name not in db.tables:
|
||||
return 0
|
||||
listed = listed_clause(db[name])
|
||||
clauses = [listed] if listed is not None else []
|
||||
return db[name].count(*clauses, **filters)
|
||||
|
||||
stats = {
|
||||
"total_members": db["users"].count() if "users" in db.tables else 0,
|
||||
"posts_today": db["posts"].count(
|
||||
created_at={">=": today_start}, deleted_at=None
|
||||
)
|
||||
if "posts" in db.tables
|
||||
else 0,
|
||||
"total_projects": db["projects"].count(deleted_at=None)
|
||||
if "projects" in db.tables
|
||||
else 0,
|
||||
"total_gists": db["gists"].count(deleted_at=None)
|
||||
if "gists" in db.tables
|
||||
else 0,
|
||||
"posts_today": listed_count(
|
||||
"posts", created_at={">=": today_start}, deleted_at=None
|
||||
),
|
||||
"total_projects": listed_count("projects", deleted_at=None),
|
||||
"total_gists": listed_count("gists", deleted_at=None),
|
||||
}
|
||||
_stats_cache.set("site", stats)
|
||||
return stats
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from sqlalchemy import or_
|
||||
from .core import db, get_table, _index
|
||||
|
||||
PUBLISHED = "published"
|
||||
HIDDEN = "hidden"
|
||||
DRAFT = "draft"
|
||||
OFFLINE = "offline"
|
||||
|
||||
VISIBILITY_VALUES = (PUBLISHED, HIDDEN, DRAFT, OFFLINE)
|
||||
VISIBILITY_CHOICES = (PUBLISHED, HIDDEN, OFFLINE)
|
||||
DEFAULT_VISIBILITY = PUBLISHED
|
||||
LISTED_VALUES = (PUBLISHED,)
|
||||
OPEN_VALUES = (PUBLISHED, HIDDEN)
|
||||
|
||||
VISIBILITY_COLUMNS = {
|
||||
"posts": "visibility",
|
||||
"gists": "visibility",
|
||||
"projects": "visibility",
|
||||
"news": "status",
|
||||
}
|
||||
|
||||
TARGET_TABLES = {
|
||||
"post": "posts",
|
||||
"gist": "gists",
|
||||
"project": "projects",
|
||||
"news": "news",
|
||||
}
|
||||
|
||||
VISIBILITY_LABELS = {
|
||||
PUBLISHED: "Published",
|
||||
HIDDEN: "Hidden",
|
||||
DRAFT: "Draft",
|
||||
OFFLINE: "Offline",
|
||||
}
|
||||
|
||||
VISIBILITY_HINTS = {
|
||||
PUBLISHED: "Listed everywhere and open to everyone.",
|
||||
HIDDEN: "Not listed or searchable; anyone with the link can still open it.",
|
||||
DRAFT: "Only you can see it. Not published yet.",
|
||||
OFFLINE: "Taken offline for now. Only you can see it; publish it again at any time.",
|
||||
}
|
||||
|
||||
def normalize_visibility(value, fallback: str = DEFAULT_VISIBILITY) -> str:
|
||||
text = str(value or "").strip().lower()
|
||||
return text if text in VISIBILITY_VALUES else fallback
|
||||
|
||||
|
||||
def visibility_column(table_name: str) -> str | None:
|
||||
return VISIBILITY_COLUMNS.get(table_name)
|
||||
|
||||
|
||||
def item_visibility(table_name: str, item: dict | None) -> str:
|
||||
column = visibility_column(table_name)
|
||||
if not column or not item:
|
||||
return DEFAULT_VISIBILITY
|
||||
return normalize_visibility(item.get(column))
|
||||
|
||||
|
||||
def is_listed(table_name: str, item: dict | None) -> bool:
|
||||
return item_visibility(table_name, item) in LISTED_VALUES
|
||||
|
||||
|
||||
def is_open(table_name: str, item: dict | None) -> bool:
|
||||
return item_visibility(table_name, item) in OPEN_VALUES
|
||||
|
||||
|
||||
def listed_clause(table, own_uid: str | None = None):
|
||||
name = getattr(table, "name", None) or getattr(table.table, "name", None)
|
||||
column_name = visibility_column(name)
|
||||
if not column_name or not table.exists:
|
||||
return None
|
||||
columns = table.table.columns
|
||||
if column_name not in columns:
|
||||
return None
|
||||
column = columns[column_name]
|
||||
clause = or_(column.is_(None), column.in_(LISTED_VALUES))
|
||||
if own_uid and "user_uid" in columns:
|
||||
clause = or_(clause, columns.user_uid == own_uid)
|
||||
return clause
|
||||
|
||||
|
||||
def listed_sql(table_name: str, alias: str = "") -> str:
|
||||
column = visibility_column(table_name)
|
||||
if not column or table_name not in db.tables:
|
||||
return "1 = 1"
|
||||
if not get_table(table_name).has_column(column):
|
||||
return "1 = 1"
|
||||
prefix = f"{alias}." if alias else ""
|
||||
return f"COALESCE({prefix}{column}, '{DEFAULT_VISIBILITY}') = '{PUBLISHED}'"
|
||||
|
||||
|
||||
def ensure_visibility_columns() -> None:
|
||||
for table_name, column in VISIBILITY_COLUMNS.items():
|
||||
if table_name == "news" or table_name not in db.tables:
|
||||
continue
|
||||
table = get_table(table_name)
|
||||
if not table.has_column(column):
|
||||
table.create_column_by_example(column, "")
|
||||
_index(db, table_name, f"idx_{table_name}_visibility", [column])
|
||||
|
||||
|
||||
def visibility_options(values=VISIBILITY_CHOICES) -> list[dict]:
|
||||
return [
|
||||
{"value": v, "label": VISIBILITY_LABELS[v], "hint": VISIBILITY_HINTS[v]}
|
||||
for v in values
|
||||
]
|
||||
|
||||
|
||||
def visibility_label(value) -> str:
|
||||
return VISIBILITY_LABELS[normalize_visibility(value)]
|
||||
@@ -99,3 +99,71 @@ NON_BODY_ENDPOINTS = {
|
||||
"notifications-open",
|
||||
"admin-bots-frame",
|
||||
}
|
||||
|
||||
|
||||
VISIBILITY_VALUES = ["published", "hidden", "offline", "draft"]
|
||||
VISIBILITY_DESCRIPTION = (
|
||||
"published (default): listed and open to everyone. hidden: not listed, searched or indexed, "
|
||||
"anyone with the link can open it. offline: only the owner and administrators can open it. "
|
||||
"draft: only the owner, not yet published."
|
||||
)
|
||||
|
||||
|
||||
VISIBILITY_OMITTED = {
|
||||
"create": " Optional and backwards compatible: when omitted the item is published.",
|
||||
"edit": " Optional and backwards compatible: when omitted the current value is kept.",
|
||||
"value": "",
|
||||
}
|
||||
|
||||
|
||||
def visibility_field(mode="create"):
|
||||
return field(
|
||||
"value" if mode == "value" else "visibility",
|
||||
"form",
|
||||
"enum",
|
||||
mode == "value",
|
||||
"published",
|
||||
VISIBILITY_DESCRIPTION + VISIBILITY_OMITTED[mode],
|
||||
VISIBILITY_VALUES,
|
||||
)
|
||||
|
||||
|
||||
def visibility_endpoint(id, kind, path, slug_name, example, auth="user"):
|
||||
area = kind if kind == "news" else f"{kind}s"
|
||||
owner_text = (
|
||||
"Administrators only." if auth == "admin" else "Only the owner can change it."
|
||||
)
|
||||
return endpoint(
|
||||
id=id,
|
||||
method="POST",
|
||||
path=path,
|
||||
title=f"Set {kind} visibility",
|
||||
summary=(
|
||||
f"Set the visibility of a {kind} to published, hidden, offline or draft. "
|
||||
f"{owner_text} Hidden items disappear from every listing, search and sitemap but "
|
||||
"open by direct link; offline and draft items are visible only to the owner and "
|
||||
"administrators (everyone else gets a 404)."
|
||||
),
|
||||
auth=auth,
|
||||
encoding="form",
|
||||
params=[
|
||||
field(
|
||||
slug_name,
|
||||
"path",
|
||||
"string",
|
||||
True,
|
||||
example,
|
||||
f"Slug or UID of the {kind}.",
|
||||
),
|
||||
visibility_field("value"),
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"redirect": f"/{area}/example-slug",
|
||||
"data": {
|
||||
"uid": "0190c0de-0000-7000-8000-000000000000",
|
||||
"visibility": "hidden",
|
||||
"url": f"/{area}/example-slug",
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .._shared import endpoint, field
|
||||
from .._shared import endpoint, field, visibility_endpoint
|
||||
|
||||
GROUP = {
|
||||
"slug": "admin",
|
||||
@@ -482,6 +482,14 @@ four ways to sign requests.
|
||||
field("uid", "path", "string", True, "NEWS_UID", "Article UID.")
|
||||
],
|
||||
),
|
||||
visibility_endpoint(
|
||||
"admin-news-visibility",
|
||||
"news",
|
||||
"/admin/news/{uid}/visibility",
|
||||
"uid",
|
||||
"NEWS_UID",
|
||||
auth="admin",
|
||||
),
|
||||
endpoint(
|
||||
id="admin-news-landing",
|
||||
method="POST",
|
||||
@@ -718,6 +726,7 @@ four ways to sign requests.
|
||||
field("target_model", "json", "string", True, "x-ai/grok-4.3", "Model actually sent upstream."),
|
||||
field("provider", "json", "string", False, "openrouter", "Provider name to route through. Blank = the default provider."),
|
||||
field("fallback_model", "json", "string", False, "molodetz-pro", "Another already-configured source_model of the same kind, tried once automatically when this route fails after its own retries are exhausted. Blank = no fallback."),
|
||||
field("tools_mode", "json", "string", False, "emulated", "'native' (default) forwards tools untouched. 'emulated' declares that this model does not support tool calling: tools are rewritten into the system prompt before sending and the reply is converted back into OpenAI tool_calls."),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
@@ -787,6 +796,49 @@ four ways to sign requests.
|
||||
field("app_reference", "json", "string", False, "typosaurus", "App label (the X-App-Reference header). Blank = every app."),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="admin-gateway-learned",
|
||||
method="GET",
|
||||
path="/admin/gateway/learned",
|
||||
title="List learned gateway request shapes",
|
||||
summary=(
|
||||
"Every request shape the gateway learned automatically after an upstream rejected the "
|
||||
"default one with an unrecognized-key 400, per provider and target model, plus the "
|
||||
"resolved/exhausted/healed counters."
|
||||
),
|
||||
auth="admin",
|
||||
sample_response={
|
||||
"learned": [
|
||||
{
|
||||
"provider": "vaids",
|
||||
"target_model": "deepseek-v4-1-flash-1017-uncensored",
|
||||
"method": "none",
|
||||
"thinking_dialect": "none",
|
||||
"drop_keys": [],
|
||||
"learned_at": "2026-11-13T00:55:00+00:00",
|
||||
}
|
||||
],
|
||||
"count": 1,
|
||||
"stats": {"resolved": 1, "exhausted": 0, "healed": 0},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-gateway-learned-reset",
|
||||
method="POST",
|
||||
path="/admin/gateway/learned/reset",
|
||||
title="Reset a learned gateway request shape",
|
||||
summary=(
|
||||
"Forget the learned method for one provider and target model so the gateway starts from "
|
||||
"the default shape again. Use provider 'default' for the base upstream."
|
||||
),
|
||||
auth="admin",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("provider", "json", "string", False, "vaids", "Provider name, or default."),
|
||||
field("target_model", "json", "string", True, "deepseek-v4-1-flash-1017-uncensored", "Upstream target model."),
|
||||
],
|
||||
sample_response={"ok": True},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-gateway-quota-rule-delete",
|
||||
method="DELETE",
|
||||
|
||||
@@ -23,8 +23,10 @@ The POST endpoints are **actions**: they accept form fields, set or clear the `s
|
||||
and return a `302` redirect (or the JSON envelope for JSON callers).
|
||||
|
||||
**Sign-up requires a unique `username` and `email`** plus a `confirm_password` that matches the
|
||||
password; **you log in with your `email` and password**. JSON callers receive validation errors
|
||||
as a `422` with the shape `{ "fields": {...}, "messages": [...] }`.
|
||||
password; **you log in with your `username` or your `email`, plus your password** (the `email`
|
||||
field accepts either identifier; usernames are case-sensitive, email addresses are not). The
|
||||
same identifier rule applies to `POST /auth/token` and HTTP Basic. JSON callers receive
|
||||
validation errors as a `422` with the shape `{ "fields": {...}, "messages": [...] }`.
|
||||
""",
|
||||
"endpoints": [
|
||||
endpoint(
|
||||
@@ -89,20 +91,20 @@ as a `422` with the shape `{ "fields": {...}, "messages": [...] }`.
|
||||
method="POST",
|
||||
path="/auth/login",
|
||||
title="Log in",
|
||||
summary="Authenticate with email and password. Sets the session cookie.",
|
||||
summary="Authenticate with username (or email) and password. Sets the session cookie.",
|
||||
auth="public",
|
||||
encoding="form",
|
||||
params=[
|
||||
field("email", "form", "string", True, "alice@example.com", "Your registered email."),
|
||||
field("email", "form", "string", True, "alice@example.com", "Your username or your registered email."),
|
||||
field("password", "form", "string", True, "mysecret", "Your password."),
|
||||
field("remember_me", "form", "string", False, "on", "Send 'on' to extend the session to the remember-me lifetime."),
|
||||
field("next", "form", "string", False, "", "Redirect target after login."),
|
||||
],
|
||||
notes=[
|
||||
"JSON callers receive the account's `email`, `api_key`, and "
|
||||
"`is_admin` flag in `data`, so a script can trade one email/password "
|
||||
"login for a reusable key. Failed logins return 401 and never "
|
||||
"include the key.",
|
||||
"JSON callers receive the account's `username`, `email`, `api_key`, "
|
||||
"and `is_admin` flag in `data`, so a script can trade one "
|
||||
"username/password or email/password login for a reusable key. "
|
||||
"Failed logins return 401 and never include the key.",
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
@@ -115,6 +117,31 @@ as a `422` with the shape `{ "fields": {...}, "messages": [...] }`.
|
||||
},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="auth-token",
|
||||
method="POST",
|
||||
path="/auth/token",
|
||||
title="Issue access token",
|
||||
summary="Exchange username (or email) and password for a short-lived access token. No session cookie is set.",
|
||||
auth="public",
|
||||
encoding="form",
|
||||
params=[
|
||||
field("email", "form", "string", True, "alice@example.com", "Your username or your registered email."),
|
||||
field("password", "form", "string", True, "mysecret", "Your password."),
|
||||
],
|
||||
notes=[
|
||||
"Accepts a JSON body with the same fields. The returned token "
|
||||
"works as an `Authorization: Bearer` or `X-API-KEY` credential "
|
||||
"until it expires (default 7 days). Missing fields return 400, "
|
||||
"wrong credentials 401, a deactivated account 403; failures "
|
||||
"never include a token.",
|
||||
],
|
||||
sample_response={
|
||||
"access_token": "ACCESS_TOKEN",
|
||||
"token_type": "bearer",
|
||||
"expires_in": 604800,
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="auth-forgot-password",
|
||||
method="GET",
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .._shared import COMMENT_TARGETS, GIST_LANGUAGES, PROJECT_TYPES, endpoint, field
|
||||
from .._shared import (
|
||||
COMMENT_TARGETS,
|
||||
GIST_LANGUAGES,
|
||||
PROJECT_TYPES,
|
||||
endpoint,
|
||||
field,
|
||||
visibility_field,
|
||||
)
|
||||
from devplacepy.constants import TOPICS
|
||||
|
||||
GROUP = {
|
||||
@@ -143,6 +150,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field(),
|
||||
field(
|
||||
"content",
|
||||
"form",
|
||||
@@ -276,6 +284,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field("edit"),
|
||||
field(
|
||||
"post_slug",
|
||||
"path",
|
||||
@@ -548,6 +557,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field(),
|
||||
field(
|
||||
"title",
|
||||
"form",
|
||||
@@ -649,6 +659,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field("edit"),
|
||||
field(
|
||||
"project_slug",
|
||||
"path",
|
||||
@@ -771,7 +782,7 @@ four ways to sign requests.
|
||||
id="projects-private",
|
||||
method="POST",
|
||||
path="/projects/{project_slug}/private",
|
||||
title="Set project visibility",
|
||||
title="Make a project private or public",
|
||||
summary="Mark a project you own private or public. Send value=1 for private, value=0 for public. A project you hide as a member stays visible to administrators; a project you hide as an administrator is visible only to you, not to other administrators (this also hides its files and any attached containers).",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
@@ -1023,6 +1034,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field(),
|
||||
field(
|
||||
"title",
|
||||
"form",
|
||||
@@ -1068,6 +1080,7 @@ four ways to sign requests.
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
visibility_field("edit"),
|
||||
field(
|
||||
"gist_slug",
|
||||
"path",
|
||||
|
||||
@@ -19,10 +19,11 @@ Every example uses your current host:
|
||||
## Authentication
|
||||
|
||||
Each endpoint is tagged **public**, **user**, or **admin**. Authenticate user and admin
|
||||
endpoints with any of the four methods in [Authentication](/docs/authentication.html): the
|
||||
`session` cookie, an `X-API-KEY` header, a `Bearer` token, or HTTP Basic credentials. The
|
||||
interactive panels on this site pre-fill your own API key, so you can run user-level calls
|
||||
immediately.
|
||||
endpoints with any of the methods in [Authentication](/docs/authentication.html): the
|
||||
`session` cookie, an `X-API-KEY` header, a `Bearer` token, or HTTP Basic credentials
|
||||
(username or email, plus password; the headers also accept native access tokens and
|
||||
DevRant auth tokens). The interactive panels on this site pre-fill your own API key, so
|
||||
you can run user-level calls immediately.
|
||||
|
||||
## Request bodies
|
||||
|
||||
|
||||
@@ -79,6 +79,31 @@ four ways to sign requests.
|
||||
),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="messages-delete",
|
||||
method="POST",
|
||||
path="/messages/{message_uid}/delete",
|
||||
title="Delete a sent message",
|
||||
summary="Soft-delete a direct message. Administrators only, and only messages they sent themselves; the bubble is removed live for both participants.",
|
||||
auth="admin",
|
||||
encoding="none",
|
||||
destructive=True,
|
||||
params=[
|
||||
field(
|
||||
"message_uid",
|
||||
"path",
|
||||
"string",
|
||||
True,
|
||||
"MESSAGE_UID",
|
||||
"Uid of the message to delete.",
|
||||
),
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"redirect": "/messages?with_uid=RECEIVER_UID",
|
||||
"data": {"deleted_uid": "MESSAGE_UID"},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="messages-conversations",
|
||||
method="GET",
|
||||
|
||||
@@ -106,7 +106,7 @@ page. Every endpoint follows the shared [Conventions & Errors](/docs/conventions
|
||||
],
|
||||
sample_response={"device": DEVICE_SAMPLE, "secret": "dpn_EXAMPLE_SECRET_SHOWN_ONCE"},
|
||||
notes=[
|
||||
"Refused with `403` and a `reason` (`kill_switch`, `no_consent`, `not_enabled`, `not_adult`, `deletion_pending`, `suspended`, `account_inactive`) unless donation is switched on, you are an adult, you have given the `bandwidth_sharing` consent and enabled donation for yourself.",
|
||||
"Refused with `403` and a `reason` (`kill_switch`, `no_consent`, `not_enabled`, `deletion_pending`, `suspended`, `account_inactive`) unless donation is switched on, you have given the `bandwidth_sharing` consent and enabled donation for yourself.",
|
||||
"Refused with `409` when you already have the maximum number of devices.",
|
||||
"The response is `201` and is never cached.",
|
||||
],
|
||||
|
||||
@@ -8,8 +8,7 @@ GROUP = {
|
||||
"intro": """
|
||||
# Notifications
|
||||
|
||||
Read your notification feed and mark items read. The unread counts endpoint backs the badges
|
||||
in the navigation bar. Deliver these to the browser with [Web Push](/docs/push.html).
|
||||
Read your notification feed and mark items read. With grouping on (the default, a profile setting), similar events share one entry in `event_entries` and the unread counts count groups; with it off, entries stay one per event. Repeat external alerts are throttled to one per similar event group per 5 minutes unless the user turned throttling off; in-app rows are never throttled. The unread counts endpoint backs the badges in the navigation bar. Deliver these to the browser with [Web Push](/docs/push.html).
|
||||
|
||||
Every endpoint follows the shared [Conventions & Errors](/docs/conventions.html) (auth, content
|
||||
negotiation, pagination, status codes); see [Authentication](/docs/authentication.html) for the
|
||||
|
||||
@@ -379,7 +379,11 @@ four ways to sign requests.
|
||||
"> Running this invalidates the key these documentation panels use. Do it from your "
|
||||
"[profile page](/profile/{{ username }}) instead, then reload these docs.",
|
||||
],
|
||||
sample_response={"api_key": "NEW_UUID"},
|
||||
sample_response={
|
||||
"api_key": "NEW_UUID",
|
||||
"username": "alice",
|
||||
"email": "alice@example.com",
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="profile-give-award",
|
||||
@@ -500,6 +504,48 @@ four ways to sign requests.
|
||||
),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="profile-enter-to-submit",
|
||||
method="POST",
|
||||
path="/profile/{username}/enter-to-submit",
|
||||
title="Toggle Enter to send",
|
||||
summary="Enter posts comments and sends messages; Shift+Enter adds a new line. Admins may target any user.",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("username", "path", "string", True, "bob_test", "Profile owner. Must be yourself unless you are an admin."),
|
||||
field("value", "form", "boolean", False, "1", "1 for Enter to send, 0 for Enter adds a new line."),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="profile-notification-grouping",
|
||||
method="POST",
|
||||
path="/profile/{username}/notifications/grouping",
|
||||
title="Toggle grouped notifications",
|
||||
summary="Group similar notifications into one card; the badge counts groups. Admins may target any user.",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("username", "path", "string", True, "bob_test", "Profile owner. Must be yourself unless you are an admin."),
|
||||
field("value", "form", "boolean", False, "1", "1 to group, 0 to show every event."),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="profile-notification-throttling",
|
||||
method="POST",
|
||||
path="/profile/{username}/notifications/throttling",
|
||||
title="Toggle throttled repeat alerts",
|
||||
summary="At most one push, Telegram message and toast per similar event group per 5 minutes. In-app is never throttled. Admins may target any user.",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("username", "path", "string", True, "bob_test", "Profile owner. Must be yourself unless you are an admin."),
|
||||
field("value", "form", "boolean", False, "1", "1 to throttle repeat alerts, 0 to alert on every event."),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="profile-notification-toggle",
|
||||
method="POST",
|
||||
|
||||
@@ -20,6 +20,11 @@ its build. A registration body without a `provider` field is a `webpush` registr
|
||||
existing clients need no change. An APNs body may include a stable `client_id` so a later
|
||||
token rotation updates the same device instead of inserting another row.
|
||||
|
||||
Every delivered notification carries the recipient's unread count, the same number as
|
||||
`notifications` in `GET /notifications/counts` ([Notifications](/docs/notifications.html)). `apns` presents it as `aps.badge`,
|
||||
so a native app icon updates while the app is in the background; `webpush` forwards it as the
|
||||
`unread` field of the payload.
|
||||
|
||||
`DELETE /push.json` removes a single registration by the same identity used to create it
|
||||
(`endpoint` for webpush, `token` or `client_id` for apns). The server also stops delivering to a
|
||||
subscription once its push endpoint reports it as gone. These mirror the in-app
|
||||
|
||||
@@ -11,7 +11,7 @@ GROUP = {
|
||||
Attachment storage. Upload a file - or hand the server a public URL to fetch - to receive an
|
||||
attachment record, then reference its `uid` in an `attachment_uids` field when creating a post,
|
||||
comment, project, gist, message, or issue - see
|
||||
[Posts, Comments, Projects, Gists & News](/docs/content.html). Images and videos embed and
|
||||
[Posts, Comments, Projects, Gists & News](/docs/content.html). GIFs from GIPHY are attached the same way, through `/giphy/attach`. Images and videos embed and
|
||||
play inline once posted; other types render as download links. The record's `is_image` and
|
||||
`is_video` flags indicate how the file is displayed.
|
||||
|
||||
@@ -112,6 +112,82 @@ four ways to sign requests.
|
||||
"mime_type": "image/png",
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="giphy-status",
|
||||
method="GET",
|
||||
path="/giphy/status",
|
||||
title="GIF picker availability",
|
||||
summary="Tell whether an administrator has configured GIPHY, so a client knows whether to offer a GIF picker.",
|
||||
auth="user",
|
||||
sample_response={"configured": True},
|
||||
),
|
||||
endpoint(
|
||||
id="giphy-search",
|
||||
method="GET",
|
||||
path="/giphy/search",
|
||||
title="Search GIFs",
|
||||
summary="Search GIPHY by keyword through the server, so the API key never reaches a browser.",
|
||||
auth="user",
|
||||
params=[
|
||||
field("q", "query", "string", True, "happy dance", "Search keywords, 1-200 characters."),
|
||||
field("limit", "query", "integer", False, "24", "Number of results, 1-50 (default 24)."),
|
||||
],
|
||||
notes=[
|
||||
"Results are capped to the content rating an administrator configured (default `pg`) and cached for five minutes.",
|
||||
"Each member may make 30 GIF searches or trending requests per minute; beyond that the answer is `429` with `Retry-After`. `503` means GIPHY is not configured, `400` a missing `q`, `502` an upstream failure.",
|
||||
],
|
||||
sample_response={"query": "happy dance", "results": [{
|
||||
"id": "xT9IgDEI1iZyb2wqo8",
|
||||
"title": "happy dance GIF",
|
||||
"url": "https://giphy.com/gifs/xT9IgDEI1iZyb2wqo8",
|
||||
"preview": {"url": "https://media.giphy.com/media/xT9IgDEI1iZyb2wqo8/200w.gif", "width": 200, "height": 150, "size": 90210},
|
||||
"original": {"url": "https://media.giphy.com/media/xT9IgDEI1iZyb2wqo8/giphy.gif", "width": 480, "height": 360, "size": 1048576},
|
||||
}]},
|
||||
),
|
||||
endpoint(
|
||||
id="giphy-trending",
|
||||
method="GET",
|
||||
path="/giphy/trending",
|
||||
title="Trending GIFs",
|
||||
summary="List the GIFs currently trending on GIPHY, in the same shape as Search GIFs.",
|
||||
auth="user",
|
||||
params=[
|
||||
field("limit", "query", "integer", False, "24", "Number of results, 1-50 (default 24)."),
|
||||
],
|
||||
notes=["Shares the per-member rate limit, the cache and the error codes of Search GIFs."],
|
||||
sample_response={"results": [{
|
||||
"id": "xT9IgDEI1iZyb2wqo8",
|
||||
"title": "happy dance GIF",
|
||||
"url": "https://giphy.com/gifs/xT9IgDEI1iZyb2wqo8",
|
||||
"preview": {"url": "https://media.giphy.com/media/xT9IgDEI1iZyb2wqo8/200w.gif", "width": 200, "height": 150, "size": 90210},
|
||||
"original": {"url": "https://media.giphy.com/media/xT9IgDEI1iZyb2wqo8/giphy.gif", "width": 480, "height": 360, "size": 1048576},
|
||||
}]},
|
||||
),
|
||||
endpoint(
|
||||
id="giphy-attach",
|
||||
method="POST",
|
||||
path="/giphy/attach",
|
||||
title="Attach a GIF",
|
||||
summary="Download a GIPHY GIF on the server and store it as an attachment, exactly like an upload.",
|
||||
auth="user",
|
||||
encoding="json",
|
||||
params=[
|
||||
field("gif_id", "json", "string", True, "xT9IgDEI1iZyb2wqo8", "GIPHY id from Search GIFs or Trending GIFs."),
|
||||
field("title", "json", "string", False, "happy dance", "Optional title used as the attachment's filename."),
|
||||
],
|
||||
notes=[
|
||||
"Accepts a JSON body or a form. Returns `201` with the same record as Upload a file; reference its `uid` in `attachment_uids`. `413` if larger than the upload limit, `415` when GIF uploads are disabled, `404` for an unknown id.",
|
||||
],
|
||||
sample_response={
|
||||
"uid": "ATTACHMENT_UID",
|
||||
"original_filename": "happy dance.gif",
|
||||
"url": "/static/uploads/attachments/ab/cd/ATTACHMENT_UID.gif",
|
||||
"file_size": 1048576,
|
||||
"mime_type": "image/gif",
|
||||
"is_image": True,
|
||||
"is_video": False,
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="uploads-list",
|
||||
method="GET",
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from devplacepy.database import (
|
||||
count_unread_groups,
|
||||
db,
|
||||
get_table,
|
||||
get_setting,
|
||||
@@ -71,8 +72,8 @@ def _user_facts(user: dict) -> dict:
|
||||
"followers": _count("follows", following_uid=uid),
|
||||
"following": _count("follows", follower_uid=uid),
|
||||
"bookmarks": _count("bookmarks", user_uid=uid),
|
||||
"unread_notifications": _count("notifications", user_uid=uid, read=False),
|
||||
"unread_messages": _count("messages", receiver_uid=uid, read=False),
|
||||
"unread_notifications": count_unread_groups(uid),
|
||||
"unread_messages": _count("messages", receiver_uid=uid, read=False, deleted_at=None),
|
||||
},
|
||||
"recent_posts": [
|
||||
{
|
||||
|
||||
@@ -0,0 +1,163 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from urllib.parse import parse_qs, urlencode, urlparse
|
||||
|
||||
EMBED_JS_PATH = Path(__file__).parent / "static" / "js" / "embed-spec.js"
|
||||
|
||||
YOUTUBE_HOSTS = frozenset(
|
||||
{
|
||||
"youtube.com",
|
||||
"www.youtube.com",
|
||||
"m.youtube.com",
|
||||
"youtube-nocookie.com",
|
||||
"www.youtube-nocookie.com",
|
||||
}
|
||||
)
|
||||
MUSIC_HOSTS = frozenset({"music.youtube.com"})
|
||||
SHORT_HOSTS = frozenset({"youtu.be", "www.youtu.be"})
|
||||
|
||||
VIDEO_ID_PATTERN = "[A-Za-z0-9_-]{11}"
|
||||
LIST_ID_PATTERN = "[A-Za-z0-9_-]{13,64}"
|
||||
START_PATTERN = "[0-9]{1,6}s?"
|
||||
MAX_START_SECONDS = 86400
|
||||
MAX_MUSIC_EMBEDS = 3
|
||||
|
||||
EMBED_BASE = "https://www.youtube.com/embed"
|
||||
PLAYLIST_SLUG = "videoseries"
|
||||
EMBED_ALLOW = (
|
||||
"accelerometer; autoplay; clipboard-write; encrypted-media; "
|
||||
"gyroscope; picture-in-picture"
|
||||
)
|
||||
MUSIC_CAPTION = "Open in YouTube Music"
|
||||
PATH_VIDEO_PREFIXES = ("embed", "v", "shorts")
|
||||
|
||||
_VIDEO_ID_RE = re.compile(VIDEO_ID_PATTERN)
|
||||
_LIST_ID_RE = re.compile(LIST_ID_PATTERN)
|
||||
_START_RE = re.compile(START_PATTERN)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Embed:
|
||||
kind: str
|
||||
video_id: str
|
||||
list_id: str
|
||||
start: int
|
||||
music: bool
|
||||
|
||||
|
||||
class EmbedParser:
|
||||
@staticmethod
|
||||
def parse(url: str) -> Embed | None:
|
||||
try:
|
||||
parts = urlparse(url)
|
||||
except ValueError:
|
||||
return None
|
||||
if parts.scheme not in ("http", "https"):
|
||||
return None
|
||||
host = (parts.hostname or "").lower()
|
||||
query = parse_qs(parts.query)
|
||||
segments = [segment for segment in parts.path.split("/") if segment]
|
||||
start = EmbedParser._start(query)
|
||||
if host in SHORT_HOSTS:
|
||||
if len(segments) != 1 or not _VIDEO_ID_RE.fullmatch(segments[0]):
|
||||
return None
|
||||
return Embed("video", segments[0], "", start, False)
|
||||
music = host in MUSIC_HOSTS
|
||||
if not music and host not in YOUTUBE_HOSTS:
|
||||
return None
|
||||
list_id = EmbedParser._list_id(query)
|
||||
if segments == ["watch"]:
|
||||
return EmbedParser._from_watch(query, list_id, start, music)
|
||||
if segments == ["playlist"]:
|
||||
return EmbedParser._playlist(list_id, music)
|
||||
if music:
|
||||
return None
|
||||
if segments == ["embed"] and query.get("listType", [""])[0] == "playlist":
|
||||
return EmbedParser._playlist(list_id, False)
|
||||
if len(segments) == 2 and segments[0] in PATH_VIDEO_PREFIXES:
|
||||
return EmbedParser._from_path(segments, list_id, start)
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def src(embed: Embed) -> str:
|
||||
if embed.kind == "playlist":
|
||||
return f"{EMBED_BASE}/{PLAYLIST_SLUG}?{urlencode({'list': embed.list_id})}"
|
||||
params: dict[str, str | int] = {}
|
||||
if embed.list_id:
|
||||
params["list"] = embed.list_id
|
||||
if embed.start:
|
||||
params["start"] = embed.start
|
||||
suffix = f"?{urlencode(params)}" if params else ""
|
||||
return f"{EMBED_BASE}/{embed.video_id}{suffix}"
|
||||
|
||||
@staticmethod
|
||||
def spec() -> dict:
|
||||
return {
|
||||
"youtubeHosts": sorted(YOUTUBE_HOSTS),
|
||||
"musicHosts": sorted(MUSIC_HOSTS),
|
||||
"shortHosts": sorted(SHORT_HOSTS),
|
||||
"videoIdPattern": VIDEO_ID_PATTERN,
|
||||
"listIdPattern": LIST_ID_PATTERN,
|
||||
"startPattern": START_PATTERN,
|
||||
"maxStartSeconds": MAX_START_SECONDS,
|
||||
"maxMusicEmbeds": MAX_MUSIC_EMBEDS,
|
||||
"embedBase": EMBED_BASE,
|
||||
"playlistSlug": PLAYLIST_SLUG,
|
||||
"allow": EMBED_ALLOW,
|
||||
"musicCaption": MUSIC_CAPTION,
|
||||
"pathVideoPrefixes": list(PATH_VIDEO_PREFIXES),
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def write_module(path: Path = EMBED_JS_PATH) -> int:
|
||||
spec = EmbedParser.spec()
|
||||
body = json.dumps(spec, ensure_ascii=False, sort_keys=True, indent=4)
|
||||
path.write_text(
|
||||
f"// retoor <retoor@molodetz.nl>\n\nexport const EMBED_SPEC = {body};\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
return len(spec)
|
||||
|
||||
@staticmethod
|
||||
def _first(query: dict, key: str) -> str:
|
||||
values = query.get(key)
|
||||
return values[0] if values else ""
|
||||
|
||||
@staticmethod
|
||||
def _list_id(query: dict) -> str:
|
||||
value = EmbedParser._first(query, "list")
|
||||
return value if _LIST_ID_RE.fullmatch(value) else ""
|
||||
|
||||
@staticmethod
|
||||
def _start(query: dict) -> int:
|
||||
raw = EmbedParser._first(query, "start") or EmbedParser._first(query, "t")
|
||||
if not _START_RE.fullmatch(raw):
|
||||
return 0
|
||||
seconds = int(raw.rstrip("s"))
|
||||
return seconds if seconds <= MAX_START_SECONDS else 0
|
||||
|
||||
@staticmethod
|
||||
def _playlist(list_id: str, music: bool) -> Embed | None:
|
||||
if not list_id:
|
||||
return None
|
||||
return Embed("playlist", "", list_id, 0, music)
|
||||
|
||||
@staticmethod
|
||||
def _from_watch(query: dict, list_id: str, start: int, music: bool) -> Embed | None:
|
||||
video_id = EmbedParser._first(query, "v")
|
||||
if _VIDEO_ID_RE.fullmatch(video_id):
|
||||
return Embed("video", video_id, list_id, start, music)
|
||||
return EmbedParser._playlist(list_id, music)
|
||||
|
||||
@staticmethod
|
||||
def _from_path(segments: list[str], list_id: str, start: int) -> Embed | None:
|
||||
slug = segments[1]
|
||||
if slug == PLAYLIST_SLUG:
|
||||
return EmbedParser._playlist(list_id, False)
|
||||
if not _VIDEO_ID_RE.fullmatch(slug):
|
||||
return None
|
||||
return Embed("video", slug, list_id, start, False)
|
||||
@@ -31,9 +31,11 @@ def _post_pool() -> list[str]:
|
||||
if cached is not None:
|
||||
return cached
|
||||
slugs: list[str] = []
|
||||
from devplacepy.database import listed_sql
|
||||
|
||||
if "posts" in db.tables:
|
||||
rows = db.query(
|
||||
"SELECT slug, uid FROM posts WHERE deleted_at IS NULL ORDER BY RANDOM() LIMIT :limit",
|
||||
f"SELECT slug, uid FROM posts WHERE deleted_at IS NULL AND {listed_sql('posts')} ORDER BY RANDOM() LIMIT :limit",
|
||||
limit=POOL_SIZE,
|
||||
)
|
||||
slugs = [row["slug"] or row["uid"] for row in rows]
|
||||
|
||||
@@ -24,6 +24,7 @@ from devplacepy.services.containers.service import ContainerService
|
||||
from devplacepy.services.containers.workspace_service import WorkspaceService
|
||||
from devplacepy.services.dbapi.service import DbApiJobService
|
||||
from devplacepy.services.devii import DeviiService
|
||||
from devplacepy.services.giphy.service import GiphyService
|
||||
from devplacepy.services.gitea.service import IssueTrackerService
|
||||
from devplacepy.services.jobs.award_service import AwardService
|
||||
from devplacepy.services.jobs.deepsearch.service import DeepsearchService
|
||||
@@ -73,6 +74,7 @@ SERVICE_CLASSES = [
|
||||
IssueCreateService,
|
||||
PlanningReportService,
|
||||
IssueTrackerService,
|
||||
GiphyService,
|
||||
ContainerService,
|
||||
WorkspaceService,
|
||||
XmlrpcService,
|
||||
|
||||
@@ -38,6 +38,7 @@ from devplacepy.routers import (
|
||||
gists,
|
||||
quizzes,
|
||||
uploads,
|
||||
giphy,
|
||||
media,
|
||||
push,
|
||||
leaderboard,
|
||||
@@ -72,9 +73,19 @@ logging.basicConfig(
|
||||
level=getattr(logging, LOG_LEVEL, logging.INFO),
|
||||
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
|
||||
)
|
||||
logging.getLogger("alembic.runtime.migration").setLevel(logging.WARNING)
|
||||
|
||||
app = FastAPI(
|
||||
title="DevPlace",
|
||||
description=(
|
||||
"Authentication: username+password, email+password (login form, JSON "
|
||||
"POST /auth/login, POST /auth/token, and HTTP Basic all accept the "
|
||||
"username or the email address), or the personal API key from the "
|
||||
"profile page sent as X-API-KEY or Authorization: Bearer "
|
||||
"(conventionally carried in the DEVPLACE_API_KEY environment variable). "
|
||||
"GET /profile with an API key returns the account username, email, and "
|
||||
"API key. Full reference: /docs/authentication.html"
|
||||
),
|
||||
docs_url="/swagger",
|
||||
redoc_url=None,
|
||||
openapi_url="/openapi.json",
|
||||
@@ -125,6 +136,7 @@ app.include_router(issues.router, prefix="/issues")
|
||||
app.include_router(gists.router, prefix="/gists")
|
||||
app.include_router(news.router, prefix="/news")
|
||||
app.include_router(uploads.router, prefix="/uploads")
|
||||
app.include_router(giphy.router, prefix="/giphy")
|
||||
app.include_router(media.router, prefix="/media")
|
||||
app.include_router(zips.router, prefix="/zips")
|
||||
app.include_router(forks.router, prefix="/forks")
|
||||
|
||||
@@ -6,6 +6,9 @@ from devplacepy.models.shared import (
|
||||
normalize_poll_options,
|
||||
_declared_age,
|
||||
normalize_index_list,
|
||||
VisibilityForm,
|
||||
VisibilityCreateMixin,
|
||||
VisibilityEditMixin,
|
||||
)
|
||||
from devplacepy.models.auth import (
|
||||
SignupForm,
|
||||
@@ -41,6 +44,7 @@ from devplacepy.models.projects import (
|
||||
)
|
||||
from devplacepy.models.profile import (
|
||||
CustomizationToggleForm,
|
||||
EnterToSubmitForm,
|
||||
AwardGiveForm,
|
||||
NotificationPrefForm,
|
||||
AiCorrectionForm,
|
||||
@@ -78,6 +82,7 @@ from devplacepy.models.uploads import (
|
||||
UploadUrlForm,
|
||||
GenerateImageForm,
|
||||
AttachmentRenameForm,
|
||||
GiphyAttachForm,
|
||||
)
|
||||
from devplacepy.models.tools import (
|
||||
PlanningForm,
|
||||
@@ -161,6 +166,9 @@ __all__ = [
|
||||
"normalize_poll_options",
|
||||
"_declared_age",
|
||||
"normalize_index_list",
|
||||
"VisibilityForm",
|
||||
"VisibilityCreateMixin",
|
||||
"VisibilityEditMixin",
|
||||
"SignupForm",
|
||||
"LoginForm",
|
||||
"ForgotPasswordForm",
|
||||
@@ -188,6 +196,7 @@ __all__ = [
|
||||
"ProjectFileDeleteLinesForm",
|
||||
"ProjectFileAppendForm",
|
||||
"CustomizationToggleForm",
|
||||
"EnterToSubmitForm",
|
||||
"AwardGiveForm",
|
||||
"NotificationPrefForm",
|
||||
"AiCorrectionForm",
|
||||
@@ -217,6 +226,7 @@ __all__ = [
|
||||
"IssueAttachmentForm",
|
||||
"IssueStatusForm",
|
||||
"UploadUrlForm",
|
||||
"GiphyAttachForm",
|
||||
"GenerateImageForm",
|
||||
"AttachmentRenameForm",
|
||||
"PlanningForm",
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from typing import Literal, Optional
|
||||
from pydantic import BaseModel, Field
|
||||
from devplacepy.models.shared import VisibilityCreateMixin, VisibilityEditMixin
|
||||
|
||||
|
||||
class SkillInstallForm(BaseModel):
|
||||
@@ -23,7 +24,7 @@ class MessageForm(BaseModel):
|
||||
client_id: Optional[str] = Field(default=None, max_length=64)
|
||||
|
||||
|
||||
class GistForm(BaseModel):
|
||||
class GistForm(VisibilityCreateMixin):
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
description: str = Field(default="", max_length=5000)
|
||||
source_code: str = Field(min_length=1, max_length=400000)
|
||||
@@ -31,7 +32,7 @@ class GistForm(BaseModel):
|
||||
attachment_uids: list[str] = []
|
||||
|
||||
|
||||
class GistEditForm(BaseModel):
|
||||
class GistEditForm(VisibilityEditMixin):
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
description: str = Field(default="", max_length=5000)
|
||||
source_code: str = Field(min_length=1, max_length=400000)
|
||||
|
||||
@@ -4,10 +4,14 @@ from typing import Literal
|
||||
from pydantic import BaseModel, Field, field_validator, model_validator
|
||||
from devplacepy.constants import TOPICS
|
||||
from devplacepy.rendering import is_single_emoji
|
||||
from devplacepy.models.shared import normalize_poll_options
|
||||
from devplacepy.models.shared import (
|
||||
normalize_poll_options,
|
||||
VisibilityCreateMixin,
|
||||
VisibilityEditMixin,
|
||||
)
|
||||
|
||||
|
||||
class PostForm(BaseModel):
|
||||
class PostForm(VisibilityCreateMixin):
|
||||
content: str = Field(min_length=10, max_length=125000)
|
||||
title: str = Field(default="", max_length=500)
|
||||
topic: str = "random"
|
||||
@@ -50,7 +54,7 @@ class WarJoinForm(BaseModel):
|
||||
return value
|
||||
|
||||
|
||||
class PostEditForm(BaseModel):
|
||||
class PostEditForm(VisibilityEditMixin):
|
||||
content: str = Field(min_length=10, max_length=125000)
|
||||
title: str = Field(default="", max_length=500)
|
||||
topic: str = "random"
|
||||
|
||||
@@ -12,6 +12,10 @@ class CustomizationToggleForm(BaseModel):
|
||||
value: bool = False
|
||||
|
||||
|
||||
class EnterToSubmitForm(BaseModel):
|
||||
value: bool = True
|
||||
|
||||
|
||||
class AwardGiveForm(BaseModel):
|
||||
description: str = Field(min_length=1, max_length=125)
|
||||
|
||||
|
||||
@@ -2,10 +2,15 @@
|
||||
|
||||
from typing import Literal
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from devplacepy.models.shared import normalize_european_date, normalize_website_url
|
||||
from devplacepy.models.shared import (
|
||||
normalize_european_date,
|
||||
normalize_website_url,
|
||||
VisibilityCreateMixin,
|
||||
VisibilityEditMixin,
|
||||
)
|
||||
|
||||
|
||||
class ProjectForm(BaseModel):
|
||||
class ProjectForm(VisibilityCreateMixin):
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
description: str = Field(min_length=1, max_length=5000)
|
||||
release_date: str = ""
|
||||
@@ -33,7 +38,7 @@ class ProjectForm(BaseModel):
|
||||
return normalize_website_url(value)
|
||||
|
||||
|
||||
class ProjectEditForm(BaseModel):
|
||||
class ProjectEditForm(VisibilityEditMixin):
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
description: str = Field(min_length=1, max_length=5000)
|
||||
release_date: str = ""
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
from urllib.parse import urlparse
|
||||
from pydantic import BaseModel, field_validator
|
||||
from devplacepy.database.visibility import VISIBILITY_VALUES, DEFAULT_VISIBILITY
|
||||
|
||||
|
||||
def normalize_european_date(value):
|
||||
@@ -73,3 +76,41 @@ def normalize_index_list(value):
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
return indexes
|
||||
|
||||
|
||||
def parse_visibility(value, default):
|
||||
text = str(value if value is not None else "").strip().lower()
|
||||
if not text:
|
||||
return default
|
||||
if text not in VISIBILITY_VALUES:
|
||||
raise ValueError(
|
||||
"Visibility must be one of: " + ", ".join(VISIBILITY_VALUES)
|
||||
)
|
||||
return text
|
||||
|
||||
|
||||
class VisibilityCreateMixin(BaseModel):
|
||||
visibility: str = DEFAULT_VISIBILITY
|
||||
|
||||
@field_validator("visibility", mode="before")
|
||||
@classmethod
|
||||
def valid_visibility(cls, value):
|
||||
return parse_visibility(value, DEFAULT_VISIBILITY)
|
||||
|
||||
|
||||
class VisibilityEditMixin(BaseModel):
|
||||
visibility: Optional[str] = None
|
||||
|
||||
@field_validator("visibility", mode="before")
|
||||
@classmethod
|
||||
def valid_visibility(cls, value):
|
||||
return parse_visibility(value, None)
|
||||
|
||||
|
||||
class VisibilityForm(BaseModel):
|
||||
value: str
|
||||
|
||||
@field_validator("value", mode="before")
|
||||
@classmethod
|
||||
def valid_value(cls, value):
|
||||
return parse_visibility(value, None) or DEFAULT_VISIBILITY
|
||||
|
||||
@@ -16,3 +16,8 @@ class GenerateImageForm(BaseModel):
|
||||
|
||||
class AttachmentRenameForm(BaseModel):
|
||||
filename: str = Field(min_length=1, max_length=255)
|
||||
|
||||
|
||||
class GiphyAttachForm(BaseModel):
|
||||
gif_id: str = Field(min_length=1, max_length=64)
|
||||
title: Optional[str] = Field(default=None, max_length=255)
|
||||
|
||||
@@ -76,5 +76,6 @@ A match **updates** the row (token, client_id, environment) and clears `deleted_
|
||||
- A `.p8` that does not parse is cached as a failure for the same window, so a misconfiguration costs one error log per window rather than one parse per notification.
|
||||
- **`DEAD_REASONS` is deliberately narrow: only `BadDeviceToken` (400), `Unregistered` (410) and `ExpiredToken` (410).** These are the only reasons in Apple's documented table that describe the TOKEN itself as permanently invalid. `DeviceTokenNotForTopic` and `TopicDisallowed` are 400 errors about the **topic/provisioning matching the connection**, not the token - they fire identically for every token when `push_apns_topic` is misconfigured or the certificate/entitlements don't match, so treating them as dead would soft-delete the entire APNs subscriber base (a table deliberately kept OUT of `SOFT_DELETE_TABLES`, hence unrestorable) on the first delivery after a one-field admin typo. Never add a topic/provisioning-shaped reason to `DEAD_REASONS`.
|
||||
- **A 410's `timestamp` is honored before deleting.** Apple's 410 body carries `{"reason": "Unregistered", "timestamp": <ms epoch>}` - the instant Apple last confirmed the token dead, which can trail real device state by "several days" per Apple engineering guidance (410 delivery is intentionally non-deterministic; do not use it to infer app-uninstall timing). `apns._dead_before` converts it to ISO and it rides on `Delivery.dead_before`; `store.mark_dead(id, dead_before)` compares it against the row's `registered_at` and **skips the delete** (logs at INFO instead) if the registration was re-registered/revived after that instant - closing the race where an in-flight delivery against a stale row state would otherwise undo a concurrent revival. `dead_before` is only ever set for a 410; the `BadDeviceToken`/`ExpiredToken` paths pass `None` (unconditional delete, as before), since those reasons carry no `timestamp`.
|
||||
- The shared payload dict (`title`, `message`, `icon`, `url`) is translated once per batch into `aps.alert` plus the custom `url`/`icon` keys, mirroring what `service-worker.js` does for Web Push. `thread-id` mirrors the service worker's notification `tag`.
|
||||
- The shared payload dict (`title`, `message`, `icon`, `url`, `unread`) is translated once per batch into `aps.alert` plus the custom `url`/`icon` keys, mirroring what `service-worker.js` does for Web Push. `thread-id` mirrors the service worker's notification `tag`.
|
||||
- **`unread` becomes `aps.badge`.** `unread` is the recipient's unread notification count, `count_unread_groups(user_uid)` (groups when grouping is on, rows when off: the same number as `/notifications/counts`), computed by `utils/notifications.py` `_deliver_notification` right after the in-app insert and only when a push is actually sent. `apns._badge` accepts a non-negative integer (a `bool` is refused) and otherwise omits `aps.badge`, which leaves the app icon untouched; the welcome probe carries no `unread`. iOS applies `aps.badge` itself, so the icon updates while the app is backgrounded or terminated. The key is named `unread`, never `badge`, because `badge` already means the monochrome icon URL in the Notifications API. Web Push forwards the payload unchanged and the service worker ignores the extra key. A throttled push is not sent, so the badge shows the count at the last delivered push until the app is opened.
|
||||
- `POST /push.json` probes a created or revived row immediately via `notify_registration`. The JSON is `{registered: true, delivered: bool, error?: string}`. `registered` stays true so existing clients keep working; `delivered`/`error` surface Apple's reason instead of killing the row silently from the client's point of view. The row is still marked dead on a `DEAD` outcome (subject to the `dead_before` guard above) so later `notify_user` calls do not keep hitting a known-bad token.
|
||||
|
||||
@@ -230,6 +230,13 @@ def _dead_before(response: httpx.Response) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _badge(payload: dict[str, Any]) -> int | None:
|
||||
unread = payload.get("unread")
|
||||
if isinstance(unread, bool) or not isinstance(unread, int) or unread < 0:
|
||||
return None
|
||||
return unread
|
||||
|
||||
|
||||
class ApnsProvider(PushProvider):
|
||||
name = "apns"
|
||||
label = PROVIDER_LABEL
|
||||
@@ -323,16 +330,20 @@ class ApnsProvider(PushProvider):
|
||||
return fields
|
||||
|
||||
def prepare(self, payload: dict[str, Any]) -> str:
|
||||
aps: dict[str, Any] = {
|
||||
"alert": {
|
||||
"title": payload.get("title") or "DevPlace",
|
||||
"body": payload.get("message") or "",
|
||||
},
|
||||
"sound": "default",
|
||||
"thread-id": THREAD_ID,
|
||||
}
|
||||
badge = _badge(payload)
|
||||
if badge is not None:
|
||||
aps["badge"] = badge
|
||||
return json.dumps(
|
||||
{
|
||||
"aps": {
|
||||
"alert": {
|
||||
"title": payload.get("title") or "DevPlace",
|
||||
"body": payload.get("message") or "",
|
||||
},
|
||||
"sound": "default",
|
||||
"thread-id": THREAD_ID,
|
||||
},
|
||||
"aps": aps,
|
||||
"url": payload.get("url") or DEFAULT_PUSH_URL,
|
||||
"icon": payload.get("icon") or PUSH_ICON,
|
||||
}
|
||||
|
||||
@@ -100,6 +100,7 @@ def ensure_certificates() -> None:
|
||||
):
|
||||
return
|
||||
lock_path = VAPID_PRIVATE_KEY_FILE.parent / ".vapid.lock"
|
||||
lock_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(lock_path, "w") as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX)
|
||||
try:
|
||||
|
||||
+43
-22
@@ -12,6 +12,8 @@ import emoji
|
||||
import mistune
|
||||
from markupsafe import Markup
|
||||
|
||||
from devplacepy.embeds import EMBED_ALLOW, MAX_MUSIC_EMBEDS, MUSIC_CAPTION, EmbedParser
|
||||
|
||||
EMOJI_JS_PATH = Path(__file__).parent / "static" / "js" / "emoji-shortcodes.js"
|
||||
|
||||
CUSTOM_SHORTCODES = {
|
||||
@@ -55,11 +57,6 @@ _WIDGET_RE = re.compile(r"<dp-widget>(.*?)</dp-widget>", re.DOTALL | re.IGNORECA
|
||||
_WIDGET_PH = "\x00WIDGET_{}\x00"
|
||||
|
||||
_SHORTCODE_RE = re.compile(r":([A-Za-z0-9_+\-]+):")
|
||||
_YOUTUBE_RE = re.compile(
|
||||
r"(?:https?://)?(?:www\.)?"
|
||||
r"(?:youtube\.com/(?:watch\?v=|embed/|v/|shorts/)|youtu\.be/)"
|
||||
r"([A-Za-z0-9_-]{11})"
|
||||
)
|
||||
_IMAGE_RE = re.compile(r"\.(jpg|jpeg|png|gif|webp|svg|bmp)(\?.*)?$", re.I)
|
||||
_VIDEO_RE = re.compile(r"\.(mp4|webm|ogg|ogv|mov|m4v)(\?.*)?$", re.I)
|
||||
_AUDIO_RE = re.compile(r"\.(mp3|wav|flac|ogg|aac|m4a|wma|opus)(\?.*)?$", re.I)
|
||||
@@ -90,11 +87,6 @@ def _split_trailing_punct(url: str) -> tuple[str, str]:
|
||||
break
|
||||
return url, "".join(reversed(trailing))
|
||||
|
||||
_YOUTUBE_ALLOW = (
|
||||
"accelerometer; autoplay; clipboard-write; encrypted-media; "
|
||||
"gyroscope; picture-in-picture"
|
||||
)
|
||||
|
||||
_EMAIL_RE = re.compile(r"\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b")
|
||||
_EMAIL_KEEP_DOMAIN = "molodetz.nl"
|
||||
|
||||
@@ -192,16 +184,44 @@ def _alt_from_url(url: str) -> str:
|
||||
return name if name else "Embedded image"
|
||||
|
||||
|
||||
def _embed_url(url: str) -> str:
|
||||
youtube = _YOUTUBE_RE.search(url)
|
||||
class _EmbedBudget:
|
||||
def __init__(self) -> None:
|
||||
self.music = 0
|
||||
|
||||
def allows(self, music: bool) -> bool:
|
||||
if not music:
|
||||
return True
|
||||
if self.music >= MAX_MUSIC_EMBEDS:
|
||||
return False
|
||||
self.music += 1
|
||||
return True
|
||||
|
||||
|
||||
def _youtube_embed(url: str, budget: _EmbedBudget) -> str | None:
|
||||
embed = EmbedParser.parse(url)
|
||||
if embed is None or not budget.allows(embed.music):
|
||||
return None
|
||||
lazy = " loading=\"lazy\"" if embed.music else ""
|
||||
frame = (
|
||||
f'<div class="embed-youtube"><iframe '
|
||||
f'src="{html.escape(EmbedParser.src(embed), quote=True)}" '
|
||||
f'frameborder="0" allowfullscreen{lazy} allow="{EMBED_ALLOW}">'
|
||||
f"</iframe></div>"
|
||||
)
|
||||
if not embed.music:
|
||||
return frame
|
||||
return (
|
||||
f'<div class="embed-music">{frame}'
|
||||
f'<a class="embed-music-caption" href="{html.escape(url, quote=True)}" '
|
||||
f'target="_blank" rel="noopener noreferrer">{html.escape(MUSIC_CAPTION)}</a>'
|
||||
f"</div>"
|
||||
)
|
||||
|
||||
|
||||
def _embed_url(url: str, budget: _EmbedBudget) -> str:
|
||||
youtube = _youtube_embed(url, budget)
|
||||
if youtube:
|
||||
video_id = youtube.group(1)
|
||||
return (
|
||||
f'<div class="embed-youtube"><iframe '
|
||||
f'src="https://www.youtube.com/embed/{video_id}" '
|
||||
f'frameborder="0" allowfullscreen allow="{_YOUTUBE_ALLOW}">'
|
||||
f"</iframe></div>"
|
||||
)
|
||||
return youtube
|
||||
escaped = html.escape(url, quote=True)
|
||||
if _IMAGE_RE.search(url):
|
||||
alt = html.escape(_alt_from_url(url), quote=True)
|
||||
@@ -230,7 +250,7 @@ def _mask_emails(text: str) -> str:
|
||||
return _EMAIL_RE.sub(_mask_email, text)
|
||||
|
||||
|
||||
def _transform_text(text: str) -> str:
|
||||
def _transform_text(text: str, budget: _EmbedBudget) -> str:
|
||||
out: list[str] = []
|
||||
pos = 0
|
||||
for match in _TOKEN_RE.finditer(text):
|
||||
@@ -238,7 +258,7 @@ def _transform_text(text: str) -> str:
|
||||
out.append(html.escape(_mask_emails(text[pos:match.start()])))
|
||||
if match.group("url"):
|
||||
url, trailing = _split_trailing_punct(match.group("url"))
|
||||
out.append(_embed_url(url))
|
||||
out.append(_embed_url(url, budget))
|
||||
if trailing:
|
||||
out.append(html.escape(trailing))
|
||||
else:
|
||||
@@ -269,6 +289,7 @@ class _MediaProcessor(HTMLParser):
|
||||
super().__init__(convert_charrefs=True)
|
||||
self._out: list[str] = []
|
||||
self._skip_depth = 0
|
||||
self._budget = _EmbedBudget()
|
||||
|
||||
def handle_starttag(self, tag: str, attrs: list) -> None:
|
||||
self._out.append(_emit_tag(tag, attrs))
|
||||
@@ -287,7 +308,7 @@ class _MediaProcessor(HTMLParser):
|
||||
if self._skip_depth > 0:
|
||||
self._out.append(html.escape(_mask_emails(data)))
|
||||
else:
|
||||
self._out.append(_transform_text(data))
|
||||
self._out.append(_transform_text(data, self._budget))
|
||||
|
||||
def result(self) -> str:
|
||||
return "".join(self._out)
|
||||
|
||||
+11
-2
@@ -19,7 +19,14 @@ def wants_json(request: Request) -> bool:
|
||||
return "application/json" in accept and "text/html" not in accept
|
||||
|
||||
|
||||
def respond(request: Request, template: str, context: dict, *, model: type[BaseModel]):
|
||||
def respond(
|
||||
request: Request,
|
||||
template: str,
|
||||
context: dict,
|
||||
*,
|
||||
model: type[BaseModel],
|
||||
status_code: int = 200,
|
||||
):
|
||||
if wants_json(request):
|
||||
try:
|
||||
payload = model.model_validate(context).model_dump(mode="json")
|
||||
@@ -29,7 +36,9 @@ def respond(request: Request, template: str, context: dict, *, model: type[BaseM
|
||||
return JSONResponse(payload)
|
||||
from devplacepy.templating import templates
|
||||
|
||||
return templates.TemplateResponse(request, template, context)
|
||||
return templates.TemplateResponse(
|
||||
request, template, context, status_code=status_code
|
||||
)
|
||||
|
||||
|
||||
def action_result(
|
||||
|
||||
@@ -34,6 +34,7 @@ Prefixes are wired in `main.py`:
|
||||
| `/issues` | issues/ package - issue tracker backed by Gitea (no local issue store): `index.py` (list `?state=`/`?page=`, detail `/{number}` with comments), `create.py` (async AI-enhanced filing `/create` enqueues a `issue_create` job, status at `/jobs/{uid}`), `comment.py` (synchronous, pushes to Gitea + notifies admins), `status.py` (admin open/closed), `attachments.py` (file attachments on open issues + comments, mirrored to Gitea native assets; add/list/delete with owner-or-admin + open-state guards) |
|
||||
| `/gists` | gists.py |
|
||||
| `/news` | news.py |
|
||||
| `/giphy` | giphy.py - GIF picker backend: `GET /giphy/status`, `GET /giphy/search`, `GET /giphy/trending` (cached, 30 calls/min/user, pure JSON) and `POST /giphy/attach` (stores the GIF as an attachment, audit `attachment.giphy`); see `services/giphy/CLAUDE.md` |
|
||||
| `/uploads` | uploads.py - attachment management CRUD for the signed-in user over the ONE `attachments` table (the same rows that appear on posts/comments/projects/gists/issues). Create: `POST /upload` (multipart), `POST /upload-url` (server-side fetch). Read: `GET ""` (own attachments, paginated 24/page newest-first, `?page=`, `?linked=true|false` via `database.get_user_attachments`), `GET /{attachment_uid}` (one, via `database.get_user_attachment`). Update: `PATCH /{attachment_uid}` (rename display filename via `attachments.rename_attachment`; the original file extension is ALWAYS preserved - renaming can never change the file type, the upload-time security control - audit `attachment.rename`). Delete: `DELETE /delete/{attachment_uid}` (soft delete). All `require_user_api` (401 for guests); read/rename/delete of another user's row is owner-or-admin. JSON-only router (no HTML/`respond`); list uses `UploadsListOut`, single/rename return `UploadItemOut`. Devii tools mirror every face: `upload_file`/`attach_url`/`list_attachments`/`get_attachment`/`rename_attachment`/`delete_attachment` |
|
||||
| `/media` | media.py - profile media gallery item soft delete/restore: `POST /media/{uid}/delete` and `POST /media/{uid}/restore` (owner or admin) |
|
||||
| `/openai` | openai_gateway.py |
|
||||
@@ -178,6 +179,10 @@ All dates displayed to users use European DD/MM/YYYY format. Implemented via:
|
||||
- **`time_ago()`** returns `DD/MM/YYYY` for items older than 30 days (instead of `"Xmo ago"`)
|
||||
- Services page has a JS `formatDate()` function for live polling updates
|
||||
|
||||
### Admin page specification
|
||||
|
||||
Every admin route and template conforms to `devplacepy/routers/admin/CLAUDE.md` (A1-A11). Read it before adding or changing anything under `/admin`; it is enforced by `tests/unit/admin_spec.py`.
|
||||
|
||||
### Admin pagination
|
||||
|
||||
Both `/admin/users` and `/admin/news` use offset-based pagination via a reusable component:
|
||||
@@ -293,10 +298,10 @@ All SEO features are implemented across the following locations:
|
||||
|
||||
### Post page layout (three columns, mirrors `/feed`)
|
||||
|
||||
`post.html` reuses `/feed`'s exact layout building blocks rather than inventing new ones, wrapped in `.post-page-layout` (`static/css/post.css`, `grid-template-columns: var(--sidebar-width) minmax(0, 1fr) 280px`, collapsing to one column at 1024px):
|
||||
`post.html` reuses `/feed`'s exact layout building blocks rather than inventing new ones, wrapped in `.post-page-layout` (`static/css/post.css`). The grid is `var(--sidebar-width) minmax(0, 1fr) 280px` only when both asides are present. Each column is placed explicitly (sidebar column 1, `.post-page` column 2, `.feed-right` column 3), and a missing aside drops its track instead of shifting the article into the 240px sidebar column. It collapses to one column at 1024px:
|
||||
|
||||
- **Left column** - `<aside class="post-page-sidebar">`, sticky, holding up to three separate `.sidebar-card` blocks (never merged into one card - each is its own bordered panel): "Gists from {author}" and "Projects from {author}" (`content.get_user_sidebar_gists`/`get_user_sidebar_projects`, up to 5 each, most-recently-modified first, private projects filtered via `can_view_project`), and "Related Discussions" (same-topic posts). Every card title is a `.sidebar-heading` div (the same class `feed.html`'s left sidebar uses for "Topics"/"Resources"/"Online now" - `.sidebar-card .sidebar-heading` in `sidebar.css`, so it only styles correctly nested inside a `.sidebar-card`, never bare). Each list item reuses the plain `.related-list`/`.related-link`/`.related-title`/`.related-meta` classes (`base.css`) with `content_preview()` for the ellipsis-truncated description.
|
||||
- **Middle column** - `.post-page` (`max-width: 720px`): the post article, comments.
|
||||
- **Middle column** - `.post-page` (`width: 100%`, `max-width: 720px`, centered in its column): the post article, comments. Below 1024px it fills the single column instead of shrink-wrapping to the post text.
|
||||
- **Right column** - `<aside class="feed-right">`, the exact same class `feed.html` uses for its Daily Topic widget (sticky, hidden below 1024px via `feed.css`'s own media query - no post-page-specific override needed). Holds the "Featured" cards (`database.get_featured_topics`, up to 3, cached pool + per-request random sample): each is a `.daily-topic-card` with its own `.daily-topic-label` ("Featured") - matching the single Daily Topic card's internal label, since there is no section-level heading here (a `.sidebar-heading` div placed directly in `.feed-right` would NOT be styled, as noted above - the fix used is a per-card label instead of a bare heading).
|
||||
|
||||
All three columns are populated by `routers/posts.py` `post_page_context()`, the single context builder shared by the real `/posts/{slug}` route and `happy404.render()` (see the root `CLAUDE.md`), so a decoy happy-404 post page renders with the identical sidebar/featured layout as a real one.
|
||||
@@ -377,3 +382,25 @@ The profile **Activity** tab (`/profile/{username}?tab=activity`, public) interl
|
||||
- **Comment items** get `"url": f"{resolve_object_url(target_type, target_uid)}#comment-{c['uid']}"`, the SEO-friendly parent detail URL plus the `#comment-{uid}` anchor. This is **byte-identical to a comment notification's `target_url`** (`content.create_comment_record`), so clicking an activity comment reproduces the notification click exactly: it lands on the parent post and `NotificationManager.js` scrolls to / highlights the `id="comment-{uid}"` element (`_comment.html`). Reuse `resolve_object_url` (`database.py`) - never rebuild this URL by hand, and do not call `resolve_object_url("comment", uid)` here (it re-fetches the row the loop already has).
|
||||
- **Frontend:** the card is wrapped in the shared full-card overlay link (`card-link-host` + `_card_link.html`), the same pattern `_post_card.html` uses; inner content links (mentions, URLs rendered by `render_content`) stay clickable via the existing `card-link-host a:not(.card-link)` z-index rule (`base.css`). No new CSS.
|
||||
- **API:** `ProfileOut.activities` is `list[Any]`, so the added `url` (and, on comments, the existing `target_type` + `uid` = parent target uid) flow into the JSON response with no schema change - this is the parent-post reference exposed on each comment item.
|
||||
|
||||
## Content visibility routes
|
||||
|
||||
Posts, gists, projects and news share one four-value status (published, hidden, draft, offline); the specification, truth table and choke points are in `devplacepy/database/CLAUDE.md` ("Content visibility"). It is an ordinary field of the create and edit forms of posts, gists and projects (`visibility`, via the `_visibility_field.html` partial), handled by `create_content_item` and `edit_content_item` like every other field; there are no dedicated toggle routes. The only dedicated route is the administrator's `POST /admin/news/{uid}/visibility` (`VisibilityForm{value}`), because news has no edit form. Hidden items are `noindex,nofollow`. Comments, votes, reactions, bookmarks and notes refuse (404) a target the viewer cannot open (`content.can_view_target`), and the saved/notes lists skip such targets. The devRant API resolves rants through `post_by_id(rant_id, viewer)` and lists listed rants only. Audit key: `<kind>.visibility.set`. Devii: the `visibility` parameter on the create and edit tools, plus `admin_set_news_visibility`.
|
||||
|
||||
## Notifications grouping and profile settings (requested_changes items 1, 4, 5)
|
||||
|
||||
- `POST /profile/{username}/enter-to-submit`, `POST /profile/{username}/notifications/grouping` and `POST /profile/{username}/notifications/throttling` are owner-or-admin toggles through `routers/profile/_shared.resolve_customization_target`, audited as `profile.enter_to_submit.toggle`, `profile.notification.grouping.toggle` and `profile.notification.throttle.toggle`.
|
||||
- `GET /notifications` groups events with `notification_group_key` when the recipient has grouping on (250 row fetch cap, 25 groups per page, date groups outside event groups). Opening or dismissing a grouped card marks every member read. With grouping off the page and badge keep the old one-row behaviour.
|
||||
|
||||
## Feature checklist (one data source, four faces)
|
||||
|
||||
Every feature is one data source fanning out into HTML, JSON (`respond(..., model=XOut)`; a context key not on `XOut` is silently dropped from JSON), a Devii agent action, and a documented endpoint. A change touching one file must confirm none of the four faces were missed. Checklist, ordered by data flow:
|
||||
|
||||
1. **Understand.** Read the router, template, matching tests, and the nested `CLAUDE.md`. Trace input model, router, data helper, response (HTML and JSON).
|
||||
2. **Data layer.** `database/` for query/batch helpers (never inline N+1; guard raw SQL with `if "table" in db.tables`). `models/` for the Pydantic `Form` (add to the domain module, re-export list, and `__all__`). `schemas/` for the `*Out` JSON model; every context key a JSON route exposes must exist on it. Indexes via `CREATE INDEX IF NOT EXISTS` in `init_db()`.
|
||||
3. **Server layer.** Guard matches the read (public), write (member), or admin surface. Specific paths before catch-alls. HTML+JSON via `respond(...)` or pure JSON via `JSONResponse`. Register any new router in `main.py`.
|
||||
4. **View layer.** Extend `base.html`; page CSS/JS via `extra_head`/`extra_js`. Reuse partials and template globals. ES6 modules, one class per file, instantiated on `app`.
|
||||
5. **Safety layer (skip only when the feature stores and shows nothing).** New user-generated surface: register in `REPORTABLE_TARGETS` (or `UNREPORTABLE_TABLES` with reason), resolve in `resolve_object_url`, include `_report_button.html`. New personal-data category or third-party transmission: update `/docs/privacy.html` and the app-store declarations in the same change.
|
||||
6. **Agent + docs layer.** Devii `Action` where a user could ask for this (`requires_auth` mirrors the guard). `docs_api.py` `endpoint()` with params and `sample_response`. `DOCS_PAGES` prose. `seo.py` context and sitemap entry for new public pages.
|
||||
7. **Document.** `README.md` for product-facing surface; the nested `CLAUDE.md` for any new mechanic or cross-layer wiring; root `CLAUDE.md` only for a new architectural rule or workflow step.
|
||||
8. **Validate.** Import gate, per-language manual checks, no em dashes, `pyflakes`/`ruff check` on every touched file, the four-layer rigor procedure for spendable/state-machine/race shapes, then full `make test` green before done. Failures at any step block the workflow.
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
# CLAUDE.md
|
||||
|
||||
The specification every `/admin` page follows. Claude Code loads it whenever a file under `routers/admin/` is touched; the root, `routers/`, `templates/`, `static/js/` and `static/css/` `CLAUDE.md` files point here. A page that breaks a rule below is a defect, not a style choice. `tests/unit/admin_spec.py` enforces the mechanical rules (A1, A2, A5, A6, A7, A9, A10) against the real templates and routers, so a violation fails the suite.
|
||||
|
||||
Code references: the section registry is `devplacepy/admin_sections.py`; the caps and floors are `ADMIN_LIST_CAP` and `ADMIN_POLL_FLOOR_SECONDS` in `routers/admin/_shared.py`; the cap note is `templates/_admin_cap_note.html`.
|
||||
|
||||
## A1. Backend rendered, always
|
||||
|
||||
A GET of an admin page returns the complete content in the HTML: every table, card, key figure, tab label and empty state is rendered by Jinja from data the handler computed. A placeholder such as "Loading..." in a template is forbidden. The page must be readable and correct with JavaScript off.
|
||||
|
||||
JavaScript only enhances: live refresh, charts drawn on a `<canvas>`, sorting, modals. The numbers and tables around a chart are server rendered; the chart is drawn from JSON embedded in the page.
|
||||
|
||||
The only permitted live-first surface is a page whose whole purpose is a live channel (WebSocket) such as the Bot Monitor and the container terminal, and even those render their initial frame on the server.
|
||||
|
||||
## A2. One handler, one builder, one model
|
||||
|
||||
- The HTML route calls `respond(request, "<template>.html", ctx, model=XOut)` with `base_seo_context(..., robots="noindex,nofollow")` and a breadcrumb trail `Home > Admin > <Page>`. Never `templates.TemplateResponse` directly: that drops the JSON face and the `*Out` schema.
|
||||
- The data the page shows is produced by ONE builder function. The HTML route and the refresh endpoint (`GET /admin/<page>/data`, `require_admin`, `JSONResponse`) both call it. The two must never compute the same figures twice with different code.
|
||||
- Context keys never collide with a Jinja global (root rule: `viewer_is_admin`, not `is_admin`).
|
||||
- Every admin page sets `"admin_section": "<key>"` where the key exists in `ADMIN_SECTIONS`.
|
||||
|
||||
## A3. Page anatomy
|
||||
|
||||
Every page extends `admin_base.html` and, in this order, renders:
|
||||
|
||||
1. `<div class="admin-toolbar">` holding the page title as `<h2>` (never `<h1>`; the layout owns the document heading), then `admin-count` spans, filters and the primary action.
|
||||
2. Optional `<nav class="admin-tabs" data-overflow-tabs>` for page tabs. A tab is a real link (`?tab=`), rendered on the server.
|
||||
3. The content: `admin-table-wrap > table.admin-table`, or cards, or a form `admin-settings-form`.
|
||||
4. The empty state: `<p class="admin-empty">`, never `empty-state`, never an empty table.
|
||||
5. Pagination via `_pagination.html` when the list is paged (A6).
|
||||
|
||||
Tables carry `<caption class="sr-only">`, every `<th>` has `scope="col"`, every icon-only or repeated row button has an `aria-label`. Page CSS goes in `{% block extra_head %}` after `{{ super() }}` (never re-link `admin.css` or `sidebar.css`; `admin_base.html` already loads them). No inline `<style>`, no `style=` attributes, no inline event handlers.
|
||||
|
||||
## A4. Class vocabulary
|
||||
|
||||
Buttons are `admin-btn` with `admin-btn-primary`, `admin-btn-danger`, `admin-btn-sm`. Fields are `admin-field`, selects `admin-select`, forms `admin-settings-form` or `admin-inline-form`, hints `admin-hint`. Status chips reuse the page family badge (`service-status`, `cm-badge`, `workspace-badge`). The public-site classes `btn`, `btn-primary`, `btn-danger`, `hint-text`, `empty-state`, `admin-header` are forbidden in admin templates. A missing admin class is added to `admin.css`, never worked around with a public class. Destructive buttons carry `data-confirm` (and `data-confirm-danger` for role/quota style changes).
|
||||
|
||||
## A5. Time
|
||||
|
||||
Every instant shown to a viewer is `local_dt(iso, mode)` or `dt_ago(iso)`; never `format_date`/`time_ago` for an instant (those stay for JSON and non-timestamp date fields). When client JS re-renders a row it emits the same `<time data-dt="...">` markup, and `LocalTime` localizes it through its MutationObserver; JS never formats an instant itself with `toLocaleString`/`toLocaleTimeString`.
|
||||
|
||||
## A6. Everything is capped
|
||||
|
||||
No list is unbounded.
|
||||
|
||||
- A list that can grow with users or content is paginated: `build_pagination(page, total)` plus `_limit/_offset` in the query and `_pagination.html`. Sorting is in SQL (`order_by`), never `rows.sort()` over the whole table.
|
||||
- A list that is naturally small (services, gateway providers) still reads with `_limit=ADMIN_LIST_CAP` and renders `_admin_cap_note.html` (`_shown`, `_total`) so truncation is visible, never silent.
|
||||
- Per-row work (queries, `await`s, service calls) is done for the visible page only, in batch (`get_users_by_uids` and friends), never per row of the whole table.
|
||||
- Query parameters that size a computation (`hours`, `top_n`, `limit`) are clamped on the server to a declared constant.
|
||||
|
||||
## A7. Heavy and remote work
|
||||
|
||||
A GET handler must not wait on a remote service. Gitea, providers and any network call go through a cache with a fixed TTL and a hard item cap; the handler renders the cached value at once and keeps the page's explicit error state (`*_error`) when the source is down. A computation costing more than one page of plain queries (statistics, usage analytics) is computed once per request: the HTML route embeds its result, and a tab or window change either navigates (server render) or fetches `/data`, never both.
|
||||
|
||||
## A8. Live refresh
|
||||
|
||||
- The shared `Poller` only: `new Poller(fn, ms, { pauseHidden: true, immediate: false })`. `immediate: false` because the first paint is already on the page (A1). Raw `setInterval` is forbidden.
|
||||
- Interval floor is `ADMIN_POLL_FLOOR_SECONDS` (15 s); a page that needs faster is a live channel.
|
||||
- Fetch through `Http.getJson`; never a bare `fetch`.
|
||||
- The refresh response is the `/data` JSON of A2, or, for a page made of server-rendered sections, HTML fetched with `Http.getHtml` (the page itself, or a `/fragment` route returning the same partial) and swapped into the `[data-live]` regions (Backups, Community pages, AI usage, Gateway stats). The markup then lives only in Jinja and the one builder feeds the page, the fragment and `/data`. The client re-renders only the regions that changed and preserves open modals, focus, scroll position and input values.
|
||||
- Initial data for charts is embedded as `<script type="application/json" id="<root>-initial">{{ initial | tojson }}</script>` and read synchronously; the client does not issue a second request to learn what the server already knew.
|
||||
|
||||
## A9. Assets
|
||||
|
||||
- One copy of each vendored library. Chart.js is `/static/vendor/chartjs/chart.umd.min.js`, loaded with `defer` through `static_url(...)` from `{% block extra_js %}` only on pages that draw charts.
|
||||
- Page JS is one ES6 module per class in `static/js/`, started from a three-line module block in `extra_js`. No page logic inline.
|
||||
|
||||
## A10. Navigation is one registry
|
||||
|
||||
The admin sections are declared once in `devplacepy/admin_sections.py` (`ADMIN_SECTIONS`, exposed as the `admin_sections` Jinja global). `admin_base.html` (desktop sidebar) and the admin block of `base.html` (the phone menu, because the sidebar is hidden at 1024 px and below) both loop over it. A new admin page adds one registry entry and nothing else; hand-writing a nav link in either template is forbidden. Every `admin_section` value passed by a route must exist in the registry.
|
||||
|
||||
## A11. Responsive
|
||||
|
||||
Everything works at 360 px. Tables sit in `admin-table-wrap` (horizontal scroll inside the wrap, never the page). A toolbar with more than three controls wraps; a button group of more than three becomes a `<select>` or a `dp-action-menu`. Touch targets are at least 44 px (`admin.css` already enforces it for `admin-btn`/`admin-select`).
|
||||
|
||||
## Checklist for a new or changed admin page
|
||||
|
||||
1. Registry entry in `ADMIN_SECTIONS`, `admin_section` matches.
|
||||
2. Builder function, `respond(... model=XOut)`, `/data` endpoint if it refreshes, audit event for every mutation.
|
||||
3. Template follows A3, A4, A5; no placeholder text; caps and pagination per A6.
|
||||
4. JS (only if needed) per A8 and A9.
|
||||
5. `tests/api/admin/...` asserts the first response already contains the data (not "Loading"), and `tests/unit/admin_spec.py` still passes.
|
||||
@@ -8,6 +8,10 @@ from devplacepy.responses import action_result
|
||||
from devplacepy.services.audit import record as audit
|
||||
|
||||
|
||||
ADMIN_LIST_CAP = 200
|
||||
ADMIN_POLL_FLOOR_SECONDS = 15
|
||||
|
||||
|
||||
def seniority_key(user: dict) -> tuple[str, int]:
|
||||
return (user.get("created_at") or "", user.get("id") or 0)
|
||||
|
||||
|
||||
@@ -8,16 +8,46 @@ from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.responses import respond
|
||||
from devplacepy.schemas import GatewayUsageOut
|
||||
from devplacepy.services.manager import service_manager
|
||||
from devplacepy.services.openai_gateway.analytics import build_analytics
|
||||
from devplacepy.services.openai_gateway.analytics import MAX_WINDOW_HOURS, build_analytics
|
||||
from devplacepy.services.openai_gateway.usage import pricing_from_cfg
|
||||
from devplacepy.templating import templates
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
DEFAULT_WINDOW_HOURS = 48
|
||||
MAX_TOP_N = 50
|
||||
WINDOW_CHOICES = (
|
||||
(1, "1 hour"),
|
||||
(24, "24 hours"),
|
||||
(48, "48 hours"),
|
||||
(168, "7 days"),
|
||||
(0, "All time"),
|
||||
)
|
||||
|
||||
|
||||
def _clamp_hours(hours: int) -> int:
|
||||
if hours <= 0:
|
||||
return 0
|
||||
return min(hours, MAX_WINDOW_HOURS)
|
||||
|
||||
|
||||
def _clamp_top_n(top_n: int) -> int:
|
||||
return max(1, min(top_n, MAX_TOP_N))
|
||||
|
||||
|
||||
def build_usage(hours: int, top_n: int) -> dict:
|
||||
service = service_manager.get_service("openai")
|
||||
pricing = pricing_from_cfg(service.get_config()) if service is not None else None
|
||||
return build_analytics(_clamp_hours(hours), top_n=_clamp_top_n(top_n), pricing=pricing)
|
||||
|
||||
|
||||
@router.get("/ai-usage", response_class=HTMLResponse)
|
||||
async def admin_ai_usage(request: Request):
|
||||
async def admin_ai_usage(
|
||||
request: Request, hours: int = DEFAULT_WINDOW_HOURS, top_n: int = 10
|
||||
):
|
||||
admin = require_admin(request)
|
||||
usage = build_usage(hours, top_n)
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
@@ -39,14 +69,28 @@ async def admin_ai_usage(request: Request):
|
||||
"request": request,
|
||||
"user": admin,
|
||||
"admin_section": "ai-usage",
|
||||
"usage": usage,
|
||||
"window_choices": WINDOW_CHOICES,
|
||||
"window_hours": usage["window_hours"],
|
||||
**usage,
|
||||
},
|
||||
model=GatewayUsageOut,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/ai-usage/data")
|
||||
async def admin_ai_usage_json(request: Request, hours: int = 48, top_n: int = 10):
|
||||
async def admin_ai_usage_json(
|
||||
request: Request, hours: int = DEFAULT_WINDOW_HOURS, top_n: int = 10
|
||||
):
|
||||
require_admin(request)
|
||||
service = service_manager.get_service("openai")
|
||||
pricing = pricing_from_cfg(service.get_config()) if service is not None else None
|
||||
return JSONResponse(build_analytics(hours, top_n=top_n, pricing=pricing))
|
||||
return JSONResponse(build_usage(hours, top_n))
|
||||
|
||||
|
||||
@router.get("/ai-usage/fragment", response_class=HTMLResponse)
|
||||
async def admin_ai_usage_fragment(
|
||||
request: Request, hours: int = DEFAULT_WINDOW_HOURS, top_n: int = 10
|
||||
):
|
||||
require_admin(request)
|
||||
usage = build_usage(hours, top_n)
|
||||
html = templates.get_template("_ai_usage_report.html").render(usage=usage)
|
||||
return HTMLResponse(html)
|
||||
|
||||
@@ -20,6 +20,8 @@ from devplacepy.services.jobs import queue
|
||||
from devplacepy.utils import is_primary_admin, not_found, require_admin
|
||||
from fastapi import Depends, HTTPException
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.database import db, get_table
|
||||
from devplacepy.routers.admin._shared import ADMIN_LIST_CAP
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
@@ -63,12 +65,16 @@ def _targets() -> list[dict]:
|
||||
]
|
||||
|
||||
async def _dashboard(can_download: bool) -> dict:
|
||||
backups = [_backup_payload(row, can_download) for row in store.list_backups()]
|
||||
backups = [
|
||||
_backup_payload(row, can_download)
|
||||
for row in store.list_backups(limit=ADMIN_LIST_CAP)
|
||||
]
|
||||
schedules = store.list_schedules()
|
||||
storage = await asyncio.to_thread(store.compute_storage_stats)
|
||||
return {
|
||||
"storage": storage,
|
||||
"backups": backups,
|
||||
"backups_total": get_table("backups").count() if "backups" in db.tables else 0,
|
||||
"schedules": schedules,
|
||||
"targets": _targets(),
|
||||
"metrics": _metrics(backups),
|
||||
|
||||
@@ -10,7 +10,7 @@ from urllib.parse import urlencode
|
||||
from fastapi import APIRouter, Form, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from devplacepy.database import get_setting, get_table, get_users_by_uids, resolve_object_url
|
||||
from devplacepy.database import build_pagination, get_setting, get_table, get_users_by_uids, resolve_object_url
|
||||
from devplacepy.database.sync import count_links, identity_map
|
||||
from devplacepy.responses import action_result, json_error, respond
|
||||
from devplacepy.schemas import AdminCommunitiesOut, AdminCommunityLedgerOut, AdminCommunityOut, AdminCommunitySchemaOut
|
||||
@@ -174,8 +174,10 @@ def feed_summary(feed: dict) -> dict:
|
||||
|
||||
|
||||
@router.get("/communities", response_class=HTMLResponse)
|
||||
async def admin_communities(request: Request):
|
||||
async def admin_communities(request: Request, page: int = 1):
|
||||
admin = require_admin(request)
|
||||
pagination = build_pagination(page, store.count())
|
||||
offset = (pagination["page"] - 1) * pagination["per_page"]
|
||||
return respond(
|
||||
request,
|
||||
"admin_communities.html",
|
||||
@@ -183,7 +185,8 @@ async def admin_communities(request: Request):
|
||||
**seo(request, "Communities", LIST_URL, "Communities"),
|
||||
"request": request,
|
||||
"user": admin,
|
||||
"items": [public(row) for row in store.list()],
|
||||
"items": [public(row) for row in store.list(pagination["per_page"], offset)],
|
||||
"pagination": pagination,
|
||||
"sync_enabled": sync_enabled(),
|
||||
**service_flags(),
|
||||
"admin_section": "communities",
|
||||
|
||||
@@ -6,6 +6,7 @@ from fastapi import Depends, APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
|
||||
|
||||
from devplacepy.database import (
|
||||
build_pagination,
|
||||
db,
|
||||
get_table,
|
||||
get_users_by_uids,
|
||||
@@ -111,10 +112,20 @@ def _audit_admin(request: Request, admin: dict, event_key: str, inst: dict, summ
|
||||
result=result,
|
||||
)
|
||||
|
||||
def _containers_payload(admin: dict, page: int) -> dict:
|
||||
visible = _decorate(store.all_instances(), admin)
|
||||
pagination = build_pagination(page, len(visible))
|
||||
start = (pagination["page"] - 1) * pagination["per_page"]
|
||||
return {
|
||||
"instances": visible[start : start + pagination["per_page"]],
|
||||
"pagination": pagination,
|
||||
}
|
||||
|
||||
|
||||
@router.get("", response_class=HTMLResponse)
|
||||
async def containers_index(request: Request):
|
||||
async def containers_index(request: Request, page: int = 1):
|
||||
admin = require_admin(request)
|
||||
instances = _decorate(store.all_instances(), admin)
|
||||
payload = _containers_payload(admin, page)
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
@@ -136,7 +147,7 @@ async def containers_index(request: Request):
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": admin,
|
||||
"instances": instances,
|
||||
**payload,
|
||||
"boot_languages": BOOT_LANGUAGES,
|
||||
"restart_policies": RESTART_POLICIES,
|
||||
"admin_section": "containers",
|
||||
@@ -145,9 +156,9 @@ async def containers_index(request: Request):
|
||||
)
|
||||
|
||||
@router.get("/data")
|
||||
async def containers_index_json(request: Request):
|
||||
async def containers_index_json(request: Request, page: int = 1):
|
||||
admin = require_admin(request)
|
||||
return JSONResponse({"instances": _decorate(store.all_instances(), admin)})
|
||||
return JSONResponse(_containers_payload(admin, page))
|
||||
|
||||
@router.get("/projects/search")
|
||||
async def project_search(request: Request, q: str = ""):
|
||||
|
||||
@@ -5,7 +5,13 @@ import logging
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from devplacepy.database import db, get_admin_uids, get_int_setting, get_users_by_uids
|
||||
from devplacepy.database import (
|
||||
build_pagination,
|
||||
db,
|
||||
get_admin_uids,
|
||||
get_int_setting,
|
||||
get_users_by_uids,
|
||||
)
|
||||
from devplacepy.responses import action_result, respond
|
||||
from devplacepy.schemas import AdminDeviiTasksOut
|
||||
from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
@@ -32,17 +38,26 @@ def _schedule_text(row: dict) -> str:
|
||||
return f"once {row.get('run_at') or ''}".strip()
|
||||
|
||||
|
||||
def _rows(state: str) -> list[dict]:
|
||||
def _page(state: str, page: int) -> tuple[list[dict], dict]:
|
||||
if TABLE not in db.tables:
|
||||
return []
|
||||
return [], build_pagination(page, 0)
|
||||
criteria: dict = {"deleted_at": None}
|
||||
if state == "active":
|
||||
criteria["enabled"] = True
|
||||
elif state == "inactive":
|
||||
criteria["enabled"] = False
|
||||
rows = list(db[TABLE].find(**criteria))
|
||||
rows.sort(key=lambda row: int(row.get("run_count") or 0), reverse=True)
|
||||
return rows
|
||||
table = db[TABLE]
|
||||
pagination = build_pagination(page, table.count(**criteria))
|
||||
offset = (pagination["page"] - 1) * pagination["per_page"]
|
||||
rows = list(
|
||||
table.find(
|
||||
order_by=["-run_count", "-created_at"],
|
||||
_limit=pagination["per_page"],
|
||||
_offset=offset,
|
||||
**criteria,
|
||||
)
|
||||
)
|
||||
return rows, pagination
|
||||
|
||||
|
||||
def _quotas(owner_uids: set[str]) -> dict[str, dict]:
|
||||
@@ -98,11 +113,11 @@ def _require_task(uid: str) -> dict:
|
||||
|
||||
|
||||
@router.get("/devii-tasks", response_class=HTMLResponse)
|
||||
async def admin_devii_tasks(request: Request, state: str = "active"):
|
||||
async def admin_devii_tasks(request: Request, state: str = "active", page: int = 1):
|
||||
admin = require_admin(request)
|
||||
if state not in STATES:
|
||||
state = "active"
|
||||
rows = _rows(state)
|
||||
rows, pagination = _page(state, page)
|
||||
items = _items(rows)
|
||||
bounds = {
|
||||
"max_concurrent": get_int_setting(
|
||||
@@ -149,6 +164,7 @@ async def admin_devii_tasks(request: Request, state: str = "active"):
|
||||
"request": request,
|
||||
"user": admin,
|
||||
"items": items,
|
||||
"pagination": pagination,
|
||||
"state": state,
|
||||
"tabs": tabs,
|
||||
"limits": bounds,
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
from urllib.parse import quote
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi import APIRouter, Query, Request
|
||||
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
|
||||
from pydantic import ValidationError
|
||||
|
||||
@@ -15,9 +17,21 @@ from devplacepy.schemas import (
|
||||
AdminGatewayQuotaFormOut,
|
||||
)
|
||||
from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.routers.admin._shared import ADMIN_LIST_CAP
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.manager import service_manager
|
||||
from devplacepy.services.openai_gateway import model_stats_query, quota, routing
|
||||
from devplacepy.services.openai_gateway import (
|
||||
health,
|
||||
model_stats_query,
|
||||
quota,
|
||||
routing,
|
||||
)
|
||||
from devplacepy.services.openai_gateway.learned import (
|
||||
DEFAULT_PROVIDER,
|
||||
LearnedResetIn,
|
||||
learned_store,
|
||||
)
|
||||
from devplacepy.services.openai_gateway.shape import shape_stats
|
||||
from devplacepy.templating import templates
|
||||
from devplacepy.utils import not_found, require_admin
|
||||
|
||||
@@ -26,6 +40,7 @@ router = APIRouter()
|
||||
|
||||
INDEX_URL = "/admin/gateway"
|
||||
TABS = ("models", "providers", "quota", "stats")
|
||||
DEFAULT_STATS_RANGE = "24h"
|
||||
|
||||
|
||||
def _default_provider_summary() -> dict:
|
||||
@@ -137,27 +152,69 @@ def _error_response(
|
||||
):
|
||||
if wants_json(request):
|
||||
return json_error(status_code, message)
|
||||
return templates.TemplateResponse(
|
||||
return respond(
|
||||
request,
|
||||
template,
|
||||
{**context, "request": request, "error": message},
|
||||
model=AdminGatewayOut,
|
||||
status_code=status_code,
|
||||
)
|
||||
|
||||
|
||||
def _models_with_learned(limit: int = ADMIN_LIST_CAP) -> list[dict]:
|
||||
models = routing.model_store.list()[:limit]
|
||||
for model in models:
|
||||
method = learned_store.get(
|
||||
model["provider"] or DEFAULT_PROVIDER, model["target_model"]
|
||||
)
|
||||
model["learned_shape"] = method.label if method is not None else ""
|
||||
model["learned_provider"] = model["provider"] or DEFAULT_PROVIDER
|
||||
return models
|
||||
|
||||
|
||||
def _reset_learned(request: Request, admin: dict, payload: LearnedResetIn) -> bool:
|
||||
provider = payload.provider or DEFAULT_PROVIDER
|
||||
existed = learned_store.forget(provider, payload.target_model)
|
||||
if existed:
|
||||
audit.record(
|
||||
request,
|
||||
"gateway.learned.reset",
|
||||
user=admin,
|
||||
target_type="gateway_learned_method",
|
||||
target_uid=f"{provider}/{payload.target_model}",
|
||||
target_label=f"{provider}/{payload.target_model}",
|
||||
summary=(
|
||||
f"admin {admin['username']} reset the learned request shape "
|
||||
f"for {provider}/{payload.target_model}"
|
||||
),
|
||||
)
|
||||
return existed
|
||||
|
||||
|
||||
# --- Index page (tabs: models / providers / quota) --------------------------
|
||||
|
||||
|
||||
@router.get("/gateway", response_class=HTMLResponse)
|
||||
async def gateway_config_page(request: Request, tab: str = "models"):
|
||||
async def gateway_config_page(
|
||||
request: Request,
|
||||
tab: str = "models",
|
||||
tested: str = "",
|
||||
test_image: str = "",
|
||||
stats_range: str = Query(DEFAULT_STATS_RANGE, alias="range"),
|
||||
):
|
||||
admin = require_admin(request)
|
||||
if tab not in TABS:
|
||||
tab = "models"
|
||||
quota_rules = quota.quota_rule_store.list()
|
||||
if not test_image.startswith("http"):
|
||||
test_image = ""
|
||||
all_quota_rules = quota.quota_rule_store.list()
|
||||
quota_rules = all_quota_rules[:ADMIN_LIST_CAP]
|
||||
for rule in quota_rules:
|
||||
rule["spent_24h_usd"] = round(
|
||||
quota.spent_24h(rule["owner_kind"], rule["owner_id"], rule["app_reference"]), 6
|
||||
)
|
||||
all_providers = routing.provider_store.list()
|
||||
stats_context = _stats_context(stats_range) if tab == "stats" else {}
|
||||
return respond(
|
||||
request,
|
||||
"admin_gateway.html",
|
||||
@@ -166,12 +223,18 @@ async def gateway_config_page(request: Request, tab: str = "models"):
|
||||
"user": admin,
|
||||
"admin_section": "gateway",
|
||||
"tab": tab,
|
||||
"providers": routing.provider_store.list(),
|
||||
"models": routing.model_store.list(),
|
||||
"providers": all_providers[:ADMIN_LIST_CAP],
|
||||
"providers_total": len(all_providers),
|
||||
"models": _models_with_learned(),
|
||||
"models_total": routing.model_store.count(),
|
||||
"quota_rules": quota_rules,
|
||||
"quota_rules_total": len(all_quota_rules),
|
||||
**stats_context,
|
||||
"default_provider": _default_provider_summary(),
|
||||
"quota_defaults": _quota_defaults_summary(),
|
||||
"stats_ranges": list(model_stats_query.RANGE_SECONDS.keys()),
|
||||
"tested": tested,
|
||||
"test_image": test_image,
|
||||
},
|
||||
model=AdminGatewayOut,
|
||||
)
|
||||
@@ -376,6 +439,44 @@ async def list_models(request: Request):
|
||||
)
|
||||
|
||||
|
||||
@router.get("/gateway/learned")
|
||||
async def list_learned(request: Request):
|
||||
require_admin(request)
|
||||
rows = learned_store.list()
|
||||
return JSONResponse(
|
||||
{"learned": rows, "count": len(rows), "stats": shape_stats.snapshot()}
|
||||
)
|
||||
|
||||
|
||||
@router.post("/gateway/learned/reset")
|
||||
async def reset_learned_json(request: Request):
|
||||
admin = require_admin(request)
|
||||
body = await _payload(request)
|
||||
try:
|
||||
payload = LearnedResetIn(**body)
|
||||
except ValidationError as exc:
|
||||
return _validation_error(exc)
|
||||
if not _reset_learned(request, admin, payload):
|
||||
return JSONResponse(
|
||||
{"ok": False, "error": "No learned method for that route"},
|
||||
status_code=404,
|
||||
)
|
||||
return JSONResponse({"ok": True})
|
||||
|
||||
|
||||
@router.post("/gateway/learned/reset-page")
|
||||
async def reset_learned_page(request: Request):
|
||||
admin = require_admin(request)
|
||||
body = await _payload(request)
|
||||
try:
|
||||
payload = LearnedResetIn(**body)
|
||||
except ValidationError as exc:
|
||||
return _validation_error(exc)
|
||||
if not _reset_learned(request, admin, payload):
|
||||
raise not_found("No learned method for that route")
|
||||
return RedirectResponse(url=f"{INDEX_URL}?tab=models", status_code=302)
|
||||
|
||||
|
||||
@router.get("/gateway/provider-models")
|
||||
async def provider_models(request: Request, provider: str = ""):
|
||||
require_admin(request)
|
||||
@@ -472,6 +573,7 @@ MODEL_FORM_DEFAULTS = {
|
||||
"off_peak_end": "",
|
||||
"off_peak_discount_pct": "0",
|
||||
"fallback_model": "",
|
||||
"tools_mode": "native",
|
||||
"is_active": "1",
|
||||
}
|
||||
|
||||
@@ -505,6 +607,7 @@ def _model_form_values_from_route(route: dict) -> dict:
|
||||
"off_peak_end": _minutes_to_time(route.get("off_peak_end_minute")),
|
||||
"off_peak_discount_pct": _num("off_peak_discount_pct"),
|
||||
"fallback_model": route.get("fallback_model", ""),
|
||||
"tools_mode": route.get("tools_mode", "native"),
|
||||
"is_active": _bool_str(route.get("is_active", True)),
|
||||
}
|
||||
|
||||
@@ -548,6 +651,7 @@ def _model_payload_kwargs(data: dict) -> dict:
|
||||
"off_peak_end_minute": _time_to_minutes(data.get("off_peak_end", "")),
|
||||
"off_peak_discount_pct": _num("off_peak_discount_pct"),
|
||||
"fallback_model": data.get("fallback_model", ""),
|
||||
"tools_mode": data.get("tools_mode", "native"),
|
||||
"is_active": data.get("is_active", "1"),
|
||||
}
|
||||
|
||||
@@ -675,6 +779,58 @@ async def delete_model_page(request: Request, source_model: str):
|
||||
return RedirectResponse(url=f"{INDEX_URL}?tab=models", status_code=302)
|
||||
|
||||
|
||||
@router.post("/gateway/models/test-all")
|
||||
async def test_all_models_page(request: Request):
|
||||
admin = require_admin(request)
|
||||
tested, failed = await health.test_all()
|
||||
audit.record(
|
||||
request,
|
||||
"gateway.models.test_all",
|
||||
user=admin,
|
||||
target_type="gateway_models",
|
||||
target_uid="all",
|
||||
target_label=f"{tested} routes",
|
||||
summary=(
|
||||
f"admin {admin['username']} tested all gateway model routes: "
|
||||
f"{tested - failed} ok, {failed} failed"
|
||||
),
|
||||
)
|
||||
return RedirectResponse(
|
||||
url=f"{INDEX_URL}?tab=models&tested=all", status_code=302
|
||||
)
|
||||
|
||||
|
||||
@router.post("/gateway/models/{source_model:path}/test")
|
||||
async def test_model_page(request: Request, source_model: str):
|
||||
admin = require_admin(request)
|
||||
route = routing.model_store.get(source_model)
|
||||
if route is None:
|
||||
raise not_found("Model route not found")
|
||||
result = await health.probe_route(source_model)
|
||||
health.record_health(source_model, result)
|
||||
audit.record(
|
||||
request,
|
||||
"gateway.model.test",
|
||||
user=admin,
|
||||
target_type="gateway_model",
|
||||
target_uid=source_model,
|
||||
target_label=source_model,
|
||||
summary=(
|
||||
f"admin {admin['username']} tested gateway model route "
|
||||
f"{source_model}: {'ok' if result.ok else result.code}"
|
||||
),
|
||||
)
|
||||
url = f"{INDEX_URL}?tab=models&tested={quote(source_model, safe='')}"
|
||||
if (
|
||||
result.ok
|
||||
and route.kind == "image"
|
||||
and result.image_url.startswith("http")
|
||||
and len(result.image_url) <= 2000
|
||||
):
|
||||
url += f"&test_image={quote(result.image_url, safe='')}"
|
||||
return RedirectResponse(url=url, status_code=302)
|
||||
|
||||
|
||||
# --- Quota rules: JSON API (Devii + programmatic clients) -------------------
|
||||
|
||||
|
||||
@@ -970,8 +1126,28 @@ async def reset_quota_rule_page(request: Request, uid: str):
|
||||
# --- Stats: JSON API for the Stats tab's charts ------------------------------
|
||||
|
||||
|
||||
def _stats_context(range_key: str) -> dict:
|
||||
if range_key not in model_stats_query.RANGE_SECONDS:
|
||||
range_key = DEFAULT_STATS_RANGE
|
||||
summary = model_stats_query.compute_summary(range_key)
|
||||
return _stats_view(summary, range_key)
|
||||
|
||||
|
||||
def _stats_view(summary: dict, range_key: str) -> dict:
|
||||
per_model = summary["per_model"]
|
||||
generated = datetime.fromtimestamp(summary["generated_at"], tz=timezone.utc)
|
||||
return {
|
||||
"stats": summary,
|
||||
"stats_range": range_key,
|
||||
"stats_generated_iso": generated.isoformat(),
|
||||
"stats_models": model_stats_query.list_known_models()[:ADMIN_LIST_CAP],
|
||||
"stats_table_models": per_model[:ADMIN_LIST_CAP],
|
||||
"stats_table_models_total": len(per_model),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/gateway/stats/data")
|
||||
async def gateway_stats_data(request: Request, range: str = "24h"):
|
||||
async def gateway_stats_data(request: Request, range: str = DEFAULT_STATS_RANGE):
|
||||
require_admin(request)
|
||||
try:
|
||||
return JSONResponse(model_stats_query.compute_summary(range))
|
||||
@@ -979,6 +1155,17 @@ async def gateway_stats_data(request: Request, range: str = "24h"):
|
||||
return json_error(400, f"Unknown range: {range!r}")
|
||||
|
||||
|
||||
@router.get("/gateway/stats/fragment", response_class=HTMLResponse)
|
||||
async def gateway_stats_fragment(request: Request, range: str = DEFAULT_STATS_RANGE):
|
||||
require_admin(request)
|
||||
if range not in model_stats_query.RANGE_SECONDS:
|
||||
return json_error(400, f"Unknown range: {range!r}")
|
||||
context = _stats_context(range)
|
||||
return HTMLResponse(
|
||||
templates.get_template("_gateway_stats_fragment.html").render(**context)
|
||||
)
|
||||
|
||||
|
||||
@router.get("/gateway/stats/models")
|
||||
async def gateway_stats_models(request: Request):
|
||||
require_admin(request)
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from devplacepy.cache import TTLCache
|
||||
from devplacepy.responses import respond
|
||||
from devplacepy.schemas import AdminIssuesPlanningOut
|
||||
from devplacepy.seo import base_seo_context
|
||||
@@ -16,10 +18,20 @@ from devplacepy.utils import require_admin
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
TICKETS_CACHE_SECONDS = 60
|
||||
TICKETS_CACHE_KEY = "open"
|
||||
TICKETS_FETCH_TIMEOUT_SECONDS = 8
|
||||
_tickets_cache = TTLCache(TICKETS_CACHE_SECONDS, max_size=1)
|
||||
|
||||
|
||||
async def _open_tickets() -> tuple[list[dict], bool]:
|
||||
cached = _tickets_cache.get(TICKETS_CACHE_KEY)
|
||||
if cached is not None:
|
||||
return cached, False
|
||||
try:
|
||||
issues = await collect_open_issues(runtime.get_client())
|
||||
issues = await asyncio.wait_for(
|
||||
collect_open_issues(runtime.get_client()), TICKETS_FETCH_TIMEOUT_SECONDS
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("Could not load open tickets for planning: %s", exc)
|
||||
return [], True
|
||||
@@ -35,6 +47,7 @@ async def _open_tickets() -> tuple[list[dict], bool]:
|
||||
}
|
||||
for issue in issues
|
||||
]
|
||||
_tickets_cache.set(TICKETS_CACHE_KEY, tickets)
|
||||
return tickets, False
|
||||
|
||||
|
||||
@@ -49,7 +62,7 @@ async def admin_issues_planning(request: Request):
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
{"name": "Admin", "url": "/admin"},
|
||||
{"name": "Issues", "url": "/issues"},
|
||||
{"name": "Ticket Planning", "url": "/admin/issues/planning"},
|
||||
],
|
||||
)
|
||||
configured = gitea_config().is_configured
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from fastapi import APIRouter, Request
|
||||
from typing import Annotated
|
||||
from fastapi import Depends, APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
from devplacepy.database import (
|
||||
get_table,
|
||||
@@ -15,6 +16,9 @@ from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.responses import respond, action_result
|
||||
from devplacepy.schemas import AdminNewsOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.models import VisibilityForm
|
||||
from devplacepy.content import set_visibility_item
|
||||
from devplacepy.dependencies import json_or_form
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
@@ -182,3 +186,21 @@ async def admin_news_delete(request: Request, uid: str):
|
||||
links=[audit.target("news", uid, article.get("title"))],
|
||||
)
|
||||
return action_result(request, "/admin/news")
|
||||
|
||||
|
||||
@router.post("/news/{uid}/visibility")
|
||||
async def admin_news_visibility(
|
||||
request: Request,
|
||||
uid: str,
|
||||
data: Annotated[VisibilityForm, Depends(json_or_form(VisibilityForm))],
|
||||
):
|
||||
admin = require_admin(request)
|
||||
return set_visibility_item(
|
||||
request,
|
||||
"news",
|
||||
"news",
|
||||
admin,
|
||||
uid,
|
||||
data.value,
|
||||
"/admin/news",
|
||||
)
|
||||
|
||||
@@ -15,6 +15,8 @@ from devplacepy.database.nodes import (
|
||||
DEVICE_BREAKDOWN_COLUMNS,
|
||||
LIMITS,
|
||||
add_event,
|
||||
count_all_devices,
|
||||
count_flags,
|
||||
count_open_requests,
|
||||
device_breakdown,
|
||||
cache_stats,
|
||||
@@ -234,7 +236,7 @@ def overview() -> dict:
|
||||
lifetime = lifetime_counters()
|
||||
outcomes = {name: value for name, value in lifetime.items() if name.startswith(OUTCOME_PREFIX)}
|
||||
devices = recent_devices(ADMIN_LIST_LIMIT)
|
||||
flags = list_flags()[:ADMIN_LIST_LIMIT]
|
||||
flags = list_flags(limit=ADMIN_LIST_LIMIT)
|
||||
device_by_uid = {row["uid"]: row for row in devices}
|
||||
missing = [row["device_uid"] for row in flags if row.get("device_uid") and row["device_uid"] not in device_by_uid]
|
||||
for uid in missing:
|
||||
@@ -251,7 +253,9 @@ def overview() -> dict:
|
||||
**breakdowns,
|
||||
"usage_today": usage_day_totals(),
|
||||
"flags": flag_views(flags, owners, device_by_uid),
|
||||
"flags_total": count_flags(),
|
||||
"devices": device_views(devices, owners),
|
||||
"devices_total": count_all_devices(),
|
||||
"settings": settings_view(),
|
||||
"limits": LIMITS.describe(),
|
||||
"circuits": circuit_views(),
|
||||
|
||||
@@ -4,7 +4,9 @@ import logging
|
||||
from typing import Any
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from devplacepy.templating import templates
|
||||
from devplacepy.responses import respond
|
||||
from devplacepy.routers.admin._shared import ADMIN_LIST_CAP
|
||||
from devplacepy.schemas import ServiceDetailOut, ServicesOut
|
||||
from devplacepy.utils import require_admin, not_found
|
||||
from devplacepy.services.manager import service_manager
|
||||
from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
@@ -15,10 +17,24 @@ logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _services_payload() -> dict:
|
||||
services = service_manager.describe_all()
|
||||
return {
|
||||
"services": services[:ADMIN_LIST_CAP],
|
||||
"services_total": len(services),
|
||||
}
|
||||
|
||||
|
||||
def _service_payload(name: str) -> dict | None:
|
||||
svc = service_manager.get_service(name)
|
||||
if svc is None:
|
||||
return None
|
||||
return {"service": svc.describe()}
|
||||
|
||||
|
||||
@router.get("", response_class=HTMLResponse)
|
||||
async def services_page(request: Request):
|
||||
admin = require_admin(request)
|
||||
services = service_manager.describe_all()
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
@@ -32,32 +48,33 @@ async def services_page(request: Request):
|
||||
],
|
||||
schemas=[website_schema(base)],
|
||||
)
|
||||
return templates.TemplateResponse(
|
||||
return respond(
|
||||
request,
|
||||
"services.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": admin,
|
||||
"services": services,
|
||||
**_services_payload(),
|
||||
"admin_section": "services",
|
||||
},
|
||||
model=ServicesOut,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/data")
|
||||
async def services_json(request: Request):
|
||||
require_admin(request)
|
||||
return JSONResponse({"services": service_manager.describe_all()})
|
||||
return JSONResponse({"services": _services_payload()["services"]})
|
||||
|
||||
|
||||
@router.get("/{name}", response_class=HTMLResponse)
|
||||
async def service_detail(request: Request, name: str):
|
||||
admin = require_admin(request)
|
||||
svc = service_manager.get_service(name)
|
||||
if svc is None:
|
||||
payload = _service_payload(name)
|
||||
if payload is None:
|
||||
raise not_found("Service not found")
|
||||
info = svc.describe()
|
||||
info = payload["service"]
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
@@ -73,27 +90,28 @@ async def service_detail(request: Request, name: str):
|
||||
schemas=[website_schema(base)],
|
||||
)
|
||||
tool_groups = tool_prefs.group_overview() if name == "devii" else None
|
||||
return templates.TemplateResponse(
|
||||
return respond(
|
||||
request,
|
||||
"service_detail.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": admin,
|
||||
"service": info,
|
||||
**payload,
|
||||
"tool_groups": tool_groups,
|
||||
"admin_section": "services",
|
||||
},
|
||||
model=ServiceDetailOut,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{name}/data")
|
||||
async def service_detail_json(request: Request, name: str):
|
||||
require_admin(request)
|
||||
svc = service_manager.get_service(name)
|
||||
if svc is None:
|
||||
payload = _service_payload(name)
|
||||
if payload is None:
|
||||
return JSONResponse({"error": "unknown service"}, status_code=404)
|
||||
return JSONResponse({"service": svc.describe()})
|
||||
return JSONResponse(payload)
|
||||
|
||||
|
||||
def _audit_service(request: Request, admin: dict, name: str, event_key: str, summary: str, **kwargs: Any) -> None:
|
||||
|
||||
@@ -7,7 +7,7 @@ from devplacepy.responses import respond
|
||||
from devplacepy.schemas.statistics import StatisticsOut
|
||||
from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.services.statistics.build import build_statistics_tab
|
||||
from devplacepy.services.statistics.common import VALID_TABS
|
||||
from devplacepy.services.statistics.common import MAX_WINDOW_HOURS, VALID_TABS
|
||||
from devplacepy.utils import require_admin
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -32,10 +32,45 @@ TAB_LABELS = (
|
||||
)
|
||||
|
||||
|
||||
@router.get("/statistics", response_class=HTMLResponse)
|
||||
async def admin_statistics(request: Request, tab: str = "overview", hours: int = 168):
|
||||
admin = require_admin(request)
|
||||
STATISTICS_DEFAULT_HOURS = 168
|
||||
STATISTICS_DEFAULT_TOP_N = 10
|
||||
STATISTICS_MAX_TOP_N = 50
|
||||
STATISTICS_WINDOWS = (
|
||||
(24, "Last 24 hours"),
|
||||
(168, "Last 7 days"),
|
||||
(720, "Last 30 days"),
|
||||
(2160, "Last 90 days"),
|
||||
(0, "All time"),
|
||||
)
|
||||
|
||||
|
||||
def clamp_hours(hours: int) -> int:
|
||||
if hours <= 0:
|
||||
return 0
|
||||
return min(hours, MAX_WINDOW_HOURS)
|
||||
|
||||
|
||||
def statistics_view(tab: str, hours: int, compare: int, top_n: int) -> dict:
|
||||
active = tab if tab in VALID_TABS else "overview"
|
||||
bounded_top = max(1, min(top_n, STATISTICS_MAX_TOP_N))
|
||||
return build_statistics_tab(
|
||||
active,
|
||||
clamp_hours(hours),
|
||||
compare=bool(compare),
|
||||
top_n=bounded_top,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/statistics", response_class=HTMLResponse)
|
||||
async def admin_statistics(
|
||||
request: Request,
|
||||
tab: str = "overview",
|
||||
hours: int = STATISTICS_DEFAULT_HOURS,
|
||||
compare: int = 1,
|
||||
):
|
||||
admin = require_admin(request)
|
||||
initial = statistics_view(tab, hours, compare, STATISTICS_DEFAULT_TOP_N)
|
||||
active = initial["tab"]
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
@@ -49,7 +84,6 @@ async def admin_statistics(request: Request, tab: str = "overview", hours: int =
|
||||
],
|
||||
schemas=[website_schema(base)],
|
||||
)
|
||||
initial = build_statistics_tab(active, hours, compare=True, top_n=10)
|
||||
tabs = [
|
||||
{"key": key, "label": label, "icon": icon, "active": key == active}
|
||||
for key, label, icon in TAB_LABELS
|
||||
@@ -64,7 +98,9 @@ async def admin_statistics(request: Request, tab: str = "overview", hours: int =
|
||||
"admin_section": "statistics",
|
||||
"tabs": tabs,
|
||||
"active_tab": active,
|
||||
"window_hours": hours,
|
||||
"window_hours": initial["window_hours"],
|
||||
"compare": initial["compare"],
|
||||
"windows": STATISTICS_WINDOWS,
|
||||
"initial": initial,
|
||||
},
|
||||
model=StatisticsOut,
|
||||
@@ -75,16 +111,9 @@ async def admin_statistics(request: Request, tab: str = "overview", hours: int =
|
||||
async def admin_statistics_data(
|
||||
request: Request,
|
||||
tab: str = "overview",
|
||||
hours: int = 168,
|
||||
hours: int = STATISTICS_DEFAULT_HOURS,
|
||||
compare: int = 1,
|
||||
top_n: int = 10,
|
||||
top_n: int = STATISTICS_DEFAULT_TOP_N,
|
||||
):
|
||||
require_admin(request)
|
||||
return JSONResponse(
|
||||
build_statistics_tab(
|
||||
tab,
|
||||
hours,
|
||||
compare=bool(compare),
|
||||
top_n=top_n,
|
||||
)
|
||||
)
|
||||
return JSONResponse(statistics_view(tab, hours, compare, top_n))
|
||||
|
||||
@@ -6,7 +6,12 @@ from typing import Annotated
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from devplacepy.database import get_projects_by_uids, get_table, get_users_by_uids
|
||||
from devplacepy.database import (
|
||||
build_pagination,
|
||||
get_projects_by_uids,
|
||||
get_table,
|
||||
get_users_by_uids,
|
||||
)
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import (
|
||||
EditorPrefsForm,
|
||||
@@ -15,7 +20,7 @@ from devplacepy.models import (
|
||||
WorkspaceSuspendForm,
|
||||
)
|
||||
from devplacepy.responses import action_result, json_error, respond
|
||||
from devplacepy.routers.admin._shared import deny_senior, is_senior_admin
|
||||
from devplacepy.routers.admin._shared import ADMIN_LIST_CAP, deny_senior, is_senior_admin
|
||||
from devplacepy.schemas import AdminWorkspacesOut
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.services.audit import record as audit
|
||||
@@ -49,8 +54,32 @@ async def _decorate(rows: list[dict]) -> list[dict]:
|
||||
return decorated
|
||||
|
||||
|
||||
def _all_workspaces() -> list[dict]:
|
||||
return list(get_table("instances").find(is_workspace=1, deleted_at=None))
|
||||
def _workspace_page(page: int) -> tuple[list[dict], dict]:
|
||||
table = get_table("instances")
|
||||
total = table.count(is_workspace=1, deleted_at=None)
|
||||
pagination = build_pagination(page, total)
|
||||
offset = (pagination["page"] - 1) * pagination["per_page"]
|
||||
rows = list(
|
||||
table.find(
|
||||
is_workspace=1,
|
||||
deleted_at=None,
|
||||
order_by=["-created_at"],
|
||||
_limit=pagination["per_page"],
|
||||
_offset=offset,
|
||||
)
|
||||
)
|
||||
return rows, pagination
|
||||
|
||||
|
||||
async def _workspaces_payload(page: int) -> dict:
|
||||
rows, pagination = _workspace_page(page)
|
||||
open_flags = flags.list_flags()
|
||||
return {
|
||||
"workspaces": await _decorate(rows),
|
||||
"flags": open_flags[:ADMIN_LIST_CAP],
|
||||
"flags_total": len(open_flags),
|
||||
"pagination": pagination,
|
||||
}
|
||||
|
||||
|
||||
def _instance_or_404(uid: str) -> dict:
|
||||
@@ -75,13 +104,12 @@ def _audit(request: Request, admin: dict, event_key: str, instance: dict, **extr
|
||||
|
||||
|
||||
@router.get("/workspaces")
|
||||
async def admin_workspaces(request: Request):
|
||||
async def admin_workspaces(request: Request, page: int = 1):
|
||||
admin = require_admin(request)
|
||||
if not isinstance(admin, dict):
|
||||
return admin
|
||||
context = {
|
||||
"workspaces": await _decorate(_all_workspaces()),
|
||||
"flags": flags.list_flags(),
|
||||
**await _workspaces_payload(page),
|
||||
"admin_section": "workspaces",
|
||||
"user": admin,
|
||||
**base_seo_context(
|
||||
@@ -100,13 +128,11 @@ async def admin_workspaces(request: Request):
|
||||
|
||||
|
||||
@router.get("/workspaces/data")
|
||||
async def admin_workspaces_data(request: Request):
|
||||
async def admin_workspaces_data(request: Request, page: int = 1):
|
||||
admin = require_admin(request)
|
||||
if not isinstance(admin, dict):
|
||||
return admin
|
||||
return JSONResponse(
|
||||
{"workspaces": await _decorate(_all_workspaces()), "flags": flags.list_flags()}
|
||||
)
|
||||
return JSONResponse(await _workspaces_payload(page))
|
||||
|
||||
|
||||
@router.post("/workspaces/{uid}/suspend")
|
||||
|
||||
@@ -52,13 +52,14 @@ async def login_page(request: Request, next: str | None = None):
|
||||
|
||||
@router.post("/login")
|
||||
async def login(request: Request, data: Annotated[LoginForm, Depends(json_or_form(LoginForm))]):
|
||||
email = data.email.strip().lower()
|
||||
identifier = data.email.strip()
|
||||
email = identifier.lower()
|
||||
password = data.password
|
||||
remember_me = data.remember_me == "on"
|
||||
|
||||
errors = []
|
||||
users = get_table("users")
|
||||
user = users.find_one(email=email)
|
||||
user = users.find_one(email=email) or users.find_one(username=identifier)
|
||||
|
||||
if not user or not await verify_password_async(password, user["password_hash"]):
|
||||
errors.append("Invalid email or password")
|
||||
|
||||
@@ -22,7 +22,7 @@ async def token(
|
||||
request: Request,
|
||||
data: Annotated[LoginForm, Depends(json_or_form(LoginForm))],
|
||||
):
|
||||
identifier = data.email.strip().lower()
|
||||
identifier = data.email.strip()
|
||||
password = data.password
|
||||
|
||||
if not identifier or not password:
|
||||
@@ -32,7 +32,7 @@ async def token(
|
||||
)
|
||||
|
||||
users = get_table("users")
|
||||
user = users.find_one(email=identifier) or users.find_one(
|
||||
user = users.find_one(email=identifier.lower()) or users.find_one(
|
||||
username=identifier
|
||||
)
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ from fastapi.responses import RedirectResponse, JSONResponse, HTMLResponse
|
||||
from devplacepy.database import get_table, db, paginate, resolve_object_url, _now_iso
|
||||
from devplacepy.utils import generate_uid, require_user, time_ago, redirect_back
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.content import can_view_item, can_view_target
|
||||
from devplacepy.responses import respond
|
||||
from devplacepy.schemas import SavedOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
@@ -50,7 +51,8 @@ async def saved_page(request: Request, before: str = None):
|
||||
if table.has_column("deleted_at"):
|
||||
clauses.append(table.table.columns.deleted_at.is_(None))
|
||||
for obj in table.find(*clauses):
|
||||
resolved[(target_type, obj["uid"])] = obj
|
||||
if can_view_item(table_name, obj, user):
|
||||
resolved[(target_type, obj["uid"])] = obj
|
||||
|
||||
items = []
|
||||
for row in rows:
|
||||
@@ -95,6 +97,8 @@ async def toggle_bookmark(request: Request, target_type: str, target_uid: str):
|
||||
user = require_user(request)
|
||||
if target_type not in BOOKMARKABLE:
|
||||
return JSONResponse({"error": "Invalid target"}, status_code=400)
|
||||
if not can_view_target(target_type, target_uid, user):
|
||||
return JSONResponse({"error": "Not found"}, status_code=404)
|
||||
|
||||
bookmarks = get_table("bookmarks")
|
||||
existing = bookmarks.find_one(
|
||||
|
||||
@@ -8,11 +8,12 @@ from fastapi.responses import JSONResponse
|
||||
from devplacepy.database import get_table, resolve_object_url
|
||||
from devplacepy.content import (
|
||||
is_owner,
|
||||
can_view_target,
|
||||
create_comment_record,
|
||||
delete_comment_record,
|
||||
edit_comment_record,
|
||||
)
|
||||
from devplacepy.utils import require_user, is_admin
|
||||
from devplacepy.utils import require_user, is_admin, not_found
|
||||
from devplacepy.models import CommentForm, CommentEditForm
|
||||
from devplacepy.responses import action_result, wants_json, json_error
|
||||
from devplacepy.schemas import CommentEditOut
|
||||
@@ -26,6 +27,8 @@ router = APIRouter()
|
||||
async def create_comment(request: Request, data: Annotated[CommentForm, Depends(json_or_form(CommentForm))]):
|
||||
user = require_user(request)
|
||||
target_uid = data.target_uid or data.post_uid
|
||||
if not can_view_target(data.target_type, target_uid, user):
|
||||
raise not_found("Not found")
|
||||
comment_uid, comment_url = create_comment_record(
|
||||
request,
|
||||
user,
|
||||
|
||||
@@ -114,7 +114,7 @@ async def create_rant(request: Request):
|
||||
async def get_rant(request: Request, rant_id: str):
|
||||
params = await merge_params(request)
|
||||
viewer = resolve_actor(request, params)
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, viewer)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
detail = load_rant_detail(post, viewer)
|
||||
@@ -127,7 +127,7 @@ async def edit_rant(request: Request, rant_id: str):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.", fail_reason="not_found")
|
||||
if not is_owner(post, user):
|
||||
@@ -166,7 +166,7 @@ async def delete_rant(request: Request, rant_id: str):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
if not (is_owner(post, user) or is_admin(user)):
|
||||
@@ -181,7 +181,7 @@ async def vote_rant(request: Request, rant_id: str):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
value = as_int(params.get("vote"))
|
||||
@@ -208,7 +208,7 @@ async def _set_favorite(request: Request, rant_id: str, saved: bool):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
set_bookmark(request, user, "post", post["uid"], saved)
|
||||
@@ -221,7 +221,7 @@ async def comment_rant(request: Request, rant_id: str):
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
post = post_by_id(rant_id, user)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
text = (params.get("comment") or "").strip()
|
||||
|
||||
@@ -14,7 +14,7 @@ The **Legal** section (`SECTION_LEGAL`, in the `AUDIENCE_START` tier so it is on
|
||||
|
||||
`DOCS_PAGES` entries take optional `admin: True` (hidden + 404 for non-admins, but still indexed and surfaced only to admins by `docs_search`) and `section: "..."` (a nested sidebar group rendered by `docs_base.html`). The sidebar groups `section`s under four ordered **audience tiers** (`AUDIENCES` in `routers/docs/pages.py`): `Start here` (General), `Build with the API` (API, Components, Styles), `Contribute and internals` (Architecture, Services, Devii internals, Bots internals, Testing, Claude Code), and `Operate` (Administration, Production). `nav_groups(visible_pages)` builds the `[(audience, [(section, [pages])])]` tree from the flat visible-page list (so a section's pages collect under one heading regardless of `DOCS_PAGES` order or the API/Administration interleave from `api_doc_pages()`); `views.py` passes it as `nav`, and `docs_base.html` renders an audience super-header (`.sidebar-tier`) above each section subheading (`.sidebar-subheading`). `DOCS_PAGES` stays the canonical list for search/export/routing - the tiering is sidebar-only.
|
||||
|
||||
The public `getting-started` page (`SECTION_GENERAL`) is the new-contributor on-ramp (install/run, the four-faces workflow, validation); gate its deep-internals links with `{% if is_admin(user) %}` so guests get no 404s. Keep one canonical home per concept: the `auth` API group intro in `docs_api.py` defers method detail to the `authentication` prose page rather than re-listing the four methods. The member-facing `devii` prose page is functional; admins also get a `Devii internals` section of `devii-*` technical subpages.
|
||||
The public `getting-started` page (`SECTION_GENERAL`) is the new-contributor on-ramp (install/run, the four-faces workflow, validation); gate its deep-internals links with `{% if is_admin(user) %}` so guests get no 404s. Keep one canonical home per concept: the `auth` API group intro in `docs_api.py` defers method detail to the `authentication` prose page rather than re-listing the auth methods (three account credentials: username+password, email+password, personal API key; the prose page is canonical for precedence, `DEVPLACE_API_KEY`, the `GET /profile` account fetch, and the separate DevRant/WebSocket auth domains). The member-facing `devii` prose page is functional; admins also get a `Devii internals` section of `devii-*` technical subpages.
|
||||
|
||||
## Single source of truth
|
||||
|
||||
|
||||
@@ -69,6 +69,12 @@ DOCS_PAGES = [
|
||||
"kind": "prose",
|
||||
"section": SECTION_GENERAL,
|
||||
},
|
||||
{
|
||||
"slug": "content-visibility",
|
||||
"title": "Content visibility",
|
||||
"kind": "prose",
|
||||
"section": SECTION_GENERAL,
|
||||
},
|
||||
{
|
||||
"slug": "code-farm",
|
||||
"title": "Code Farm",
|
||||
@@ -292,6 +298,12 @@ DOCS_PAGES = [
|
||||
"kind": "prose",
|
||||
"section": SECTION_COMPONENTS,
|
||||
},
|
||||
{
|
||||
"slug": "component-dp-gif-picker",
|
||||
"title": "dp-gif-picker",
|
||||
"kind": "prose",
|
||||
"section": SECTION_COMPONENTS,
|
||||
},
|
||||
{
|
||||
"slug": "component-dp-upload",
|
||||
"title": "dp-upload",
|
||||
@@ -316,6 +328,12 @@ DOCS_PAGES = [
|
||||
"kind": "prose",
|
||||
"section": SECTION_COMPONENTS,
|
||||
},
|
||||
{
|
||||
"slug": "component-dp-action-menu",
|
||||
"title": "dp-action-menu",
|
||||
"kind": "prose",
|
||||
"section": SECTION_COMPONENTS,
|
||||
},
|
||||
{
|
||||
"slug": "component-dp-lightbox",
|
||||
"title": "dp-lightbox",
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from typing import Annotated
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from fastapi import APIRouter, Depends, Query, Request
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from devplacepy.attachments import (
|
||||
RemoteFetchError,
|
||||
is_extension_allowed,
|
||||
store_attachment_from_url,
|
||||
)
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import GiphyAttachForm
|
||||
from devplacepy.responses import json_error
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.giphy import runtime
|
||||
from devplacepy.services.giphy.client import MAX_LIMIT, GiphyError
|
||||
from devplacepy.services.giphy.config import giphy_config
|
||||
from devplacepy.services.giphy.limits import UserRateLimiter
|
||||
from devplacepy.utils import require_user_api, track_action
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
limiter = UserRateLimiter()
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
async def giphy_status(request: Request):
|
||||
require_user_api(request)
|
||||
return JSONResponse({"configured": giphy_config().is_configured})
|
||||
|
||||
|
||||
def _limit_value(limit: int) -> int:
|
||||
return max(1, min(int(limit), MAX_LIMIT))
|
||||
|
||||
|
||||
def _error(exc: GiphyError) -> JSONResponse:
|
||||
status = exc.status if 400 <= exc.status < 600 else 502
|
||||
return json_error(status, exc.message)
|
||||
|
||||
|
||||
def _throttled(user: dict) -> JSONResponse | None:
|
||||
if limiter.allow(user["uid"]):
|
||||
return None
|
||||
response = json_error(429, "Too many GIF searches, slow down")
|
||||
response.headers["Retry-After"] = str(int(limiter.window_seconds))
|
||||
return response
|
||||
|
||||
|
||||
@router.get("/search")
|
||||
async def search_gifs(
|
||||
request: Request, q: str = Query(default="", max_length=200), limit: int = 24
|
||||
):
|
||||
user = require_user_api(request)
|
||||
query = q.strip()
|
||||
if not query:
|
||||
return json_error(400, "q is required")
|
||||
if (refused := _throttled(user)) is not None:
|
||||
return refused
|
||||
try:
|
||||
gifs = await runtime.get_client().search(query, limit=_limit_value(limit))
|
||||
except GiphyError as exc:
|
||||
return _error(exc)
|
||||
return JSONResponse({"query": query, "results": gifs})
|
||||
|
||||
|
||||
@router.get("/trending")
|
||||
async def trending_gifs(request: Request, limit: int = 24):
|
||||
user = require_user_api(request)
|
||||
if (refused := _throttled(user)) is not None:
|
||||
return refused
|
||||
try:
|
||||
gifs = await runtime.get_client().trending(limit=_limit_value(limit))
|
||||
except GiphyError as exc:
|
||||
return _error(exc)
|
||||
return JSONResponse({"results": gifs})
|
||||
|
||||
|
||||
@router.post("/attach")
|
||||
async def attach_gif(
|
||||
request: Request,
|
||||
data: Annotated[GiphyAttachForm, Depends(json_or_form(GiphyAttachForm))],
|
||||
):
|
||||
user = require_user_api(request)
|
||||
client = runtime.get_client()
|
||||
try:
|
||||
gif = await client.get_gif(data.gif_id)
|
||||
except GiphyError as exc:
|
||||
return _error(exc)
|
||||
|
||||
url = (gif.get("original") or {}).get("url") or (gif.get("preview") or {}).get("url")
|
||||
if not url:
|
||||
return json_error(502, "GIPHY returned no downloadable file")
|
||||
|
||||
stem = (data.title or gif.get("title") or gif.get("id") or "giphy").strip()
|
||||
filename = f"{stem[:200]}.gif"
|
||||
if not is_extension_allowed(".gif"):
|
||||
return json_error(415, "GIF uploads are disabled")
|
||||
|
||||
try:
|
||||
result = await store_attachment_from_url(url, user["uid"], filename)
|
||||
except RemoteFetchError as exc:
|
||||
logger.warning("Giphy attach failed for %s: %s", gif.get("id"), exc.message)
|
||||
return json_error(exc.status, exc.message)
|
||||
|
||||
audit.record(
|
||||
request,
|
||||
"attachment.giphy",
|
||||
user=user,
|
||||
target_type="attachment",
|
||||
target_uid=result.get("uid"),
|
||||
target_label=filename,
|
||||
metadata={
|
||||
"gif_id": gif.get("id"),
|
||||
"source_host": urlparse(url).hostname,
|
||||
"size": result.get("file_size"),
|
||||
"mime": result.get("mime_type"),
|
||||
},
|
||||
summary=f"{user['username']} attached a GIF from Giphy",
|
||||
links=[audit.attachment_link(result.get("uid"), filename)],
|
||||
)
|
||||
track_action(user["uid"], "upload")
|
||||
return JSONResponse(result, status_code=201)
|
||||
@@ -11,6 +11,8 @@ from devplacepy.database import (
|
||||
get_gist_languages,
|
||||
get_recent_comments_by_target_uids,
|
||||
paginate,
|
||||
listed_clause,
|
||||
is_listed,
|
||||
text_search_clause,
|
||||
resolve_object_url,
|
||||
mark_notifications_read_by_target,
|
||||
@@ -84,13 +86,17 @@ def get_gists_list(user_uid=None, language=None, search="", before=None, viewer=
|
||||
gists_table, search, ("title", "description"), author_field="user_uid"
|
||||
)
|
||||
clauses = [search_match] if search_match is not None else []
|
||||
own_uid = viewer["uid"] if viewer and viewer["uid"] == user_uid else None
|
||||
listed = listed_clause(gists_table, own_uid=own_uid)
|
||||
count_clauses = clauses + ([listed] if listed is not None else [])
|
||||
|
||||
total = gists_table.count(*clauses, deleted_at=None, **filters)
|
||||
total = gists_table.count(*count_clauses, deleted_at=None, **filters)
|
||||
gists, next_cursor = paginate(
|
||||
gists_table,
|
||||
*clauses,
|
||||
before=before,
|
||||
viewer_uid=viewer["uid"] if viewer else None,
|
||||
include_unlisted_of=own_uid,
|
||||
**filters,
|
||||
)
|
||||
|
||||
@@ -167,6 +173,7 @@ async def gist_detail(request: Request, gist_slug: str):
|
||||
title=gist.get("title", "Gist"),
|
||||
description=gist.get("description") or "",
|
||||
seo_target=("gist", gist["uid"]),
|
||||
robots="index,follow" if is_listed("gists", gist) else "noindex,nofollow",
|
||||
og_type="article",
|
||||
og_image=first_image_url(gist, detail["attachments"]),
|
||||
breadcrumbs=[
|
||||
@@ -216,6 +223,7 @@ async def create_gist(request: Request, data: Annotated[GistForm, Depends(json_o
|
||||
"description": description or None,
|
||||
"source_code": source_code,
|
||||
"language": language,
|
||||
"visibility": data.visibility,
|
||||
},
|
||||
title,
|
||||
XP_GIST,
|
||||
@@ -245,6 +253,7 @@ async def edit_gist(
|
||||
"description": data.description.strip() or None,
|
||||
"source_code": data.source_code.strip(),
|
||||
"language": language,
|
||||
"visibility": data.visibility,
|
||||
},
|
||||
"/gists",
|
||||
target_type="gist",
|
||||
@@ -254,3 +263,4 @@ async def edit_gist(
|
||||
async def delete_gist(request: Request, gist_slug: str):
|
||||
user = require_user(request)
|
||||
return delete_content_item(request, "gists", "gist", user, gist_slug, "/gists")
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ from datetime import datetime
|
||||
from typing import Annotated, Optional
|
||||
from fastapi import Depends, APIRouter, Request, WebSocket, WebSocketDisconnect
|
||||
from devplacepy.models import MessageForm
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
|
||||
from devplacepy.database import (
|
||||
get_table,
|
||||
db,
|
||||
@@ -19,12 +19,14 @@ from devplacepy.attachments import get_attachments_batch
|
||||
from devplacepy.templating import clear_messages_cache
|
||||
from devplacepy.utils import (
|
||||
require_user,
|
||||
is_admin,
|
||||
not_found,
|
||||
time_ago,
|
||||
_user_from_session,
|
||||
_user_from_api_key,
|
||||
)
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.responses import respond, action_result
|
||||
from devplacepy.responses import respond, action_result, json_error, wants_json
|
||||
from devplacepy.schemas import ConversationOut, MessagesOut
|
||||
from devplacepy.services import presence
|
||||
from devplacepy.services.audit import record as audit
|
||||
@@ -36,6 +38,8 @@ from devplacepy.services.messaging import (
|
||||
message_frame,
|
||||
message_hub,
|
||||
message_relay,
|
||||
broadcast_deleted,
|
||||
delete_message_record,
|
||||
persist_message,
|
||||
redeem_ticket,
|
||||
stamp_content_revision,
|
||||
@@ -66,7 +70,7 @@ def mark_conversation_read(user_uid: str, other_uid: str) -> None:
|
||||
return
|
||||
with db:
|
||||
db.query(
|
||||
"UPDATE messages SET read = 1 WHERE receiver_uid = :me AND sender_uid = :other AND read = 0",
|
||||
"UPDATE messages SET read = 1 WHERE receiver_uid = :me AND sender_uid = :other AND read = 0 AND deleted_at IS NULL",
|
||||
me=user_uid,
|
||||
other=other_uid,
|
||||
)
|
||||
@@ -85,7 +89,7 @@ def get_conversations(user_uid: str):
|
||||
" ORDER BY created_at DESC, id DESC"
|
||||
" ) AS rn"
|
||||
" FROM messages"
|
||||
" WHERE sender_uid = :me OR receiver_uid = :me"
|
||||
" WHERE (sender_uid = :me OR receiver_uid = :me) AND deleted_at IS NULL"
|
||||
") WHERE rn = 1 ORDER BY created_at DESC",
|
||||
me=user_uid,
|
||||
)
|
||||
@@ -127,7 +131,7 @@ def get_conversation_messages(user_uid: str, other_uid: str, before: str = ""):
|
||||
"SELECT * FROM messages"
|
||||
" WHERE ((sender_uid = :me AND receiver_uid = :other)"
|
||||
" OR (sender_uid = :other AND receiver_uid = :me))"
|
||||
" AND created_at < :before"
|
||||
" AND created_at < :before AND deleted_at IS NULL"
|
||||
" ORDER BY created_at DESC, id DESC LIMIT :lim",
|
||||
me=user_uid,
|
||||
other=other_uid,
|
||||
@@ -139,8 +143,9 @@ def get_conversation_messages(user_uid: str, other_uid: str, before: str = ""):
|
||||
msgs = list(
|
||||
db.query(
|
||||
"SELECT * FROM messages"
|
||||
" WHERE (sender_uid = :me AND receiver_uid = :other)"
|
||||
" OR (sender_uid = :other AND receiver_uid = :me)"
|
||||
" WHERE ((sender_uid = :me AND receiver_uid = :other)"
|
||||
" OR (sender_uid = :other AND receiver_uid = :me))"
|
||||
" AND deleted_at IS NULL"
|
||||
" ORDER BY created_at DESC, id DESC LIMIT :lim",
|
||||
me=user_uid,
|
||||
other=other_uid,
|
||||
@@ -293,6 +298,33 @@ async def send_message(request: Request, data: Annotated[MessageForm, Depends(js
|
||||
request, f"/messages?with_uid={receiver_uid}", data=frame
|
||||
)
|
||||
|
||||
@router.post("/{message_uid}/delete")
|
||||
async def delete_message(request: Request, message_uid: str):
|
||||
user = require_user(request)
|
||||
message = get_table("messages").find_one(uid=message_uid, deleted_at=None)
|
||||
if not is_admin(user) or (message and message["sender_uid"] != user["uid"]):
|
||||
audit.record(
|
||||
request,
|
||||
"message.delete",
|
||||
user=user,
|
||||
result="denied",
|
||||
target_type="message",
|
||||
target_uid=message_uid,
|
||||
summary=f"{user['username']} denied deleting message {message_uid}",
|
||||
)
|
||||
if wants_json(request):
|
||||
return json_error(403, "Not allowed")
|
||||
return RedirectResponse(url="/messages", status_code=302)
|
||||
if not message:
|
||||
raise not_found("Message not found")
|
||||
delete_message_record(request, user, message)
|
||||
await broadcast_deleted(message)
|
||||
return action_result(
|
||||
request,
|
||||
f"/messages?with_uid={message['receiver_uid']}",
|
||||
data={"deleted_uid": message_uid},
|
||||
)
|
||||
|
||||
async def broadcast_message(
|
||||
sender: dict, message: dict, client_id: Optional[str] = None,
|
||||
ai_processed: bool = False, ai_pending: bool = False,
|
||||
@@ -312,7 +344,7 @@ async def _await_ai_and_push(
|
||||
) -> bool:
|
||||
await asyncio.gather(*pending, return_exceptions=True)
|
||||
pending.clear()
|
||||
row = get_table("messages").find_one(uid=message["uid"])
|
||||
row = get_table("messages").find_one(uid=message["uid"], deleted_at=None)
|
||||
if not row:
|
||||
return False
|
||||
changed = row.get("content") != message.get("content")
|
||||
@@ -358,6 +390,7 @@ async def _sync_missed(user_uid: str, data: dict, websocket: WebSocket) -> None:
|
||||
" OR (sender_uid = :other AND receiver_uid = :me))"
|
||||
" AND (created_at > :since"
|
||||
" OR (updated_at IS NOT NULL AND updated_at > :since))"
|
||||
" AND deleted_at IS NULL"
|
||||
" ORDER BY created_at ASC, id ASC LIMIT :lim",
|
||||
me=user_uid,
|
||||
other=with_uid,
|
||||
@@ -372,6 +405,7 @@ async def _sync_missed(user_uid: str, data: dict, websocket: WebSocket) -> None:
|
||||
" WHERE (sender_uid = :me OR receiver_uid = :me)"
|
||||
" AND (created_at > :since"
|
||||
" OR (updated_at IS NOT NULL AND updated_at > :since))"
|
||||
" AND deleted_at IS NULL"
|
||||
" ORDER BY created_at ASC, id ASC LIMIT :lim",
|
||||
me=user_uid,
|
||||
since=since,
|
||||
|
||||
@@ -15,11 +15,12 @@ from devplacepy.database import (
|
||||
get_user_bookmarks,
|
||||
get_user_notes,
|
||||
paginate,
|
||||
is_listed,
|
||||
resolve_object_url,
|
||||
mark_notifications_read_by_target,
|
||||
)
|
||||
from devplacepy.utils import get_current_user, time_ago, not_found
|
||||
from devplacepy.content import canonical_redirect
|
||||
from devplacepy.content import canonical_redirect, can_view_item
|
||||
from devplacepy.seo import (
|
||||
base_seo_context,
|
||||
website_schema,
|
||||
@@ -105,6 +106,8 @@ async def news_detail_page(request: Request, news_slug: str):
|
||||
article = resolve_by_slug(news_table, news_slug)
|
||||
if not article:
|
||||
raise not_found("News article not found")
|
||||
if not can_view_item("news", article, user):
|
||||
raise not_found("News article not found")
|
||||
redirect = canonical_redirect("news", article, news_slug)
|
||||
if redirect:
|
||||
return redirect
|
||||
@@ -139,6 +142,7 @@ async def news_detail_page(request: Request, news_slug: str):
|
||||
title=article.get("title", "News Article"),
|
||||
description=article.get("description", "") or "",
|
||||
seo_target=("news", article["uid"]),
|
||||
robots="index,follow" if is_listed("news", article) else "noindex,nofollow",
|
||||
og_type="article",
|
||||
og_image=image_url or None,
|
||||
breadcrumbs=[
|
||||
|
||||
@@ -8,6 +8,7 @@ from devplacepy.database import get_table, db, paginate, resolve_object_url, _no
|
||||
from devplacepy.models import NoteForm
|
||||
from devplacepy.utils import generate_uid, require_user, time_ago, redirect_back
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.content import can_view_item, can_view_target
|
||||
from devplacepy.responses import respond
|
||||
from devplacepy.schemas import NotesOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
@@ -52,7 +53,8 @@ async def notes_page(request: Request, before: str = None):
|
||||
if table.has_column("deleted_at"):
|
||||
clauses.append(table.table.columns.deleted_at.is_(None))
|
||||
for obj in table.find(*clauses):
|
||||
resolved[(target_type, obj["uid"])] = obj
|
||||
if can_view_item(table_name, obj, user):
|
||||
resolved[(target_type, obj["uid"])] = obj
|
||||
|
||||
items = []
|
||||
for row in rows:
|
||||
@@ -104,6 +106,8 @@ async def set_note(
|
||||
user = require_user(request)
|
||||
if target_type not in NOTABLE:
|
||||
return JSONResponse({"error": "Invalid target"}, status_code=400)
|
||||
if not can_view_target(target_type, target_uid, user):
|
||||
return JSONResponse({"error": "Not found"}, status_code=404)
|
||||
|
||||
notes = get_table("notes")
|
||||
existing = notes.find_one(
|
||||
|
||||
@@ -4,7 +4,7 @@ import logging
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from devplacepy.database import get_table, db
|
||||
from devplacepy.database import get_table, db, get_notification_grouping_enabled, notification_group_key
|
||||
from devplacepy.templating import (
|
||||
clear_unread_cache,
|
||||
jinja_unread_count,
|
||||
@@ -20,6 +20,7 @@ logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
PAGE_SIZE = 25
|
||||
GROUP_FETCH_LIMIT = 250
|
||||
|
||||
|
||||
def _group_label(created_at: str) -> str:
|
||||
@@ -42,24 +43,28 @@ def _group_label(created_at: str) -> str:
|
||||
@router.get("", response_class=HTMLResponse)
|
||||
async def notifications_page(request: Request, before: str = None):
|
||||
user = require_user(request)
|
||||
|
||||
grouping = get_notification_grouping_enabled(user["uid"])
|
||||
next_cursor = None
|
||||
groups = []
|
||||
try:
|
||||
notifications_table = get_table("notifications")
|
||||
filters = {"user_uid": user["uid"]}
|
||||
if before:
|
||||
filters["created_at"] = {"<": before}
|
||||
limit = GROUP_FETCH_LIMIT if grouping else PAGE_SIZE + 1
|
||||
raw_notifications = list(
|
||||
notifications_table.find(
|
||||
**filters, order_by=["-created_at"], _limit=PAGE_SIZE + 1
|
||||
**filters, order_by=["-created_at"], _limit=limit
|
||||
)
|
||||
)
|
||||
|
||||
has_more = len(raw_notifications) > PAGE_SIZE
|
||||
raw_notifications = raw_notifications[:PAGE_SIZE]
|
||||
if has_more and raw_notifications:
|
||||
next_cursor = raw_notifications[-1]["created_at"]
|
||||
|
||||
if grouping:
|
||||
has_more_rows = len(raw_notifications) >= GROUP_FETCH_LIMIT
|
||||
raw_notifications = raw_notifications[:GROUP_FETCH_LIMIT]
|
||||
else:
|
||||
has_more_rows = len(raw_notifications) > PAGE_SIZE
|
||||
raw_notifications = raw_notifications[:PAGE_SIZE]
|
||||
if has_more_rows and raw_notifications:
|
||||
next_cursor = raw_notifications[-1]["created_at"]
|
||||
enriched = []
|
||||
if raw_notifications:
|
||||
from devplacepy.database import get_users_by_uids
|
||||
@@ -78,20 +83,51 @@ async def notifications_page(request: Request, before: str = None):
|
||||
"time_ago": time_ago(n["created_at"]),
|
||||
}
|
||||
)
|
||||
|
||||
groups = []
|
||||
current_label = None
|
||||
current_entries = []
|
||||
current_event_entries = []
|
||||
event_index = {}
|
||||
group_count = 0
|
||||
last_consumed = None
|
||||
for item in enriched:
|
||||
label = _group_label(item["notification"]["created_at"])
|
||||
if label != current_label:
|
||||
if current_entries:
|
||||
groups.append({"label": current_label, "entries": current_entries})
|
||||
if current_entries or current_event_entries:
|
||||
groups.append({"label": current_label, "entries": current_entries, "event_entries": current_event_entries})
|
||||
current_label = label
|
||||
current_entries = []
|
||||
current_entries.append(item)
|
||||
if current_entries:
|
||||
groups.append({"label": current_label, "entries": current_entries})
|
||||
current_event_entries = []
|
||||
event_index = {}
|
||||
if not grouping:
|
||||
current_entries.append(item)
|
||||
continue
|
||||
key = notification_group_key(item["notification"])
|
||||
key_text = "\x1f".join(key)
|
||||
existing = event_index.get(key_text)
|
||||
if existing is None:
|
||||
if group_count >= PAGE_SIZE:
|
||||
continue
|
||||
existing = {
|
||||
"key": key_text,
|
||||
"count": 0,
|
||||
"unread": False,
|
||||
"actors": [],
|
||||
"latest": item,
|
||||
}
|
||||
event_index[key_text] = existing
|
||||
current_event_entries.append(existing)
|
||||
group_count += 1
|
||||
existing["count"] += 1
|
||||
if not item["notification"].get("read"):
|
||||
existing["unread"] = True
|
||||
actor = item["actor"]
|
||||
if actor and all(a.get("uid") != actor.get("uid") for a in existing["actors"]) and len(existing["actors"]) < 3:
|
||||
existing["actors"].append(actor)
|
||||
last_consumed = item["notification"]["created_at"]
|
||||
if current_entries or current_event_entries:
|
||||
groups.append({"label": current_label, "entries": current_entries, "event_entries": current_event_entries})
|
||||
if grouping and (has_more_rows or group_count >= PAGE_SIZE) and last_consumed:
|
||||
next_cursor = last_consumed
|
||||
except Exception as e:
|
||||
logger.exception(f"Error loading notifications: {e}")
|
||||
groups = []
|
||||
@@ -114,11 +150,28 @@ async def notifications_page(request: Request, before: str = None):
|
||||
"user": user,
|
||||
"notification_groups": groups,
|
||||
"next_cursor": next_cursor,
|
||||
"grouping": grouping,
|
||||
},
|
||||
model=NotificationsOut,
|
||||
)
|
||||
|
||||
|
||||
def _mark_group_read(user_uid: str, row) -> int:
|
||||
notifications_table = get_table("notifications")
|
||||
if not get_notification_grouping_enabled(user_uid):
|
||||
if not row["read"]:
|
||||
notifications_table.update({"id": row["id"], "read": True}, ["id"])
|
||||
return 1
|
||||
return 0
|
||||
key = notification_group_key(row)
|
||||
marked = 0
|
||||
for other in notifications_table.find(user_uid=user_uid, read=False):
|
||||
if notification_group_key(other) == key:
|
||||
notifications_table.update({"id": other["id"], "read": True}, ["id"])
|
||||
marked += 1
|
||||
return marked
|
||||
|
||||
|
||||
@router.get("/counts")
|
||||
async def unread_counts(request: Request):
|
||||
user = get_current_user(request)
|
||||
@@ -140,7 +193,7 @@ async def open_notification(request: Request, notification_uid: str):
|
||||
if not n or n["user_uid"] != user["uid"]:
|
||||
return action_result(request, "/notifications")
|
||||
if not n["read"]:
|
||||
notifications_table.update({"id": n["id"], "read": True}, ["id"])
|
||||
_mark_group_read(user["uid"], n)
|
||||
clear_unread_cache(user["uid"])
|
||||
audit.record(
|
||||
request,
|
||||
@@ -160,7 +213,7 @@ async def mark_read(request: Request, notification_uid: str):
|
||||
notifications_table = get_table("notifications")
|
||||
n = notifications_table.find_one(uid=notification_uid)
|
||||
if n and n["user_uid"] == user["uid"]:
|
||||
notifications_table.update({"id": n["id"], "read": True}, ["id"])
|
||||
_mark_group_read(user["uid"], n)
|
||||
clear_unread_cache(user["uid"])
|
||||
audit.record(
|
||||
request,
|
||||
|
||||
@@ -14,6 +14,8 @@ from devplacepy.database import (
|
||||
mark_notifications_read_by_target,
|
||||
get_featured_topics,
|
||||
get_blocked_uids,
|
||||
listed_sql,
|
||||
is_listed,
|
||||
)
|
||||
from devplacepy.utils import (
|
||||
get_current_user,
|
||||
@@ -83,6 +85,7 @@ async def create_post(request: Request, data: Annotated[PostForm, Depends(json_o
|
||||
"topic": topic,
|
||||
"project_uid": project_uid or None,
|
||||
"image": image_filename,
|
||||
"visibility": data.visibility,
|
||||
},
|
||||
slug_text,
|
||||
XP_POST,
|
||||
@@ -147,7 +150,7 @@ def _next_post_url(post: dict, viewer: dict | None) -> str | None:
|
||||
blocked = get_blocked_uids(viewer["uid"]) if viewer else frozenset()
|
||||
rows = db.query(
|
||||
"SELECT slug, uid, user_uid FROM posts WHERE created_at < :created_at "
|
||||
"AND deleted_at IS NULL ORDER BY created_at DESC LIMIT :limit",
|
||||
f"AND deleted_at IS NULL AND {listed_sql('posts')} ORDER BY created_at DESC LIMIT :limit",
|
||||
created_at=post["created_at"],
|
||||
limit=10,
|
||||
)
|
||||
@@ -177,7 +180,8 @@ def post_page_context(
|
||||
request,
|
||||
title=post.get("title") or "Post",
|
||||
description=post.get("content", ""),
|
||||
robots=robots or "index,follow",
|
||||
robots=robots
|
||||
or ("index,follow" if is_listed("posts", post) else "noindex,nofollow"),
|
||||
seo_target=("post", post["uid"]),
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
@@ -207,7 +211,7 @@ def post_page_context(
|
||||
if "posts" in db.tables:
|
||||
rows = list(
|
||||
db.query(
|
||||
"SELECT p.*, u.username FROM posts p JOIN users u ON p.user_uid = u.uid WHERE p.topic = :t AND p.uid != :uid AND p.deleted_at IS NULL ORDER BY p.created_at DESC LIMIT 5",
|
||||
f"SELECT p.*, u.username FROM posts p JOIN users u ON p.user_uid = u.uid WHERE p.topic = :t AND p.uid != :uid AND p.deleted_at IS NULL AND {listed_sql('posts', 'p')} ORDER BY p.created_at DESC LIMIT 5",
|
||||
t=post.get("topic", ""),
|
||||
uid=post["uid"],
|
||||
)
|
||||
@@ -275,6 +279,7 @@ async def edit_post(
|
||||
"content": data.content.strip(),
|
||||
"title": data.title.strip() or None,
|
||||
"topic": data.topic,
|
||||
"visibility": data.visibility,
|
||||
},
|
||||
"/feed",
|
||||
target_type="post",
|
||||
@@ -297,3 +302,4 @@ async def delete_post(request: Request, post_slug: str):
|
||||
return delete_content_item(
|
||||
request, "posts", "post", user, post_slug, "/feed", inline_image_field="image"
|
||||
)
|
||||
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
import logging
|
||||
from typing import Annotated
|
||||
from fastapi import Depends, APIRouter, Request
|
||||
from devplacepy.models import CustomizationToggleForm
|
||||
from devplacepy.database import set_customization_pref
|
||||
from devplacepy.models import CustomizationToggleForm, EnterToSubmitForm
|
||||
from devplacepy.database import set_customization_pref, set_enter_to_submit
|
||||
from devplacepy.responses import action_result
|
||||
from devplacepy.services.audit import record as audit
|
||||
|
||||
@@ -47,6 +47,31 @@ async def set_customization_global(
|
||||
):
|
||||
return _toggle_customization(request, username, "global", data.value)
|
||||
|
||||
@router.post("/{username}/enter-to-submit")
|
||||
async def set_enter_to_submit_route(
|
||||
request: Request,
|
||||
username: str,
|
||||
data: Annotated[EnterToSubmitForm, Depends(json_or_form(EnterToSubmitForm))],
|
||||
):
|
||||
target, denied = resolve_customization_target(request, username)
|
||||
if denied is not None:
|
||||
return denied
|
||||
set_enter_to_submit(target["uid"], data.value)
|
||||
audit.record(
|
||||
request,
|
||||
"profile.enter_to_submit.toggle",
|
||||
target_type="user",
|
||||
target_uid=target["uid"],
|
||||
target_label=target["username"],
|
||||
old_value=0 if data.value else 1,
|
||||
new_value=1 if data.value else 0,
|
||||
summary=f"{'enabled' if data.value else 'disabled'} Enter to send for {target['username']}",
|
||||
links=[audit.target("user", target["uid"], target["username"])],
|
||||
)
|
||||
url = f"/profile/{target['username']}"
|
||||
return action_result(request, url, data={"url": url, "enter_to_submit": data.value})
|
||||
|
||||
|
||||
@router.post("/{username}/customization/pagetype")
|
||||
async def set_customization_pagetype(
|
||||
request: Request,
|
||||
|
||||
@@ -7,8 +7,13 @@ from devplacepy.models import ProfileForm
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from devplacepy.database import (
|
||||
get_setting,
|
||||
listed_clause,
|
||||
is_listed,
|
||||
get_table,
|
||||
get_customization_prefs,
|
||||
get_enter_to_submit,
|
||||
get_notification_grouping_enabled,
|
||||
get_notification_throttle_enabled,
|
||||
get_notification_prefs,
|
||||
get_user_stars,
|
||||
get_user_rank,
|
||||
@@ -138,6 +143,12 @@ async def my_profile_page(request: Request, tab: str = "posts", page: int = 1):
|
||||
user = require_user(request)
|
||||
return await profile_page(request, user["username"], tab, page)
|
||||
|
||||
def _visible_clauses(table, viewer, profile_user):
|
||||
own_uid = viewer["uid"] if viewer and viewer["uid"] == profile_user["uid"] else None
|
||||
listed = listed_clause(table, own_uid=own_uid)
|
||||
return [listed] if listed is not None else []
|
||||
|
||||
|
||||
@router.get("/{username}", response_class=HTMLResponse)
|
||||
async def profile_page(
|
||||
request: Request, username: str, tab: str = "posts", page: int = 1
|
||||
@@ -147,6 +158,7 @@ async def profile_page(
|
||||
profile_user = users.find_one(username=username)
|
||||
if not profile_user:
|
||||
raise not_found("Profile not found")
|
||||
viewing_self = bool(current_user and current_user["uid"] == profile_user["uid"])
|
||||
if current_user:
|
||||
mark_notifications_read_by_target(
|
||||
current_user["uid"], f"/profile/{profile_user['username']}"
|
||||
@@ -188,6 +200,7 @@ async def profile_page(
|
||||
posts_table = get_table("posts")
|
||||
raw_posts = list(
|
||||
posts_table.find(
|
||||
*_visible_clauses(posts_table, current_user, profile_user),
|
||||
user_uid=profile_user["uid"],
|
||||
deleted_at=None,
|
||||
order_by=["-created_at"],
|
||||
@@ -229,13 +242,20 @@ async def profile_page(
|
||||
projects = list(
|
||||
get_table("projects").find(user_uid=profile_user["uid"], deleted_at=None)
|
||||
)
|
||||
projects = [p for p in projects if can_view_project(p, current_user)]
|
||||
projects = [
|
||||
p
|
||||
for p in projects
|
||||
if (viewing_self or is_listed("projects", p))
|
||||
and can_view_project(p, current_user)
|
||||
]
|
||||
gists = []
|
||||
if tab == "gists" or include_collections:
|
||||
gists_raw = list(
|
||||
get_table("gists").find(user_uid=profile_user["uid"], deleted_at=None)
|
||||
)
|
||||
for g in gists_raw:
|
||||
if not (viewing_self or is_listed("gists", g)):
|
||||
continue
|
||||
gists.append({"gist": g, "time_ago": time_ago(g["created_at"])})
|
||||
posts_count = get_user_post_count(profile_user["uid"])
|
||||
|
||||
@@ -243,6 +263,7 @@ async def profile_page(
|
||||
if tab == "activity":
|
||||
posts_table = get_table("posts")
|
||||
for p in posts_table.find(
|
||||
*_visible_clauses(posts_table, current_user, profile_user),
|
||||
user_uid=profile_user["uid"],
|
||||
deleted_at=None,
|
||||
order_by=["-created_at"],
|
||||
@@ -304,6 +325,7 @@ async def profile_page(
|
||||
viewer_is_admin = bool(current_user and current_user.get("role") == "Admin")
|
||||
can_view_api_key = bool(current_user and (is_owner or viewer_is_admin))
|
||||
api_key = profile_user.get("api_key") if can_view_api_key else None
|
||||
email = profile_user.get("email") if can_view_api_key else None
|
||||
ai_correction_enabled = (
|
||||
bool(profile_user.get("ai_correction_enabled")) if is_owner else False
|
||||
)
|
||||
@@ -372,6 +394,19 @@ async def profile_page(
|
||||
if (tab == "notifications" and can_manage_customization)
|
||||
else []
|
||||
)
|
||||
enter_to_submit = (
|
||||
get_enter_to_submit(profile_user["uid"]) if can_manage_customization else True
|
||||
)
|
||||
notif_grouping_enabled = (
|
||||
get_notification_grouping_enabled(profile_user["uid"])
|
||||
if (tab == "notifications" and can_manage_customization)
|
||||
else True
|
||||
)
|
||||
notif_throttle_enabled = (
|
||||
get_notification_throttle_enabled(profile_user["uid"])
|
||||
if (tab == "notifications" and can_manage_customization)
|
||||
else True
|
||||
)
|
||||
notif_telegram_paired = (
|
||||
telegram_store.is_paired(profile_user["uid"])
|
||||
if (tab == "notifications" and can_manage_customization)
|
||||
@@ -467,6 +502,7 @@ async def profile_page(
|
||||
"media_pagination": media_pagination,
|
||||
"can_view_api_key": can_view_api_key,
|
||||
"api_key": api_key,
|
||||
"email": email,
|
||||
"ai_correction_enabled": ai_correction_enabled,
|
||||
"ai_correction_sync": ai_correction_sync,
|
||||
"ai_correction_prompt": ai_correction_prompt,
|
||||
@@ -482,6 +518,9 @@ async def profile_page(
|
||||
"can_manage_customization": can_manage_customization,
|
||||
"cust_disable_global": customization_prefs["disable_global"],
|
||||
"cust_disable_pagetype": customization_prefs["disable_pagetype"],
|
||||
"enter_to_submit": enter_to_submit,
|
||||
"notif_grouping_enabled": notif_grouping_enabled,
|
||||
"notif_throttle_enabled": notif_throttle_enabled,
|
||||
"notification_prefs": notification_prefs,
|
||||
"consents": consents,
|
||||
"node_me": node_me,
|
||||
@@ -576,7 +615,13 @@ async def regenerate_api_key(request: Request):
|
||||
summary=f"{user['username']} regenerated their API key",
|
||||
links=[audit.target("user", user["uid"], user["username"])],
|
||||
)
|
||||
return JSONResponse({"api_key": new_key})
|
||||
return JSONResponse(
|
||||
{
|
||||
"api_key": new_key,
|
||||
"username": user["username"],
|
||||
"email": user["email"],
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -3,8 +3,13 @@
|
||||
import logging
|
||||
from typing import Annotated
|
||||
from fastapi import Depends, APIRouter, Request
|
||||
from devplacepy.models import NotificationPrefForm
|
||||
from devplacepy.database import set_notification_pref, reset_notification_prefs
|
||||
from devplacepy.models import NotificationPrefForm, CustomizationToggleForm
|
||||
from devplacepy.database import (
|
||||
set_notification_pref,
|
||||
reset_notification_prefs,
|
||||
set_notification_grouping_enabled,
|
||||
set_notification_throttle_enabled,
|
||||
)
|
||||
from devplacepy.responses import action_result, json_error
|
||||
from devplacepy.services.audit import record as audit
|
||||
|
||||
@@ -60,6 +65,49 @@ async def set_notification_preference(
|
||||
},
|
||||
)
|
||||
|
||||
def _toggle_notification_display(request: Request, username: str, kind: str, value: bool):
|
||||
target, denied = resolve_customization_target(request, username)
|
||||
if denied is not None:
|
||||
return denied
|
||||
if kind == "grouping":
|
||||
set_notification_grouping_enabled(target["uid"], value)
|
||||
field = "notif_grouping_enabled"
|
||||
else:
|
||||
set_notification_throttle_enabled(target["uid"], value)
|
||||
field = "notif_throttle_enabled"
|
||||
audit.record(
|
||||
request,
|
||||
f"profile.notification.{kind}.toggle",
|
||||
target_type="user",
|
||||
target_uid=target["uid"],
|
||||
target_label=target["username"],
|
||||
old_value=0 if value else 1,
|
||||
new_value=1 if value else 0,
|
||||
summary=f"{'enabled' if value else 'disabled'} notification {kind} for {target['username']}",
|
||||
links=[audit.target("user", target["uid"], target["username"])],
|
||||
)
|
||||
url = f"/profile/{target['username']}?tab=notifications"
|
||||
return action_result(request, url, data={"url": url, field: value})
|
||||
|
||||
|
||||
@router.post("/{username}/notifications/grouping")
|
||||
async def set_notification_grouping(
|
||||
request: Request,
|
||||
username: str,
|
||||
data: Annotated[CustomizationToggleForm, Depends(json_or_form(CustomizationToggleForm))],
|
||||
):
|
||||
return _toggle_notification_display(request, username, "grouping", data.value)
|
||||
|
||||
|
||||
@router.post("/{username}/notifications/throttling")
|
||||
async def set_notification_throttling(
|
||||
request: Request,
|
||||
username: str,
|
||||
data: Annotated[CustomizationToggleForm, Depends(json_or_form(CustomizationToggleForm))],
|
||||
):
|
||||
return _toggle_notification_display(request, username, "throttle", data.value)
|
||||
|
||||
|
||||
@router.post("/{username}/notifications/reset")
|
||||
async def reset_notification_preferences(request: Request, username: str):
|
||||
target, denied = resolve_customization_target(request, username)
|
||||
|
||||
@@ -11,7 +11,7 @@ Each project card links to `/projects/{project_uid}` showing full project detail
|
||||
|
||||
**Project overview page.** The detail page is a dedicated project showcase: one encompassing dark card (`.project-shell`, the site `--bg-card` surface with clipped corners) wraps the hero, the section tab bar and the two-column body, and every inner panel (tab bar, sidebar cards, devlog post cards, empty state, comments section) sits one elevation lighter on `--bg-secondary`. The hero's cover banner is the attachment referenced by `projects.cover_attachment_uid`, falling back to the first image attachment (brand-gradient band when neither exists); the title block, type/platform chips and author row render OVERLAID on the banner behind a bottom scrim (dark text-shadow for readability) beside the optional `projects.logo_attachment_uid` tile, with an owner-set **Visit Website** CTA (`projects.website_url`). Cover and logo ride the ONE existing upload pipeline: `dp-upload` widgets (`name="cover_attachment_uid"`/`"logo_attachment_uid"`, `max-files="1"`) in the create/edit modals upload to `/uploads/upload`, the route validates each uid via `database.get_user_attachment` (must exist, belong to the actor, be an image - `_hero_attachment_uid`) and links it to the project through `attachments.link_attachments`; an empty value on edit keeps the current image (no removal control). `website_url`/`repo_url` are normalized by `models.normalize_website_url` (scheme-less input gets `https://`, non-http(s) rejected) and render with `rel="noopener nofollow"`. Below the hero an anchor **section tab bar** (`.project-tabs`, underline style, Overview `.active`) links `#about` / `#devlog` / `#screenshots` (only when gallery images exist) / `#comments` / the Files page - server-rendered anchors, no JS tab state. The main column holds **About** (description + non-image attachments), the **Devlog** (every post whose `project_uid` points at the project via `_post_card.html` - the template loads `feed.css` for the card styles alongside `post.css`, the same rule as `news.html`) with `devlog_count` (`content.count_project_devlog`) and an owner **Post update** button (`.project-devlog-post-btn`) opening the shared composer preset to `topic=devlog` + this project (the form lives ONCE in `templates/_post_composer_form.html`, locals `_composer_topic`/`_composer_project`, included by `feed.html` and `project_detail.html` - never fork a second copy), a **Screenshots** gallery (image attachments minus the cover/logo, thumbnails, `data-lightbox`, capped at 12 rendered), and the comment thread; the sidebar holds Links (website/repository/files/fork source), the Stats card (5 `.project-stat` entries + a last-update line) and the Author card. Owners add gallery images via the More-menu **Add screenshots** modal: `_attachment_form.html` uploads, then `POST /projects/{slug}/screenshots` (`ProjectScreenshotsForm`, owner-only, audit `project.screenshots.add`) links the uids through the same `link_attachments` choke point; Devii action `project_add_screenshots`, docs id `projects-screenshots`. `comment_count`/`devlog_count` ride `ProjectDetailOut`; the new project fields ride `ProjectOut`; the page og:image prefers the cover attachment. **Locator discipline:** the page has several `Files` anchors (action row, tab bar, sidebar) and, for owners, a second hidden `textarea[name='content']`/Post button inside the composer modal - tests MUST scope (`.project-detail-actions a:has-text('Files')`, `.comment-form textarea[name='content']`).
|
||||
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest into a `dp-action-menu` component. The trigger is `<button data-action-menu-trigger>` labelled **More**; the secondary actions (Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) are real elements inside `<div data-action-menu-source hidden>`, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/components/AppActionMenu.js` (auto-defined via `components/index.js`) builds the menu through `app.contextMenu` and each item's `onSelect` activates the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), the `data-modal` Edit trigger, and the `data-confirm`/`data-confirm-danger` forms, which the component confirms itself before re-clicking (the element sits in a hidden container, so delegated handlers would never see it). Reuse `<dp-action-menu>` for any future action row that overflows; do not write a bespoke controller.
|
||||
|
||||
**Owner editing.** Mirrors post editing exactly: an owner-only **Edit** menu item (`data-modal="edit-project-modal"`) opens the `modal()` macro's `edit-project-modal`, a plain `POST` form to `/projects/edit/{slug}` (route `edit_project` in `routers/projects/index.py`, body `ProjectEditForm`, owner-gated through the shared `content.edit_content_item` which returns 403 JSON / redirect for non-owners and stamps `updated_at`). The modal is the create modal pre-filled from the `project` row (title, description, type/status radios pre-checked, dates via `format_date()` back to DD/MM/YYYY). `is_private`/`read_only` are NOT edited here - they stay on their dedicated toggles. The platforms tag widget reuses the create modal's `platforms-input`/`platforms`/`platforms-tags` ids; `ProfileEditor.initPlatformTags` now **seeds existing tags** from the hidden `#platforms` value on load, so both the empty create form and the pre-filled edit form work from the same code. Devii tool `edit_project`; documented in `docs_api.py` (`projects-edit`).
|
||||
|
||||
@@ -31,7 +31,11 @@ Each project carries a virtual filesystem so it can hold a whole software projec
|
||||
|
||||
## Project visibility and read-only
|
||||
|
||||
Two owner-controlled flags on the `projects` row, both integer `0`/`1` (default `0`, set on the create insert so the columns always exist): `is_private` hides the project from other viewers, `read_only` freezes its filesystem against every mutation. **The full security predicates** (`content.can_view_project`, `owns_instance`, `can_view_project_containers`, `can_view_instance`, `can_manage_instance`, the primary-administrator container isolation rules) are documented once in the root CLAUDE.md's "Project visibility (`is_private`) and read-only" section - this file covers only the project-page/filesystem UI-level behavior that sits on top of those predicates.
|
||||
Two owner-controlled flags on the `projects` row, both integer `0`/`1` (default `0`, set on the create insert so the columns always exist): `is_private` hides the project from other viewers, `read_only` freezes its filesystem against every mutation. **The full security predicates** (documented once, here):
|
||||
|
||||
- Project visibility (`is_private`) and read-only (`read_only`) are owner-controlled flags on `projects`. Read access is gated by the single `content.can_view_project(project, user)` predicate at EVERY read surface, never re-implemented inline. Predicate: `not is_private OR is_owner OR (is_admin AND owner is not an admin)`.
|
||||
- Containers have their own, stricter isolation predicates (`owns_instance`, `can_view_project_containers`, `can_view_instance`, `can_manage_instance`): the primary administrator sees/manages every container; any other admin can VIEW others' containers only on public projects and can MANAGE only instances they own.
|
||||
- Read-only is enforced as a single data-layer guard `project_files._guard_writable(project_uid)` at the top of every mutation entrypoint; add it to any NEW file-mutating function. Devii may flip read-only/visibility only after explicit confirmation.
|
||||
|
||||
**Toggle routes** (owner-only, `routers/projects/index.py`): `POST /projects/{slug}/private` and `POST /projects/{slug}/readonly`, each taking `ProjectFlagForm{value: bool}` and routed through `_set_project_flag`. The create form carries an `is_private` checkbox (`ProjectForm.is_private`); `project_detail.html` shows Private/Read-only badges and owner toggle buttons (**both** the private and read-only buttons carry `data-confirm` so `ModalManager.initConfirmations` gates them through `app.dialog`), and `project_files.html` hides the editing toolbar + shows a banner when read-only. Schemas: `is_private`/`read_only` on `ProjectOut` and `ProjectDetailOut`, `read_only`/`is_private` on `ProjectFilesOut`.
|
||||
|
||||
|
||||
@@ -35,6 +35,8 @@ from devplacepy.database import (
|
||||
count_forks,
|
||||
get_top_authors,
|
||||
get_user_attachment,
|
||||
listed_clause,
|
||||
is_listed,
|
||||
)
|
||||
from devplacepy.project_files import count_files
|
||||
from devplacepy.services.jobs import queue
|
||||
@@ -116,13 +118,17 @@ def get_projects_list(
|
||||
clauses.append(visible)
|
||||
|
||||
order = ["-stars", "-created_at"] if tab == "popular" else ["-created_at"]
|
||||
total = projects.count(*clauses, **filters)
|
||||
own_uid = viewer["uid"] if viewer and viewer["uid"] == user_uid else None
|
||||
listed = listed_clause(projects, own_uid=own_uid)
|
||||
count_clauses = clauses + ([listed] if listed is not None else [])
|
||||
total = projects.count(*count_clauses, **filters)
|
||||
page, next_cursor = paginate(
|
||||
projects,
|
||||
*clauses,
|
||||
before=before,
|
||||
order=order,
|
||||
viewer_uid=viewer["uid"] if viewer else None,
|
||||
include_unlisted_of=own_uid,
|
||||
**filters,
|
||||
)
|
||||
|
||||
@@ -234,7 +240,11 @@ async def project_detail(request: Request, project_slug: str, before: str = None
|
||||
)
|
||||
|
||||
base = site_url(request)
|
||||
robots = "noindex,nofollow" if project.get("is_private") else "index,follow"
|
||||
robots = (
|
||||
"noindex,nofollow"
|
||||
if project.get("is_private") or not is_listed("projects", project)
|
||||
else "index,follow"
|
||||
)
|
||||
cover_url = next(
|
||||
(
|
||||
a["url"]
|
||||
@@ -465,6 +475,7 @@ async def create_project(request: Request, data: Annotated[ProjectForm, Depends(
|
||||
"cover_attachment_uid": cover_uid,
|
||||
"logo_attachment_uid": logo_uid,
|
||||
"is_private": 1 if data.is_private else 0,
|
||||
"visibility": data.visibility,
|
||||
"read_only": 0,
|
||||
},
|
||||
title,
|
||||
@@ -495,6 +506,7 @@ async def edit_project(
|
||||
"status": data.status,
|
||||
"website_url": data.website_url or None,
|
||||
"repo_url": data.repo_url or None,
|
||||
"visibility": data.visibility,
|
||||
}
|
||||
hero_uids = []
|
||||
for field in ("cover_attachment_uid", "logo_attachment_uid"):
|
||||
@@ -616,3 +628,4 @@ async def set_project_readonly(
|
||||
return _set_project_flag(request, project_slug, "read_only", data.value)
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ from fastapi.responses import RedirectResponse, JSONResponse
|
||||
from devplacepy.database import get_table, db, _now_iso
|
||||
from devplacepy.utils import generate_uid, require_user, track_action, redirect_back
|
||||
from devplacepy.models import ReactionForm
|
||||
from devplacepy.content import can_view_target
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.dependencies import json_or_form
|
||||
|
||||
@@ -25,6 +26,8 @@ async def react(
|
||||
user = require_user(request)
|
||||
if target_type not in REACTABLE:
|
||||
return JSONResponse({"error": "Invalid target"}, status_code=400)
|
||||
if not can_view_target(target_type, target_uid, user):
|
||||
return JSONResponse({"error": "Not found"}, status_code=404)
|
||||
|
||||
emoji = data.emoji
|
||||
reactions = get_table("reactions")
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
from devplacepy.constants import TOPICS, TOPIC_LABELS
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.database import get_table, listed_clause
|
||||
from devplacepy.routers.feed import get_feed_posts, enrich_post_cards
|
||||
from devplacepy.utils import get_current_user, not_found
|
||||
from devplacepy.seo import list_page_seo, next_page_url
|
||||
@@ -17,11 +17,13 @@ router = APIRouter()
|
||||
async def topics_hub(request: Request):
|
||||
user = get_current_user(request)
|
||||
posts_table = get_table("posts")
|
||||
listed_match = listed_clause(posts_table)
|
||||
listed = [listed_match] if listed_match is not None else []
|
||||
topics = [
|
||||
{
|
||||
"key": topic,
|
||||
"label": TOPIC_LABELS.get(topic, topic.title()),
|
||||
"post_count": posts_table.count(topic=topic, deleted_at=None),
|
||||
"post_count": posts_table.count(*listed, topic=topic, deleted_at=None),
|
||||
}
|
||||
for topic in TOPICS
|
||||
]
|
||||
|
||||
@@ -118,7 +118,7 @@ async def generate_image(request: Request, data: Annotated[GenerateImageForm, De
|
||||
}
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"X-App-Reference": "devplace-generate-image-v-1-0-0",
|
||||
"X-App-Reference": "devplace-image-gen-v-1-0-0",
|
||||
"Authorization": f"Bearer {user.get('api_key', '')}",
|
||||
}
|
||||
try:
|
||||
|
||||
@@ -5,7 +5,7 @@ from typing import Annotated
|
||||
from fastapi import Depends, APIRouter, Request
|
||||
from fastapi.responses import RedirectResponse, JSONResponse
|
||||
from devplacepy.utils import require_user, redirect_back
|
||||
from devplacepy.content import apply_vote
|
||||
from devplacepy.content import apply_vote, can_view_target
|
||||
from devplacepy.models import VoteForm
|
||||
from devplacepy.dependencies import json_or_form
|
||||
|
||||
@@ -25,6 +25,8 @@ async def vote(
|
||||
user = require_user(request)
|
||||
if target_type not in VOTABLE:
|
||||
return JSONResponse({"error": "Invalid target"}, status_code=400)
|
||||
if not can_view_target(target_type, target_uid, user):
|
||||
return JSONResponse({"error": "Not found"}, status_code=404)
|
||||
result = apply_vote(request, user, target_type, target_uid, data.value)
|
||||
|
||||
if request.headers.get("x-requested-with") == "fetch":
|
||||
|
||||
@@ -135,6 +135,9 @@ from devplacepy.schemas.admin import (
|
||||
AdminNewsOut,
|
||||
AdminNotificationsOut,
|
||||
AdminSettingsOut,
|
||||
ServiceDetailOut,
|
||||
ServiceOut,
|
||||
ServicesOut,
|
||||
AdminTrashOut,
|
||||
AdminUserOut,
|
||||
AdminUsersOut,
|
||||
|
||||
@@ -94,12 +94,43 @@ class AdminDeviiTaskItemOut(_Out):
|
||||
|
||||
class AdminDeviiTasksOut(_Out):
|
||||
items: list[AdminDeviiTaskItemOut] = []
|
||||
pagination: Optional[Any] = None
|
||||
state: str = "active"
|
||||
tabs: list[dict] = []
|
||||
limits: dict = {}
|
||||
admin_section: Optional[str] = None
|
||||
|
||||
|
||||
class ServiceOut(_Out):
|
||||
name: Optional[str] = None
|
||||
title: Optional[str] = None
|
||||
description: Optional[str] = None
|
||||
details: Optional[str] = None
|
||||
enabled: bool = False
|
||||
status: Optional[str] = None
|
||||
interval_seconds: Optional[int] = None
|
||||
last_run: Optional[str] = None
|
||||
next_run: Optional[str] = None
|
||||
heartbeat: Optional[str] = None
|
||||
uptime: Optional[str] = None
|
||||
log_buffer: list[str] = []
|
||||
metrics: dict = {}
|
||||
fields: list[Any] = []
|
||||
field_groups: list[Any] = []
|
||||
|
||||
|
||||
class ServicesOut(_Out):
|
||||
services: list[ServiceOut] = []
|
||||
services_total: Optional[int] = None
|
||||
admin_section: Optional[str] = None
|
||||
|
||||
|
||||
class ServiceDetailOut(_Out):
|
||||
service: ServiceOut
|
||||
tool_groups: Optional[list[Any]] = None
|
||||
admin_section: Optional[str] = None
|
||||
|
||||
|
||||
class AdminGameOut(_Out):
|
||||
era_active: bool = False
|
||||
era_name: str = ""
|
||||
@@ -139,6 +170,7 @@ class AdminCommunityItemOut(_Out):
|
||||
|
||||
class AdminCommunitiesOut(_Out):
|
||||
items: list[AdminCommunityItemOut] = []
|
||||
pagination: Optional[Any] = None
|
||||
sync_enabled: bool = False
|
||||
service_enabled: bool = False
|
||||
service_dry_run: bool = True
|
||||
@@ -316,7 +348,9 @@ class AdminNodesOut(_Out):
|
||||
devices_by_country: dict[str, int] = {}
|
||||
usage_today: dict[str, int] = {}
|
||||
flags: list[AdminNodeFlagOut] = []
|
||||
flags_total: int = 0
|
||||
devices: list[AdminNodeDeviceOut] = []
|
||||
devices_total: int = 0
|
||||
settings: list[AdminNodeSettingOut] = []
|
||||
limits: dict[str, int] = {}
|
||||
circuits: list[AdminNodeCircuitOut] = []
|
||||
|
||||
@@ -72,6 +72,7 @@ class BackupScheduleOut(_Out):
|
||||
class BackupDashboardOut(_Out):
|
||||
storage: dict = {}
|
||||
backups: list[BackupOut] = []
|
||||
backups_total: int = 0
|
||||
schedules: list[BackupScheduleOut] = []
|
||||
targets: list[dict] = []
|
||||
metrics: dict = {}
|
||||
|
||||
@@ -48,6 +48,7 @@ class ContainersOut(_Out):
|
||||
|
||||
class AdminContainersOut(_Out):
|
||||
instances: list = []
|
||||
pagination: Optional[Any] = None
|
||||
projects: list = []
|
||||
boot_languages: list = []
|
||||
restart_policies: list = []
|
||||
@@ -199,5 +200,7 @@ class WorkspaceOut(_Out):
|
||||
class AdminWorkspacesOut(_Out):
|
||||
workspaces: list[WorkspaceViewOut] = []
|
||||
flags: list[WorkspaceFlagOut] = []
|
||||
flags_total: int = 0
|
||||
pagination: Optional[Any] = None
|
||||
admin_section: Optional[str] = None
|
||||
user: Optional[Any] = None
|
||||
|
||||
@@ -4,11 +4,21 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any, Optional
|
||||
|
||||
from pydantic import ConfigDict, Field
|
||||
from pydantic import ConfigDict, Field, field_validator
|
||||
|
||||
from devplacepy.database.visibility import normalize_visibility
|
||||
from devplacepy.schemas.base import _Out
|
||||
|
||||
|
||||
class VisibleOut(_Out):
|
||||
visibility: str = "published"
|
||||
|
||||
@field_validator("visibility", mode="before")
|
||||
@classmethod
|
||||
def default_visibility(cls, value):
|
||||
return normalize_visibility(value)
|
||||
|
||||
|
||||
class UserOut(_Out):
|
||||
uid: str = ""
|
||||
username: str = ""
|
||||
@@ -80,7 +90,7 @@ class ProjectLinkOut(_Out):
|
||||
url: Optional[str] = None
|
||||
|
||||
|
||||
class PostOut(_Out):
|
||||
class PostOut(VisibleOut):
|
||||
uid: str = ""
|
||||
slug: Optional[str] = None
|
||||
user_uid: Optional[str] = None
|
||||
@@ -94,7 +104,7 @@ class PostOut(_Out):
|
||||
updated_at: Optional[str] = None
|
||||
|
||||
|
||||
class GistOut(_Out):
|
||||
class GistOut(VisibleOut):
|
||||
uid: str = ""
|
||||
slug: Optional[str] = None
|
||||
user_uid: Optional[str] = None
|
||||
@@ -107,7 +117,7 @@ class GistOut(_Out):
|
||||
updated_at: Optional[str] = None
|
||||
|
||||
|
||||
class ProjectOut(_Out):
|
||||
class ProjectOut(VisibleOut):
|
||||
uid: str = ""
|
||||
slug: Optional[str] = None
|
||||
user_uid: Optional[str] = None
|
||||
|
||||
@@ -50,7 +50,18 @@ class AdminGatewayOut(_Out):
|
||||
default_provider: dict = {}
|
||||
quota_defaults: dict = {}
|
||||
stats_ranges: list = []
|
||||
providers_total: int = 0
|
||||
models_total: int = 0
|
||||
quota_rules_total: int = 0
|
||||
stats: dict = {}
|
||||
stats_range: str = "24h"
|
||||
stats_generated_iso: Optional[str] = None
|
||||
stats_models: list = []
|
||||
stats_table_models: list = []
|
||||
stats_table_models_total: int = 0
|
||||
admin_section: Optional[str] = None
|
||||
tested: str = ""
|
||||
test_image: str = ""
|
||||
|
||||
|
||||
class AdminGatewayProviderFormOut(_Out):
|
||||
|
||||
@@ -85,9 +85,18 @@ class NotificationItemOut(_Out):
|
||||
time_ago: Optional[str] = None
|
||||
|
||||
|
||||
class NotificationEventGroupOut(_Out):
|
||||
key: Optional[str] = None
|
||||
count: int = 0
|
||||
unread: bool = False
|
||||
actors: list[UserOut] = []
|
||||
latest: Optional[NotificationItemOut] = None
|
||||
|
||||
|
||||
class NotificationGroupOut(_Out):
|
||||
label: Optional[str] = None
|
||||
entries: list[NotificationItemOut] = []
|
||||
event_entries: list[NotificationEventGroupOut] = []
|
||||
|
||||
|
||||
class LeaderboardEntryOut(_Out):
|
||||
@@ -262,6 +271,7 @@ class MessagesOut(_Out):
|
||||
class NotificationsOut(_Out):
|
||||
notification_groups: list[NotificationGroupOut] = []
|
||||
next_cursor: Optional[str] = None
|
||||
grouping: bool = True
|
||||
|
||||
|
||||
class LeaderboardOut(_Out):
|
||||
|
||||
@@ -45,6 +45,7 @@ class ProfileOut(_Out):
|
||||
profile_is_admin: bool = False
|
||||
can_view_api_key: bool = False
|
||||
api_key: Optional[str] = None
|
||||
email: Optional[str] = None
|
||||
ai_correction_enabled: bool = False
|
||||
ai_correction_sync: bool = False
|
||||
ai_correction_prompt: Optional[str] = None
|
||||
@@ -60,6 +61,9 @@ class ProfileOut(_Out):
|
||||
can_manage_customization: bool = False
|
||||
cust_disable_global: bool = False
|
||||
cust_disable_pagetype: bool = False
|
||||
enter_to_submit: bool = True
|
||||
notif_grouping_enabled: bool = True
|
||||
notif_throttle_enabled: bool = True
|
||||
ai_quota: Optional[dict] = None
|
||||
correction_usage: Optional[dict] = None
|
||||
modifier_usage: Optional[dict] = None
|
||||
|
||||
@@ -14,6 +14,9 @@ class StatisticsMetricOut(BaseModel):
|
||||
format: str = "int"
|
||||
delta: Optional[float] = None
|
||||
direction: Optional[str] = None
|
||||
display: Optional[str] = None
|
||||
arrow: Optional[str] = None
|
||||
delta_display: Optional[str] = None
|
||||
|
||||
|
||||
class StatisticsPointOut(BaseModel):
|
||||
@@ -32,6 +35,9 @@ class StatisticsTableOut(BaseModel):
|
||||
title: str
|
||||
columns: list[str]
|
||||
rows: list[list[Any]]
|
||||
column_classes: list[str] = []
|
||||
display_rows: list[list[str]] = []
|
||||
row_count: Optional[str] = None
|
||||
|
||||
|
||||
class StatisticsHighlightOut(BaseModel):
|
||||
|
||||
+8
-2
@@ -421,9 +421,13 @@ def make_sitemap(base_url):
|
||||
|
||||
|
||||
def _collect(table, limit, label, **query):
|
||||
from devplacepy.database import listed_clause
|
||||
|
||||
if table.has_column("deleted_at") and "deleted_at" not in query:
|
||||
query["deleted_at"] = None
|
||||
rows = list(table.find(_limit=limit, **query))
|
||||
listed = listed_clause(table)
|
||||
clauses = [listed] if listed is not None else []
|
||||
rows = list(table.find(*clauses, _limit=limit, **query))
|
||||
if len(rows) >= limit:
|
||||
logger.warning("sitemap: %s truncated at %d entries", label, limit)
|
||||
return rows
|
||||
@@ -574,11 +578,13 @@ def _build_sitemap(base_url):
|
||||
)
|
||||
)
|
||||
|
||||
from devplacepy.database import listed_sql
|
||||
|
||||
if "users" in db.tables:
|
||||
post_counts = {}
|
||||
if "posts" in db.tables:
|
||||
for row in db.query(
|
||||
"SELECT user_uid, COUNT(*) AS c FROM posts WHERE deleted_at IS NULL GROUP BY user_uid"
|
||||
f"SELECT user_uid, COUNT(*) AS c FROM posts WHERE deleted_at IS NULL AND {listed_sql('posts')} GROUP BY user_uid"
|
||||
):
|
||||
post_counts[row["user_uid"]] = row["c"]
|
||||
users = _collect(
|
||||
|
||||
@@ -132,7 +132,7 @@ devplace devii reset-quota --all # Reset every quota (users and guests)
|
||||
- **Deterministic / eventual caches skip version-sync on purpose.** Two caches need no `cache_state` name because correctness does not depend on cross-worker freshness: (1) `docs_prose._render_markdown` is an `@lru_cache` on the **static** prose source string (pure function of template content, so identical on every worker; changes only on deploy), and (2) `routers/landing.py` `_home_cache` (`TTLCache ttl=60`) memoizes the guest `/` blocks - the featured-news block (identical for everyone) and the latest-posts block **only for the no-block case**; a viewer with a non-empty block set bypasses the cache and is computed fresh, so the original block-filter semantics are preserved byte-for-byte and there is zero cross-user leakage. Worst case is a soft-deleted/edited public post lingering on `/` for <=60s. Use a plain `TTLCache`/`lru_cache` (no version name) ONLY when the value is deterministic or its staleness is purely cosmetic; anything whose staleness affects correctness or permissions MUST use the version-sync pattern above.
|
||||
- **Authoritative state lives in the DB**, memory is only a short-TTL accelerator (sessions, the Devii 24h cap via `devii_usage_ledger`, cost counters).
|
||||
- **Global rates/quotas are worker-count-aware or DB-backed.** The rate limiter enforces `ceil(limit / DEVPLACE_WEB_WORKERS)` per process so the aggregate matches the configured value with no per-request write. `DEVPLACE_WEB_WORKERS` MUST equal the real `--workers` (set in `make prod`=2 and the Dockerfile=2; default 1 for dev). Update it whenever you change worker count.
|
||||
- **First-boot side effects are flock-serialized and idempotent.** `init_db()` runs under a blocking `flock` on `devplace-init.lock` (the seed-if-missing inserts would otherwise race to duplicate rows); `ensure_certificates()` early-returns when keys exist and otherwise generates under `.vapid.lock`. Any new run-once-at-startup work follows the same pattern (flock + exists-check / `INSERT OR IGNORE`).
|
||||
- **First-boot side effects are flock-serialized and idempotent.** `init_db()` runs under a blocking `flock` on `devplace-init.lock` (the seed-if-missing inserts would otherwise race to duplicate rows); `ensure_certificates()` early-returns when keys exist and otherwise creates the key directory, then generates under `.vapid.lock`. Any new run-once-at-startup work follows the same pattern (flock + exists-check / `INSERT OR IGNORE`).
|
||||
- **Run-once background work stays behind the services lock**, never per-worker: gate it on `service_manager.owns_lock()` (held via `devplace-services.lock`), as news/bots/Devii hub do.
|
||||
|
||||
## Performance caches (request-path)
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user