devplacepy/README.md at 51832664c4b98642bfdfb1453642be102f2256f6

 # DevPlace - The Developer Social Network
 Server-rendered social network for developers. FastAPI backend serving Jinja2 templates with ES6 interactivity. Avatar generation via Multiavatar (local, no network). SQLite via `dataset` with WAL mode and concurrency tuning.
 [![CI](https://git.example.com/api/badges/DevPlace/ci/status.svg)](.gitea/workflows/test.yaml)
 ## Quick start
 ```bash
 make install        # pip install -e .
 make dev            # uvicorn --reload on port 10500
 make test           # Playwright integration + unit tests, headless, fail-fast
 make test-headed    # same tests in visible browser
 make demo           # full-journey GUI demo (headed)
 ```
 Open `http://localhost:10500`.
 ## Stack
 | Layer | Technology |
 |-------|-----------|
 | Backend | Python 3.13+, FastAPI, Uvicorn (single worker) |
 | Templates | Jinja2 (server-side rendered) |
 | Frontend | Pure ES6 JavaScript, one class per file |
 | Database | SQLite via `dataset` (auto-sync schema, `uid` PKs, WAL mode, 30s busy timeout) |
 | Auth | Session cookies, SHA256+SALT via passlib |
 | Avatars | Multiavatar (local SVG generation, no external API, <5ms) |
 | Validation | `hawk` (Python/JS/CSS/HTML) |
 | Load testing | Locust (locustfile.py) |
 ## Project structure
 ```
 devplacepy/
   main.py             # FastAPI app, router registration
   config.py           # Settings from env vars + .env
   database.py         # dataset connection, index creation
   templating.py       # Shared Jinja2 environment + globals
   avatar.py           # Multiavatar generation, URL builder
   utils.py            # Password hashing, session mgmt, time_ago, notification hook
   models.py           # Pydantic schemas
   push.py             # Web push crypto, VAPID keys, encrypt/send/register
   routers/            # One file per domain (auth, feed, posts, push, ...)
   templates/          # Jinja2 HTML templates
   static/css/         # Page-specific CSS files
   static/js/          # Application.js (ES6 module)
   services/           # Background service framework (base, manager, news)
 ```
 ## Routes
 | Prefix | Purpose |
 |--------|---------|
 | `/auth` | Signup, login, logout, forgot/reset password |
 | `/feed` | Post feed with topic/tab filtering (public) |
 | `/news` | Developer news listing, detail page with comments |
 | `/posts` | Post detail, creation |
 | `/comments` | Comment creation, deletion |
 | `/projects` | Project listing, creation |
 | `/profile` | Profile view, editing |
 | `/messages` | Direct messaging |
 | `/notifications` | Notification list, mark read |
 | `/votes` | Upvote/downvote on posts, comments, projects |
 | `/follow` | Follow/unfollow users |
 | `/avatar` | Multiavatar proxy with in-memory cache |
 | `/bugs` | Bug reports listing, creation |
 | `/services` | Background service monitoring (status, logs) |
 | `/admin` | Admin panel (user management, news curation, settings) |
 | `(none)` | `/robots.txt`, `/sitemap.xml` (SEO) |
 | `(none)` | `/push.json` (VAPID key + subscribe), `/service-worker.js`, `/manifest.json` (push + PWA) |
 ## Configuration
 | Env var | Default | Purpose |
 |---------|---------|---------|
 | `DEVPLACE_DATABASE_URL` | `sqlite:///devplace.db` | Database connection string |
 | `SECRET_KEY` | hardcoded fallback | Session signing key |
 | `DEVPLACE_VAPID_SUB` | `mailto:retoor@molodetz.nl` | Contact address in the VAPID JWT `sub` claim |
 ## Background Services
 `devplacepy/services/` provides a generic async service framework. Services start on server boot, run at configurable intervals, and are gracefully cancelled on shutdown.
 ### Service framework
 - **`BaseService`** - abstract class with async run loop, `deque(maxlen=20)` log buffer, `name`, `interval_seconds`
 - **`ServiceManager`** - singleton that registers, starts, and stops all services
 - **`NewsService`** - fetches news from `news.app.molodetz.nl/api`, grades with AI, stores articles >= threshold
 ### Adding a service
 Create a class extending `BaseService`, override `run_once()`, register in `main.py`:
 ```python
 service_manager.register(YourService())
 ```
 The service appears at `/services` with live status and log tail.
 ### News service
 Fetches news from a configurable API, grades each article via AI, stores ALL articles in the `news` table (nothing is silently skipped). Articles with grade >= the configurable threshold are auto-published; the rest go to draft. Admin can manually publish/draft, toggle for landing page appearance, and delete. Images are extracted from article URLs.
 Configuration via admin site settings:
 | Setting key | Default | Purpose |
 |-------------|---------|---------|
 | `news_grade_threshold` | `7` | Minimum AI grade for auto-publish |
 | `news_api_url` | `https://news.app.molodetz.nl/api` | News source |
 | `news_ai_url` | `https://openai.app.molodetz.nl/v1/chat/completions` | AI grading endpoint |
 | `news_ai_model` | `molodetz` | AI model |
 News articles have detail pages at `/news/{slug}` with full comment support (same component as posts/projects). The landing page can display curated articles toggled from admin.
 CLI: `devplace news clear` - delete all news from local database.
 ## Push notifications & PWA
 Authenticated users can receive native web push notifications, and the site is an
 installable Progressive Web App. Push uses only standard libraries (`cryptography`,
 `PyJWT`, `httpx`) against the Web Push Protocol — no third-party push wrapper.
 ### Events
 Every event that already produces an in-app notification also sends a web push,
 because both share a single funnel — `create_notification()` in `utils.py`:
 | Event | Recipient |
 |-------|-----------|
 | Direct message received | receiver |
 | Comment on your post | post author |
 | Reply to your comment | comment author |
 | `@mention` in any content | mentioned user |
 | Upvote on your content | content owner |
 | New follower | followed user |
 `create_notification` schedules delivery as a fire-and-forget async task, so a dead
 subscription or push-service error never blocks the triggering request. Delivery
 (`push.notify_user`) iterates a user's subscriptions, encrypts the payload
 (legacy `aesgcm` content encoding), and POSTs to each endpoint; subscriptions that
 return `404`/`410` are soft-deleted.
 ### VAPID keys
 The server identity is three PEM files generated once at startup in the repository
 root: `notification-private.pem`, `notification-private.pkcs8.pem`,
 `notification-public.pem`. They are git-ignored.
 **These keys are the application's identity to the push services. If they are lost or
 regenerated, every existing subscription becomes permanently undeliverable.** Persist
 them across deployments and back them up; do not regenerate them.
 ### Opt-in
 The browser requires the first permission prompt to originate from a user gesture, so
 opt-in is exposed as a button in both the top navigation (bell-with-slash icon) and on
 the `/notifications` page. After opt-in, the subscription is refreshed silently on
 every page load. `PushManager.js` owns registration, subscription, and the opt-in UI.
 ### PWA
 `manifest.json` (192/512 and maskable icons), `service-worker.js`, and an install
 button (`PwaInstaller.js`) make the app installable. The service worker uses a
 network-first strategy for navigations and falls back to `static/offline.html` when
 offline. Installation requires a secure origin (HTTPS, or `localhost` for development).
 | File | Role |
 |------|------|
 | `devplacepy/push.py` | VAPID keys, payload encryption, send, register |
 | `devplacepy/routers/push.py` | `/push.json`, `/service-worker.js`, `/manifest.json` |
 | `static/js/PushManager.js` | Service-worker registration + subscribe + opt-in UI |
 | `static/js/PwaInstaller.js` | `beforeinstallprompt` capture + install button |
 | `static/service-worker.js` | Receives push, shows notification, offline fallback |
 | `static/manifest.json` | PWA manifest (icons, display, theme) |
 | `static/offline.html` | Offline fallback page |
 ## Database
 SQLite via `dataset` with production-oriented pragmas set on every connection:
 ```sql
 PRAGMA journal_mode=WAL;      -- concurrent readers + writers
 PRAGMA synchronous=NORMAL;     -- safe with WAL, faster than FULL
 PRAGMA busy_timeout=30000;     -- wait 30s instead of failing on lock
 PRAGMA cache_size=-8000;       -- 8MB page cache
 PRAGMA temp_store=MEMORY;      -- temp tables in memory
 PRAGMA mmap_size=268435456;    -- 256MB memory map for reads
 ```
 All indexes are created via `CREATE INDEX IF NOT EXISTS` wrapped in try/except - safe to run on every startup regardless of table state.
 ## Testing
 - **274 tests** across 23 files: Playwright integration + unit tests
 - Playwright (NOT pytest-playwright plugin - conflicts, uninstall it)
 - Server starts as subprocess on port 10501 with isolated temp database
 - Test users `alice_test` / `bob_test` seeded via HTTP at session start
 - Tests stop at first failure (`-x` flag)
 - Failure screenshots auto-save to `/tmp/devplace_test_screenshots/`
 - Headed mode: `PLAYWRIGHT_HEADLESS=0 make test`
 ### Key test patterns
 - Every `page.goto()` and `page.wait_for_url()` uses `wait_until="domcontentloaded"` - avatar images don't block test execution
 - Session-scoped browser context with per-test cookie clearing
 - `page.locator(...).wait_for(state="visible")` preferred over bare selectors
 ## Avatars
 Uses [Multiavatar](https://github.com/multiavatar/multiavatar-python) - generates deterministic SVG avatars locally from a seed string (the username). No external API calls, no network dependency. Generation takes <5ms. Results are cached in-memory (cleared on server restart).
 Avatar URL format: `/avatar/multiavatar/{username}?size={size}`
 ## CI/CD
 Gitea Actions workflow at `.gitea/workflows/test.yaml`:
 - Runs on push/PR to main
 - Sets up Python 3.13, installs dependencies + Playwright
 - Validates all source files with `hawk`
 - Runs all tests with fail-fast
 - Uploads failure screenshots as artifacts
 ## Feature workflow
 . Implement the feature (router + template + CSS + JS)
 . `hawk .` - validate all source files
 . `make test` - run all tests (fail-fast)
 . Add Playwright tests in `tests/test_*.py` for new functionality
 . For visual features: `falcon take --output /tmp/verify.png && falcon describe /tmp/verify.png`
 . Update `AGENTS.md` and `README.md` if new conventions were introduced
 ## License
 MIT - DevPlace