Make the presence roster the single source of truth for online status

Online status had three server-side candidate populations and two
client-side deciders, so the feed roster and the /messages indicators
could legitimately disagree.

PresenceRelayService built its online set from whichever topics happened
to be subscribed on a given tick: the roster candidates on /feed, only
the per-uid dot rows on /messages. Different populations meant a
different hysteresis baseline, so the same user could be online in one
place and offline in the other. On top of that, PresenceManager re-derived
online status client-side from a frozen data-presence-last-seen with a
strict timeout and no hysteresis, re-evaluated every 20s, so any element
whose relay frame was missed drifted grey after the timeout and stayed
there. AppChat carried a third renderer with its own PubSubClient that
only ever wrote online/offline, plus hand-built dot markup duplicating
_presence_dot.html.

The relay now collapses to one set on one topic. Each tick it reads the
online population in a single indexed query (online_candidates, capped by
the new PRESENCE_TRACK_LIMIT), applies hysteresis once, and publishes
{count, online, users} on public.presence.roster only when the uid set
changes. online is the authority for every avatar dot; users is the same
set trimmed to PRESENCE_ONLINE_LIMIT for the feed panel. The per-uid
public.presence.{uid} topics are gone, which also removes one
subscription per distinct author on a page.

is_online(user) is now stays_online(seconds_since(last_seen), False), so
the server-rendered initial state and the live set apply one formula.

PresenceManager makes one subscription and renders every
[data-presence-uid] element as membership of that set, with no clock and
no expiry timer; before the first frame the server-rendered state stands.
Relative "last seen" text is a <time data-dt data-dt-mode="ago"> handled
by the shared LocalTime. AppChat lost its presence code entirely, and the
new Avatar.badgeElement is the JS twin of _presence_dot.html, so dot
markup now lives in exactly two places.

Also fix the awards column ensure-block, which the awards tests exposed.
backfill_api_keys opened with an "if users not in db.tables" guard, but
on a brand-new database that is precisely the state at init_db time, so
the whole users ensure-block was skipped. The first signup then created
users with only the columns of that INSERT, leaving every ensured-but-
unwritten column absent from the server's reflected metadata for the rest
of the process lifetime. That is why the awards tab, the prominent award
banner and the avatar award badge were invisible on a fresh database. It
now calls get_table("users") unconditionally.

test_avatar_badge_on_feed_when_prominent asserted that any online user
carries an award badge while only awarding a user who was never active,
so it passed only by accident. It now makes the awarded user active and
asserts the badge on that user's roster entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

CLAUDE.md

@@ -119,6 +119,7 @@ Tests run on port 10501 with a tempfile SQLite DB and a dedicated `DEVPLACE_DATA
 | `DEVPLACE_WEB_WORKERS` / `--workers` | `nproc` (prod) | Uvicorn worker count; `make prod WEB_WORKERS=N` to override. |
 | `DEVPLACE_PRESENCE_TIMEOUT_SECONDS` | `60` | How long after a user's last activity they still count as online. `config.PRESENCE_WRITE_SECONDS` (half of it) throttles `last_seen` writes per worker. |
 | `DEVPLACE_PRESENCE_ONLINE_LIMIT` | `30` | Max avatars in the feed's live "Online now" panel. |
+| `DEVPLACE_PRESENCE_TRACK_LIMIT` | `500` | Size of the online set the presence relay tracks and publishes as the authority for every avatar dot. `PRESENCE_ONLINE_LIMIT` only caps how many of them the feed panel *displays*. |
 | `DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS` | `20` | Grace margin (hysteresis) before an online user is dropped, kills dot/roster flicker at the boundary. |
 | `DEVPLACE_DATA_DIR` | `<repo>/data` | Single root for ALL runtime/user-generated data OUTSIDE the package and OUTSIDE `/static`. Point at a volume in prod. |
 | `DEVPLACE_OUTBOUND_PROXY_URL` | unset | Fallback for the `outbound_proxy_url` site setting (below) when the DB/settings row is unavailable (early CLI contexts). Prefer configuring the setting via `/admin/settings` - it applies live with no restart. |

README.md

@@ -227,6 +227,7 @@ The log is **administrator-only**. `/admin/audit-log` is a paginated, filterable
 | `DEVPLACE_PRESENCE_TIMEOUT_SECONDS` | `60` | Online-presence window: a user counts as online for this many seconds after their last activity. `last_seen` is refreshed by a throttled in-place update at most once per half this interval per worker (no per-load inserts, no data growth) |
 | `DEVPLACE_PRESENCE_ONLINE_LIMIT` | `30` | Maximum avatars shown in the feed's live "Online now" panel (ordered alphabetically by username) |
 | `DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS` | `20` | Grace margin before an online user drops to offline (hysteresis): online at the timeout, offline only after timeout + this. Prevents online/offline flicker for users hovering at the boundary |
+| `DEVPLACE_PRESENCE_TRACK_LIMIT` | `500` | Size of the online set the presence relay tracks and publishes on `public.presence.roster`. That one set is the single source of truth behind every avatar presence dot on every page; `DEVPLACE_PRESENCE_ONLINE_LIMIT` only caps how many of them the feed's Online now panel displays |
 
 ### Runtime settings
 

devplacepy/config.py

@@ -50,6 +50,7 @@ SITE_URL = environ.get("DEVPLACE_SITE_URL", "").rstrip("/")
 PRESENCE_TIMEOUT_SECONDS = int(environ.get("DEVPLACE_PRESENCE_TIMEOUT_SECONDS", "60"))
 PRESENCE_WRITE_SECONDS = max(1, PRESENCE_TIMEOUT_SECONDS // 2)
 PRESENCE_ONLINE_LIMIT = int(environ.get("DEVPLACE_PRESENCE_ONLINE_LIMIT", "30"))
+PRESENCE_TRACK_LIMIT = int(environ.get("DEVPLACE_PRESENCE_TRACK_LIMIT", "500"))
 PRESENCE_ONLINE_MARGIN_SECONDS = int(
     environ.get("DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS", "20")
 )

devplacepy/database/CLAUDE.md

@@ -28,6 +28,8 @@ _index(db, "news", "idx_news_status", ["status"])   # now safe - column exists
 
 Do this with `get_table(name)` (NOT `if name in db.tables`): `get_table` + `create_column_by_example` creates the table if it is absent, so the schema is guaranteed regardless of which process or which insert touches the table first. Symptom to watch for: a page renders fine in isolation but shows empty data or 500s only after a CLI/maintenance test runs earlier in the suite (it created the table partially).
 
+**This applies to `users` too, and an `if "users" not in db.tables: return` guard silently defeats it.** `backfill_api_keys()` is the `users` ensure-block (every non-signup column: `api_key`, `last_seen`, the AI correction/modifier settings, `avatar_seed`, `award_count`/`last_award_at`/`last_award_slug`/`last_award_uid`, ...). It used to early-return when `users` was absent, which is exactly the state on a **brand-new database**: `init_db()` runs before the first signup, so the ensure-block was skipped entirely, and the first `/auth/signup` then created `users` with only the columns of that INSERT. Every ensured-but-unwritten column was therefore missing from the long-running server's reflected metadata, so `users.find_one(...)` returned rows without them **for the whole process lifetime** - the feature reading them looked simply switched off (this is what made the awards tab, the prominent-award banner, and the avatar award badge invisible on a fresh DB, and it is invisible in production only because the columns happen to exist from an older boot). It now calls `get_table("users")` unconditionally, so the table is born with the full ensured column set. Never reintroduce a `db.tables` guard in front of a column-ensure block.
+
 The `_index(...)` helper supports `where=` (partial) and `unique=` indexes; every table with a `uid` column gets a UNIQUE `idx_<table>_uid`, soft-delete tables get a PARTIAL `idx_<table>_trash` (`WHERE deleted_at IS NOT NULL`) and NEVER a bare `deleted_at` index (it mis-steers the planner on live reads), and "live newest-first" listings need a composite/live-partial index that carries the sort column (see "Indexing conventions" below). `init_db()` finishes with `ANALYZE`/`PRAGMA optimize`. Verify any index change with `EXPLAIN QUERY PLAN`.
 
 **SQLite is synchronous by design and will never be made async (hard rule).** The `dataset`/SQLite layer is synchronous and is called directly inside the async route handlers. This is intentional and more than fast enough for this platform: the database is a local file tuned with WAL, `synchronous=NORMAL`, a 30s busy timeout, an 8MB page cache, and a 256MB mmap, so queries are sub-millisecond and never meaningfully stall the event loop. Do NOT convert the database layer to async, do NOT wrap DB calls in a threadpool/`run_in_executor`/`to_thread`, and do NOT raise the synchronous DB access as a performance, blocking-I/O, or "requests aren't fully async" concern. This is a settled, sophisticated design choice and is not open for revisiting.

devplacepy/database/schema.py

@@ -1720,9 +1720,7 @@ def migrate_ai_gateway_settings() -> None:
 
 
 def backfill_api_keys() -> int:
-    if "users" not in db.tables:
+    users = get_table("users")
-        return 0
-    users = db["users"]
     if not users.has_column("api_key"):
         users.create_column_by_example("api_key", "")
     if not users.has_column("created_at"):

devplacepy/services/CLAUDE.md

@@ -179,12 +179,12 @@ Online status is a single **`users.last_seen`** UTC-ISO column (ensured in `data
 
 **Read path (any worker):** `presence.is_online(user_row)` = `now - last_seen < PRESENCE_TIMEOUT_SECONDS` (env `DEVPLACE_PRESENCE_TIMEOUT_SECONDS`, default 60). Profile (`routers/profile/index.py` -> `profile_online`) and messages (`routers/messages.py` seed) read `last_seen` off the user row they already loaded - no extra query. Exposed as the Jinja global `is_online(user)` (`templating.py`), on `UserOut.last_seen` and `ProfileOut.profile_online`. This is the **only** cross-worker-correct approach here because pub/sub is in-process.
 
-**Live path (lock owner only), change-only + hysteresis:** `PresenceRelayService` (`services/presence_relay.py`, `BaseService`, default-enabled, 2s tick, registered in `main.py`) is a sibling of `NotificationRelayService`/`LiveViewRelayService`. Each tick it recomputes ONE global online set `self._online` (dot-subscribed uids batch-read via `get_users_by_uids` + roster candidates) and drives BOTH the per-user dots and the feed roster from that one set, so they can never disagree. The set uses **hysteresis** via `presence.stays_online(elapsed, was_online)`: a user becomes online at `PRESENCE_TIMEOUT_SECONDS` but only drops after `+ PRESENCE_ONLINE_MARGIN_SECONDS` (env `DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS`, default 20) - **quick to go online, slow (grace margin) to go offline** - which kills boundary flicker for a user hovering near the timeout. It publishes `{online, last_seen}` to `public.presence.{uid}` **only when a topic's `online` bool changed OR the topic is newly subscribed (first-seen)** - never on a fixed interval, so a steady page emits nothing after the initial frame (`self._published[topic] -> bool`, pruned to active topics). `public.presence.{uid}` is subscribable by any logged-in user (`pubsub/policy.py` allows `public.*`); guests fall back to the server-rendered initial state. The one-directional grace also means dots and roster stay consistent across viewers (the shared `self._online` is the single authority). To keep it lightweight the relay reads all due users in **one batched `get_users_by_uids`** per tick.
+**Live path (lock owner only), ONE set on ONE topic, change-only + hysteresis:** `PresenceRelayService` (`services/presence_relay.py`, `BaseService`, default-enabled, 2s tick, registered in `main.py`) is a sibling of `NotificationRelayService`/`LiveViewRelayService` and is **the single source of truth for live online status**. Each tick, only while the roster topic has subscribers, it reads the online population in ONE indexed query (`presence.online_candidates()`, capped at `config.PRESENCE_TRACK_LIMIT`, env `DEVPLACE_PRESENCE_TRACK_LIMIT`, default 500) and recomputes ONE set `self._online` with **hysteresis** via `presence.stays_online(elapsed, was_online)`: a user becomes online at `PRESENCE_TIMEOUT_SECONDS` but only drops after `+ PRESENCE_ONLINE_MARGIN_SECONDS` (env `DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS`, default 20) - **quick to go online, slow (grace margin) to go offline** - which kills boundary flicker. It publishes that set on the ONE shared topic `public.presence.roster` (`roster_payload`: `{count, online: [uid...], users: [display rows]}`) **only when the set of online uids changes** (a `frozenset` compare, so reordering never republishes), never on a fixed interval, so an idle site emits nothing. `online` is the authority for EVERY avatar dot; `users` is the same set trimmed to `PRESENCE_ONLINE_LIMIT` for the feed's avatar panel, and `count` matches it. **There are no per-user `public.presence.{uid}` topics** - they were removed precisely because their candidate population differed from the roster's, so dots and roster could disagree (the /messages-vs-feed bug). One set, one topic, one frame. `public.presence.roster` is subscribable by any logged-in user (`pubsub/policy.py` allows `public.*`); guests keep the server-rendered initial state.
 
-**Frontend:** `static/js/PresenceManager.js` (`app.presence`, constructed with `this.pubsub` in `Application.js`, mirroring `LocalTime`/`CounterManager`) scans `[data-presence-uid]` elements, subscribes each uid to `public.presence.{uid}` (deduped per uid, so a repeated author is one subscription), and treats each frame's `online` flag as **authoritative** (`entry.online`), toggling the `online` class + (for `data-presence-label` elements) the "online / last seen X / offline" text. Because the relay is change-only and authoritative, a live-subscribed dot is **not** expired by the client clock (no false-offline flicker for an active user whose `last_seen` the client cannot see advancing); the 20s `last_seen` staleness timer only applies to entries that never received a frame (guests / degraded). The window is read from `<body data-presence-timeout>`. Both the profile `.profile-presence` dot and the messages `#messages-presence` span carry `data-presence-uid`/`data-presence-last-seen`.
+**Frontend:** `static/js/PresenceManager.js` (`app.presence`, constructed with `this.pubsub` in `Application.js`, mirroring `LocalTime`/`CounterManager`) makes ONE subscription to `public.presence.roster` and keeps the pushed `online` uid set. It scans `[data-presence-uid]` elements (on load and via a `MutationObserver`, so dynamically inserted markup is covered) and renders each one purely as membership of that set - toggling the `online` class and, for `data-presence-label` elements, the "online / last seen X / offline" text, where the relative time is a `<time data-dt data-dt-mode="ago">` formatted by the shared `LocalTime` (presence never formats a date itself). **There is NO client-side clock, no `data-presence-timeout`, and no expiry timer** - the old `isOnline(lastSeen)` fallback was a second decider that silently drifted a dot to grey after the timeout whenever a frame was missed, which is exactly how /messages diverged from the feed. Before the first frame arrives the server-rendered state simply stands. The constructor takes an optional `root` (`new PresenceManager(pubsub, root)`) so a detached widget can scope it; `dp-chat mode="embed"` uses that only when no page-global `app.presence` exists. Both the profile `.profile-presence` dot and the messages `#messages-presence` span carry `data-presence-uid`/`data-presence-last-seen`.
 
-**Online-now roster (feed):** the same relay maintains ONE shared topic `public.presence.roster`, republished **only when the SET of online uids changes** (a `frozenset` compare, so pure reordering never republishes). `services/presence.py` `online_users(limit)` (strict, feed initial render) and `online_candidates(limit)` (grace window, relay hysteresis) both go through `database.get_online_users(cutoff_iso, limit)`, which reads users with `last_seen >= cutoff` via the `idx_users_last_seen` index (the one place presence is queried by `last_seen`; `config.PRESENCE_ONLINE_LIMIT`, env `DEVPLACE_PRESENCE_ONLINE_LIMIT`, default 30). **The list is ordered ALPHABETICALLY by username** (`get_online_users` `order_by=["username"]` + case-insensitive `presence.sort_by_username`), NOT by recency, so avatars keep a stable position and do not needlessly reshuffle as people's `last_seen` ticks. `routers/feed.py` puts `online_users` on the context (`FeedOut.online_users`) and `feed.html` renders the initial **Online now** panel as a `.sidebar-section` at the bottom of the left feed sidebar (`aside.sidebar-card`); `static/js/OnlineUsers.js` (`app.onlineUsers`) subscribes to `public.presence.roster` and re-renders the avatar list + count live. Roster avatars use a plain green `.presence-dot` with NO `data-presence-uid` (list membership IS the presence, so no per-user subscription - the relay drops a user from the roster when they go offline).
+**Online-now roster (feed):** the feed panel is just the display face of the same frame. `services/presence.py` `online_users(limit=PRESENCE_ONLINE_LIMIT)` (strict cutoff, feed initial render) and `online_candidates(limit=PRESENCE_TRACK_LIMIT)` (grace window, the relay's authority population) both go through `database.get_online_users(cutoff_iso, limit)`, which reads users with `last_seen >= cutoff` via the `idx_users_last_seen` index (the one place presence is queried by `last_seen`). **The list is ordered ALPHABETICALLY by username** (`get_online_users` `order_by=["username"]` + case-insensitive `presence.sort_by_username`), NOT by recency, so avatars keep a stable position. `routers/feed.py` puts `online_users` on the context (`FeedOut.online_users`) and `feed.html` renders the initial **Online now** panel as a `.sidebar-section` at the bottom of the left feed sidebar; `static/js/OnlineUsers.js` (`app.onlineUsers`) subscribes to `public.presence.roster` and re-renders the avatar list + count from `users`/`count`. **A roster avatar is NOT a special case** - it renders the same `_presence_dot.html` (server) / `Avatar.badgeElement` (client) as every other avatar, subscribed like every other dot, so it cannot disagree with the rest of the page.
 
-**Avatar presence dot (sitewide, DRY):** a small corner dot on **every** user avatar (green online, muted grey offline) comes from ONE reusable partial `templates/_presence_dot.html` - `<span class="presence-dot" data-presence-uid data-presence-last-seen>` guarded on `_user.get('uid')` (a partial-dict author, e.g. the issues includes, renders no dot). It carries **no** `data-presence-label`, so `PresenceManager` colours it with zero extra JS. It is included by the shared avatar partial `templates/_avatar_link.html` (its `.user-avatar-link` anchor is the positioning host, covering ~19 sites) and by the handful of raw-`<img class="avatar-img">` sites wrapped in a positioned `<span class="avatar-badge">` (the two `base.html` nav avatars, the `profile.html` hero + followers list, the `messages.html` conversation list). CSS in `static/css/base.css` (`.user-avatar-link`/`.avatar-badge` `position:relative;display:inline-flex`, `.presence-dot` sized `30%` of the avatar clamped 8-14px with a `--bg-card` ring, `.online` -> `--success`), so it is proportional and responsive at every avatar size with no per-size class. `database/follows.py` `get_follow_list` now carries `last_seen` in its trimmed dict so the followers/following dots resolve (all other author dicts are full `get_users_by_uids` rows). `dp-avatar` (`AppAvatar.js`) is docs-demo only (no real user avatars) and is intentionally out of scope. Reuse `_presence_dot.html` + the `.avatar-badge` wrapper for any new avatar surface - never hand-roll a presence dot.
+**Avatar presence dot (sitewide, DRY):** a small corner dot on **every** user avatar (green online, muted grey offline) comes from ONE reusable partial `templates/_presence_dot.html` - `<span class="presence-dot" data-presence-uid data-presence-last-seen>` guarded on `_user.get('uid')` (a partial-dict author, e.g. the issues includes, renders no dot). It carries **no** `data-presence-label`, so `PresenceManager` colours it with zero extra JS. It is included by the shared avatar partial `templates/_avatar_link.html` (its `.user-avatar-link` anchor is the positioning host, covering ~19 sites) and by the handful of raw-`<img class="avatar-img">` sites wrapped in a positioned `<span class="avatar-badge">` (the two `base.html` nav avatars, the `profile.html` hero + followers list, the `messages.html` conversation list). CSS in `static/css/base.css` (`.user-avatar-link`/`.avatar-badge` `position:relative;display:inline-flex`, `.presence-dot` sized `30%` of the avatar clamped 8-14px with a `--bg-card` ring, `.online` -> `--success`), so it is proportional and responsive at every avatar size with no per-size class. `database/follows.py` `get_follow_list` now carries `last_seen` in its trimmed dict so the followers/following dots resolve (all other author dicts are full `get_users_by_uids` rows). `dp-avatar` (`AppAvatar.js`) is docs-demo only (no real user avatars) and is intentionally out of scope. In JavaScript the matching builder is `Avatar.badgeElement(user)` (`static/js/Avatar.js`), which emits the `.avatar-badge` + `.presence-dot` + award-badge trio and is used by `OnlineUsers` and `AppChat._buildConversationItem`, so dot markup lives in exactly two places: the partial and that helper. Reuse `_presence_dot.html` (server) or `Avatar.badgeElement` (client) for any new avatar surface - never hand-roll a presence dot.
 
-**Messaging refactor:** the old presence was per-worker and WS-connect-based (`message_hub.is_online`/`last_seen`, `_announce_presence`, the WS `presence` frame) and broke with >1 worker. That display path was **removed**; `message_hub` keeps only its socket connection tracking for message delivery. The messages header presence is now the shared `PresenceManager`, so chat presence is finally cross-worker correct. **Never re-implement WS-connect presence** - reuse `presence.is_online`, the `public.presence.{uid}` topic, and `PresenceManager`.
+**Messaging refactor:** the old presence was per-worker and WS-connect-based (`message_hub.is_online`/`last_seen`, `_announce_presence`, the WS `presence` frame) and broke with >1 worker. That display path was **removed**; `message_hub` keeps only its socket connection tracking for message delivery. The messages header presence is now the shared `PresenceManager`, so chat presence is finally cross-worker correct. **Never re-implement WS-connect presence** - reuse `presence.is_online`, the `public.presence.roster` topic, and `PresenceManager`. `dp-chat` likewise no longer carries its own presence renderer or `PubSubClient`: that private duplicate only ever showed online/offline (never `last seen`) and was a third source of truth.

devplacepy/services/messaging/CLAUDE.md

@@ -57,7 +57,7 @@ Client -> server: `{type:"send", receiver_uid, content, attachment_uids?, client
 
 ## Presence is NOT part of the messaging WS
 
-Presence is handled entirely by the generic, cross-worker-correct mechanism documented in `devplacepy/services/CLAUDE.md` ("Online presence"): a single `users.last_seen` column, written by the `track_presence` HTTP middleware, read server-side at page load via `presence.is_online(user)`, and kept live client-side by `PresenceManager` (`static/js/PresenceManager.js`) subscribing any `[data-presence-uid]` element to the pub/sub topic `public.presence.{uid}`. The messages page's `#messages-presence` span carries `data-presence-uid`/`data-presence-last-seen` like every other avatar-presence site in the app - it is not messaging-specific code. An earlier design had WS-connect-based presence living in `message_hub` (`is_online`/`last_seen`, an `_announce_presence` helper, and a WS `presence` frame); that design was per-worker only and broke with more than one uvicorn worker, so it was removed outright. `message_hub` today tracks ONLY socket connections for message delivery - it has no presence state, and there is no `presence` frame anywhere in the current WS protocol (client or server). **Never re-implement WS-connect presence on this socket** - reuse `presence.is_online`, the `public.presence.{uid}` topic, and `PresenceManager`, exactly as every other presence surface in the app does.
+Presence is handled entirely by the generic, cross-worker-correct mechanism documented in `devplacepy/services/CLAUDE.md` ("Online presence"): a single `users.last_seen` column, written by the `track_presence` HTTP middleware, read server-side at page load via `presence.is_online(user)`, and kept live client-side by `PresenceManager` (`static/js/PresenceManager.js`) subscribing any `[data-presence-uid]` element to the single pub/sub topic `public.presence.roster`. The messages page's `#messages-presence` span carries `data-presence-uid`/`data-presence-last-seen` like every other avatar-presence site in the app - it is not messaging-specific code. An earlier design had WS-connect-based presence living in `message_hub` (`is_online`/`last_seen`, an `_announce_presence` helper, and a WS `presence` frame); that design was per-worker only and broke with more than one uvicorn worker, so it was removed outright. `message_hub` today tracks ONLY socket connections for message delivery - it has no presence state, and there is no `presence` frame anywhere in the current WS protocol (client or server). **Never re-implement WS-connect presence on this socket** - reuse `presence.is_online`, the `public.presence.roster` topic, and `PresenceManager`, exactly as every other presence surface in the app does. `dp-chat` also no longer owns a private presence renderer or `PubSubClient` - that duplicate disagreed with the feed roster and was removed.
 
 ## Renderer integration (XSS control)
 

devplacepy/services/presence.py

@@ -10,6 +10,7 @@ from devplacepy.config import (
     PRESENCE_ONLINE_LIMIT,
     PRESENCE_ONLINE_MARGIN_SECONDS,
     PRESENCE_TIMEOUT_SECONDS,
+    PRESENCE_TRACK_LIMIT,
     PRESENCE_WRITE_SECONDS,
 )
 from devplacepy.database import get_online_users, set_last_seen
@@ -39,13 +40,6 @@ def seconds_since(last_seen: Optional[str]) -> Optional[float]:
     return (datetime.now(timezone.utc) - seen).total_seconds()
 
 
-def is_online(user: Optional[dict]) -> bool:
-    if not user:
-        return False
-    elapsed = seconds_since(user.get("last_seen"))
-    return elapsed is not None and elapsed < PRESENCE_TIMEOUT_SECONDS
-
-
 def stays_online(elapsed: Optional[float], was_online: bool) -> bool:
     if elapsed is None:
         return False
@@ -53,6 +47,12 @@ def stays_online(elapsed: Optional[float], was_online: bool) -> bool:
     return elapsed < (grace if was_online else PRESENCE_TIMEOUT_SECONDS)
 
 
+def is_online(user: Optional[dict]) -> bool:
+    if not user:
+        return False
+    return stays_online(seconds_since(user.get("last_seen")), was_online=False)
+
+
 def _cutoff_iso(seconds: int) -> str:
     return (datetime.now(timezone.utc) - timedelta(seconds=seconds)).isoformat()
 
@@ -69,7 +69,7 @@ def online_users(limit: int = PRESENCE_ONLINE_LIMIT) -> list:
     return sort_by_username(get_online_users(online_cutoff_iso(), limit))
 
 
-def online_candidates(limit: int = PRESENCE_ONLINE_LIMIT) -> list:
+def online_candidates(limit: int = PRESENCE_TRACK_LIMIT) -> list:
     return sort_by_username(
         get_online_users(
             _cutoff_iso(PRESENCE_TIMEOUT_SECONDS + PRESENCE_ONLINE_MARGIN_SECONDS), limit

devplacepy/services/presence_relay.py

@@ -2,11 +2,10 @@
 
 from __future__ import annotations
 
-import re
 from typing import Optional
 
 from devplacepy.config import PRESENCE_ONLINE_LIMIT
-from devplacepy.database import db, get_users_by_uids
+from devplacepy.database import db
 from devplacepy.database.awards import award_is_prominent
 from devplacepy.services import presence
 from devplacepy.services.base import BaseService
@@ -14,108 +13,73 @@ from devplacepy.services.pubsub import publish as pubsub_publish
 from devplacepy.services.pubsub.hub import pubsub
 
 ROSTER_TOPIC = "public.presence.roster"
-TOPIC_PATTERN = re.compile(r"^public\.presence\.(?P<uid>[A-Za-z0-9_-]{1,128})$")
+
+
+def roster_payload(rows: list, online: set) -> dict:
+    users = [row for row in rows if row["uid"] in online]
+    listed = users[:PRESENCE_ONLINE_LIMIT]
+    return {
+        "count": len(listed),
+        "online": [row["uid"] for row in users],
+        "users": [
+            {
+                "uid": row["uid"],
+                "username": row["username"],
+                "avatar_seed": row.get("avatar_seed") or row["username"],
+                "last_award_slug": row.get("last_award_slug") or "",
+                "award_prominent": award_is_prominent(row),
+            }
+            for row in listed
+        ],
+    }
 
 
 class PresenceRelayService(BaseService):
     title = "Presence relay"
     description = (
-        "Single source of truth for LIVE online status. Each tick it recomputes one online "
+        "Single source of truth for LIVE online status. Each tick it recomputes ONE online "
         "set with hysteresis - a user is online at the timeout window but only drops after an "
-        "extra grace margin - and drives BOTH the per-user avatar dots (public.presence.{uid}) "
+        "extra grace margin - and publishes that whole set on the single shared topic "
-        "and the shared feed roster (public.presence.roster) from that one set, so they never "
+        "public.presence.roster, which drives BOTH the feed's Online now avatars and every "
-        "disagree. It publishes ONLY on change (a real online<->offline transition or a first-seen "
+        "avatar presence dot on every page. One set, one topic, one frame, so no two "
-        "subscriber), reads due users in one batched query per tick, and does nothing when idle. "
+        "indicators can ever disagree. It publishes ONLY when the set of online users changes, "
-        "Runs on the service lock owner where every subscriber converges."
+        "reads the online population in one indexed query per tick, and does nothing when no "
+        "one is subscribed. Runs on the service lock owner where every subscriber converges."
     )
     default_enabled = True
 
     def __init__(self):
         super().__init__(name="presence_relay", interval_seconds=2)
         self._online: set[str] = set()
-        self._published: dict[str, bool] = {}
+        self._published: Optional[frozenset] = None
-        self._roster_uids: Optional[frozenset] = None
+
+    def _subscribed(self) -> bool:
+        return any(
+            entry["subscribers"] and entry["topic"] == ROSTER_TOPIC
+            for entry in pubsub.topics()
+        )
 
     async def run_once(self) -> None:
-        if "users" not in db.tables:
+        if "users" not in db.tables or not self._subscribed():
+            self._published = None
             return
-        dot_pairs: list[tuple[str, str]] = []
-        roster_subscribed = False
-        for entry in pubsub.topics():
-            topic = entry["topic"]
-            if not entry["subscribers"] or "*" in topic:
-                continue
-            if topic == ROSTER_TOPIC:
-                roster_subscribed = True
-                continue
-            match = TOPIC_PATTERN.match(topic)
-            if match is not None:
-                dot_pairs.append((topic, match.group("uid")))
-
-        roster_rows = presence.online_candidates() if roster_subscribed else []
-        rows: dict[str, dict] = {}
-        dot_uids = {uid for _, uid in dot_pairs}
-        if dot_uids:
-            rows.update(get_users_by_uids(list(dot_uids)))
-        for row in roster_rows:
-            rows[row["uid"]] = row
 
+        rows = presence.online_candidates()
         prev = self._online
-        online: set[str] = set()
+        self._online = {
-        for uid, row in rows.items():
+            row["uid"]
-            elapsed = presence.seconds_since(row.get("last_seen"))
+            for row in rows
-            if presence.stays_online(elapsed, uid in prev):
+            if presence.stays_online(
-                online.add(uid)
+                presence.seconds_since(row.get("last_seen")), row["uid"] in prev
-        self._online = online
-
-        await self._publish_dots(dot_pairs, online, rows)
-        await self._publish_roster(roster_subscribed, roster_rows, online)
-
-    async def _publish_dots(self, dot_pairs, online, rows) -> None:
-        active = {topic for topic, _ in dot_pairs}
-        self._published = {t: v for t, v in self._published.items() if t in active}
-        published = 0
-        for topic, uid in dot_pairs:
-            is_on = uid in online
-            if self._published.get(topic) == is_on:
-                continue
-            row = rows.get(uid) or {}
-            published += await pubsub_publish(
-                topic, {"online": is_on, "last_seen": row.get("last_seen")}
             )
-            self._published[topic] = is_on
+        }
-        if published:
-            self.log(f"pushed {published} presence change(s)")
 
-    async def _publish_roster(self, subscribed, roster_rows, online) -> None:
+        uids = frozenset(self._online)
-        if not subscribed:
+        if uids == self._published:
-            self._roster_uids = None
             return
-        users = [row for row in roster_rows if row["uid"] in online][:PRESENCE_ONLINE_LIMIT]
+        self._published = uids
-        uids = frozenset(row["uid"] for row in users)
+        await pubsub_publish(ROSTER_TOPIC, roster_payload(rows, self._online))
-        if uids == self._roster_uids:
+        self.log(f"roster changed: {len(uids)} online")
-            return
-        self._roster_uids = uids
-        await pubsub_publish(
-            ROSTER_TOPIC,
-            {
-                "count": len(users),
-                "users": [
-                    {
-                        "uid": row["uid"],
-                        "username": row["username"],
-                        "avatar_seed": row.get("avatar_seed") or row["username"],
-                        "last_award_slug": row.get("last_award_slug") or "",
-                        "award_prominent": award_is_prominent(row),
-                    }
-                    for row in users
-                ],
-            },
-        )
-        self.log(f"roster changed: {len(users)} online")
 
     def collect_metrics(self) -> dict:
-        return {
+        return {"online": len(self._online)}
-            "online": len(self._online),
-            "tracked_dots": len(self._published),
-        }

devplacepy/static/js/Avatar.js

@@ -12,6 +12,37 @@ export class Avatar {
         img.loading = "lazy";
         return img;
     }
+
+    static badgeElement(user, size = 32, sizeClass = "sm") {
+        const badge = document.createElement("span");
+        badge.className = "avatar-badge";
+        const username = user.username || "";
+        const img = document.createElement("img");
+        img.src = `/avatar/multiavatar/${encodeURIComponent(user.avatar_seed || username)}?size=${size}`;
+        img.className = `avatar-img avatar-${sizeClass}`;
+        img.alt = username;
+        img.loading = "lazy";
+        badge.appendChild(img);
+        badge.appendChild(Avatar.presenceDot(user));
+        if (user.award_prominent && user.last_award_slug) {
+            const award = document.createElement("img");
+            award.className = "award-badge";
+            award.src = `/awards/${encodeURIComponent(user.last_award_slug)}/64`;
+            award.alt = "Latest award";
+            award.loading = "lazy";
+            badge.appendChild(award);
+        }
+        return badge;
+    }
+
+    static presenceDot(user) {
+        const dot = document.createElement("span");
+        dot.className = "presence-dot";
+        dot.setAttribute("data-presence-uid", user.uid || "");
+        dot.setAttribute("data-presence-last-seen", user.last_seen || "");
+        dot.setAttribute("aria-hidden", "true");
+        return dot;
+    }
 }
 
 window.Avatar = Avatar;

devplacepy/static/js/CLAUDE.md

@@ -20,7 +20,7 @@ Self-contained, presentational UI is built as custom elements with the `dp-` pre
 
 **Prompt seeding on the shared Devii opener (`data-devii-prompt`).** `DeviiTerminal.bindTriggers` binds every `[data-devii-open]` element; it now reads `trigger.dataset.deviiPrompt` and passes it to `DeviiTerminal.open(prompt)`, which calls `this.element.open()` and then `devii-terminal.prefill(text)` (sets `this.input.value`, moves the caret to the end, focuses). **It never auto-sends** - the member reads the request and presses Enter, keeping the assistant's first action explicitly user-initiated. This is a platform-wide capability available to any page, not a quiz-only shim: add `data-devii-prompt="..."` beside `data-devii-open` and the terminal opens pre-filled.
 
-`dp-chat` (`AppChat.js`) is the Slack-like DM chat widget that fully replaced the old page-controller trio `MessagesLayout.js`/`MessagesSocket.js`/`MessageSearch.js` (deleted) and `static/css/messages.css` (superseded by `static/css/chat.css`) on `/messages`. Unlike every other component here, it does NOT build its DOM from scratch on `connectedCallback` when server-rendered light-DOM children already exist (`mode="page"`) - it **adopts** the existing `.messages-list`/`.messages-main`/`.messages-thread`/`.message-bubble` markup `templates/messages.html` still renders (so no-JS and crawler fallback both keep working) and only builds a from-scratch skeleton when none is present (`mode="embed"`, a standalone `<dp-chat mode="embed" self-uid="..." with-uid="..." send-url="..." ws-url="...">` usable outside `/messages`). It owns its own `ChatSocket` instance, the conversation search dropdown (absorbed from the old `MessageSearch.js`, still built on the shared `ListNav` utility), consecutive-message grouping (`chat/MessageGrouping.js`), a working optimistic send with `.pending`/`.failed` (tap-to-retry) bubble states, and an opt-in (`ai-indicator` attribute) "Adjusted by AI" caption shown when a reconciled echo's `ai_processed` frame flag is true and the content changed from what was locally typed - never a diff/revert UI, the backend keeps no pre-correction copy to diff against. Presence for `mode="page"` needs no component code at all: the adopted markup keeps its `data-presence-uid` attributes, so the page-global `PresenceManager` (below) already drives it; `dp-chat` opens its own scoped `PubSubClient` subscription only in `mode="embed"`, where no page-global instance exists. The message body itself keeps rendering through `<dp-content>` (see "CLIENT-only now" below) - the redesign only removed the historical `no-copy` attribute on message bubbles so `dp-content`'s own existing `.content-copy-btn` doubles as the hover/focus-reveal action toolbar's Copy button (§6.3 of the design doc it was built from), instead of a second copy mechanism being hand-rolled.
+`dp-chat` (`AppChat.js`) is the Slack-like DM chat widget that fully replaced the old page-controller trio `MessagesLayout.js`/`MessagesSocket.js`/`MessageSearch.js` (deleted) and `static/css/messages.css` (superseded by `static/css/chat.css`) on `/messages`. Unlike every other component here, it does NOT build its DOM from scratch on `connectedCallback` when server-rendered light-DOM children already exist (`mode="page"`) - it **adopts** the existing `.messages-list`/`.messages-main`/`.messages-thread`/`.message-bubble` markup `templates/messages.html` still renders (so no-JS and crawler fallback both keep working) and only builds a from-scratch skeleton when none is present (`mode="embed"`, a standalone `<dp-chat mode="embed" self-uid="..." with-uid="..." send-url="..." ws-url="...">` usable outside `/messages`). It owns its own `ChatSocket` instance, the conversation search dropdown (absorbed from the old `MessageSearch.js`, still built on the shared `ListNav` utility), consecutive-message grouping (`chat/MessageGrouping.js`), a working optimistic send with `.pending`/`.failed` (tap-to-retry) bubble states, and an opt-in (`ai-indicator` attribute) "Adjusted by AI" caption shown when a reconciled echo's `ai_processed` frame flag is true and the content changed from what was locally typed - never a diff/revert UI, the backend keeps no pre-correction copy to diff against. Presence needs no component code at all: the adopted markup keeps its `data-presence-uid` attributes, so the page-global `PresenceManager` (below) already drives it, and the conversation list it builds live uses the shared `Avatar.badgeElement(user)` rather than hand-rolled dot markup. In `mode="embed"`, where no page-global instance may exist, it constructs a root-scoped `PresenceManager` instead of re-implementing presence - `dp-chat` owns no presence logic of its own. The message body itself keeps rendering through `<dp-content>` (see "CLIENT-only now" below) - the redesign only removed the historical `no-copy` attribute on message bubbles so `dp-content`'s own existing `.content-copy-btn` doubles as the hover/focus-reveal action toolbar's Copy button (§6.3 of the design doc it was built from), instead of a second copy mechanism being hand-rolled.
 
 `static/js/chat/ChatSocket.js` and `static/js/chat/MessageGrouping.js` are chat-scoped helpers used only by `AppChat.js` - `ChatSocket` is modeled on `PubSubClient.js`'s real exponential backoff (200ms doubling to a 5000ms cap, reset on a successful connect) rather than the flat-delay retry the deleted `MessagesSocket.js` used, and preserves the same `4013` "wrong worker" fast-retry special case. `MessageGrouping.shouldGroup(previous, current)` is the one pure predicate (<=300s gap, same sender) shared by both the initial-history grouping pass and the live-append path, so the two can never disagree. Neither file is a general-purpose "do not reimplement" utility for the rest of the app (see the out-of-scope note in the design document this shipped from) - they are not added to the do-not-reimplement list below; a future non-chat WebSocket feature should keep hand-rolling its own client (as `DeviiSocket.js`/`DeepsearchProgressSocket.js`/`SeoProgressSocket.js`/`AppDeepsearchChat.js`'s inline socket already do) rather than reaching into `chat/`.
 
@@ -48,6 +48,7 @@ Detail on each utility:
 - **`OptimisticAction` (`static/js/OptimisticAction.js`).** Base with one method, `submit(url, params, errorTarget, render)`: `Http.sendForm` -> `render(result)` on success -> `console.error` + (when `errorTarget` is given) `Toast.flash(errorTarget, "Error", 1500)` on failure. `VoteManager`/`ReactionBar`/`BookmarkManager`/`PollManager` `extend` it and call `this.submit(...)` for their POST, **keeping their own event wiring** (so `ReactionBar`'s palette toggle and `PollManager`'s multi-action handlers and `VoteManager`'s per-button `stopPropagation` are untouched). Pass `errorTarget` only where the old code toasted (`VoteManager`); the others pass `null` to keep their console-only behaviour.
 - **`EmojiPickerElement` (`static/js/EmojiPickerElement.js`).** The single wrapper around the vendored `emoji-picker-element`: `EmojiPickerElement.load()` lazily imports the vendor module once (shared promise, failures swallowed) and `EmojiPickerElement.create(onSelect)` returns a configured `<emoji-picker>` (data source `static/vendor/emoji-picker-element/data.json`) that calls `onSelect(unicode)` on `emoji-click`. Both consumers use it: `EmojiPicker` (insert at cursor in a textarea) and `ReactionBar` (react with any emoji). Never build an `<emoji-picker>`, re-import the vendor module, or repeat the data-source path elsewhere.
 - **`FloatingWindow` / `WindowManager` (`static/js/components/`).** The draggable window base and shared z-order manager - documented in the Container manager section; both the container terminals and the Devii terminal extend `FloatingWindow`.
+- **`PresenceManager` (`static/js/PresenceManager.js`, `app.presence`) and `Avatar` (`static/js/Avatar.js`).** The single online-status renderer and the single avatar-markup builder. `PresenceManager` makes ONE subscription to `public.presence.roster` and drives EVERY `[data-presence-uid]` element from the pushed online uid set - there is no client-side clock and no expiry timer, so nothing can disagree with the feed's Online now panel. `Avatar.badgeElement(user)` builds the `.avatar-badge` + `.presence-dot` + award-badge trio (the JS twin of `templates/_presence_dot.html`) and is used by `OnlineUsers` and `dp-chat`. Never re-derive online status from a timestamp in feature code, and never hand-build a presence dot.
 - **`ScrollMemory` (`static/js/ScrollMemory.js`, `app.scrollMemory`).** Site-wide, per-tab scroll restoration - the fix for the "back to feed jumps to top" class of bugs. It sets `history.scrollRestoration = "manual"` once (never rely on the browser's auto-restore, which fires before late-loading content settles and never applies to normal link navigations), so ALL scroll restoration flows through this one module. State lives in `sessionStorage` (per-tab by definition, exactly the required scope): a position map keyed by exact `pathname + search` (hash ignored; saved by a throttled passive scroll listener plus a final write on `pagehide`/hidden `visibilitychange`, pruned to 50 entries / 60 min), a visited-URL trail (capped at 20), and a one-shot click-intent flag (30s validity, consumed on every load).
   **When it restores** - only when the situation is genuinely "going back": (1) navigation type `back_forward` (browser back without bfcache; with bfcache - `pageshow` `persisted` - the frozen page already has its scroll, so it only re-syncs the trail and clears the intent flag), (2) `reload`, (3) a same-origin click on `a.back-link` / `a[data-scroll-back]` / a breadcrumb link / any link whose target equals the trail's previous URL, which stamps the intent flag the next load matches. A fresh visit (topnav, address bar, redirect after POST) never restores, and a URL with a `#fragment` always wins over restoration.
   **How it restores reliably**: a `requestAnimationFrame` loop re-applies the target position (clamped to the current `scrollHeight`, `behavior: "instant"` so the global `html { scroll-behavior: smooth }` never animates it) until the document height has been stable at the target for 10 frames or a 4s deadline passes, and aborts instantly on the first `wheel`/`touchstart`/`keydown`/`pointerdown` so it never fights the user. It also **upgrades bare back-links on load**: when the previous trail URL has the same pathname as a query-less `a.back-link`/`[data-scroll-back]` href (post page's `/feed` vs the `/feed?tab=recent&before=...` the user actually came from), the href is rewritten to the exact previous URL so tab/topic/cursor AND scroll survive the round trip. Give any new "back to X" anchor the `back-link` class (or `data-scroll-back`) and it participates automatically - never hand-roll `scrollTo` persistence per page. Guarded by `tests/e2e/feed.py::test_feed_scroll_restored_via_back_link` / `_via_browser_back` / `_not_restored_on_fresh_visit`.

devplacepy/static/js/OnlineUsers.js

@@ -1,5 +1,7 @@
 // retoor <retoor@molodetz.nl>
 
+import { Avatar } from "./Avatar.js";
+
 const ROSTER_TOPIC = "public.presence.roster";
 
 export class OnlineUsers {
@@ -13,51 +15,30 @@ export class OnlineUsers {
         }
     }
 
-    render(data) {
+    render(roster) {
-        if (!data || !Array.isArray(data.users)) return;
+        if (!roster || !Array.isArray(roster.users)) return;
         if (this.countEl) {
-            this.countEl.textContent = data.count != null ? data.count : data.users.length;
+            this.countEl.textContent = roster.count != null ? roster.count : roster.users.length;
         }
         this.list.textContent = "";
-        if (!data.users.length) {
+        if (!roster.users.length) {
             const empty = document.createElement("span");
             empty.className = "online-empty";
             empty.textContent = this.emptyText;
             this.list.appendChild(empty);
             return;
         }
-        for (const user of data.users) {
+        for (const user of roster.users) {
             this.list.appendChild(this.buildItem(user));
         }
     }
 
     buildItem(user) {
-        const seed = user.avatar_seed || user.username;
         const link = document.createElement("a");
         link.href = "/profile/" + user.username;
         link.className = "online-user";
         link.title = user.username;
-        const badge = document.createElement("span");
+        link.appendChild(Avatar.badgeElement(user));
-        badge.className = "avatar-badge";
-        const img = document.createElement("img");
-        img.className = "avatar-img avatar-sm";
-        img.src = "/avatar/multiavatar/" + encodeURIComponent(seed) + "?size=32";
-        img.alt = user.username;
-        img.loading = "lazy";
-        const dot = document.createElement("span");
-        dot.className = "presence-dot online";
-        dot.setAttribute("aria-hidden", "true");
-        badge.appendChild(img);
-        badge.appendChild(dot);
-        if (user.award_prominent && user.last_award_slug) {
-            const award = document.createElement("img");
-            award.className = "award-badge";
-            award.src = `/awards/${encodeURIComponent(user.last_award_slug)}/64`;
-            award.alt = "Latest award";
-            award.loading = "lazy";
-            badge.appendChild(award);
-        }
-        link.appendChild(badge);
         return link;
     }
 }

devplacepy/static/js/PresenceManager.js

@@ -1,15 +1,12 @@
 // retoor <retoor@molodetz.nl>
 
-const TOPIC_PREFIX = "public.presence.";
+const ROSTER_TOPIC = "public.presence.roster";
-const REFRESH_MS = 20000;
 
 export class PresenceManager {
-    constructor(pubsub) {
+    constructor(pubsub, root = document) {
-        this.pubsub = pubsub;
+        this.online = null;
-        const seconds = parseInt(document.body.dataset.presenceTimeout, 10);
-        this.timeoutMs = (Number.isFinite(seconds) && seconds > 0 ? seconds : 60) * 1000;
         this.tracked = new Map();
-        this.scan(document);
+        this.scan(root);
         this.observer = new MutationObserver((mutations) => {
             for (const mutation of mutations) {
                 for (const node of mutation.addedNodes) {
@@ -17,10 +14,13 @@ export class PresenceManager {
                 }
             }
         });
-        if (document.body) {
+        const host = root === document ? document.body : root;
-            this.observer.observe(document.body, { childList: true, subtree: true });
+        if (host) this.observer.observe(host, { childList: true, subtree: true });
-        }
+        if (pubsub) pubsub.subscribe(ROSTER_TOPIC, (data) => this.apply(data));
-        window.setInterval(() => this.refresh(), REFRESH_MS);
+    }
+
+    stop() {
+        this.observer.disconnect();
     }
 
     scan(root) {
@@ -37,62 +37,67 @@ export class PresenceManager {
         if (!uid) return;
         let entry = this.tracked.get(uid);
         if (!entry) {
-            entry = { els: new Set(), lastSeen: null, online: null };
+            entry = { els: new Set(), lastSeen: null };
             this.tracked.set(uid, entry);
-            if (this.pubsub) {
-                this.pubsub.subscribe(TOPIC_PREFIX + uid, (data) => {
-                    this.update(uid, data);
-                });
-            }
         }
         if (entry.els.has(el)) return;
         entry.els.add(el);
         const seed = el.dataset.presenceLastSeen || null;
         if (seed) entry.lastSeen = seed;
-        this.render(el, entry);
+        this.render(uid, entry);
     }
 
-    update(uid, data) {
+    apply(roster) {
-        const entry = this.tracked.get(uid);
+        if (!roster || !Array.isArray(roster.online)) return;
-        if (!entry || !data) return;
+        const previous = this.online;
-        if (data.online != null) entry.online = !!data.online;
+        this.online = new Set(roster.online);
-        if (data.last_seen) entry.lastSeen = data.last_seen;
+        const wentOffline = new Date().toISOString();
-        for (const el of entry.els) this.render(el, entry);
+        for (const [uid, entry] of this.tracked.entries()) {
+            if (previous && previous.has(uid) && !this.online.has(uid)) {
+                entry.lastSeen = wentOffline;
+            }
+            this.render(uid, entry);
+        }
     }
 
-    isOnline(lastSeen) {
+    isOnline(uid) {
-        if (!lastSeen) return false;
+        return !!this.online && this.online.has(uid);
-        const then = new Date(lastSeen).getTime();
-        if (Number.isNaN(then)) return false;
-        return Date.now() - then < this.timeoutMs;
     }
 
-    render(el, entry) {
+    relative(iso) {
-        const online = entry.online != null ? entry.online : this.isOnline(entry.lastSeen);
+        const localTime = window.app && window.app.localTime;
-        el.classList.toggle("online", online);
+        if (localTime) return localTime.format(iso, "ago");
-        const label = online
+        const date = new Date(iso);
-            ? "online"
+        return Number.isNaN(date.getTime()) ? null : date.toLocaleDateString("en-GB");
-            : entry.lastSeen
-                ? "last seen " + this.formatLastSeen(entry.lastSeen)
-                : "offline";
-        el.title = label;
-        if (el.hasAttribute("data-presence-label")) el.textContent = label;
     }
 
-    formatLastSeen(iso) {
+    label(uid, entry) {
-        const then = new Date(iso);
+        if (this.isOnline(uid)) return { text: "online" };
-        if (Number.isNaN(then.getTime())) return "recently";
+        const relative = entry.lastSeen ? this.relative(entry.lastSeen) : null;
-        const seconds = Math.max(0, Math.floor((Date.now() - then.getTime()) / 1000));
+        if (!relative) return { text: "offline" };
-        if (seconds < 60) return "just now";
+        return { text: "last seen ", iso: entry.lastSeen, relative };
-        if (seconds < 3600) return Math.floor(seconds / 60) + "m ago";
-        if (seconds < 86400) return Math.floor(seconds / 3600) + "h ago";
-        return then.toLocaleDateString("en-GB");
     }
 
-    refresh() {
+    timeElement(iso, relative) {
-        if (document.hidden) return;
+        const time = document.createElement("time");
-        for (const entry of this.tracked.values()) {
+        time.setAttribute("datetime", iso);
-            for (const el of entry.els) this.render(el, entry);
+        time.dataset.dt = "";
+        time.dataset.dtMode = "ago";
+        time.textContent = relative;
+        return time;
+    }
+
+    render(uid, entry) {
+        if (!this.online) return;
+        const online = this.isOnline(uid);
+        const label = this.label(uid, entry);
+        const title = label.iso ? label.text + label.relative : label.text;
+        for (const el of entry.els) {
+            el.classList.toggle("online", online);
+            el.title = title;
+            if (!el.hasAttribute("data-presence-label")) continue;
+            el.textContent = label.text;
+            if (label.iso) el.appendChild(this.timeElement(label.iso, label.relative));
         }
     }
 }

devplacepy/static/js/components/AppChat.js

@@ -10,6 +10,7 @@ import { DomUtils } from "../DomUtils.js";
 import { ListNav } from "../ListNav.js";
 import { contentRenderer } from "../ContentRenderer.js";
 import { PubSubClient } from "../PubSubClient.js";
+import { PresenceManager } from "../PresenceManager.js";
 
 const CHAT_CSS_ID = "chat-css";
 const TYPING_THROTTLE_MS = 1500;
@@ -39,8 +40,7 @@ export class AppChat extends Component {
         super();
         this._built = false;
         this.socket = null;
-        this._pubsub = null;
+        this._presence = null;
-        this._presenceUnsubs = null;
         this._pendingSends = new Map();
         this._failedSends = new Map();
         this._uploading = false;
@@ -101,15 +101,7 @@ export class AppChat extends Component {
         for (const entry of this._pendingSends.values()) clearTimeout(entry.timeoutId);
         this._pendingSends.clear();
         this._failedSends.clear();
-        if (this._presenceUnsubs) {
+        if (this._presence) this._presence.stop();
-            this._presenceUnsubs.forEach((unsub) => {
-                try {
-                    unsub();
-                } catch {
-                    /* ignore */
-                }
-            });
-        }
     }
 
     _readConfig() {
@@ -148,7 +140,6 @@ export class AppChat extends Component {
         this.typingEl = this.querySelector("#typing-indicator");
         this.backBtn = this.querySelector("#messages-back-btn");
         this.searchInput = this.querySelector("#message-search");
-        this.presenceEl = this.querySelector("#messages-presence");
     }
 
     _ensureSkeleton() {
@@ -167,10 +158,10 @@ export class AppChat extends Component {
         presence.setAttribute("role", "status");
         presence.setAttribute("aria-live", "polite");
         presence.setAttribute("data-presence-label", "");
+        presence.setAttribute("data-presence-uid", this.withUid || "");
         presence.textContent = "offline";
         headerInfo.appendChild(presence);
         header.appendChild(headerInfo);
-        this.presenceEl = presence;
 
         this.thread = document.createElement("div");
         this.thread.className = "messages-thread";
@@ -783,18 +774,7 @@ export class AppChat extends Component {
         item.setAttribute("role", "listitem");
         if (user.uid && user.uid === this.withUid) item.setAttribute("aria-current", "true");
 
-        const badge = document.createElement("span");
+        item.appendChild(Avatar.badgeElement(user));
-        badge.className = "avatar-badge";
-        const img = Avatar.imgElement(user.avatar_seed || user.username || "", 32);
-        img.alt = user.username || "";
-        badge.appendChild(img);
-        const dot = document.createElement("span");
-        dot.className = "presence-dot";
-        dot.setAttribute("data-presence-uid", user.uid || "");
-        dot.setAttribute("data-presence-last-seen", user.last_seen || "");
-        dot.setAttribute("aria-hidden", "true");
-        badge.appendChild(dot);
-        item.appendChild(badge);
 
         const info = document.createElement("div");
         info.className = "conversation-info";
@@ -1066,24 +1046,9 @@ export class AppChat extends Component {
     }
 
     _initPresence() {
-        if (this.mode !== "embed" || !this.withUid) return;
+        if (this.mode !== "embed") return;
-        this._presenceUnsubs = [];
+        if (window.app && window.app.presence) return;
-        try {
+        this._presence = new PresenceManager(new PubSubClient(), this);
-            this._pubsub = new PubSubClient();
-            const unsub = this._pubsub.subscribe(`public.presence.${this.withUid}`, (data) => this._onPresence(data));
-            this._presenceUnsubs.push(unsub);
-        } catch {
-            this._pubsub = null;
-        }
-    }
-
-    _onPresence(data) {
-        if (!data || !this.presenceEl) return;
-        const online = !!data.online;
-        this.presenceEl.classList.toggle("online", online);
-        if (this.presenceEl.hasAttribute("data-presence-label")) {
-            this.presenceEl.textContent = online ? "online" : "offline";
-        }
     }
 }
 

devplacepy/templates/base.html

@@ -62,7 +62,7 @@
     </script>
     {% endif %}
 </head>
-<body data-user-uid="{{ user['uid'] if user else '' }}" data-presence-timeout="{{ presence_timeout }}">
+<body data-user-uid="{{ user['uid'] if user else '' }}">
     <a class="skip-link" href="#main-content">Skip to main content</a>
     <nav class="topnav" aria-label="Primary">
         <div class="topnav-inner">

devplacepy/templates/docs/presence.html

@@ -8,7 +8,7 @@ DevPlace shows whether a user is **online** right now. You see it on every profi
 
 - Every user **avatar** carries a small dot in its bottom-right corner: **green** when the person is online, **muted grey** when they are not, so presence is visible everywhere an avatar appears (the feed, comments, the navbar, message lists, and more). Hover the dot for the exact status.
 - On a **profile** and at the top of a **conversation** you also get the word `online`, or **`last seen ...`** with a relative time, or `offline` if the person has not been seen since the feature started tracking them.
-- The indicator **updates on its own** while you have the page open: if someone comes online while you are looking at their profile, the dot turns green within a few seconds, and it fades back to grey shortly after they go idle. You never need to refresh.
+- The indicator **updates on its own** while you have the page open: if someone comes online while you are looking at their profile, the dot turns green within a few seconds, and it fades back to grey shortly after they go idle. You never need to refresh. Live updating needs you to be signed in; a signed-out visitor sees the status as it was when the page loaded.
 - The **feed** shows an **Online now** panel at the bottom of the left sidebar: the avatars of everyone currently online, ordered alphabetically so they keep a stable spot, with a live count. People appear and disappear from it in real time as they come and go, again with no refresh.
 
 "Active" simply means loading any page on the site. There is nothing to switch on and no busy or away status to set; presence is automatic.
@@ -33,10 +33,10 @@ Presence is one timestamp plus a lightweight push, built to touch the database a
 
 **Reading it.** `presence.is_online(user_row)` is exposed as the Jinja global `is_online(user)` and rendered from the user row a page already loaded, so no extra query runs. The value is also on `UserOut.last_seen` and `ProfileOut.profile_online` for JSON clients.
 
-**Live updates (change-only, hysteresis).** `PresenceRelayService` (`services/presence_relay.py`) runs on the service-lock owner and recomputes ONE global online set each tick that drives BOTH the avatar dots and the feed roster, so they can never disagree. Membership uses hysteresis (`presence.stays_online`): a user goes online at `PRESENCE_TIMEOUT_SECONDS` but only drops after an extra `PRESENCE_ONLINE_MARGIN_SECONDS` grace (`DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS`, default 20) - quick on, slow off - which is what prevents boundary flicker. It pushes to `public.presence.{uid}` **only when a watched user's online bool actually changes** (or a new subscriber first appears) - never on a fixed interval - reading watched users in **one batched query per tick**, so an idle page costs zero messages. The frontend `PresenceManager` (`static/js/PresenceManager.js`, `app.presence`) subscribes any element carrying `data-presence-uid` (deduped per uid) and treats each frame's `online` flag as **authoritative** - it does not expire a live dot from its own clock, so an active user never flickers to grey. The staleness timer remains only as a fallback for elements with no live subscription (e.g. guests). Reuse `is_online`, `presence.stays_online`, the `public.presence.{uid}` topic, and `PresenceManager` for any new online indicator rather than re-implementing presence.
+**Live updates (one set, one topic, change-only, hysteresis).** `PresenceRelayService` (`services/presence_relay.py`) runs on the service-lock owner and recomputes ONE online set each tick from ONE indexed query over the online population (`presence.online_candidates()`, capped at `config.PRESENCE_TRACK_LIMIT`, env `DEVPLACE_PRESENCE_TRACK_LIMIT`, default 500). Membership uses hysteresis (`presence.stays_online`): a user goes online at `PRESENCE_TIMEOUT_SECONDS` but only drops after an extra `PRESENCE_ONLINE_MARGIN_SECONDS` grace (`DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS`, default 20) - quick on, slow off - which is what prevents boundary flicker. That single set is published on the single shared topic `public.presence.roster` as `{count, online: [uid, ...], users: [...]}`, **only when the set of online users changes** (a `frozenset` compare, so reordering never republishes) and never on a fixed interval, so an idle site costs zero messages. `online` is the authority for every avatar dot anywhere on the page; `users` is the same set trimmed to `PRESENCE_ONLINE_LIMIT` for the feed's avatar panel. There are no per-user presence topics: one set, one topic, one frame, so no two indicators can drift apart. `services/presence.py` `is_online(user)` is literally `stays_online(seconds_since(last_seen), was_online=False)`, so the server-rendered initial state and the live set apply the same rule.
 
-**Online-now roster.** The same relay maintains one shared topic, `public.presence.roster`, republished **only when the set of online users changes** (a `frozenset` compare, so reordering never republishes). `services/presence.py` `online_users()`/`online_candidates()` read it with an indexed `last_seen` query (`database.get_online_users`; `idx_users_last_seen`), ordered **alphabetically by username** so an avatar keeps a stable position instead of jumping around as activity ticks. The feed renders the initial list server-side and `static/js/OnlineUsers.js` (`app.onlineUsers`) subscribes to the topic and re-renders the avatar list live. Roster avatars are online by definition, so their dots are a plain green `.presence-dot` with no per-user subscription - list membership *is* the presence.
+**The frontend.** `static/js/PresenceManager.js` (`app.presence`) makes ONE subscription to `public.presence.roster` and keeps the pushed `online` uid set. Every element carrying `data-presence-uid` - anywhere on any page, discovered on load and through a `MutationObserver` for markup inserted later - resolves its state as membership of that one set, toggling the `online` class and, for `data-presence-label` elements, the "online / last seen X / offline" text (the relative time is a `<time data-dt data-dt-mode="ago">` formatted by the shared `LocalTime`, so presence never formats a date itself). There is **no client-side clock and no expiry timer**: the relay is the only thing that decides who is online, and before the first frame arrives the server-rendered state simply stands. `static/js/OnlineUsers.js` (`app.onlineUsers`) renders the feed's avatar panel from `users` in that same frame, so the panel and the dots are two views of one payload. `services/presence.py` `online_users()` renders the initial panel server-side with the same indexed `last_seen` query (`database.get_online_users`; `idx_users_last_seen`), ordered **alphabetically by username** so an avatar keeps a stable position instead of jumping around as activity ticks.
 
-**The avatar dot.** The corner dot is one reusable partial, `templates/_presence_dot.html`, which emits a `<span class="presence-dot" data-presence-uid=... data-presence-last-seen=...>` (guarded on `uid`, so an author with no resolvable user renders no dot). It is included by the shared avatar partial `templates/_avatar_link.html` (which covers most avatars) and by a handful of raw-avatar sites wrapped in a positioned `.avatar-badge` span. Because the dot carries `data-presence-uid` but no `data-presence-label`, `PresenceManager` drives its colour with **no extra JavaScript**. The dot sizes itself as a percentage of the avatar (clamped 8-14px), so it stays proportional at every avatar size. Reuse `is_online`, `_presence_dot.html`, the `public.presence.{uid}` topic, and `PresenceManager` for any new online indicator rather than re-implementing presence.
+**The avatar dot.** The corner dot is one reusable partial, `templates/_presence_dot.html`, which emits a `<span class="presence-dot" data-presence-uid=... data-presence-last-seen=...>` (guarded on `uid`, so an author with no resolvable user renders no dot). It is included by the shared avatar partial `templates/_avatar_link.html` (which covers most avatars) and by the raw-avatar sites wrapped in a positioned `.avatar-badge` span, including the feed's Online now panel - a roster avatar is not a special case, it is the same subscribed dot as everywhere else. In JavaScript the matching builder is `Avatar.badgeElement(user)` (`static/js/Avatar.js`), used by `OnlineUsers` and by the chat's conversation list, so dot markup exists in exactly two places: the partial and that helper. Because the dot carries `data-presence-uid` but no `data-presence-label`, `PresenceManager` drives its colour with **no extra JavaScript**. The dot sizes itself as a percentage of the avatar (clamped 8-14px), so it stays proportional at every avatar size. Reuse `is_online`, `_presence_dot.html`/`Avatar.badgeElement`, the `public.presence.roster` topic, and `PresenceManager` for any new online indicator rather than re-implementing presence.
 {% endraw %}
 </div>

devplacepy/templates/feed.html

@@ -66,7 +66,7 @@
             <div class="online-users-list" data-online-users-list data-empty-text="No one online right now">
                 {% for ou in online_users %}
                 <a href="/profile/{{ ou['username'] }}" class="online-user" title="{{ ou['username'] }}">
-                    <span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(ou), 32) }}" class="avatar-img avatar-sm" alt="{{ ou['username'] }}" loading="lazy"><span class="presence-dot online" aria-hidden="true"></span>{% set _user = ou %}{% include "_award_badge.html" %}</span>
+                    <span class="avatar-badge"><img src="{{ avatar_url('multiavatar', avatar_seed(ou), 32) }}" class="avatar-img avatar-sm" alt="{{ ou['username'] }}" loading="lazy">{% set _user = ou %}{% include "_presence_dot.html" %}{% include "_award_badge.html" %}</span>
                 </a>
                 {% else %}
                 <span class="online-empty">No one online right now</span>

devplacepy/templates/messages.html

@@ -46,7 +46,7 @@
             {% set _user = other_user %}{% set _size = 32 %}{% set _size_class = "sm" %}{% include "_avatar_link.html" %}
             <div class="messages-header-info">
                 <h3>{% set _user = other_user %}{% set _class = none %}{% include "_user_link.html" %}</h3>
-                <span class="messages-presence{% if is_online(other_user) %} online{% endif %}" id="messages-presence" role="status" aria-live="polite" data-presence-uid="{{ other_user['uid'] }}" data-presence-last-seen="{{ other_user.get('last_seen') or '' }}" data-presence-label>{% if is_online(other_user) %}online{% elif other_user.get('last_seen') %}last seen {{ format_date(other_user['last_seen']) }}{% else %}offline{% endif %}</span>
+                <span class="messages-presence{% if is_online(other_user) %} online{% endif %}" id="messages-presence" role="status" aria-live="polite" data-presence-uid="{{ other_user['uid'] }}" data-presence-last-seen="{{ other_user.get('last_seen') or '' }}" data-presence-label>{% if is_online(other_user) %}online{% elif other_user.get('last_seen') %}last seen {{ dt_ago(other_user['last_seen']) }}{% else %}offline{% endif %}</span>
             </div>
         </div>
 

devplacepy/templates/profile.html

@@ -18,7 +18,7 @@
             </div>
             <h1 class="profile-name">{{ profile_user['username'] }}</h1>
 
-            <div class="profile-presence{% if profile_online %} online{% endif %}" role="status" data-presence-uid="{{ profile_user['uid'] }}" data-presence-last-seen="{{ profile_user.get('last_seen') or '' }}" data-presence-label>{% if profile_online %}online{% elif profile_user.get('last_seen') %}last seen {{ format_date(profile_user['last_seen']) }}{% else %}offline{% endif %}</div>
+            <div class="profile-presence{% if profile_online %} online{% endif %}" role="status" data-presence-uid="{{ profile_user['uid'] }}" data-presence-last-seen="{{ profile_user.get('last_seen') or '' }}" data-presence-label>{% if profile_online %}online{% elif profile_user.get('last_seen') %}last seen {{ dt_ago(profile_user['last_seen']) }}{% else %}offline{% endif %}</div>
 
             <div class="profile-stats">
                 <div class="profile-stat">

devplacepy/templating.py

@@ -6,7 +6,7 @@ import jinja2
 from fastapi.templating import Jinja2Templates
 from markupsafe import Markup, escape
 from devplacepy.cache import TTLCache
-from devplacepy.config import STATIC_VERSION, TEMPLATES_DIR, TEMPLATE_AUTO_RELOAD, PRESENCE_TIMEOUT_SECONDS
+from devplacepy.config import STATIC_VERSION, TEMPLATES_DIR, TEMPLATE_AUTO_RELOAD
 from devplacepy.constants import TOPICS, REACTION_EMOJI
 from devplacepy.database import get_int_setting, get_setting, get_table
 from devplacepy.avatar import avatar_url, avatar_seed
@@ -55,7 +55,6 @@ templates.env.globals["owns"] = _owns
 templates.env.globals["is_self"] = is_self
 templates.env.globals["guest_disabled"] = guest_disabled
 templates.env.globals["is_online"] = presence.is_online
-templates.env.globals["presence_timeout"] = PRESENCE_TIMEOUT_SECONDS
 
 from devplacepy.docs_devrant import devrant_endpoints
 

tests/api/feed.py

@@ -381,6 +381,29 @@ def test_feed_shows_online_now_section(app_server):
     assert f'class="online-user" title="{name}"' in html
 
 
+def test_online_now_avatar_uses_the_shared_presence_dot(app_server):
+    name = _unique("0ros")
+    s = requests.Session()
+    s.post(
+        f"{BASE_URL}/auth/signup",
+        data={
+            "username": name,
+            "email": f"{name}@t.dev",
+            "password": "secret123",
+            "confirm_password": "secret123",
+        },
+        allow_redirects=True,
+    )
+    uid = _db_user(name)["uid"]
+    html = s.get(f"{BASE_URL}/feed").text
+    panel = html.split('data-online-users-list', 1)[1].split("</div>", 2)[0]
+    # the roster avatar is driven by the same subscribed dot as every other avatar,
+    # never a hardcoded green marker
+    assert f'data-presence-uid="{uid}"' in panel
+    assert "data-presence-last-seen" in panel
+    assert '<span class="presence-dot online" aria-hidden="true"></span>' not in panel
+
+
 def test_feed_shows_poll_results_without_voting(app_server):
     s, _ = _session_polls()
     title = f"feedpoll-{int(time.time() * 1000)}"

tests/api/messages/index.py

@@ -180,3 +180,31 @@ def test_messages_conversation_list_avatar_has_presence_dot(app_server):
     assert "conversation-item" in html
     assert "presence-dot" in html
     assert f'data-presence-uid="{other_uid}"' in html
+
+
+def test_chat_presence_agrees_with_the_feed_online_roster(app_server):
+    sender, _ = _member()
+    # the roster is alphabetical and capped, so name the partner to sort first
+    other_name = _unique("0chat")
+    other = requests.Session()
+    other.post(
+        f"{BASE_URL}/auth/signup",
+        data={
+            "username": other_name,
+            "email": f"{other_name}@t.dev",
+            "password": "secret123",
+            "confirm_password": "secret123",
+        },
+        allow_redirects=True,
+    )
+    other_uid = _db_user(other_name)["uid"]
+
+    # the partner is active, so the feed roster - the single source of truth - lists them
+    roster = sender.get(f"{BASE_URL}/feed").text
+    assert f'class="online-user" title="{other_name}"' in roster
+
+    # ...and the chat header must say exactly the same thing
+    chat = sender.get(f"{BASE_URL}/messages?with_uid={other_uid}").text
+    header = chat.split('id="messages-presence"', 1)[1].split("</span>", 1)[0]
+    assert header.endswith("online"), header
+    assert f'data-presence-uid="{other_uid}"' in header

tests/e2e/messages.py

@@ -188,6 +188,25 @@ def test_messages_page_loads(alice):
     assert page.is_visible(".messages-layout")
 
 
+def test_chat_presence_is_driven_by_the_shared_roster(alice, bob):
+    bob_page, _ = bob
+    bob_page.goto(f"{BASE_URL}/feed", wait_until="domcontentloaded")
+    bob_uid = get_table("users").find_one(username="bob_test")["uid"]
+    page, _ = alice
+    page.goto(
+        f"{BASE_URL}/messages?with_uid={bob_uid}", wait_until="domcontentloaded"
+    )
+    presence = page.locator("#messages-presence")
+    expect(presence).to_be_visible()
+    expect(presence).to_have_class(re.compile(r"\bonline\b"))
+    # the chat header and the feed roster read the same subscribed uid, never a
+    # second client-side clock over a frozen last_seen
+    assert presence.get_attribute("data-presence-uid") == bob_uid
+    dots = page.locator(f".presence-dot[data-presence-uid='{bob_uid}']")
+    assert dots.count() >= 1
+    expect(dots.first).to_have_class(re.compile(r"\bonline\b"))
+
+
 def test_messages_search_input(alice):
     page, _ = alice
     page.goto(f"{BASE_URL}/messages", wait_until="domcontentloaded")

tests/e2e/profile/awards.py

@@ -104,6 +104,15 @@ def test_guest_can_read_awards_tab(browser):
 def test_avatar_badge_on_feed_when_prominent(alice):
     page, _ = alice
     _seed_published("bob_test", "alice_test", "Badge feed")
+    # the Online now roster only lists active users, so make the awarded user active
+    get_table("users").update(
+        {
+            "uid": get_table("users").find_one(username="bob_test")["uid"],
+            "last_seen": datetime.now(timezone.utc).isoformat(),
+        },
+        ["uid"],
+    )
     page.goto(f"{BASE_URL}/feed", wait_until="domcontentloaded")
-    if page.locator(".online-user").count() > 0:
+    entry = page.locator(".online-user[title='bob_test']")
-        expect(page.locator(".online-user .award-badge").first).to_be_visible()
+    expect(entry).to_be_visible()
+    expect(entry.locator(AWARD_BADGE)).to_be_visible()

tests/unit/services/presence.py

@@ -31,6 +31,18 @@ def test_is_online_unparseable_is_offline():
     assert presence.is_online({"last_seen": "not-a-timestamp"}) is False
 
 
+def test_is_online_is_the_first_observation_of_stays_online():
+    from devplacepy.config import PRESENCE_ONLINE_MARGIN_SECONDS
+
+    for seconds in (0, 5, PRESENCE_TIMEOUT_SECONDS - 1, PRESENCE_TIMEOUT_SECONDS + 1,
+                    PRESENCE_TIMEOUT_SECONDS + PRESENCE_ONLINE_MARGIN_SECONDS + 1):
+        row = {"last_seen": _iso(seconds)}
+        expected = presence.stays_online(
+            presence.seconds_since(row["last_seen"]), was_online=False
+        )
+        assert presence.is_online(row) is expected
+
+
 def test_seconds_since_none_on_missing_or_bad():
     assert presence.seconds_since(None) is None
     assert presence.seconds_since("") is None
@@ -151,3 +163,22 @@ def test_online_candidates_uses_grace_window(local_db):
     assert band not in strict
     # a user past the grace window is in neither
     assert beyond not in candidates
+
+
+def test_online_candidates_tracks_beyond_the_display_limit(local_db):
+    from devplacepy.config import PRESENCE_ONLINE_LIMIT, PRESENCE_TRACK_LIMIT
+
+    # the display roster is capped for the feed panel, the tracked authority set is not
+    assert PRESENCE_TRACK_LIMIT > PRESENCE_ONLINE_LIMIT
+    now = datetime.now(timezone.utc).isoformat()
+    users = get_table("users")
+    made = []
+    for index in range(PRESENCE_ONLINE_LIMIT + 5):
+        uid = str(uuid_utils.uuid7())
+        users.insert(
+            {"uid": uid, "username": f"trk{index:03d}{uid[:6]}",
+             "email": f"{uid[:8]}@o.test", "last_seen": now}
+        )
+        made.append(uid)
+    tracked = {u["uid"] for u in presence.online_candidates()}
+    assert set(made) <= tracked

tests/unit/services/presence_relay.py

@@ -0,0 +1,45 @@
+# retoor <retoor@molodetz.nl>
+
+from devplacepy.config import PRESENCE_ONLINE_LIMIT
+from devplacepy.services.presence_relay import ROSTER_TOPIC, roster_payload
+
+
+def _rows(count):
+    return [
+        {"uid": f"u{index:03d}", "username": f"user{index:03d}", "avatar_seed": None}
+        for index in range(count)
+    ]
+
+
+def test_roster_topic_is_the_single_presence_channel():
+    assert ROSTER_TOPIC == "public.presence.roster"
+
+
+def test_payload_only_carries_the_online_set(local_db):
+    rows = _rows(4)
+    payload = roster_payload(rows, {"u000", "u002"})
+    assert payload["online"] == ["u000", "u002"]
+    assert [user["uid"] for user in payload["users"]] == ["u000", "u002"]
+    assert payload["count"] == 2
+
+
+def test_dots_and_display_roster_come_from_one_set(local_db):
+    rows = _rows(PRESENCE_ONLINE_LIMIT + 7)
+    online = {row["uid"] for row in rows}
+    payload = roster_payload(rows, online)
+    # every online user is authoritative for their avatar dot...
+    assert set(payload["online"]) == online
+    # ...while the feed's avatar panel stays capped for display
+    assert len(payload["users"]) == PRESENCE_ONLINE_LIMIT
+    assert payload["count"] == PRESENCE_ONLINE_LIMIT
+    assert set(payload["online"]) >= {user["uid"] for user in payload["users"]}
+
+
+def test_avatar_seed_falls_back_to_username(local_db):
+    payload = roster_payload([{"uid": "u1", "username": "ada", "avatar_seed": None}], {"u1"})
+    assert payload["users"][0]["avatar_seed"] == "ada"
+
+
+def test_offline_users_are_absent_everywhere(local_db):
+    payload = roster_payload(_rows(3), set())
+    assert payload == {"count": 0, "online": [], "users": []}