Document the unread count in the push payload

Specifies unread and aps.badge in the push and utils notes, records
the notifications ensure-block and the index that serves the unread
read in the database note, and updates the README and the push API
docs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-15 04:05:28 +01:00
co-authored by Claude Sonnet 5.5
parent 2452f9e85b
commit 585fab74a6
6 changed files with 13 additions and 5 deletions
+3 -1
View File
@@ -1075,7 +1075,9 @@ in real time, bridged onto the in-process pub/sub bus by a lock-owner relay:
| Issue-tracker update | reporter / admins | | Issue-tracker update | reporter / admins |
`create_notification` schedules delivery as a fire-and-forget async task, so a dead `create_notification` schedules delivery as a fire-and-forget async task, so a dead
subscription or push-service error never blocks the triggering request. Delivery subscription or push-service error never blocks the triggering request. The payload carries the recipient's unread count (`unread`, the same number as
`/notifications/counts`), which APNs presents as `aps.badge` so the app icon updates even
while the app is closed. Delivery
(`push.notify_user`) reads a user's subscriptions once, groups them by provider, builds (`push.notify_user`) reads a user's subscriptions once, groups them by provider, builds
each provider's payload once, and sends over that provider's own HTTP client (stealth for each provider's payload once, and sends over that provider's own HTTP client (stealth for
Web Push, a dedicated HTTP/2 client for APNs). A subscription the push service reports as Web Push, a dedicated HTTP/2 client for APNs). A subscription the push service reports as
+1 -1
View File
@@ -272,4 +272,4 @@ Posts, gists, projects and news carry one status from `VISIBILITY_VALUES`: `publ
## Data CLI ## Data CLI
`devplace apikey backfill` assigns keys to users that lack one. `devplace attachments prune` removes orphan attachment records/files. `devplace system prune` (with `--dry-run`) purges soft-deleted attachment/project-file blobs, sweeps blob files with zero DB reference, and GCs orphaned container workspace dirs. `database/notifications.py` owns the single grouping specification: `NOTIFICATION_GROUP_KEYS` (actor / target / single per type, unlisted types are single), `notification_group_key(row)`, and `count_unread_groups(user_uid)` (groups when enabled, rows when off). Every badge producer calls `count_unread_groups`; none counts rows itself. Delivery throttling (item 5) reuses the same key. `devplace apikey backfill` assigns keys to users that lack one. `devplace attachments prune` removes orphan attachment records/files. `devplace system prune` (with `--dry-run`) purges soft-deleted attachment/project-file blobs, sweeps blob files with zero DB reference, and GCs orphaned container workspace dirs. `database/notifications.py` owns the single grouping specification: `NOTIFICATION_GROUP_KEYS` (actor / target / single per type, unlisted types are single), `notification_group_key(row)`, and `count_unread_groups(user_uid)` (groups when enabled, rows when off). Every badge producer calls `count_unread_groups`; none counts rows itself. The push payload (`unread`, presented as `aps.badge`) is one of them. The `notifications` column set is ensured in `init_db` like `messages` (listed in `tests/unit/database/schema.py` `FILTERED_COLUMNS`), and the unread read `find(user_uid=..., read=False)` is served by `idx_notifications_user_read (user_uid, read)`. Delivery throttling (item 5) reuses the same key.
+5
View File
@@ -20,6 +20,11 @@ its build. A registration body without a `provider` field is a `webpush` registr
existing clients need no change. An APNs body may include a stable `client_id` so a later existing clients need no change. An APNs body may include a stable `client_id` so a later
token rotation updates the same device instead of inserting another row. token rotation updates the same device instead of inserting another row.
Every delivered notification carries the recipient's unread count, the same number as
`notifications` in `GET /notifications/counts` ([Notifications](/docs/notifications.html)). `apns` presents it as `aps.badge`,
so a native app icon updates while the app is in the background; `webpush` forwards it as the
`unread` field of the payload.
`DELETE /push.json` removes a single registration by the same identity used to create it `DELETE /push.json` removes a single registration by the same identity used to create it
(`endpoint` for webpush, `token` or `client_id` for apns). The server also stops delivering to a (`endpoint` for webpush, `token` or `client_id` for apns). The server also stops delivering to a
subscription once its push endpoint reports it as gone. These mirror the in-app subscription once its push endpoint reports it as gone. These mirror the in-app
+2 -1
View File
@@ -76,5 +76,6 @@ A match **updates** the row (token, client_id, environment) and clears `deleted_
- A `.p8` that does not parse is cached as a failure for the same window, so a misconfiguration costs one error log per window rather than one parse per notification. - A `.p8` that does not parse is cached as a failure for the same window, so a misconfiguration costs one error log per window rather than one parse per notification.
- **`DEAD_REASONS` is deliberately narrow: only `BadDeviceToken` (400), `Unregistered` (410) and `ExpiredToken` (410).** These are the only reasons in Apple's documented table that describe the TOKEN itself as permanently invalid. `DeviceTokenNotForTopic` and `TopicDisallowed` are 400 errors about the **topic/provisioning matching the connection**, not the token - they fire identically for every token when `push_apns_topic` is misconfigured or the certificate/entitlements don't match, so treating them as dead would soft-delete the entire APNs subscriber base (a table deliberately kept OUT of `SOFT_DELETE_TABLES`, hence unrestorable) on the first delivery after a one-field admin typo. Never add a topic/provisioning-shaped reason to `DEAD_REASONS`. - **`DEAD_REASONS` is deliberately narrow: only `BadDeviceToken` (400), `Unregistered` (410) and `ExpiredToken` (410).** These are the only reasons in Apple's documented table that describe the TOKEN itself as permanently invalid. `DeviceTokenNotForTopic` and `TopicDisallowed` are 400 errors about the **topic/provisioning matching the connection**, not the token - they fire identically for every token when `push_apns_topic` is misconfigured or the certificate/entitlements don't match, so treating them as dead would soft-delete the entire APNs subscriber base (a table deliberately kept OUT of `SOFT_DELETE_TABLES`, hence unrestorable) on the first delivery after a one-field admin typo. Never add a topic/provisioning-shaped reason to `DEAD_REASONS`.
- **A 410's `timestamp` is honored before deleting.** Apple's 410 body carries `{"reason": "Unregistered", "timestamp": <ms epoch>}` - the instant Apple last confirmed the token dead, which can trail real device state by "several days" per Apple engineering guidance (410 delivery is intentionally non-deterministic; do not use it to infer app-uninstall timing). `apns._dead_before` converts it to ISO and it rides on `Delivery.dead_before`; `store.mark_dead(id, dead_before)` compares it against the row's `registered_at` and **skips the delete** (logs at INFO instead) if the registration was re-registered/revived after that instant - closing the race where an in-flight delivery against a stale row state would otherwise undo a concurrent revival. `dead_before` is only ever set for a 410; the `BadDeviceToken`/`ExpiredToken` paths pass `None` (unconditional delete, as before), since those reasons carry no `timestamp`. - **A 410's `timestamp` is honored before deleting.** Apple's 410 body carries `{"reason": "Unregistered", "timestamp": <ms epoch>}` - the instant Apple last confirmed the token dead, which can trail real device state by "several days" per Apple engineering guidance (410 delivery is intentionally non-deterministic; do not use it to infer app-uninstall timing). `apns._dead_before` converts it to ISO and it rides on `Delivery.dead_before`; `store.mark_dead(id, dead_before)` compares it against the row's `registered_at` and **skips the delete** (logs at INFO instead) if the registration was re-registered/revived after that instant - closing the race where an in-flight delivery against a stale row state would otherwise undo a concurrent revival. `dead_before` is only ever set for a 410; the `BadDeviceToken`/`ExpiredToken` paths pass `None` (unconditional delete, as before), since those reasons carry no `timestamp`.
- The shared payload dict (`title`, `message`, `icon`, `url`) is translated once per batch into `aps.alert` plus the custom `url`/`icon` keys, mirroring what `service-worker.js` does for Web Push. `thread-id` mirrors the service worker's notification `tag`. - The shared payload dict (`title`, `message`, `icon`, `url`, `unread`) is translated once per batch into `aps.alert` plus the custom `url`/`icon` keys, mirroring what `service-worker.js` does for Web Push. `thread-id` mirrors the service worker's notification `tag`.
- **`unread` becomes `aps.badge`.** `unread` is the recipient's unread notification count, `count_unread_groups(user_uid)` (groups when grouping is on, rows when off: the same number as `/notifications/counts`), computed by `utils/notifications.py` `_deliver_notification` right after the in-app insert and only when a push is actually sent. `apns._badge` accepts a non-negative integer (a `bool` is refused) and otherwise omits `aps.badge`, which leaves the app icon untouched; the welcome probe carries no `unread`. iOS applies `aps.badge` itself, so the icon updates while the app is backgrounded or terminated. The key is named `unread`, never `badge`, because `badge` already means the monochrome icon URL in the Notifications API. Web Push forwards the payload unchanged and the service worker ignores the extra key. A throttled push is not sent, so the badge shows the count at the last delivered push until the app is opened.
- `POST /push.json` probes a created or revived row immediately via `notify_registration`. The JSON is `{registered: true, delivered: bool, error?: string}`. `registered` stays true so existing clients keep working; `delivered`/`error` surface Apple's reason instead of killing the row silently from the client's point of view. The row is still marked dead on a `DEAD` outcome (subject to the `dead_before` guard above) so later `notify_user` calls do not keep hitting a known-bad token. - `POST /push.json` probes a created or revived row immediately via `notify_registration`. The JSON is `{registered: true, delivered: bool, error?: string}`. `registered` stays true so existing clients keep working; `delivered`/`error` surface Apple's reason instead of killing the row silently from the client's point of view. The row is still marked dead on a `DEAD` outcome (subject to the `dead_before` guard above) so later `notify_user` calls do not keep hitting a known-bad token.
+1 -1
View File
@@ -76,7 +76,7 @@ Every notification type is independently toggleable on three channels - **in-app
- **Channels are data-driven.** Adding a channel is a constant change, never scattered edits: `NOTIFICATION_CHANNELS` (the tuple), `_NOTIFICATION_CHANNEL_COLUMNS` (channel -> table column), and `_NOTIFICATION_CHANNEL_DEFAULTS` (channel -> per-type fallback when no admin default row exists). `in_app`/`push` default `1` (on); **`telegram` defaults `0` (off for every type)**. The four helpers (`_notification_overrides`, `get_notification_prefs`, `set_notification_pref`, `_notification_default`) iterate these maps generically, so a new channel needs only the constant entry + the `notification_preferences` column in `init_db`. - **Channels are data-driven.** Adding a channel is a constant change, never scattered edits: `NOTIFICATION_CHANNELS` (the tuple), `_NOTIFICATION_CHANNEL_COLUMNS` (channel -> table column), and `_NOTIFICATION_CHANNEL_DEFAULTS` (channel -> per-type fallback when no admin default row exists). `in_app`/`push` default `1` (on); **`telegram` defaults `0` (off for every type)**. The four helpers (`_notification_overrides`, `get_notification_prefs`, `set_notification_pref`, `_notification_default`) iterate these maps generically, so a new channel needs only the constant entry + the `notification_preferences` column in `init_db`.
- **Resolution** (`database.notification_enabled(user_uid, type, channel)`): a live `notification_preferences` row for `(user_uid, type)` wins; else the global default `get_int_setting("notif_default_{type}_{channel}", _NOTIFICATION_CHANNEL_DEFAULTS[channel])`; else the channel fallback. So a user who never touched a setting inherits the admin default (or the channel fallback when no default row exists); a user who customized keeps their choice. - **Resolution** (`database.notification_enabled(user_uid, type, channel)`): a live `notification_preferences` row for `(user_uid, type)` wins; else the global default `get_int_setting("notif_default_{type}_{channel}", _NOTIFICATION_CHANNEL_DEFAULTS[channel])`; else the channel fallback. So a user who never touched a setting inherits the admin default (or the channel fallback when no default row exists); a user who customized keeps their choice.
- **Storage**: table `notification_preferences` (`uid`, `user_uid`, `notification_type`, `in_app_enabled`, `push_enabled`, `telegram_enabled`, timestamps, soft-delete columns), in `SOFT_DELETE_TABLES`, columns+indexes ensured in `init_db`. Cached under the `"notif_prefs"` cache-version name (`sync_local_cache`/`bump_cache_version`), keyed by `user_uid`. Helpers: `get_notification_prefs(user_uid)` (full resolved list for the UI/JSON, one boolean per channel keyed by channel name), `set_notification_pref(user_uid, type, channel, enabled)` (upsert one channel, leaving the others at their resolved value; revives a soft-deleted row), `reset_notification_prefs(user_uid)` (soft-delete the user's overrides -> fall back to defaults), `get_notification_default`/`set_notification_default` (the `notif_default_*` `site_settings` keys; defaults are NOT seeded so an untouched type reads its channel fallback: `1` for in_app/push, `0` for telegram). - **Storage**: table `notification_preferences` (`uid`, `user_uid`, `notification_type`, `in_app_enabled`, `push_enabled`, `telegram_enabled`, timestamps, soft-delete columns), in `SOFT_DELETE_TABLES`, columns+indexes ensured in `init_db`. Cached under the `"notif_prefs"` cache-version name (`sync_local_cache`/`bump_cache_version`), keyed by `user_uid`. Helpers: `get_notification_prefs(user_uid)` (full resolved list for the UI/JSON, one boolean per channel keyed by channel name), `set_notification_pref(user_uid, type, channel, enabled)` (upsert one channel, leaving the others at their resolved value; revives a soft-deleted row), `reset_notification_prefs(user_uid)` (soft-delete the user's overrides -> fall back to defaults), `get_notification_default`/`set_notification_default` (the `notif_default_*` `site_settings` keys; defaults are NOT seeded so an untouched type reads its channel fallback: `1` for in_app/push, `0` for telegram).
- **Enforcement (`create_notification` single funnel) is in `utils.create_notification`**: the in-app `notifications.insert` + `clear_unread_cache` run only when `notification_enabled(..., "in_app")`, `_schedule_push` runs only when `notification_enabled(..., "push")`, and `_schedule_telegram` runs only when `notification_enabled(..., "telegram")`. The `notification.create` audit row is unconditional and records which channels fired (`metadata.in_app`/`push`/`telegram`). **Because the DM path in `messages.py` now routes through `create_notification` (type `message`)** instead of inserting directly, direct messages are gated like everything else and gained a (default-on) push channel. **`create_notification` is a thin wrapper that defers the whole body** (preference reads + insert + push + telegram + audit) to the **background task queue** via `background.submit(_deliver_notification, ...)`, so it never blocks the request (inline under tests); call it directly, never wrap it in `background.submit`. See `devplacepy/services/CLAUDE.md` -> Background task queue. - **Enforcement (`create_notification` single funnel) is in `utils.create_notification`**: the in-app `notifications.insert` + `clear_unread_cache` run only when `notification_enabled(..., "in_app")`, `_schedule_push` runs only when `notification_enabled(..., "push")`, and `_schedule_telegram` runs only when `notification_enabled(..., "telegram")`. The `notification.create` audit row is unconditional and records which channels fired (`metadata.in_app`/`push`/`telegram`). **Because the DM path in `messages.py` now routes through `create_notification` (type `message`)** instead of inserting directly, direct messages are gated like everything else and gained a (default-on) push channel. **`create_notification` is a thin wrapper that defers the whole body** (preference reads + insert + push + telegram + audit) to the **background task queue** via `background.submit(_deliver_notification, ...)`, so it never blocks the request (inline under tests); call it directly, never wrap it in `background.submit`. When the push channel fires, `_schedule_push` also receives `count_unread_groups(user_uid)` (read after the in-app insert) and puts it in the shared push payload as `unread`; the APNs provider presents it as `aps.badge` (`devplacepy/push/CLAUDE.md`). See `devplacepy/services/CLAUDE.md` -> Background task queue.
- **Telegram channel (cross-worker delivery).** `_schedule_telegram(user_uid, message)` calls `services/telegram/store.enqueue_outbox`, which no-ops when the user is not paired (`link_for_user` is None) and otherwise inserts a row into the `telegram_outbox` queue (`uid`, `user_uid`, `chat_id`, `text`, `status` pending|sent|failed, `attempts`, timestamps; transient, NOT in `SOFT_DELETE_TABLES`, columns+index ensured in `init_db`). This is needed because `_deliver_notification` runs on the background queue of **any** worker, but the Telegram bot worker subprocess (and `TelegramService.send_markdown`) lives **only on the service-lock owner**. `services/telegram/outbox_service.py` `TelegramOutboxService` (lock-owner `BaseService`, default-enabled, ~2s interval, registered in `main.py` after `TelegramService`) drains pending rows on the lock owner: if the Telegram service is missing/disabled/`not worker_alive()` it returns (rows wait, nothing is lost), else it sends each via `send_markdown`, `mark_outbox_sent` on success or `mark_outbox_failed` (retries up to `MAX_OUTBOX_ATTEMPTS=3`, then `failed`), and `prune_outbox` clears old sent/failed rows. The DB is the cross-worker bridge here exactly as in `NotificationRelayService`/`MessageRelay`; a dedicated service (not folded into the 1s toast relay) keeps the blocking 30s-per-send `send_markdown` off the toast path. Telegram defaults off, so absent any pairing the platform sends nothing until a paired user opts a type in. The profile Telegram column is disabled until the user pairs Telegram. - **Telegram channel (cross-worker delivery).** `_schedule_telegram(user_uid, message)` calls `services/telegram/store.enqueue_outbox`, which no-ops when the user is not paired (`link_for_user` is None) and otherwise inserts a row into the `telegram_outbox` queue (`uid`, `user_uid`, `chat_id`, `text`, `status` pending|sent|failed, `attempts`, timestamps; transient, NOT in `SOFT_DELETE_TABLES`, columns+index ensured in `init_db`). This is needed because `_deliver_notification` runs on the background queue of **any** worker, but the Telegram bot worker subprocess (and `TelegramService.send_markdown`) lives **only on the service-lock owner**. `services/telegram/outbox_service.py` `TelegramOutboxService` (lock-owner `BaseService`, default-enabled, ~2s interval, registered in `main.py` after `TelegramService`) drains pending rows on the lock owner: if the Telegram service is missing/disabled/`not worker_alive()` it returns (rows wait, nothing is lost), else it sends each via `send_markdown`, `mark_outbox_sent` on success or `mark_outbox_failed` (retries up to `MAX_OUTBOX_ATTEMPTS=3`, then `failed`), and `prune_outbox` clears old sent/failed rows. The DB is the cross-worker bridge here exactly as in `NotificationRelayService`/`MessageRelay`; a dedicated service (not folded into the 1s toast relay) keeps the blocking 30s-per-send `send_markdown` off the toast path. Telegram defaults off, so absent any pairing the platform sends nothing until a paired user opts a type in. The profile Telegram column is disabled until the user pairs Telegram.
- **Live pairing update (pub/sub).** When a user completes pairing from their phone, the bridge (`services/telegram/bridge.py` `_handle_unpaired`) runs on the **service-lock owner** - exactly where every `/pubsub/ws` subscriber converges - so right after the `telegram.pair.success` audit it calls `_publish_pairing(user_uid, True)` -> `services.pubsub.publish(f"user.{uid}.telegram", {"paired": True})` (best-effort, lazy import). No new table, no cross-worker hop. The topic is already authorized for that user by `services/pubsub/policy.py` `_own_namespace` (`user.{uid}.*`). Frontend: `TelegramPairing` and `NotificationPrefs` (both built with `this.pubsub` in `Application.js`, after `PubSubClient`) read `document.body.dataset.userUid` and subscribe to `user.{uid}.telegram` (the `LiveNotifications`/`CounterManager` pattern); on `{paired:true}` the pairing panel flips to Connected (`setPaired`/`hideCode`/`setMessage`) and the notifications Telegram column enables live (drop `disabled`+`notif-switch-disabled`, hide the hint). Cross-device **unpair** live update is intentionally not wired (HTTP unpair runs on any worker; same-page unpair updates locally). - **Live pairing update (pub/sub).** When a user completes pairing from their phone, the bridge (`services/telegram/bridge.py` `_handle_unpaired`) runs on the **service-lock owner** - exactly where every `/pubsub/ws` subscriber converges - so right after the `telegram.pair.success` audit it calls `_publish_pairing(user_uid, True)` -> `services.pubsub.publish(f"user.{uid}.telegram", {"paired": True})` (best-effort, lazy import). No new table, no cross-worker hop. The topic is already authorized for that user by `services/pubsub/policy.py` `_own_namespace` (`user.{uid}.*`). Frontend: `TelegramPairing` and `NotificationPrefs` (both built with `this.pubsub` in `Application.js`, after `PubSubClient`) read `document.body.dataset.userUid` and subscribe to `user.{uid}.telegram` (the `LiveNotifications`/`CounterManager` pattern); on `{paired:true}` the pairing panel flips to Connected (`setPaired`/`hideCode`/`setMessage`) and the notifications Telegram column enables live (drop `disabled`+`notif-switch-disabled`, hide the hint). Cross-device **unpair** live update is intentionally not wired (HTTP unpair runs on any worker; same-page unpair updates locally).
- **UI:** profile **Notifications** tab (`/profile/{username}?tab=notifications`, owner-or-admin, `POST /profile/{username}/notifications` + `/reset`) and admin global defaults at `/admin/notifications`. Devii tools `notification_list`/`notification_set`/`notification_reset` (`handler="notification"`, owner-scoped, `set`/`reset` in `CONFIRM_REQUIRED`). Reuse this canonical-list + resolver + override-table + single-funnel pattern for future per-user per-channel preferences. - **UI:** profile **Notifications** tab (`/profile/{username}?tab=notifications`, owner-or-admin, `POST /profile/{username}/notifications` + `/reset`) and admin global defaults at `/admin/notifications`. Devii tools `notification_list`/`notification_set`/`notification_reset` (`handler="notification"`, owner-scoped, `set`/`reset` in `CONFIRM_REQUIRED`). Reuse this canonical-list + resolver + override-table + single-funnel pattern for future per-user per-channel preferences.
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "devplacepy" name = "devplacepy"
version = "1.1.31" version = "1.1.32"
description = "DevPlace - The Developer Social Network" description = "DevPlace - The Developer Social Network"
requires-python = ">=3.12" requires-python = ">=3.12"
dependencies = [ dependencies = [