From 585fab74a6ee7c220ff15ee4d7317e25b451ab16 Mon Sep 17 00:00:00 2001 From: retoor Date: Sun, 15 Nov 2026 04:05:28 +0100 Subject: [PATCH] Document the unread count in the push payload Specifies unread and aps.badge in the push and utils notes, records the notifications ensure-block and the index that serves the unread read in the database note, and updates the README and the push API docs. Co-Authored-By: Claude Sonnet 5.5 --- README.md | 4 +++- devplacepy/database/CLAUDE.md | 2 +- devplacepy/docs_api/groups/push.py | 5 +++++ devplacepy/push/CLAUDE.md | 3 ++- devplacepy/utils/CLAUDE.md | 2 +- pyproject.toml | 2 +- 6 files changed, 13 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 26f8e430..d54a6822 100644 --- a/README.md +++ b/README.md @@ -1075,7 +1075,9 @@ in real time, bridged onto the in-process pub/sub bus by a lock-owner relay: | Issue-tracker update | reporter / admins | `create_notification` schedules delivery as a fire-and-forget async task, so a dead -subscription or push-service error never blocks the triggering request. Delivery +subscription or push-service error never blocks the triggering request. The payload carries the recipient's unread count (`unread`, the same number as +`/notifications/counts`), which APNs presents as `aps.badge` so the app icon updates even +while the app is closed. Delivery (`push.notify_user`) reads a user's subscriptions once, groups them by provider, builds each provider's payload once, and sends over that provider's own HTTP client (stealth for Web Push, a dedicated HTTP/2 client for APNs). A subscription the push service reports as diff --git a/devplacepy/database/CLAUDE.md b/devplacepy/database/CLAUDE.md index a6cd2727..f51181d2 100644 --- a/devplacepy/database/CLAUDE.md +++ b/devplacepy/database/CLAUDE.md @@ -272,4 +272,4 @@ Posts, gists, projects and news carry one status from `VISIBILITY_VALUES`: `publ ## Data CLI -`devplace apikey backfill` assigns keys to users that lack one. `devplace attachments prune` removes orphan attachment records/files. `devplace system prune` (with `--dry-run`) purges soft-deleted attachment/project-file blobs, sweeps blob files with zero DB reference, and GCs orphaned container workspace dirs. `database/notifications.py` owns the single grouping specification: `NOTIFICATION_GROUP_KEYS` (actor / target / single per type, unlisted types are single), `notification_group_key(row)`, and `count_unread_groups(user_uid)` (groups when enabled, rows when off). Every badge producer calls `count_unread_groups`; none counts rows itself. Delivery throttling (item 5) reuses the same key. +`devplace apikey backfill` assigns keys to users that lack one. `devplace attachments prune` removes orphan attachment records/files. `devplace system prune` (with `--dry-run`) purges soft-deleted attachment/project-file blobs, sweeps blob files with zero DB reference, and GCs orphaned container workspace dirs. `database/notifications.py` owns the single grouping specification: `NOTIFICATION_GROUP_KEYS` (actor / target / single per type, unlisted types are single), `notification_group_key(row)`, and `count_unread_groups(user_uid)` (groups when enabled, rows when off). Every badge producer calls `count_unread_groups`; none counts rows itself. The push payload (`unread`, presented as `aps.badge`) is one of them. The `notifications` column set is ensured in `init_db` like `messages` (listed in `tests/unit/database/schema.py` `FILTERED_COLUMNS`), and the unread read `find(user_uid=..., read=False)` is served by `idx_notifications_user_read (user_uid, read)`. Delivery throttling (item 5) reuses the same key. diff --git a/devplacepy/docs_api/groups/push.py b/devplacepy/docs_api/groups/push.py index c75c91af..53b4270b 100644 --- a/devplacepy/docs_api/groups/push.py +++ b/devplacepy/docs_api/groups/push.py @@ -20,6 +20,11 @@ its build. A registration body without a `provider` field is a `webpush` registr existing clients need no change. An APNs body may include a stable `client_id` so a later token rotation updates the same device instead of inserting another row. +Every delivered notification carries the recipient's unread count, the same number as +`notifications` in `GET /notifications/counts` ([Notifications](/docs/notifications.html)). `apns` presents it as `aps.badge`, +so a native app icon updates while the app is in the background; `webpush` forwards it as the +`unread` field of the payload. + `DELETE /push.json` removes a single registration by the same identity used to create it (`endpoint` for webpush, `token` or `client_id` for apns). The server also stops delivering to a subscription once its push endpoint reports it as gone. These mirror the in-app diff --git a/devplacepy/push/CLAUDE.md b/devplacepy/push/CLAUDE.md index 4d776fc8..1eaee79a 100644 --- a/devplacepy/push/CLAUDE.md +++ b/devplacepy/push/CLAUDE.md @@ -76,5 +76,6 @@ A match **updates** the row (token, client_id, environment) and clears `deleted_ - A `.p8` that does not parse is cached as a failure for the same window, so a misconfiguration costs one error log per window rather than one parse per notification. - **`DEAD_REASONS` is deliberately narrow: only `BadDeviceToken` (400), `Unregistered` (410) and `ExpiredToken` (410).** These are the only reasons in Apple's documented table that describe the TOKEN itself as permanently invalid. `DeviceTokenNotForTopic` and `TopicDisallowed` are 400 errors about the **topic/provisioning matching the connection**, not the token - they fire identically for every token when `push_apns_topic` is misconfigured or the certificate/entitlements don't match, so treating them as dead would soft-delete the entire APNs subscriber base (a table deliberately kept OUT of `SOFT_DELETE_TABLES`, hence unrestorable) on the first delivery after a one-field admin typo. Never add a topic/provisioning-shaped reason to `DEAD_REASONS`. - **A 410's `timestamp` is honored before deleting.** Apple's 410 body carries `{"reason": "Unregistered", "timestamp": }` - the instant Apple last confirmed the token dead, which can trail real device state by "several days" per Apple engineering guidance (410 delivery is intentionally non-deterministic; do not use it to infer app-uninstall timing). `apns._dead_before` converts it to ISO and it rides on `Delivery.dead_before`; `store.mark_dead(id, dead_before)` compares it against the row's `registered_at` and **skips the delete** (logs at INFO instead) if the registration was re-registered/revived after that instant - closing the race where an in-flight delivery against a stale row state would otherwise undo a concurrent revival. `dead_before` is only ever set for a 410; the `BadDeviceToken`/`ExpiredToken` paths pass `None` (unconditional delete, as before), since those reasons carry no `timestamp`. -- The shared payload dict (`title`, `message`, `icon`, `url`) is translated once per batch into `aps.alert` plus the custom `url`/`icon` keys, mirroring what `service-worker.js` does for Web Push. `thread-id` mirrors the service worker's notification `tag`. +- The shared payload dict (`title`, `message`, `icon`, `url`, `unread`) is translated once per batch into `aps.alert` plus the custom `url`/`icon` keys, mirroring what `service-worker.js` does for Web Push. `thread-id` mirrors the service worker's notification `tag`. +- **`unread` becomes `aps.badge`.** `unread` is the recipient's unread notification count, `count_unread_groups(user_uid)` (groups when grouping is on, rows when off: the same number as `/notifications/counts`), computed by `utils/notifications.py` `_deliver_notification` right after the in-app insert and only when a push is actually sent. `apns._badge` accepts a non-negative integer (a `bool` is refused) and otherwise omits `aps.badge`, which leaves the app icon untouched; the welcome probe carries no `unread`. iOS applies `aps.badge` itself, so the icon updates while the app is backgrounded or terminated. The key is named `unread`, never `badge`, because `badge` already means the monochrome icon URL in the Notifications API. Web Push forwards the payload unchanged and the service worker ignores the extra key. A throttled push is not sent, so the badge shows the count at the last delivered push until the app is opened. - `POST /push.json` probes a created or revived row immediately via `notify_registration`. The JSON is `{registered: true, delivered: bool, error?: string}`. `registered` stays true so existing clients keep working; `delivered`/`error` surface Apple's reason instead of killing the row silently from the client's point of view. The row is still marked dead on a `DEAD` outcome (subject to the `dead_before` guard above) so later `notify_user` calls do not keep hitting a known-bad token. diff --git a/devplacepy/utils/CLAUDE.md b/devplacepy/utils/CLAUDE.md index 55684c48..5d19f526 100644 --- a/devplacepy/utils/CLAUDE.md +++ b/devplacepy/utils/CLAUDE.md @@ -76,7 +76,7 @@ Every notification type is independently toggleable on three channels - **in-app - **Channels are data-driven.** Adding a channel is a constant change, never scattered edits: `NOTIFICATION_CHANNELS` (the tuple), `_NOTIFICATION_CHANNEL_COLUMNS` (channel -> table column), and `_NOTIFICATION_CHANNEL_DEFAULTS` (channel -> per-type fallback when no admin default row exists). `in_app`/`push` default `1` (on); **`telegram` defaults `0` (off for every type)**. The four helpers (`_notification_overrides`, `get_notification_prefs`, `set_notification_pref`, `_notification_default`) iterate these maps generically, so a new channel needs only the constant entry + the `notification_preferences` column in `init_db`. - **Resolution** (`database.notification_enabled(user_uid, type, channel)`): a live `notification_preferences` row for `(user_uid, type)` wins; else the global default `get_int_setting("notif_default_{type}_{channel}", _NOTIFICATION_CHANNEL_DEFAULTS[channel])`; else the channel fallback. So a user who never touched a setting inherits the admin default (or the channel fallback when no default row exists); a user who customized keeps their choice. - **Storage**: table `notification_preferences` (`uid`, `user_uid`, `notification_type`, `in_app_enabled`, `push_enabled`, `telegram_enabled`, timestamps, soft-delete columns), in `SOFT_DELETE_TABLES`, columns+indexes ensured in `init_db`. Cached under the `"notif_prefs"` cache-version name (`sync_local_cache`/`bump_cache_version`), keyed by `user_uid`. Helpers: `get_notification_prefs(user_uid)` (full resolved list for the UI/JSON, one boolean per channel keyed by channel name), `set_notification_pref(user_uid, type, channel, enabled)` (upsert one channel, leaving the others at their resolved value; revives a soft-deleted row), `reset_notification_prefs(user_uid)` (soft-delete the user's overrides -> fall back to defaults), `get_notification_default`/`set_notification_default` (the `notif_default_*` `site_settings` keys; defaults are NOT seeded so an untouched type reads its channel fallback: `1` for in_app/push, `0` for telegram). -- **Enforcement (`create_notification` single funnel) is in `utils.create_notification`**: the in-app `notifications.insert` + `clear_unread_cache` run only when `notification_enabled(..., "in_app")`, `_schedule_push` runs only when `notification_enabled(..., "push")`, and `_schedule_telegram` runs only when `notification_enabled(..., "telegram")`. The `notification.create` audit row is unconditional and records which channels fired (`metadata.in_app`/`push`/`telegram`). **Because the DM path in `messages.py` now routes through `create_notification` (type `message`)** instead of inserting directly, direct messages are gated like everything else and gained a (default-on) push channel. **`create_notification` is a thin wrapper that defers the whole body** (preference reads + insert + push + telegram + audit) to the **background task queue** via `background.submit(_deliver_notification, ...)`, so it never blocks the request (inline under tests); call it directly, never wrap it in `background.submit`. See `devplacepy/services/CLAUDE.md` -> Background task queue. +- **Enforcement (`create_notification` single funnel) is in `utils.create_notification`**: the in-app `notifications.insert` + `clear_unread_cache` run only when `notification_enabled(..., "in_app")`, `_schedule_push` runs only when `notification_enabled(..., "push")`, and `_schedule_telegram` runs only when `notification_enabled(..., "telegram")`. The `notification.create` audit row is unconditional and records which channels fired (`metadata.in_app`/`push`/`telegram`). **Because the DM path in `messages.py` now routes through `create_notification` (type `message`)** instead of inserting directly, direct messages are gated like everything else and gained a (default-on) push channel. **`create_notification` is a thin wrapper that defers the whole body** (preference reads + insert + push + telegram + audit) to the **background task queue** via `background.submit(_deliver_notification, ...)`, so it never blocks the request (inline under tests); call it directly, never wrap it in `background.submit`. When the push channel fires, `_schedule_push` also receives `count_unread_groups(user_uid)` (read after the in-app insert) and puts it in the shared push payload as `unread`; the APNs provider presents it as `aps.badge` (`devplacepy/push/CLAUDE.md`). See `devplacepy/services/CLAUDE.md` -> Background task queue. - **Telegram channel (cross-worker delivery).** `_schedule_telegram(user_uid, message)` calls `services/telegram/store.enqueue_outbox`, which no-ops when the user is not paired (`link_for_user` is None) and otherwise inserts a row into the `telegram_outbox` queue (`uid`, `user_uid`, `chat_id`, `text`, `status` pending|sent|failed, `attempts`, timestamps; transient, NOT in `SOFT_DELETE_TABLES`, columns+index ensured in `init_db`). This is needed because `_deliver_notification` runs on the background queue of **any** worker, but the Telegram bot worker subprocess (and `TelegramService.send_markdown`) lives **only on the service-lock owner**. `services/telegram/outbox_service.py` `TelegramOutboxService` (lock-owner `BaseService`, default-enabled, ~2s interval, registered in `main.py` after `TelegramService`) drains pending rows on the lock owner: if the Telegram service is missing/disabled/`not worker_alive()` it returns (rows wait, nothing is lost), else it sends each via `send_markdown`, `mark_outbox_sent` on success or `mark_outbox_failed` (retries up to `MAX_OUTBOX_ATTEMPTS=3`, then `failed`), and `prune_outbox` clears old sent/failed rows. The DB is the cross-worker bridge here exactly as in `NotificationRelayService`/`MessageRelay`; a dedicated service (not folded into the 1s toast relay) keeps the blocking 30s-per-send `send_markdown` off the toast path. Telegram defaults off, so absent any pairing the platform sends nothing until a paired user opts a type in. The profile Telegram column is disabled until the user pairs Telegram. - **Live pairing update (pub/sub).** When a user completes pairing from their phone, the bridge (`services/telegram/bridge.py` `_handle_unpaired`) runs on the **service-lock owner** - exactly where every `/pubsub/ws` subscriber converges - so right after the `telegram.pair.success` audit it calls `_publish_pairing(user_uid, True)` -> `services.pubsub.publish(f"user.{uid}.telegram", {"paired": True})` (best-effort, lazy import). No new table, no cross-worker hop. The topic is already authorized for that user by `services/pubsub/policy.py` `_own_namespace` (`user.{uid}.*`). Frontend: `TelegramPairing` and `NotificationPrefs` (both built with `this.pubsub` in `Application.js`, after `PubSubClient`) read `document.body.dataset.userUid` and subscribe to `user.{uid}.telegram` (the `LiveNotifications`/`CounterManager` pattern); on `{paired:true}` the pairing panel flips to Connected (`setPaired`/`hideCode`/`setMessage`) and the notifications Telegram column enables live (drop `disabled`+`notif-switch-disabled`, hide the hint). Cross-device **unpair** live update is intentionally not wired (HTTP unpair runs on any worker; same-page unpair updates locally). - **UI:** profile **Notifications** tab (`/profile/{username}?tab=notifications`, owner-or-admin, `POST /profile/{username}/notifications` + `/reset`) and admin global defaults at `/admin/notifications`. Devii tools `notification_list`/`notification_set`/`notification_reset` (`handler="notification"`, owner-scoped, `set`/`reset` in `CONFIRM_REQUIRED`). Reuse this canonical-list + resolver + override-table + single-funnel pattern for future per-user per-channel preferences. diff --git a/pyproject.toml b/pyproject.toml index f5986b1d..d92f837b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "devplacepy" -version = "1.1.31" +version = "1.1.32" description = "DevPlace - The Developer Social Network" requires-python = ">=3.12" dependencies = [