Compare commits

..

5 Commits

Author SHA1 Message Date
0ec3e61118 Move image pixel reads to get_flattened_data and add the font libraries
Some checks are pending
DevPlace CI / test (push) Waiting to run
Pillow 12 renames Image.getdata to get_flattened_data; the award image
normaliser and the isslop hue histogram both read pixels that way. The image
stack also needs pango, harfbuzz, fontconfig and a base font in the container,
so text rendering has glyphs to work with.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 22:50:54 +02:00
a78b656ef9 Document the push providers in the README and the audit catalogue
README covers the provider model, the two providers and their transports, the
admin configuration surface at /admin/services/push, the delivery loop and the
updated file map. events.md records that push.subscribe and push.update now
carry the provider in their metadata, with endpoint_host set only for
endpoint-based providers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 22:50:54 +02:00
7674dac628 Cover the push providers with tests and document the subsystem
Unit tests for the provider registry, both providers' registration parsing, the
APNs payload translation, provider token signing and caching, header and status
mapping against a mock transport, provider grouping and the delivery timeout
clamp, plus service tests for the configuration surface, the retention sweep and
the per-provider metrics. Api tests cover the provider listing on GET
/push.json, registration with and without an explicit provider, idempotency and
the rejection of an unknown or unconfigured provider.

Provider settings in unit tests are supplied by monkeypatching the provider's
setting reader rather than writing site_settings, because the unit tier shares
its database with the running api-tier server.

devplacepy/push/CLAUDE.md documents the protocol, how to add a provider, the
invariants and the APNs specifics; the root, routers and services files point at
it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 22:47:20 +02:00
53ddf4f233 Add push provider architecture with Apple Push Notification support
Split the push delivery library into a provider architecture. devplacepy/push
becomes a package: a PushProvider protocol with a registry, the existing Web
Push implementation moved unchanged behind it, a new APNs provider, a store
owning every push_registration access, and a delivery loop that groups a user's
subscriptions by provider, prepares each provider's payload once and sends over
a single shared client.

APNs delivers over HTTP/2 with an ES256 provider token cached per credential
fingerprint, so a worker signs at most one token per 45 minutes. Registrations
carry a hexadecimal device token; 410 and the Unregistered class of reasons soft
delete the subscription exactly like a gone Web Push endpoint.

All provider configuration is edited at /admin/services/push through the same
ConfigField surface every other subsystem uses, assembled from the registry so a
future provider needs no edit to the service. A provider that is disabled,
unconfigured or holding an unusable credential accepts no registrations and is
skipped during delivery, never failing the other providers.

POST /push.json accepts a registration for any active provider; a body without a
provider field is a Web Push body, so existing clients are unchanged. GET
/push.json keeps publicKey at the top level and adds the active providers.
push_registration gains provider and token columns, ensured in init_db with a
converging backfill; existing rows are never rewritten.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 22:43:10 +02:00
5079f40f46 Merge pull request 'Fix #146: Add missing icon field to badges API response' (#147) from typosaurus/ticket-146 into master
Some checks failed
DevPlace CI / test (push) Failing after 1h25m1s
Reviewed-on: #147
2026-07-27 12:35:48 +02:00
26 changed files with 1411 additions and 110 deletions

View File

@ -148,6 +148,7 @@ Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file i
| `devplacepy/services/bot/CLAUDE.md` | `BotsService` fleet |
| `devplacepy/services/dbapi/CLAUDE.md` | `/dbapi` primary-admin-only read-only database API |
| `devplacepy/services/pubsub/CLAUDE.md` | Database-free pub/sub bus |
| `devplacepy/push/CLAUDE.md` | Push notification providers: the `PushProvider` protocol, the registry, Web Push and APNs, registration storage |
| `devplacepy/services/game/CLAUDE.md` | Code Farm idle game (economy invariants, raids, the one-pure-function rule) |
| `devplacepy/services/quiz/CLAUDE.md` | Quizzes (the terminal publish lock, attempt atomicity, answer-key withholding, AI free-text grading, the best-attempt scoreboard) |
| `devplacepy/services/CLAUDE.md` | Background task queue, AI correction/modifier, presence, live view relay, `BaseService`/`ServiceManager` |

View File

@ -4,6 +4,8 @@ WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
curl ca-certificates \
libglib2.0-0 libpango-1.0-0 libpangoft2-1.0-0 libharfbuzz0b libfontconfig1 \
fonts-dejavu-core \
&& rm -rf /var/lib/apt/lists/*
# Optional: the docker CLI so the (admin-only) container manager can drive the host

View File

@ -15,6 +15,12 @@ make test-headed # same tests in visible browser
Open `http://localhost:10500`.
PDF export (DeepSearch reports, via weasyprint) needs the Pango text stack installed at system level. The production image installs it; on a development host install it once:
```bash
sudo apt-get install -y libglib2.0-0 libpango-1.0-0 libpangoft2-1.0-0 libharfbuzz0b libfontconfig1 fonts-dejavu-core
```
## Stack
| Layer | Technology |
@ -40,7 +46,7 @@ devplacepy/
avatar.py # Multiavatar generation, URL builder
utils/ # Password hashing, session mgmt, time_ago, notification hook (package)
models.py # Pydantic schemas
push.py # Web push crypto, VAPID keys, encrypt/send/register
push/ # Push delivery: provider protocol, Web Push, APNs, registrations
routers/ # One file per domain (auth, feed, posts, push, ...)
templates/ # Jinja2 HTML templates
static/css/ # Page-specific CSS files
@ -811,9 +817,37 @@ calling itself.
## Push notifications & PWA
Authenticated users can receive native web push notifications, and the site is an
Authenticated users can receive native push notifications, and the site is an
installable Progressive Web App. Push uses only standard libraries (`cryptography`,
`PyJWT`, `httpx`) against the Web Push Protocol - no third-party push wrapper.
`PyJWT`, `httpx`) against the Web Push Protocol and the Apple Push Notification service -
no third-party push wrapper.
### Providers
Delivery is split into providers behind one protocol (`devplacepy/push/providers/`). A user
receives a notification through every provider they hold a live subscription for.
| Provider | Registration | Transport |
|----------|--------------|-----------|
| `webpush` | `PushSubscription` from the browser `PushManager` (endpoint + `p256dh`/`auth` keys) | Web Push Protocol, VAPID signed, `aesgcm` encrypted payload |
| `apns` | Hexadecimal device token | `POST https://api.push.apple.com/3/device/{token}` over HTTP/2, ES256 provider token |
`POST /push.json` accepts a registration for any active provider; a body without a
`provider` field is a `webpush` body, so browsers need no change. `GET /push.json` returns
the VAPID public key plus the providers currently accepting registrations. A provider that
is disabled or not fully configured accepts no registrations and is skipped during
delivery, so an unconfigured provider is inert rather than an error.
Every provider setting is edited at **`/admin/services/push`**: per provider an `Enabled`
toggle, the VAPID subject for `webpush`, and team id, key id, `.p8` auth key (stored as a
masked secret), topic and environment (production or sandbox) for `apns`. The same page
holds the shared delivery timeout and the retention window after which dead subscriptions
are removed. Push delivery does not depend on that service running; stopping it only stops
the pruning sweep.
Adding a third provider is one file plus one registry entry: the registration route, the
delivery loop, the admin page, the audit record and the metrics are all written against the
provider protocol.
### Events
@ -836,9 +870,11 @@ in real time, bridged onto the in-process pub/sub bus by a lock-owner relay:
`create_notification` schedules delivery as a fire-and-forget async task, so a dead
subscription or push-service error never blocks the triggering request. Delivery
(`push.notify_user`) iterates a user's subscriptions, encrypts the payload
(legacy `aesgcm` content encoding), and POSTs to each endpoint; subscriptions that
return `404`/`410` are soft-deleted.
(`push.notify_user`) reads a user's subscriptions once, groups them by provider, builds
each provider's payload once, and sends over a single shared HTTP client. A subscription
the push service reports as gone (`404`/`410` for Web Push, `410` or an `Unregistered`
class reason for APNs) is soft-deleted; any other failure is logged and the subscription is
kept.
A notification is also **marked read automatically when you open the page that shows its
content** - viewing a post clears its comment, reply, upvote and mention notifications;
@ -895,7 +931,10 @@ offline. Installation requires a secure origin (HTTPS, or `localhost` for develo
| File | Role |
|------|------|
| `devplacepy/push.py` | VAPID keys, payload encryption, send, register |
| `devplacepy/push/providers/` | Provider protocol, Web Push (VAPID keys, payload encryption), APNs |
| `devplacepy/push/store.py` | `push_registration` reads and writes |
| `devplacepy/push/delivery.py` | `notify_user` - group by provider, deliver, reap dead subscriptions |
| `devplacepy/services/push/service.py` | Provider configuration at `/admin/services/push`, retention sweep, metrics |
| `devplacepy/routers/push.py` | `/push.json`, `/service-worker.js`, `/manifest.json` |
| `static/js/PushManager.js` | Service-worker registration + subscribe + opt-in UI |
| `static/service-worker.js` | Receives push, shows notification, offline fallback |

View File

@ -12,7 +12,7 @@ def enforce_rgba_png(file_bytes: bytes) -> bytes:
corner = img.getpixel((0, 0))
if len(corner) == 4 and corner[3] == 255:
bg = corner[:3]
data = img.getdata()
data = img.get_flattened_data()
cleaned = []
for pixel in data:
if pixel[:3] == bg:

View File

@ -134,7 +134,28 @@ def init_db():
)
_index(db, "notifications", "idx_notifications_user", ["user_uid"])
_index(db, "notifications", "idx_notifications_user_read", ["user_uid", "read"])
push_registration = get_table("push_registration")
for column, example in (
("uid", ""),
("user_uid", ""),
("provider", "webpush"),
("endpoint", ""),
("key_auth", ""),
("key_p256dh", ""),
("token", ""),
("created_at", ""),
("deleted_at", ""),
):
if not push_registration.has_column(column):
push_registration.create_column_by_example(column, example)
_index(db, "push_registration", "idx_push_registration_user", ["user_uid"])
_index(db, "push_registration", "idx_push_registration_provider", ["provider"])
if "push_registration" in db.tables:
with db:
db.query(
"UPDATE push_registration SET provider = 'webpush' "
"WHERE provider IS NULL OR provider = ''"
)
_index(db, "sessions", "idx_sessions_token", ["session_token"])
projects = get_table("projects")
for column, example in (

View File

@ -8,8 +8,15 @@ GROUP = {
"intro": """
# Web Push
Browser push notifications via the Web Push protocol. Fetch the public VAPID key, then
register a `PushSubscription` obtained from the browser's `PushManager`.
Push notifications are delivered by one or more providers. `webpush` is the default and
implements the Web Push protocol: fetch the public VAPID key, then register a
`PushSubscription` obtained from the browser's `PushManager`. `apns` delivers to an Apple
Push Notification service device token and is only offered when an administrator has
configured it.
`GET /push.json` lists the providers that currently accept registrations. A registration
body without a `provider` field is a `webpush` registration, so existing clients need no
change.
There is no server-side unsubscribe endpoint: unsubscription is handled entirely in the
browser by calling `PushManager.unsubscribe()` on the subscription. The server stops delivering
@ -26,39 +33,60 @@ four ways to sign requests.
method="GET",
path="/push.json",
title="Get the public key",
summary="Return the VAPID public key for subscribing.",
summary="Return the VAPID public key and the providers that accept registrations.",
auth="public",
sample_response={"publicKey": "BASE64_VAPID_KEY"},
sample_response={
"publicKey": "BASE64_VAPID_KEY",
"providers": {"webpush": {"publicKey": "BASE64_VAPID_KEY"}},
},
),
endpoint(
id="push-register",
method="POST",
path="/push.json",
title="Register a subscription",
summary="Register a browser push subscription. Sends a welcome notification.",
summary="Register a push subscription. Sends a welcome notification.",
auth="user",
encoding="json",
interactive=False,
params=[
field(
"provider",
"json",
"string",
False,
"webpush",
"Provider to register with. Omit for webpush.",
),
field(
"endpoint",
"json",
"string",
True,
False,
"https://fcm.googleapis.com/...",
"Subscription endpoint URL.",
"Subscription endpoint URL. Required for webpush.",
),
field(
"keys",
"json",
"string",
True,
False,
'{"p256dh":"...","auth":"..."}',
"Subscription keys object.",
"Subscription keys object. Required for webpush.",
),
field(
"token",
"json",
"string",
False,
"a1b2c3...",
"Hexadecimal device token. Required for apns.",
),
],
notes=[
'The body must be JSON: `{"endpoint": "...", "keys": {"p256dh": "...", "auth": "..."}}`.'
'A webpush body is JSON: `{"endpoint": "...", "keys": {"p256dh": "...", "auth": "..."}}`.',
'An APNs body is JSON: `{"provider": "apns", "token": "..."}`.',
"A provider that is unknown, disabled or unconfigured returns 400.",
],
sample_response={"registered": True},
),

View File

@ -115,6 +115,7 @@ from devplacepy.services.containers.service import ContainerService
from devplacepy.services.xmlrpc import XmlrpcService
from devplacepy.services.audit import AuditService
from devplacepy.services.audit import record as audit
from devplacepy.services.push import PushService
from devplacepy.services.telegram import TelegramService
from devplacepy.services.telegram.outbox_service import TelegramOutboxService
@ -275,6 +276,7 @@ async def lifespan(app: FastAPI):
service_manager.register(ContainerService())
service_manager.register(XmlrpcService())
service_manager.register(AuditService())
service_manager.register(PushService())
service_manager.register(TelegramService())
service_manager.register(TelegramOutboxService())
if not os.environ.get("DEVPLACE_DISABLE_SERVICES"):

52
devplacepy/push/CLAUDE.md Normal file
View File

@ -0,0 +1,52 @@
This file documents `devplacepy/push/` - push notification delivery and its provider architecture. Claude Code loads it automatically whenever a file under this directory is read or edited.
## What this package is
One delivery library behind one public surface. `devplacepy.push` exports `notify_user`, `register`, `ensure_certificates`, `public_key_standard_b64` and the Web Push crypto helpers; every caller in the codebase (`main.py`, `utils/notifications.py`, `routers/push.py`) imports only those names. Everything else is internal to the package.
| Module | Role |
|---|---|
| `providers/base.py` | `PushProvider` protocol, the `Delivery` outcome and the three outcome constants |
| `providers/webpush.py` | VAPID key material, `aesgcm` payload encryption, the Web Push provider |
| `providers/apns.py` | Apple Push Notification service provider (token based, HTTP/2) |
| `providers/__init__.py` | `PROVIDERS` registry, `get`, `active`, `is_active`, `admin_fields`, `client_config` |
| `store.py` | Every `push_registration` read and write |
| `delivery.py` | `notify_user`: group by provider, one shared client, one prepared body per provider |
The admin configuration surface lives in `devplacepy/services/push/service.py` (`PushService`), not here.
## Adding a provider
1. Write `providers/<name>.py` with a `PushProvider` subclass: `name`, `label`, `config_fields`, `is_configured`, `parse_registration`, `prepare`, `deliver`, optionally `client_config`.
2. Add one entry to `PROVIDERS` in `providers/__init__.py`.
That is the whole change. The registration route, the delivery loop, the admin page, the audit record, the metrics and the docs are written against the protocol and need no edit. The `Enabled` toggle (`push_<name>_enabled`) comes from the base class, so a provider never declares its own.
## Invariants
- **Zero cost for the request.** Delivery is reached only through `utils/notifications.py` `_schedule_push`, a fire-and-forget task. Never make a route await `notify_user`, and never add a queue or a table to this path.
- **`deliver` never raises.** Return `Delivery(REJECTED, detail)` instead. `delivery.py` guards anyway, but a raising provider costs a log line per subscription.
- **A provider that is not configured is inert, never an error.** `is_configured()` is false, `is_active()` is false, the delivery loop skips it, and `POST /push.json` refuses a registration for it with 400. Nothing else in the platform notices.
- **`DEAD` is the only outcome that touches the database.** It soft-deletes the registration (`deleted_at`), exactly like a `404`/`410` Web Push endpoint always did. `REJECTED` keeps the row.
- **Every insert writes `deleted_at: None`,** and every read filters `deleted_at IS NULL`. `push_registration` deliberately stays out of `SOFT_DELETE_TABLES` (no `deleted_by`, not restorable from Trash) - a dead device token has no owner action to undo.
- **A row without a provider is a Web Push row.** `store.provider_of` resolves `None`/`""` to `DEFAULT_PROVIDER`, so a row written by an old worker during a deploy still delivers. `init_db` backfills the column once with a single converging `UPDATE`.
## Storage
`push_registration` columns are ensured in `init_db` (`database/schema.py`) because `dataset` only creates the columns of a table's first insert, and `find(provider=...)` against a missing column matches nothing.
| Column | webpush | apns |
|---|---|---|
| `provider` | `webpush` | `apns` |
| `endpoint`, `key_auth`, `key_p256dh` | set | `NULL` |
| `token` | `NULL` | device token |
Deduplication is generic: `store.register` looks up `user_uid` + `provider` + exactly the fields the provider's `parse_registration` returned, so a provider never writes its own identity rule.
## APNs specifics
- `POST https://{host}/3/device/{token}` over HTTP/2, host from `push_apns_environment` (`api.push.apple.com` or `api.sandbox.push.apple.com`; an unrecognised value falls back to production). HTTP/2 comes from `stealth_async_client` because the origin is `https` - the cleartext downgrade in `curl_transport` does not apply.
- Provider token: `ES256`, header `kid` = key id, claims `iss` = team id and `iat`. Cached per credential fingerprint for 45 minutes, so a worker signs at most one token per 45 minutes; Apple refuses tokens regenerated faster than every 20 minutes. Editing any credential changes the fingerprint and takes effect on the next delivery, with no restart.
- A `.p8` that does not parse is cached as a failure for the same window, so a misconfiguration costs one error log per window rather than one parse per notification.
- `410`, or any status carrying reason `BadDeviceToken`, `Unregistered`, `ExpiredToken`, `DeviceTokenNotForTopic` or `TopicDisallowed`, is `DEAD`. Everything else is `REJECTED`.
- The shared payload dict (`title`, `message`, `icon`, `url`) is translated once per batch into `aps.alert` plus the custom `url`/`icon` keys, mirroring what `service-worker.js` does for Web Push. `thread-id` mirrors the service worker's notification `tag`.

View File

@ -0,0 +1,29 @@
# retoor <retoor@molodetz.nl>
from devplacepy.push.delivery import notify_user
from devplacepy.push.providers.webpush import (
browser_base64,
create_notification_authorization,
create_notification_info_with_payload,
ensure_certificates,
generate_pkcs8_private_key,
generate_private_key,
generate_public_key,
hkdf,
public_key_standard_b64,
)
from devplacepy.push.store import register
__all__ = [
"browser_base64",
"create_notification_authorization",
"create_notification_info_with_payload",
"ensure_certificates",
"generate_pkcs8_private_key",
"generate_private_key",
"generate_public_key",
"hkdf",
"notify_user",
"public_key_standard_b64",
"register",
]

View File

@ -0,0 +1,83 @@
# retoor <retoor@molodetz.nl>
import logging
from typing import Any
from devplacepy import stealth
from devplacepy.database import get_int_setting
from devplacepy.push import providers, store
logger = logging.getLogger(__name__)
TIMEOUT_KEY = "push_delivery_timeout_seconds"
DEFAULT_TIMEOUT_SECONDS = 10
MIN_TIMEOUT_SECONDS = 1
MAX_TIMEOUT_SECONDS = 120
def timeout_seconds() -> float:
seconds = get_int_setting(TIMEOUT_KEY, DEFAULT_TIMEOUT_SECONDS)
return float(min(max(seconds, MIN_TIMEOUT_SECONDS), MAX_TIMEOUT_SECONDS))
def group_by_provider(
registrations: list[dict[str, Any]],
) -> dict[str, list[dict[str, Any]]]:
grouped: dict[str, list[dict[str, Any]]] = {}
for registration in registrations:
grouped.setdefault(store.provider_of(registration), []).append(registration)
return grouped
async def notify_user(user_uid: str, payload: dict[str, Any]) -> None:
registrations = store.active_for_user(user_uid)
if not registrations:
logger.debug("No active push subscriptions for user %s", user_uid)
return
grouped = group_by_provider(registrations)
async with stealth.stealth_async_client(timeout=timeout_seconds()) as client:
for name, rows in grouped.items():
provider = providers.PROVIDERS.get(name)
if provider is None:
logger.warning(
"Unknown push provider %s on %s subscriptions of user %s",
name,
len(rows),
user_uid,
)
continue
if not providers.is_active(provider):
logger.debug(
"Push provider %s is not active; skipping %s subscriptions",
name,
len(rows),
)
continue
try:
prepared = provider.prepare(payload)
except Exception as exc:
logger.error("Push provider %s could not build a payload: %s", name, exc)
continue
for registration in rows:
await _deliver_one(provider, client, registration, prepared, user_uid)
async def _deliver_one(provider, client, registration, prepared, user_uid) -> None:
try:
outcome = await provider.deliver(client, registration, prepared)
except Exception as exc:
logger.error("Push provider %s raised for %s: %s", provider.name, user_uid, exc)
return
if outcome.status == providers.ACCEPTED:
logger.debug("Push delivered to %s via %s", user_uid, provider.name)
return
if outcome.status == providers.DEAD:
try:
store.mark_dead(registration["id"])
except Exception as exc:
logger.error("Could not soft-delete push subscription: %s", exc)
return
logger.warning(
"Push rejected by %s for %s: %s", provider.name, user_uid, outcome.detail
)

View File

@ -0,0 +1,76 @@
# retoor <retoor@molodetz.nl>
import logging
from typing import Any
from devplacepy.push.providers.apns import ApnsProvider
from devplacepy.push.providers.base import (
ACCEPTED,
DEAD,
REJECTED,
Delivery,
PushProvider,
)
from devplacepy.push.providers.webpush import WebPushProvider
logger = logging.getLogger(__name__)
DEFAULT_PROVIDER = WebPushProvider.name
PROVIDERS: dict[str, PushProvider] = {
provider.name: provider for provider in (WebPushProvider(), ApnsProvider())
}
__all__ = [
"ACCEPTED",
"DEAD",
"DEFAULT_PROVIDER",
"Delivery",
"PROVIDERS",
"PushProvider",
"REJECTED",
"active",
"admin_fields",
"client_config",
"get",
"is_active",
"names",
]
def get(name: str | None) -> PushProvider | None:
if not isinstance(name, str):
name = ""
return PROVIDERS.get(name.strip().lower() or DEFAULT_PROVIDER)
def names() -> list[str]:
return list(PROVIDERS)
def active() -> list[PushProvider]:
return [provider for provider in PROVIDERS.values() if is_active(provider)]
def admin_fields() -> list:
return [field for provider in PROVIDERS.values() for field in provider.all_fields()]
def client_config() -> dict[str, Any]:
return {provider.name: _client_config(provider) for provider in active()}
def is_active(provider: PushProvider) -> bool:
try:
return provider.is_active()
except Exception as exc:
logger.error("Push provider %s failed its readiness check: %s", provider.name, exc)
return False
def _client_config(provider: PushProvider) -> dict[str, Any]:
try:
return provider.client_config()
except Exception as exc:
logger.error("Push provider %s failed to describe itself: %s", provider.name, exc)
return {}

View File

@ -0,0 +1,243 @@
# retoor <retoor@molodetz.nl>
import hashlib
import json
import logging
import string
import time
from typing import Any
import httpx
import jwt
from devplacepy.config import SECONDS_PER_DAY
from devplacepy.database import get_setting
from devplacepy.push.providers.base import (
ACCEPTED,
DEAD,
REJECTED,
Delivery,
PushProvider,
)
from devplacepy.services.base import ConfigField
from devplacepy.utils import DEFAULT_PUSH_URL, PUSH_ICON, generate_uid
logger = logging.getLogger(__name__)
TEAM_ID_KEY = "push_apns_team_id"
KEY_ID_KEY = "push_apns_key_id"
AUTH_KEY_KEY = "push_apns_auth_key"
TOPIC_KEY = "push_apns_topic"
ENVIRONMENT_KEY = "push_apns_environment"
PROVIDER_LABEL = "Apple Push (APNs)"
DEFAULT_ENVIRONMENT = "production"
HOSTS = {
"production": "api.push.apple.com",
"sandbox": "api.sandbox.push.apple.com",
}
ENVIRONMENT_OPTIONS = [
{"value": "production", "label": "Production"},
{"value": "sandbox", "label": "Sandbox"},
]
TOKEN_REFRESH_SECONDS = 45 * 60
TOKEN_MIN_LENGTH = 64
TOKEN_MAX_LENGTH = 200
THREAD_ID = "devplace-notification"
PUSH_TYPE = "alert"
PRIORITY = "10"
DEAD_REASONS = frozenset(
{
"BadDeviceToken",
"DeviceTokenNotForTopic",
"ExpiredToken",
"Unregistered",
"TopicDisallowed",
}
)
_token_state: dict[str, Any] = {}
def _setting(key: str) -> str:
return get_setting(key, "").strip()
def _environment() -> str:
value = _setting(ENVIRONMENT_KEY) or DEFAULT_ENVIRONMENT
return value if value in HOSTS else DEFAULT_ENVIRONMENT
def host() -> str:
return HOSTS[_environment()]
def _fingerprint(team_id: str, key_id: str, auth_key: str) -> str:
return hashlib.sha256(f"{team_id}:{key_id}:{auth_key}".encode("utf-8")).hexdigest()
def provider_token(team_id: str, key_id: str, auth_key: str) -> str:
fingerprint = _fingerprint(team_id, key_id, auth_key)
issued_at = int(time.time())
state = _token_state.get("current")
if (
state
and state["fingerprint"] == fingerprint
and issued_at - state["issued_at"] < TOKEN_REFRESH_SECONDS
):
if state["token"] is None:
raise ValueError(state["error"])
return state["token"]
try:
token = jwt.encode(
{"iss": team_id, "iat": issued_at},
auth_key,
algorithm="ES256",
headers={"kid": key_id},
)
except Exception as exc:
message = f"APNs auth key is not usable: {exc}"
_token_state["current"] = {
"token": None,
"error": message,
"issued_at": issued_at,
"fingerprint": fingerprint,
}
logger.error(message)
raise ValueError(message) from exc
_token_state["current"] = {
"token": token,
"error": "",
"issued_at": issued_at,
"fingerprint": fingerprint,
}
return token
def _reason(response: httpx.Response) -> str:
try:
body = response.json()
except ValueError:
return ""
if isinstance(body, dict):
return str(body.get("reason", ""))
return ""
class ApnsProvider(PushProvider):
name = "apns"
label = PROVIDER_LABEL
config_fields = [
ConfigField(
TEAM_ID_KEY,
"Team ID",
type="str",
default="",
help="Ten character Apple Developer team identifier, used as the token iss claim.",
group=PROVIDER_LABEL,
),
ConfigField(
KEY_ID_KEY,
"Key ID",
type="str",
default="",
help="Ten character identifier of the APNs auth key, sent as the token kid header.",
group=PROVIDER_LABEL,
),
ConfigField(
AUTH_KEY_KEY,
"Auth key (.p8)",
type="text",
default="",
secret=True,
help="Contents of the APNs .p8 signing key, including the BEGIN and END lines. Leave blank to keep the stored key.",
group=PROVIDER_LABEL,
),
ConfigField(
TOPIC_KEY,
"Topic",
type="str",
default="",
help="Bundle identifier of the receiving app, sent as the apns-topic header.",
group=PROVIDER_LABEL,
),
ConfigField(
ENVIRONMENT_KEY,
"Environment",
type="select",
default=DEFAULT_ENVIRONMENT,
options=ENVIRONMENT_OPTIONS,
help="Production delivers to App Store builds, sandbox to development builds.",
group=PROVIDER_LABEL,
),
]
def is_configured(self) -> bool:
return bool(
_setting(TEAM_ID_KEY)
and _setting(KEY_ID_KEY)
and _setting(AUTH_KEY_KEY)
and _setting(TOPIC_KEY)
)
def parse_registration(self, body: dict[str, Any]) -> dict[str, Any] | None:
token = body.get("token")
if not isinstance(token, str):
return None
token = token.strip()
if not TOKEN_MIN_LENGTH <= len(token) <= TOKEN_MAX_LENGTH:
return None
if any(character not in string.hexdigits for character in token):
return None
return {"token": token}
def prepare(self, payload: dict[str, Any]) -> str:
return json.dumps(
{
"aps": {
"alert": {
"title": payload.get("title") or "DevPlace",
"body": payload.get("message") or "",
},
"sound": "default",
"thread-id": THREAD_ID,
},
"url": payload.get("url") or DEFAULT_PUSH_URL,
"icon": payload.get("icon") or PUSH_ICON,
}
)
def headers(self) -> dict[str, str]:
return {
"authorization": f"bearer {provider_token(_setting(TEAM_ID_KEY), _setting(KEY_ID_KEY), _setting(AUTH_KEY_KEY))}",
"apns-topic": _setting(TOPIC_KEY),
"apns-push-type": PUSH_TYPE,
"apns-priority": PRIORITY,
"apns-expiration": str(int(time.time()) + SECONDS_PER_DAY),
"apns-id": generate_uid(),
"content-type": "application/json",
}
async def deliver(
self, client: httpx.AsyncClient, registration: dict[str, Any], prepared: str
) -> Delivery:
token = (registration.get("token") or "").strip()
if not token:
return Delivery(DEAD, "missing device token")
try:
headers = self.headers()
response = await client.post(
f"https://{host()}/3/device/{token}",
headers=headers,
content=prepared.encode("utf-8"),
)
except (httpx.HTTPError, ValueError) as exc:
return Delivery(REJECTED, str(exc))
if response.status_code == 200:
return Delivery(ACCEPTED)
reason = _reason(response)
detail = f"{response.status_code} {reason}".strip()
if response.status_code == 410 or reason in DEAD_REASONS:
return Delivery(DEAD, detail)
return Delivery(REJECTED, detail)

View File

@ -0,0 +1,66 @@
# retoor <retoor@molodetz.nl>
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Any
import httpx
from devplacepy.database import get_setting
from devplacepy.services.base import ConfigField
ACCEPTED = "accepted"
DEAD = "dead"
REJECTED = "rejected"
@dataclass(frozen=True)
class Delivery:
status: str
detail: str = ""
class PushProvider(ABC):
name = ""
label = ""
config_fields: list[ConfigField] = []
@property
def enabled_key(self) -> str:
return f"push_{self.name}_enabled"
def enabled_field(self) -> ConfigField:
return ConfigField(
self.enabled_key,
"Enabled",
type="bool",
default=True,
help=f"Deliver notifications through {self.label}.",
group=self.label,
)
def all_fields(self) -> list[ConfigField]:
return [self.enabled_field(), *self.config_fields]
def is_enabled(self) -> bool:
return get_setting(self.enabled_key, "1") == "1"
def is_active(self) -> bool:
return self.is_enabled() and self.is_configured()
def client_config(self) -> dict[str, Any]:
return {}
@abstractmethod
def is_configured(self) -> bool: ...
@abstractmethod
def parse_registration(self, body: dict[str, Any]) -> dict[str, Any] | None: ...
@abstractmethod
def prepare(self, payload: dict[str, Any]) -> str: ...
@abstractmethod
async def deliver(
self, client: httpx.AsyncClient, registration: dict[str, Any], prepared: str
) -> Delivery: ...

View File

@ -7,7 +7,6 @@ import logging
import os
import random
import time
from datetime import datetime, timezone
from typing import Any
from urllib.parse import urlparse
@ -20,7 +19,6 @@ from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from cryptography.hazmat.primitives.hashes import SHA256
from cryptography.hazmat.primitives.kdf.hkdf import HKDF
from devplacepy import stealth
from devplacepy.config import (
SECONDS_PER_DAY,
VAPID_PRIVATE_KEY_FILE,
@ -28,7 +26,15 @@ from devplacepy.config import (
VAPID_PUBLIC_KEY_FILE,
VAPID_SUB,
)
from devplacepy.database import get_table
from devplacepy.database import get_setting
from devplacepy.push.providers.base import (
ACCEPTED,
DEAD,
REJECTED,
Delivery,
PushProvider,
)
from devplacepy.services.base import ConfigField
from devplacepy.utils import generate_uid
logger = logging.getLogger(__name__)
@ -37,6 +43,8 @@ JWT_LIFETIME_SECONDS = 60 * 60
PUSH_TTL_SECONDS = str(SECONDS_PER_DAY)
DEAD_SUBSCRIPTION_STATUSES = (404, 410)
ACCEPTED_STATUSES = (200, 201)
SUBJECT_KEY = "push_webpush_subject"
PROVIDER_LABEL = "Web Push (VAPID)"
def generate_private_key() -> None:
@ -149,13 +157,17 @@ def public_key_standard_b64() -> str:
return base64.b64encode(point).decode("utf-8").rstrip("=")
def subject() -> str:
return get_setting(SUBJECT_KEY, "").strip() or VAPID_SUB
def create_notification_authorization(push_url: str) -> str:
target = urlparse(push_url)
audience = f"{target.scheme}://{target.netloc}"
issued_at = int(time.time())
return jwt.encode(
{
"sub": VAPID_SUB,
"sub": subject(),
"aud": audience,
"exp": issued_at + JWT_LIFETIME_SECONDS,
"nbf": issued_at,
@ -223,78 +235,76 @@ def create_notification_info_with_payload(
}
def _mark_subscription_dead(subscription_id: int) -> None:
get_table("push_registration").update(
{"id": subscription_id, "deleted_at": datetime.now(timezone.utc).isoformat()},
["id"],
)
logger.info("Soft-deleted dead push subscription id=%s", subscription_id)
class WebPushProvider(PushProvider):
name = "webpush"
label = PROVIDER_LABEL
config_fields = [
ConfigField(
SUBJECT_KEY,
"VAPID subject",
type="str",
default=VAPID_SUB,
help="Contact sent as the JWT sub claim, a mailto: or https: URL. Blank uses the built-in default.",
group=PROVIDER_LABEL,
)
]
def is_configured(self) -> bool:
return True
async def notify_user(user_uid: str, payload: dict[str, Any]) -> None:
registrations = list(
get_table("push_registration").find(user_uid=user_uid, deleted_at=None)
)
if not registrations:
logger.debug("No active push subscriptions for user %s", user_uid)
return
def client_config(self) -> dict[str, Any]:
try:
return {"publicKey": public_key_standard_b64()}
except Exception as exc:
logger.error("VAPID key material unavailable: %s", exc)
return {}
body = json.dumps(payload)
async with stealth.stealth_async_client(timeout=10.0) as client:
for subscription in registrations:
endpoint = subscription["endpoint"]
try:
notification_payload = create_notification_info_with_payload(
endpoint,
subscription["key_auth"],
subscription["key_p256dh"],
body,
)
headers = {**notification_payload["headers"], "TTL": PUSH_TTL_SECONDS}
response = await client.post(
endpoint, headers=headers, content=notification_payload["data"]
)
except (httpx.HTTPError, ValueError) as exc:
logger.warning("Push error for %s via %s: %s", user_uid, endpoint, exc)
continue
def parse_registration(self, body: dict[str, Any]) -> dict[str, Any] | None:
keys = body.get("keys")
if not isinstance(keys, dict):
return None
endpoint = body.get("endpoint")
key_auth = keys.get("auth")
key_p256dh = keys.get("p256dh")
if not (
isinstance(endpoint, str)
and isinstance(key_auth, str)
and isinstance(key_p256dh, str)
and endpoint
and key_auth
and key_p256dh
):
return None
return {
"endpoint": endpoint,
"key_auth": key_auth,
"key_p256dh": key_p256dh,
}
if response.status_code in ACCEPTED_STATUSES:
logger.debug("Push delivered to %s via %s", user_uid, endpoint)
elif response.status_code in DEAD_SUBSCRIPTION_STATUSES:
_mark_subscription_dead(subscription["id"])
else:
logger.warning(
"Push rejected (%s) for %s via %s",
response.status_code,
user_uid,
endpoint,
)
def prepare(self, payload: dict[str, Any]) -> str:
return json.dumps(payload)
async def register(
user_uid: str, endpoint: str, key_auth: str, key_p256dh: str
) -> tuple[dict[str, Any], bool]:
table = get_table("push_registration")
existing = table.find_one(
user_uid=user_uid,
endpoint=endpoint,
key_auth=key_auth,
key_p256dh=key_p256dh,
deleted_at=None,
)
if existing:
logger.debug("Push subscription already registered for user %s", user_uid)
return existing, False
record = {
"uid": generate_uid(),
"user_uid": user_uid,
"endpoint": endpoint,
"key_auth": key_auth,
"key_p256dh": key_p256dh,
"created_at": datetime.now(timezone.utc).isoformat(),
"deleted_at": None,
}
table.insert(record)
logger.info("Registered push subscription for user %s", user_uid)
return record, True
async def deliver(
self, client: httpx.AsyncClient, registration: dict[str, Any], prepared: str
) -> Delivery:
endpoint = registration.get("endpoint") or ""
if not endpoint:
return Delivery(DEAD, "missing endpoint")
try:
notification_payload = create_notification_info_with_payload(
endpoint,
registration["key_auth"],
registration["key_p256dh"],
prepared,
)
headers = {**notification_payload["headers"], "TTL": PUSH_TTL_SECONDS}
response = await client.post(
endpoint, headers=headers, content=notification_payload["data"]
)
except (httpx.HTTPError, ValueError, KeyError, TypeError) as exc:
return Delivery(REJECTED, str(exc))
if response.status_code in ACCEPTED_STATUSES:
return Delivery(ACCEPTED)
if response.status_code in DEAD_SUBSCRIPTION_STATUSES:
return Delivery(DEAD, str(response.status_code))
return Delivery(REJECTED, str(response.status_code))

83
devplacepy/push/store.py Normal file
View File

@ -0,0 +1,83 @@
# retoor <retoor@molodetz.nl>
import logging
from datetime import datetime, timezone
from typing import Any
from devplacepy.database import db, get_table
from devplacepy.push.providers import DEFAULT_PROVIDER
from devplacepy.utils import generate_uid
logger = logging.getLogger(__name__)
TABLE = "push_registration"
def table():
return get_table(TABLE)
def provider_of(registration: dict[str, Any]) -> str:
return registration.get("provider") or DEFAULT_PROVIDER
def active_for_user(user_uid: str) -> list[dict[str, Any]]:
return list(table().find(user_uid=user_uid, deleted_at=None))
def register(
user_uid: str, provider: str, fields: dict[str, Any]
) -> tuple[dict[str, Any], bool]:
registrations = table()
existing = registrations.find_one(
user_uid=user_uid, provider=provider, deleted_at=None, **fields
)
if existing:
logger.debug("Push subscription already registered for user %s", user_uid)
return existing, False
record = {
"uid": generate_uid(),
"user_uid": user_uid,
"provider": provider,
"created_at": datetime.now(timezone.utc).isoformat(),
"deleted_at": None,
**fields,
}
registrations.insert(record)
logger.info("Registered %s push subscription for user %s", provider, user_uid)
return record, True
def mark_dead(registration_id: int) -> None:
table().update(
{"id": registration_id, "deleted_at": datetime.now(timezone.utc).isoformat()},
["id"],
)
logger.info("Soft-deleted dead push subscription id=%s", registration_id)
def prune(cutoff: str) -> int:
if TABLE not in db.tables:
return 0
rows = list(table().find(deleted_at={"<": cutoff}))
if not rows:
return 0
table().delete(deleted_at={"<": cutoff})
return len(rows)
def counts() -> dict[str, int]:
if TABLE not in db.tables:
return {}
totals: dict[str, int] = {"dead": 0}
for row in db.query(
f"SELECT provider AS provider, deleted_at IS NULL AS live, COUNT(*) AS total "
f"FROM {TABLE} GROUP BY provider, deleted_at IS NULL"
):
provider = row["provider"] or DEFAULT_PROVIDER
if row["live"]:
totals[provider] = totals.get(provider, 0) + int(row["total"])
else:
totals["dead"] += int(row["total"])
return totals

View File

@ -45,7 +45,7 @@ Prefixes are wired in `main.py`:
| `/dbapi` | dbapi/ package - **primary-administrator-only, strictly READ-ONLY** generic database API over `dataset` (never inserts/updates/deletes/restores; the write surface was removed because it bypassed every per-route admin safeguard). `tables.py` (`GET /tables`, `GET /{table}/schema`), `crud.py` (read only: `GET /{table}` + `GET /{table}/{key}/{value}`, `?include_deleted`), `query.py` (`POST /query` hard SELECT-only validated read-only run; `POST /query/async` + `GET /query/{uid}` + `WS /query/{uid}/ws` via `DbApiJobService` kind `dbquery`), `nl.py` (`POST /nl` natural-language->SQL via the AI gateway, returns validated SELECT, `execute=true` runs it read-only). **Auth = the PRIMARY administrator only** (the earliest-created Admin, `utils.is_primary_admin` / `database.get_primary_admin_uid` - the same identity that gates backup downloads) by session/api_key (`services/dbapi/policy.py`); every other administrator is refused like a member, and there is **no internal-key path** - the gateway `internal_gateway_key()` is NOT accepted (no service-to-service access). SQL validated by `services/dbapi/validate.py` (sqlglot classify + suspicious-flag + read-only `EXPLAIN` dry-run). Devii tools `db_*` are read-only (list/get/query/nl; no write tools) and flagged `requires_primary_admin=True`, so they are added to the LLM tool list ONLY for the primary administrator (every other session never sees them and Devii is unaware they exist). `services/pubsub/policy.py` resolves its own admin/internal actor and does NOT reuse `dbapi.policy.caller_for`, so the primary-admin restriction does not leak onto the pub/sub bus. See `devplacepy/services/dbapi/CLAUDE.md` |
| `/quizzes` | quizzes/ package - **Quizzes**: `index.py` (hub, create, import, detail, export, edit, delete, publish, per-quiz leaderboard, cross-quiz scoreboard), `questions.py` (builder page + question CRUD + reorder), `attempts.py` (start/resume, play, answer, finish, results). The hub `GET /quizzes` is the approved three-column `.feed-layout` reused verbatim from `/feed`: filters + search left (`?filter=all|todo|done|mine|drafts`, `?search=`, `?page=`), the quiz list centre with the **New quiz** and **Create quiz with Devii** actions, and the cross-quiz **scoreboard** rail right (score per user, best attempt per quiz, 15s display cache). Publishing is terminal - every write on a published quiz is a 400 and there is no unpublish route. See `devplacepy/services/quiz/CLAUDE.md` |
| `/game` | game/ package - the **Code Farm** idle game (`index.py` + `farm.py`): `GET /game` (page), `GET /game/state`, `GET /game/leaderboard?board=`, `POST /game/{plant,harvest,buy-plot,upgrade,fertilize,daily,grant,perk,prestige,legacy,mastery,quests/claim}`, `POST /game/{defense/upgrade,defense/downgrade,infrastructure/buy,cosmetics/buy,cosmetics/equip}`, plus social `GET /game/farm/{username}` and `POST /game/farm/{username}/{water,steal}`. See `devplacepy/services/game/CLAUDE.md` |
| (none) | push.py - web push + PWA: `GET`/`POST /push.json` (VAPID public key / subscription register), `GET /service-worker.js`, `GET /manifest.json` |
| (none) | push.py - push + PWA: `GET /push.json` (VAPID public key + the providers accepting registrations), `POST /push.json` (register with any active provider; a body without `provider` is a `webpush` body, so existing clients are unchanged; an unknown, disabled or unconfigured provider is a 400), `GET /service-worker.js`, `GET /manifest.json`. Provider protocol and delivery live in `devplacepy/push/` - see `devplacepy/push/CLAUDE.md` |
| (none) | docs.py (docs/ package) - the documentation site (prose pages + API reference). See `routers/docs/CLAUDE.md` for the deep detail on this tree (`DOCS_PAGES`, audience tiers, prose rendering pipeline) |
| `/pubsub` | pubsub.py - **database-free** publish/subscribe bus. `WS /pubsub/ws` (subscribe/unsubscribe/publish frames, `foo.*` wildcards), `POST /pubsub/publish` + `GET /pubsub/topics` (admin/internal). Lock-owner-gated WS (`4013` retry) so subscribers converge on one worker; topic authz in `services/pubsub/policy.py` (`user.{uid}.*` private, `public.*` shared, admin/internal anywhere, guests opt-in). In-process `services.pubsub.publish(topic, data)` for backends; frontend `app.pubsub`. See `devplacepy/services/pubsub/CLAUDE.md` |
| (none) | seo.py - `/robots.txt`, `/sitemap.xml` |

View File

@ -5,6 +5,7 @@ from fastapi import APIRouter, Request
from fastapi.responses import FileResponse, JSONResponse
from devplacepy import push
from devplacepy.config import STATIC_DIR
from devplacepy.push import providers
from devplacepy.utils import require_user_api
from urllib.parse import urlparse
from devplacepy.services.audit import record as audit
@ -22,7 +23,11 @@ WELCOME_PAYLOAD = {
@router.get("/push.json")
async def push_public_key() -> JSONResponse:
return JSONResponse({"publicKey": push.public_key_standard_b64()})
configs = providers.client_config()
webpush = configs.get(providers.DEFAULT_PROVIDER, {})
return JSONResponse(
{"publicKey": webpush.get("publicKey", ""), "providers": configs}
)
@router.post("/push.json")
@ -33,21 +38,18 @@ async def push_register(request: Request) -> JSONResponse:
except ValueError:
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
keys = body.get("keys") if isinstance(body, dict) else None
if not (
isinstance(keys, dict)
and body.get("endpoint")
and keys.get("p256dh")
and keys.get("auth")
):
if not isinstance(body, dict):
return JSONResponse({"error": "Invalid request"}, status_code=400)
_, created = await push.register(
user_uid=user["uid"],
endpoint=body["endpoint"],
key_auth=keys["auth"],
key_p256dh=keys["p256dh"],
)
provider = providers.get(body.get("provider"))
if provider is None or not providers.is_active(provider):
return JSONResponse({"error": "Unknown provider"}, status_code=400)
fields = provider.parse_registration(body)
if fields is None:
return JSONResponse({"error": "Invalid request"}, status_code=400)
_, created = push.register(user["uid"], provider.name, fields)
if created:
try:
@ -62,7 +64,13 @@ async def push_register(request: Request) -> JSONResponse:
target_type="user",
target_uid=user["uid"],
target_label=user.get("username"),
metadata={"endpoint_host": urlparse(body["endpoint"]).hostname, "created": created},
metadata={
"provider": provider.name,
"endpoint_host": urlparse(fields["endpoint"]).hostname
if fields.get("endpoint")
else None,
"created": created,
},
summary=f"{user.get('username')} {'registered' if created else 'updated'} a push subscription",
links=[audit.target("user", user["uid"], user.get("username"))],
)

View File

@ -51,7 +51,7 @@ Generic, lightweight, **fire-and-forget** offload for non-critical side-effects
`devplacepy/services/` provides a generic framework for running background async services alongside the FastAPI server. `BaseService` provides the async run loop, a `deque(maxlen=20)` log buffer, and graceful cancellation; `ServiceManager` is a singleton that registers, starts, and stops services. Services are fully managed from the **Services admin tab** (`/admin/services`): start/stop, enable-on-boot, run-now, edit parameters, clear logs, adjustable log buffer size, with live status. All of this is generic - a new service gets it for free by declaring its config and implementing `run_once`.
This section covers only the shared machinery. The individual services built on top of it live in their own subdirectories with their own nested CLAUDE.md: `NewsService` (`services/news/`, see `devplacepy/services/news/CLAUDE.md`), `GatewayService` and provider/model routing (`services/openai_gateway/`, see `devplacepy/services/openai_gateway/CLAUDE.md`), `DeviiService` (`services/devii/`, see `devplacepy/services/devii/CLAUDE.md`), and the bot fleet service (`services/bot/`, see `devplacepy/services/bot/CLAUDE.md`).
This section covers only the shared machinery. The individual services built on top of it live in their own subdirectories with their own nested CLAUDE.md: `NewsService` (`services/news/`, see `devplacepy/services/news/CLAUDE.md`), `GatewayService` and provider/model routing (`services/openai_gateway/`, see `devplacepy/services/openai_gateway/CLAUDE.md`), `DeviiService` (`services/devii/`, see `devplacepy/services/devii/CLAUDE.md`), and the bot fleet service (`services/bot/`, see `devplacepy/services/bot/CLAUDE.md`). `PushService` (`services/push/`) is the thinnest example of the pattern: it owns no loop work beyond pruning dead subscriptions, and exists mainly so every push provider's configuration is edited through the same `ConfigField` surface as every other subsystem - its `config_fields` are assembled from `devplacepy.push.providers.admin_fields()`, so a new provider appears at `/admin/services/push` with no edit to the service. Push delivery does NOT depend on that service running (see `devplacepy/push/CLAUDE.md`).
### DB-backed state (correct across workers)

View File

@ -188,7 +188,7 @@ def _screenshot_hue_buckets(screenshot_bytes: bytes) -> dict[int, int]:
image = Image.open(io.BytesIO(screenshot_bytes)).convert("RGB")
image = image.resize((64, 64))
buckets: dict[int, int] = {}
for r, g, b in image.getdata():
for r, g, b in image.get_flattened_data():
hue, lightness, saturation = colorsys.rgb_to_hls(r / 255.0, g / 255.0, b / 255.0)
if saturation < 0.15 or lightness < 0.05 or lightness > 0.95:
continue

View File

@ -0,0 +1,9 @@
# retoor <retoor@molodetz.nl>
from devplacepy.services.push.service import (
DEFAULT_RETENTION_DAYS,
RETENTION_KEY,
PushService,
)
__all__ = ["DEFAULT_RETENTION_DAYS", "PushService", "RETENTION_KEY"]

View File

@ -0,0 +1,79 @@
# retoor <retoor@molodetz.nl>
import logging
from datetime import datetime, timedelta, timezone
from devplacepy.database import get_int_setting
from devplacepy.push import providers, store
from devplacepy.push.delivery import (
DEFAULT_TIMEOUT_SECONDS,
MAX_TIMEOUT_SECONDS,
MIN_TIMEOUT_SECONDS,
TIMEOUT_KEY,
)
from devplacepy.services.base import BaseService, ConfigField
logger = logging.getLogger(__name__)
RETENTION_KEY = "push_dead_retention_days"
DEFAULT_RETENTION_DAYS = 30
class PushService(BaseService):
title = "Push notifications"
description = (
"Configures the push providers and prunes dead subscriptions. Delivery is "
"independent of this service and continues while it is stopped."
)
details = (
"Notifications reach a user through every provider they have a live subscription "
"for. A provider delivers only while its own Enabled toggle is on and its "
"configuration is complete, so an unconfigured provider is inert."
)
default_enabled = True
min_interval = 3600
METRICS_SECONDS = 300
config_fields = [
ConfigField(
RETENTION_KEY,
"Dead subscription retention (days)",
type="int",
default=DEFAULT_RETENTION_DAYS,
minimum=0,
help="Subscriptions the push services rejected as gone are removed after this many days. 0 disables pruning.",
group="General",
),
ConfigField(
TIMEOUT_KEY,
"Delivery timeout (seconds)",
type="int",
default=DEFAULT_TIMEOUT_SECONDS,
minimum=MIN_TIMEOUT_SECONDS,
maximum=MAX_TIMEOUT_SECONDS,
help="Per request timeout used for every push provider.",
group="General",
),
*providers.admin_fields(),
]
def __init__(self) -> None:
super().__init__("push", interval_seconds=86400)
async def run_once(self) -> None:
days = get_int_setting(RETENTION_KEY, DEFAULT_RETENTION_DAYS)
if days <= 0:
self.log("Retention disabled (0 days); nothing pruned")
return
cutoff = (datetime.now(timezone.utc) - timedelta(days=days)).isoformat()
removed = store.prune(cutoff)
self.log(f"Pruned {removed} dead push subscriptions older than {days}d")
def collect_metrics(self) -> dict:
totals = store.counts()
metrics = {"dead": totals.get("dead", 0)}
for provider in providers.PROVIDERS.values():
metrics[f"{provider.name}_active"] = totals.get(provider.name, 0)
metrics[f"{provider.name}_ready"] = (
1 if providers.is_active(provider) else 0
)
return metrics

View File

@ -54,6 +54,9 @@ export class PushManager {
}
const keyData = await Http.getJson("/push.json");
if (!keyData.publicKey) {
return;
}
const applicationServerKey = Uint8Array.from(atob(keyData.publicKey), (c) => c.charCodeAt(0));
const subscription = await registration.pushManager.subscribe({

View File

@ -408,6 +408,8 @@ Every state-changing action in DevPlace records one append-only row through `dev
| `push.subscribe` | `routers/push.py` |
| `push.update` | `routers/push.py` |
Both carry `metadata.provider` (`webpush`, `apns`, ...) plus `created`; `metadata.endpoint_host` is set for endpoint-based providers and `null` for token-based ones.
## Gamification (`reward`)
| Event key | Recorded in |

View File

@ -64,3 +64,56 @@ def test_register_invalid_json_rejected(app_server):
headers={"Content-Type": "application/json"},
)
assert r.status_code == 400
def test_public_key_endpoint_lists_providers(app_server):
r = requests.get(f"{BASE_URL}/push.json")
assert r.status_code == 200
body = r.json()
assert body["publicKey"]
assert body["providers"]["webpush"]["publicKey"] == body["publicKey"]
def test_register_accepts_an_explicit_webpush_provider(app_server):
s = _session_push()
r = s.post(
f"{BASE_URL}/push.json",
json={
"provider": "webpush",
"endpoint": "https://push.example.com/explicit",
"keys": {"p256dh": "p256dh_fake", "auth": "auth_fake"},
},
)
assert r.status_code == 200, r.text
assert r.json().get("registered") is True
def test_register_is_idempotent_for_the_same_subscription(app_server):
s = _session_push()
body = {
"endpoint": "https://push.example.com/idempotent",
"keys": {"p256dh": "p256dh_fake", "auth": "auth_fake"},
}
assert s.post(f"{BASE_URL}/push.json", json=body).status_code == 200
assert s.post(f"{BASE_URL}/push.json", json=body).status_code == 200
def test_register_unknown_provider_rejected(app_server):
s = _session_push()
r = s.post(
f"{BASE_URL}/push.json",
json={"provider": "carrier-pigeon", "token": "a" * 64},
)
assert r.status_code == 400
def test_register_apns_rejected_while_unconfigured(app_server):
s = _session_push()
r = s.post(f"{BASE_URL}/push.json", json={"provider": "apns", "token": "a" * 64})
assert r.status_code == 400
def test_register_non_object_body_rejected(app_server):
s = _session_push()
r = s.post(f"{BASE_URL}/push.json", json=["nope"])
assert r.status_code == 400

View File

@ -48,3 +48,298 @@ def test_create_notification_authorization_is_jwt():
token = push.create_notification_authorization("https://push.example.com/endpoint")
assert token.count(".") == 2
def test_provider_registry_resolves_default_for_missing_name():
from devplacepy.push import providers
assert providers.get(None) is providers.PROVIDERS["webpush"]
assert providers.get("") is providers.PROVIDERS["webpush"]
assert providers.get(" APNS ") is providers.PROVIDERS["apns"]
assert providers.get("nope") is None
def test_webpush_parse_registration_accepts_subscription_shape():
from devplacepy.push import providers
webpush = providers.PROVIDERS["webpush"]
fields = webpush.parse_registration(
{
"endpoint": "https://push.example.com/sub",
"expirationTime": None,
"keys": {"p256dh": "p", "auth": "a"},
}
)
assert fields == {
"endpoint": "https://push.example.com/sub",
"key_auth": "a",
"key_p256dh": "p",
}
def test_webpush_parse_registration_rejects_incomplete_bodies():
from devplacepy.push import providers
webpush = providers.PROVIDERS["webpush"]
assert webpush.parse_registration({"endpoint": "https://push.example.com/x"}) is None
assert webpush.parse_registration({"keys": {"p256dh": "p", "auth": "a"}}) is None
assert (
webpush.parse_registration(
{"endpoint": "https://push.example.com/x", "keys": {"p256dh": "p"}}
)
is None
)
def test_apns_parse_registration_validates_device_token():
from devplacepy.push import providers
apns = providers.PROVIDERS["apns"]
token = "a1b2c3d4" * 8
assert apns.parse_registration({"token": f" {token} "}) == {"token": token}
assert apns.parse_registration({"token": "abc"}) is None
assert apns.parse_registration({"token": "z" * 64}) is None
assert apns.parse_registration({"token": "a" * 500}) is None
assert apns.parse_registration({"token": None}) is None
assert apns.parse_registration({}) is None
def test_apns_prepare_translates_the_shared_payload():
import json
from devplacepy.push import providers
body = json.loads(
providers.PROVIDERS["apns"].prepare(
{
"title": "DevPlace",
"message": "You have a new notification.",
"icon": "/static/apple-touch-icon.png",
"url": "/notifications",
}
)
)
assert body["aps"]["alert"] == {
"title": "DevPlace",
"body": "You have a new notification.",
}
assert body["aps"]["thread-id"] == "devplace-notification"
assert body["url"] == "/notifications"
assert body["icon"] == "/static/apple-touch-icon.png"
def test_apns_prepare_survives_an_empty_payload():
import json
from devplacepy.push import providers
body = json.loads(providers.PROVIDERS["apns"].prepare({}))
assert body["aps"]["alert"]["title"] == "DevPlace"
assert body["url"] == "/notifications"
def _apns_settings(monkeypatch, **values):
from devplacepy.push.providers import apns
defaults = {
apns.TEAM_ID_KEY: "",
apns.KEY_ID_KEY: "",
apns.AUTH_KEY_KEY: "",
apns.TOPIC_KEY: "",
apns.ENVIRONMENT_KEY: "",
}
defaults.update(values)
monkeypatch.setattr(apns, "_setting", lambda key: defaults.get(key, ""))
apns._token_state.clear()
return defaults
def _ec_private_key_pem():
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import ec
key = ec.generate_private_key(ec.SECP256R1())
return key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
).decode("utf-8")
def test_apns_is_configured_requires_every_credential(monkeypatch):
from devplacepy.push import providers
from devplacepy.push.providers import apns
provider = providers.PROVIDERS["apns"]
_apns_settings(monkeypatch)
assert provider.is_configured() is False
assert providers.is_active(provider) is False
_apns_settings(
monkeypatch,
**{
apns.TEAM_ID_KEY: "TEAMID1234",
apns.KEY_ID_KEY: "KEYID12345",
apns.AUTH_KEY_KEY: "pem",
},
)
assert provider.is_configured() is False
_apns_settings(
monkeypatch,
**{
apns.TEAM_ID_KEY: "TEAMID1234",
apns.KEY_ID_KEY: "KEYID12345",
apns.AUTH_KEY_KEY: "pem",
apns.TOPIC_KEY: "nl.molodetz.devplace",
},
)
assert provider.is_configured() is True
def test_apns_host_falls_back_to_production(monkeypatch):
from devplacepy.push.providers import apns
_apns_settings(monkeypatch)
assert apns.host() == "api.push.apple.com"
_apns_settings(monkeypatch, **{apns.ENVIRONMENT_KEY: "sandbox"})
assert apns.host() == "api.sandbox.push.apple.com"
_apns_settings(monkeypatch, **{apns.ENVIRONMENT_KEY: "nonsense"})
assert apns.host() == "api.push.apple.com"
def test_apns_provider_token_is_signed_and_cached(monkeypatch):
import jwt
from devplacepy.push.providers import apns
_apns_settings(monkeypatch)
pem = _ec_private_key_pem()
token = apns.provider_token("TEAMID1234", "KEYID12345", pem)
assert apns.provider_token("TEAMID1234", "KEYID12345", pem) == token
header = jwt.get_unverified_header(token)
claims = jwt.decode(token, options={"verify_signature": False})
assert header["alg"] == "ES256"
assert header["kid"] == "KEYID12345"
assert claims["iss"] == "TEAMID1234"
assert isinstance(claims["iat"], int)
other = apns.provider_token("TEAMID1234", "KEYID12345", _ec_private_key_pem())
assert other != token
def test_apns_provider_token_rejects_a_broken_auth_key(monkeypatch):
import pytest
from devplacepy.push.providers import apns
_apns_settings(monkeypatch)
with pytest.raises(ValueError):
apns.provider_token("TEAMID1234", "KEYID12345", "not-a-pem")
with pytest.raises(ValueError):
apns.provider_token("TEAMID1234", "KEYID12345", "not-a-pem")
def _apns_response_status(monkeypatch, status, body):
import httpx
from tests.conftest import run_async
from devplacepy.push import providers
from devplacepy.push.providers import apns
_apns_settings(
monkeypatch,
**{
apns.TEAM_ID_KEY: "TEAMID1234",
apns.KEY_ID_KEY: "KEYID12345",
apns.AUTH_KEY_KEY: _ec_private_key_pem(),
apns.TOPIC_KEY: "nl.molodetz.devplace",
},
)
provider = providers.PROVIDERS["apns"]
seen = {}
def handler(request):
seen["url"] = str(request.url)
seen["headers"] = dict(request.headers)
return httpx.Response(status, json=body)
async def run():
transport = httpx.MockTransport(handler)
async with httpx.AsyncClient(transport=transport) as client:
return await provider.deliver(
client, {"token": "a" * 64}, provider.prepare({"message": "hi"})
)
return run_async(run()), seen
def test_apns_delivery_maps_statuses(monkeypatch):
from devplacepy.push import providers
accepted, seen = _apns_response_status(monkeypatch, 200, {})
assert accepted.status == providers.ACCEPTED
assert seen["url"] == f"https://api.push.apple.com/3/device/{'a' * 64}"
assert seen["headers"]["apns-topic"] == "nl.molodetz.devplace"
assert seen["headers"]["apns-push-type"] == "alert"
assert seen["headers"]["apns-priority"] == "10"
assert seen["headers"]["authorization"].startswith("bearer ")
assert int(seen["headers"]["apns-expiration"]) > 0
assert seen["headers"]["apns-id"]
gone, _ = _apns_response_status(monkeypatch, 410, {"reason": "Unregistered"})
assert gone.status == providers.DEAD
bad_token, _ = _apns_response_status(monkeypatch, 400, {"reason": "BadDeviceToken"})
assert bad_token.status == providers.DEAD
payload_error, _ = _apns_response_status(
monkeypatch, 400, {"reason": "PayloadTooLarge"}
)
assert payload_error.status == providers.REJECTED
throttled, _ = _apns_response_status(monkeypatch, 429, {"reason": "TooManyRequests"})
assert throttled.status == providers.REJECTED
def test_apns_delivery_without_configuration_never_raises(monkeypatch):
import httpx
from tests.conftest import run_async
from devplacepy.push import providers
_apns_settings(monkeypatch)
provider = providers.PROVIDERS["apns"]
async def run():
transport = httpx.MockTransport(lambda request: httpx.Response(200, json={}))
async with httpx.AsyncClient(transport=transport) as client:
return await provider.deliver(client, {"token": "a" * 64}, "{}")
assert run_async(run()).status == providers.REJECTED
def test_group_by_provider_treats_a_missing_provider_as_webpush():
from devplacepy.push.delivery import group_by_provider
grouped = group_by_provider(
[
{"id": 1, "provider": None},
{"id": 2, "provider": ""},
{"id": 3, "provider": "webpush"},
{"id": 4, "provider": "apns"},
]
)
assert sorted(grouped) == ["apns", "webpush"]
assert len(grouped["webpush"]) == 3
assert len(grouped["apns"]) == 1
def test_delivery_timeout_is_clamped(monkeypatch):
from devplacepy.push import delivery
monkeypatch.setattr(delivery, "get_int_setting", lambda key, default: default)
assert delivery.timeout_seconds() == float(delivery.DEFAULT_TIMEOUT_SECONDS)
monkeypatch.setattr(delivery, "get_int_setting", lambda key, default: 0)
assert delivery.timeout_seconds() == float(delivery.MIN_TIMEOUT_SECONDS)
monkeypatch.setattr(delivery, "get_int_setting", lambda key, default: 100000)
assert delivery.timeout_seconds() == float(delivery.MAX_TIMEOUT_SECONDS)

117
tests/unit/services/push.py Normal file
View File

@ -0,0 +1,117 @@
# retoor <retoor@molodetz.nl>
from datetime import datetime, timedelta, timezone
from devplacepy.database import get_table
from devplacepy.push import store
from devplacepy.services.push import PushService
from devplacepy.utils import generate_uid
def _registration(user_uid, deleted_at=None, provider="webpush"):
record = {
"uid": generate_uid(),
"user_uid": user_uid,
"provider": provider,
"endpoint": f"https://push.example.com/{generate_uid()}",
"key_auth": "a",
"key_p256dh": "p",
"created_at": datetime.now(timezone.utc).isoformat(),
"deleted_at": deleted_at,
}
get_table("push_registration").insert(record)
return record
def test_config_fields_cover_every_provider(local_db):
keys = [field.key for field in PushService().all_fields()]
for expected in (
"service_push_enabled",
"push_dead_retention_days",
"push_delivery_timeout_seconds",
"push_webpush_enabled",
"push_webpush_subject",
"push_apns_enabled",
"push_apns_team_id",
"push_apns_key_id",
"push_apns_auth_key",
"push_apns_topic",
"push_apns_environment",
):
assert expected in keys
def test_apns_auth_key_field_is_a_masked_secret(local_db):
fields = {field.key: field for field in PushService().all_fields()}
auth_key = fields["push_apns_auth_key"]
assert auth_key.secret is True
assert auth_key.type == "text"
assert auth_key.display_value() == ""
def test_run_once_prunes_only_stale_dead_rows(local_db, monkeypatch):
from devplacepy.services import push as push_service
user_uid = f"prune_{generate_uid()}"
live = _registration(user_uid)
fresh_dead = _registration(
user_uid,
deleted_at=(datetime.now(timezone.utc) - timedelta(days=1)).isoformat(),
)
stale_dead = _registration(
user_uid,
deleted_at=(datetime.now(timezone.utc) - timedelta(days=90)).isoformat(),
)
service = PushService()
monkeypatch.setattr(
push_service.service, "get_int_setting", lambda key, default: 30
)
from tests.conftest import run_async
run_async(service.run_once())
registrations = get_table("push_registration")
assert registrations.find_one(uid=live["uid"]) is not None
assert registrations.find_one(uid=fresh_dead["uid"]) is not None
assert registrations.find_one(uid=stale_dead["uid"]) is None
def test_run_once_with_retention_disabled_prunes_nothing(local_db, monkeypatch):
from devplacepy.services import push as push_service
from tests.conftest import run_async
user_uid = f"keep_{generate_uid()}"
stale_dead = _registration(
user_uid,
deleted_at=(datetime.now(timezone.utc) - timedelta(days=900)).isoformat(),
)
service = PushService()
monkeypatch.setattr(push_service.service, "get_int_setting", lambda key, default: 0)
run_async(service.run_once())
assert get_table("push_registration").find_one(uid=stale_dead["uid"]) is not None
def test_metrics_count_live_rows_per_provider(local_db):
user_uid = f"metrics_{generate_uid()}"
_registration(user_uid)
_registration(user_uid, provider="apns")
metrics = PushService().collect_metrics()
assert metrics["webpush_active"] >= 1
assert metrics["apns_active"] >= 1
assert metrics["webpush_ready"] == 1
assert "dead" in metrics
def test_store_counts_treats_a_missing_provider_as_webpush(local_db):
user_uid = f"legacy_{generate_uid()}"
record = _registration(user_uid)
get_table("push_registration").update(
{"id": get_table("push_registration").find_one(uid=record["uid"])["id"], "provider": None},
["id"],
)
assert store.counts().get("webpush", 0) >= 1