devplacepy/devplacepy/services/nodes/CLAUDE.md

This file documents the community node network (devplacepy/services/nodes/, devplacepy/routers/nodes/): members opt in to donate a little of their connection so search can reach sites that block datacenter addresses. Claude Code auto-loads it whenever a file under devplacepy/services/nodes/ is read or edited. The design record, with the measured and unmeasured claims separated, is plan.md at the repository root.

What exists today

Phase 1 (the specification), Phase 2 (the data layer and usage ledger) and Phase 3 (registry, session, consent gate, kill switch, device and ticket routes, the lock-owner socket) are built. No job traffic flows yet. Everything else is planned and listed under "Gaps" next to the phase that closes it. Nothing here is enabled for any account.

Piece	File	State
Protocol specification	spec.py	built, tested
GET /nodes/spec.json	routers/nodes/spec.py	built, tested, public
Generated protocol docs page	templates/docs/nodes-protocol.html	built, reads the nodes_spec Jinja global
API docs group	docs_api/groups/nodes.py	built, one endpoint
Tables and helpers	database/nodes.py, NODES_COLUMNS in database/schema.py	built, tested, no route uses them yet
Usage ledger	ledger.py (UsageLedger, DonationSummary)	built, tested
bandwidth_sharing consent kind	CONSENT_KINDS in database/moderation.py	registered, never granted by default, enforced by ConsentGate
Frame codec	frames.py (decode, encode, FrameError)	built, tested, validates every frame against spec.py
Consent choke point	consent.py (ConsentGate, Verdict)	built, tested, single and batch
Socket session	session.py (NodeSession)	built, tested with a fake socket and a real one
Registry and counters	registry.py (NodeRegistry, NodeCounters)	built, tested
Supervisor	service.py (NodesService)	built, registered in main.py, default enabled
Device, ticket and socket routes	routers/nodes/devices.py, ticket.py, ws.py	built, tested
nginx upgrade block for /nodes/ws	nginx/nginx.conf.template	added

The specification is the contract

spec.py is the single source of truth. The server, the Go client, the browser extension, the docs page and the tests all read it. Nothing may restate a number, a frame field, an allowlisted host or a refusal code anywhere else: import it, or read it from /nodes/spec.json.

• ProtocolSpec holds version, min_supported, LimitSpec, FrameSpec tuple, DestinationSpec tuple and RefusalSpec tuple. Every one is a frozen dataclass. SPEC.describe() is the JSON form and the only thing the router returns.
• open_proxy is field(default=False, init=False): it cannot be set through the constructor. There is no open proxy mode and there will not be one. Changing that is a protocol change that needs a new PROTOCOL_VERSION, not a flag.
• ALLOWED_PORTS is (80, 443). A DestinationSpec may narrow it, never widen it (a unit test fails otherwise).
• DestinationSpec.matches is suffix-safe: a host matches when it equals the suffix or ends with . plus the suffix, so evilreddit.com and reddit.com.evil.net never match reddit.com.
• LimitSpec is the list of hard caps from the plan. Nothing is unbounded and nothing autotunes: a constant you can read beats magic you cannot debug. A number an admin may lower becomes a ConfigField later; the ceiling stays here.
• A frame type is unique per direction. An unknown type, a missing required field, an unexpected field or an oversize frame is a protocol error that closes the socket loudly, never an ignore.

The docs page nodes-protocol.html renders its tables from the nodes_spec Jinja global (templating.py, NODES_SPEC.describe()), so it cannot drift from the code. tests/api/nodes/spec.py asserts that every frame, destination, refusal code and limit name appears on the rendered page.

Decisions taken (recorded so they are not re-litigated)

Id	Decision
D1	devplacepy owns the node fabric; rsearch is a consumer, not the home
D2	the native client is Go, one static binary per OS and architecture
D3	the extension ships as a sideloadable zip first, store submission later
D4	rewards are XP, badges and a leaderboard only: no coins, no money, nothing purchasable
D5	donors must be adults (age_band)
D6	a datacenter address is not a donor: residential and mobile only, admin override

Data layer (Phase 2)

Seven tables, all ensured with their full column set in init_db (_ensure_nodes_tables): node_devices, node_settings, node_usage_daily, node_donors, node_flags (soft-deletable, in SOFT_DELETE_TABLES and UNREPORTABLE_TABLES with a reason), and node_requests, node_events (ephemeral, hard garbage collection only). Account deletion cascades through OWNED_TABLES for devices, settings, donors and daily usage.

• Atomic by one statement. The device cap is an INSERT ... SELECT ... WHERE (SELECT COUNT(*)...) < max, the daily bandwidth cap is an UPDATE ... WHERE COALESCE(bytes_up,0)+COALESCE(bytes_down,0)+:n <= :cap, and the daily XP cap is a conditional UPDATE that rolls the day over inside the same statement. Verified with real separate processes against fresh, unseeded rows (40 racers on a 10-job cap gave exactly 10, 24 device creations gave exactly 5, 30 XP claims gave exactly the cap).
• record_job is one transaction: the daily bucket (INSERT OR IGNORE then the conditional UPDATE) and the lifetime node_donors upsert commit together or not at all. A job refused by the cap counts nothing. A failed job counts requests_failed and never moves bytes or lifetime totals.
• Lifetime totals are maintained at write time, so pruning the daily buckets (usage_daily_retention_days) can never lower them. Prune functions take dry_run and use only the constants in LimitSpec; there is no argument that lowers a retention floor.
• Secrets: dpn_ plus 32 random bytes, shown once at create or rotate, stored as sha256 plus a 12 character lookup prefix, compared with hmac.compare_digest. IP addresses are stored only as a keyed hash (hash_ip).
• Settings are clamped on write and on read (normalise_settings): bandwidth to 1..max_donor_mb_per_day, concurrency to 1..max_concurrent_per_node, hours must be HH:MM, platforms are intersected with the spec's adapters. A member with no row is opted out.
• Site settings seeded in init_db: nodes_enabled (0), nodes_max_total, nodes_daily_quota_user, nodes_daily_quota_guest, nodes_cache_ttl_seconds, nodes_social_via_nodes (0). Like every seed, consumers pass the default to get_setting because the seed does not run on an empty database.

Connecting a node (Phase 3)

1. A member consents to bandwidth_sharing and turns donation on for themselves (the profile UI for this is Phase 9; today it is POST /profile/{username}/consent and save_node_settings).
2. POST /nodes/devices registers a device and returns its dpn_ secret once. GET /nodes/devices, POST /nodes/devices/{uid}/rotate and /delete manage it. All are require_user_api (401, never a redirect) and answer Cache-Control: no-store.
3. The node calls POST /nodes/ticket with Authorization: Bearer dpn_.... A session cookie or a member API key is not a device secret and gets 401. The ticket is a ws_tickets row with scope="nodes" and the device uid, valid ticket_ttl_seconds, redeemed by one conditional UPDATE (16 processes racing one ticket gave exactly one winner; a ticket of the wrong scope never redeems).
4. The node opens WS /nodes/ws?ticket=.... The route only runs on the service-lock owner (4013 elsewhere, the client retries fast, the /pubsub/ws precedent). Order of refusal: service disabled 1013, not the owner 4013, bad ticket or device or status 1008, consent gate 1008 with the reason, capacity 4029, then the handshake.
5. hello must arrive within hello_timeout_seconds, with a supported protocol, a kind equal to the registered device kind and a non-empty list of known modes. The server answers welcome with the full limits and allowlist from the spec. A second connection of the same device kicks the first with replaced.
6. The session pings every ping_seconds; any frame from the client resets the missed counter and missed_pings_before_evict unanswered pings close it with heartbeat. usage frames are advisory and only summed on the session. Every other client frame is a protocol error until Phase 4 assigns jobs.

The consent choke point

ConsentGate is the only place that decides whether a member's node may exist. allows_many(user_uids) decides a whole set with a fixed number of queries and allows(uid) is the one-element case of it, so the single and batch answers cannot diverge (a test compares them). Order of reasons: kill_switch, no_account, deletion_pending, account_inactive, suspended, not_adult, no_consent, not_enabled. It is called at device registration, ticket issue, socket accept and every supervisor tick. An account whose age_band is empty (created before age bands existed) is refused as not adult, loudly, rather than assumed adult.

NodesService.reconcile runs every reconcile_seconds, asks the gate once for all connected owners and loads all devices in one query, then kicks with a reason: kill_switch, revoked, suspended, or the gate's reason. This is why withdrawing the consent, switching donation off, deleting the device or flipping nodes_enabled takes effect within one tick even when the request landed on a different worker than the one holding the sockets. Measured (scratch SQLite, this machine, 2000 synthetic sessions): one tick costs about 110 ms; the per-session version of the same check cost about 1 ms per session, so 488 ms at 500 sessions and an extrapolated 2 s at 2000, which would have blocked the event loop for the whole tick. Do not reintroduce a per-session query in reconcile.

Withdrawal is member-only: MEMBER_ONLY_CONSENTS in database/moderation.py lists consents that Devii (header X-Devii-Agent: 1, resolved in routers/profile/consent.py) may withdraw but never grant, and the acceptance service lists bandwidth_sharing in EXCLUDED_CONSENTS.

Two transports, two trust levels

tunnel (native client) relays raw TCP so TLS runs end to end between DevPlace and the destination: the donor cannot read or forge content. fetch (browser extension, which has no raw sockets) lets the donor read and forge the response, so every fetch result is verified and labelled. A plain web page or a service worker cannot do either (CORS, no sockets) and is out of scope for good.

Gaps (what the next phases close)

• Phase 2 leftovers: the bandwidth_sharing consent can currently be granted through POST /profile/{username}/consent and by Devii's set_consent (Devii calls that route as the user). Nothing reads it yet, so it is inert, but Phase 3 must make Devii unable to grant it (Devii may read and withdraw) at the same time as ConsentGate lands. No XP or badge is awarded yet (UsageLedger.claim_xp only reserves the daily budget); the privacy page and app-store declarations are updated with the first phase that lets a device connect, because until then no personal data is collected.
• Phase 3 leftovers: country, asn and asn_class on a device are never filled (they stay unknown), so the datacenter exclusion (D6) is not enforced yet; it needs the offline ASN database from Phase 4. last_seen_at is written on connect only, not per heartbeat, so a device list cannot yet show who is online right now. The socket counters are per process and are shown in the service metrics only. The devplace.net front proxy idle timeout for a long-lived socket is unmeasured.
• Phase 4: broker, scheduler, dispatcher, destination policy, circuit breakers, result cache.
• Phase 5 onward: the reference client, adapters, the Go client, the extension, the profile UI, Devii tools, the CLI, the privacy and donor-terms pages, the legal gate.
• Not measured yet: Reddit and X success rates per address class, WebSocket idle timeouts through pravda.education and devplace.net, dispatcher tick cost against SQLite, socket capacity of one worker. The caps in LimitSpec are starting ceilings, not measurements.

Rules for work in this directory

• Every user-facing or protocol-facing claim in the docs must describe what runs today. A page for an unbuilt feature states its status at the top, as nodes-protocol.html does.
• A new consent is enforced at one choke point, never at call sites (see services/moderation/CLAUDE.md).
• A new table here is either in SOFT_DELETE_TABLES or documented as derived and hard-garbage-collected; it is either in REPORTABLE_TARGETS or in UNREPORTABLE_TABLES with a reason.
• The machine may suspend a node (reversible, timed). It may never ban a user or delete a ledger row on its own: it raises a flag for a human.