devplacepy/plan.md

See research_residental_proxies.md. Userss are by default opted out, but can opt in and download required tools to make it happen on their profile page. We label this as donating to the devplace community ecosystem. We have a beautiful infrastructure that is very reliable, barely costs maintenance and even better, almost free. Our search engine destroys searchxng. Also, our search engine has actually a fallback provided by jina. I bough jina in someday but then decided not to use their stuff what ended up as a nice backup system for failing searches. But what do we want? We also want reddit, X and all other social media to be part of our search results. And that is only possible when people contribute some of their resource. There are severa ways to do this. Choice is to members. We support all common options and you can configure yourself how much resource you want to donate. We will also count the amunt of data i/o / request count and display it as donated by you. Also! For the sake of our new agent system on pravda.education, we really needs this residental proy stuff to happen. Our platform destroys grok atm, but grok has the social media searches and they are important.

Please create a huge implementation plan of literally everything that needs to happen at the end of this file. Thats literally all for now.

---

IMPLEMENTATION PLAN: Community Node Network ("Donate to the DevPlace Ecosystem")

Status: plan only, nothing built. Written against the repo as of 2026-10-14 and against research_residental_proxies.md. Working name: Nodes. Code lives in devplacepy/services/nodes/, routers in devplacepy/routers/nodes/, user-facing label "Donate to the community".

0. Findings that change the shape of the plan

These come from reading the repo and the research file. Several of them contradict the framing in the brief, so they come first.

1. There is no Jina code in this repo. grep -ri jina matches only plan.md. The search engine is the external service https://rsearch.app.molodetz.nl (crawl.RSEARCH_URL, services/devii/rsearch/). The Jina fallback therefore lives in the rsearch codebase, not here. This repo only consumes rsearch from two places: services/jobs/deepsearch/crawl.py::search_queries and the Devii rsearch* tools. Where social search is implemented (here or in rsearch) is open decision D1.
2. The repo already knows social sites block it. crawl.HOSTILE_DOMAINS lists x.com, reddit.com, instagram, facebook, linkedin, tiktok, threads, youtube and skips them. Root CLAUDE.md records that reddit blocks the datacenter ASN outright and that the only fix is an IP change. So the problem is real and measured for Reddit. It is not measured for the others.
3. A residential IP fixes IP-reputation blocks, not login walls. Reddit public JSON endpoints work from a clean IP without login. X search, Instagram, LinkedIn, Facebook and TikTok mostly need a logged-in session. Donor nodes must NOT use the donor's cookies (privacy and liability), so those platforms stay mostly out of reach. X is limited to what is public (permalinks, oEmbed/syndication timelines). Promising "Reddit, X and all social media" would be a false claim. Phase 0 measures what actually works per platform before anything else is built.
4. Several "social" sources need no donors at all: Hacker News (Algolia API), Bluesky (public AppView), Mastodon (public instance APIs), Lobsters, StackExchange, GitHub, dev.to, YouTube oEmbed/transcripts. They are cheap, reliable, and cover a large part of what developers want. Build these as direct adapters first (Phase 6A). Nodes are only for the platforms that block datacenter IPs (Reddit first).
5. Browser extensions cannot do TCP. MV3 has no raw sockets. An extension can only run fetch(). So there are two fundamentally different transports, with different trust properties:
   • Tunnel mode (native client): the server opens an end-to-end TLS session to the destination THROUGH the donor's TCP socket. The donor sees only ciphertext plus the destination host. The donor cannot read or tamper with content because they cannot forge the destination's certificate. Our curl_cffi Chrome fingerprint stays intact. This is the better mode.
   • Fetch mode (extension): the donor's browser performs the HTTP request and returns the body. The donor sees the URL and plaintext response and CAN forge the response. Everything returned is untrusted and must be verified (section 5.6). TLS fingerprint is the donor's browser, which is fine.
6. A plain web page or service worker cannot do this at all. The research table rates them "Low"/"Poor". The reality is "impossible": a page cannot fetch() reddit.com cross-origin (CORS), and cannot open raw sockets. Both are dropped from scope. Documented so nobody revisits them.
7. Open SOCKS5 is the wrong product. The research recommends exposing a SOCKS5 listener. We do not need an open proxy. We need to reach a short, fixed list of public social domains for our own search. A destination-allowlisted exit (donor hard-refuses everything else) removes most of the abuse and liability surface the research file warns about (credential stuffing, ad fraud, illegal traffic). This is the single most important design decision and it is a clear rule, not a configurable maybe: no open proxy, ever; allowlist enforced on server AND on the donor client.
8. Existing seams to reuse, none to rebuild:
   • stealth.stealth_async_client(proxy=...) and outbound_proxy_url: a local node-backed forward proxy plugs into this with zero caller changes.
   • messaging/tickets.py + ws_tickets: single-use WS auth ticket flow (browsers cannot set WS headers).
   • pubsub lock-owner WS pattern (4013 retry) for the node socket.
   • BaseService + ConfigField for the dispatcher service and admin knobs.
   • LiveViewRelayService.VIEWS + private user.{uid}.* pub/sub namespace for live donor stats.
   • user_consents / CONSENT_KINDS for opt-in (default opted out is already how ai_third_party works).
   • utils.track_action / ACHIEVEMENTS / BADGE_CATALOG for donor badges.
   • GatewayUsageLedger.record_external pattern for metering consumers.
   • devii CONFIRM_REQUIRED, docs_api.endpoint(), DOCS_PAGES, audit categories + events.md.
9. Legal and ToS exposure is real and is risk number one. Reddit and X actively litigate and rate-limit scraping; ISPs restrict commercial use of residential lines; donors' IPs will get rate-limited or banned by the platforms. The plan includes a legal review gate (Phase 11) before any public launch. This is not legal advice and the plan does not assume the outcome.
10. Unknown that blocks scoping: what the "new agent system on pravda.education" is and how it will call search. The plan gives it a stable internal interface (section 6) but its requirements (latency, volume, platforms) are open question Q3.

1. Decisions needed from you before Phase 2 (everything else proceeds without them)

#	Decision	Recommendation	Why
D1	Where does social search live: in devplacepy, or in the rsearch codebase?	devplacepy owns the node fabric and exposes an internal HTTP face; rsearch calls it (or devplacepy does the social search itself)	Nodes are tied to DevPlace accounts, consent, and the ledger. rsearch has no accounts.
D2	Native client language	Go, static binary, one file per OS/arch	Research recommends Go/Rust; zero runtime for Windows users. Adds a Go toolchain to the build. Alternative: Python zipapp (no new toolchain, but needs Python on the donor machine).
D3	Extension distribution	Sideloadable zip first (audience is developers), store submission later	Stores scrutinize proxy-like extensions; sideload ships sooner and the audience can handle it.
D4	Donation reward	XP + badges + leaderboard only. No coins, no money, no purchasable anything	Root rule: no payment path, no purchasable randomness. Keeps app-store conditionals untriggered.
D5	Minimum donor age	Adults only (age_band gate)	A minor must not share an IP/line. Needs a decision on how the existing age_band is read for legacy accounts with NULL.
D6	Is a data center or a cloud VPS allowed as a "donor"?	No: residential/mobile-ASN heuristic, admin override	The whole value is residential reputation. A VPS donor is just another datacenter IP.

2. Specification first (your rule 7: everything has a declared spec)

Nothing else is written until the spec exists, because the server, the Go client, the extension, the docs and the tests all derive from it.

1. devplacepy/services/nodes/spec.py: the single source of truth, as frozen dataclasses (FrameSpec, LimitSpec, DestinationSpec, AdapterSpec, ProtocolSpec). It declares:
   • protocol version (integer, with min_supported),
   • every frame type with its fields, direction, max size,
   • every hard cap in section 3,
   • the destination allowlist (domain suffix -> adapter -> allowed methods -> allowed ports -> rate limit),
   • donor-side hard refusals (non-public IPs, ports other than 80/443, non-allowlisted hosts).
2. GET /nodes/spec.json serves the spec so clients and tests read the same numbers. A contract test fails if the Go constants, the extension constants, the docs page, or the server drift from it.
3. devplacepy/templates/docs/nodes-protocol.html (generated tables from the spec, not hand-copied) and docs/nodes.html (user-facing). Registered in DOCS_PAGES.
4. A short NODES.md is NOT created; the nested devplacepy/services/nodes/CLAUDE.md is the engineering record, per repo convention.

3. Hard caps and floors (nothing unbounded)

All values below are starting ceilings, declared in spec.py and surfaced as ConfigFields only where an admin may lower them. Static ceilings, no autotuning.

Cap	Initial value	On hit
Devices per user	5	409 with reason
Connected nodes total (lock-owner worker)	2000	WS close 4029, counter nodes_rejected_capacity
WS frame size	256 KiB	close 1009, counter
Response body per job	2 MiB	truncate + flag truncated=true, never silently
Concurrent jobs per node	2 (donor may lower to 1)	scheduler skips node
Per node, per destination domain	min 6 s between requests	scheduler skips node
Global per destination domain	admin-set rps ceiling	queue, then 429 to caller
Job timeout	25 s	mark failed, count against node health
Job queue depth	500	reject new with 429 + counter
Caller wait	15 s default, 60 s max	FabricUnavailable
Per caller-owner daily fabric requests	admin-set (default 200 user, 0 guest)	429, no donor load
Donor daily bandwidth (donor-configurable)	default 500 MB, max 20 GB	node goes idle until reset
Heartbeat	ping every 20 s, evict after 2 missed	reconnect
Reconnect backoff	1 s to 60 s, full jitter	n/a
Ticket TTL	30 s single use (existing)	n/a
Result cache	10 min TTL, 256 MiB total, 4 MiB per entry	LRU evict
Usage daily buckets	keep 400 days, rolled into lifetime totals first	prune only after rollup verified
Node event log	30 days, 50k rows	prune

Floors (machines hold, humans decide):

• The system may auto-suspend a misbehaving node (reversible, timed, e.g. circuit breaker after N consecutive failures or an integrity mismatch). It may never auto-ban a user or delete a ledger row. Hitting a ban threshold creates a flag for an admin and fires an admin notification.
• Lifetime donation totals live on the donor row and are never decreased by pruning.
• Protected destinations: the allowlist can only be changed by the primary admin and every change is audited.
• A global kill switch (nodes_enabled site setting) disconnects every node within one tick. It is the first thing built in Phase 4 and tested first.

4. Data model

All tables ensured in database/schema.py::init_db (full column set, because dataset only creates the columns of the first insert), indexes with CREATE INDEX IF NOT EXISTS, every INSERT writes deleted_at: None, deleted_by: None where the table is soft-deletable. Helpers in a new database/nodes.py, exported through database/__init__.py like skills.py.

Table	Purpose	Soft delete?	Notes
node_devices	one row per registered client install: uid, user_uid, label, kind (native/extension/container), secret_hash, secret_prefix, platform, client_version, protocol_version, status (active/suspended/revoked), suspended_until, suspend_reason, country, asn, asn_class (residential/mobile/datacenter/unknown), ip_hash, last_seen_at, created_at	yes (revoke = soft delete, cascade stamp)	index (user_uid, deleted_at), (status, last_seen_at)
node_settings	per-user donation prefs: user_uid (unique), enabled, bandwidth_mb_day, max_concurrent, active_from, active_to, timezone, allowed_platforms (JSON), pause_on_battery, updated_at	no (preference row)	enabled is subordinate to the consent, like ai_correction_enabled
node_usage_daily	per (device_uid, day) counters: requests_ok, requests_failed, bytes_up, bytes_down, cache_hits_served, integrity_failures	no (derived, hard GC)	unique (device_uid, day); upserted with one atomic INSERT ... ON CONFLICT DO UPDATE SET col = col + excluded.col (the _add_usage precedent)
node_donors	lifetime totals per user: user_uid (unique), requests_ok, bytes_total, first_donated_at, last_donated_at, xp_awarded_today, xp_day	no (derived)	survives bucket pruning; this is what the profile card and leaderboard read
node_requests	the cross-worker job queue: uid, caller_kind, caller_id, url, method, adapter, mode (tunnel/fetch/any), status (queued/assigned/done/failed/expired), device_uid, result_status, result_bytes, result_body (capped), error, created_at, assigned_at, finished_at	no (ephemeral, hard GC after 10 min)	partial index on status IN ('queued','assigned')
node_events	operational log: connects, disconnects, suspensions, verification results, refusals	no (hard GC, 30 days)	feeds admin view, not audit
node_flags	items needing a human: integrity mismatch bursts, datacenter-ASN donor, abuse reports against a node	yes	admin resolves; the floor rule's landing place

Related registrations (the repo fails the suite otherwise):

• database/moderation.py: none of these carry user-generated content visible to others, so each goes in UNREPORTABLE_TABLES WITH its reason. node_devices.label is owner-only text; confirm it is never rendered to anyone else, or it becomes reportable.
• SOFT_DELETE_TABLES: node_devices, node_flags.
• services/moderation/deletion.py::OWNED_TABLES: add node_devices, node_settings, node_donors so account deletion cascades. Add node_usage_daily via the device join. Revoke live sockets in the same call.
• user_consents: new CONSENT_KINDS["bandwidth_sharing"]. Never granted by default, never backfilled. A new consent is enforced at ONE choke point (section 5.3), never at N call sites.
• Site settings seeded in init_db operational_defaults: nodes_enabled (default 0), nodes_max_total, nodes_daily_quota_user, nodes_daily_quota_guest, nodes_cache_ttl_seconds, nodes_social_via_nodes (route hostile domains through nodes: default 0).

5. Server

5.1 Package layout (OO, one responsibility per class, one class per file where it is a class)

devplacepy/services/nodes/
  CLAUDE.md                 engineering record (written in Phase 12, started in Phase 3)
  spec.py                   ProtocolSpec and friends (section 2)
  registry.py               NodeRegistry: connected sessions, capacity, lookup by uid/user
  session.py                NodeSession: one WS, frame codec, in-flight jobs, health, heartbeat
  frames.py                 Frame encode/decode + validation against spec
  consent.py                ConsentGate: THE single choke point (5.3)
  policy.py                 DestinationPolicy: allowlist, port/IP rules, per-domain limits
  scheduler.py              NodeScheduler: pick a node (fairness, diversity, caps)
  broker.py                 JobBroker: enqueue, assign, await result, expire (DB-backed)
  dispatcher.py             NodeDispatcher: lock-owner loop moving node_requests to sessions
  tunnel.py                 TunnelBridge: local forward-proxy listener -> tunnel frames
  fetcher.py                FetchBridge: fetch-mode request/response framing
  verifier.py               NodeVerifier: canary, ASN class, redundancy checks
  ledger.py                 UsageLedger: atomic counters, rollups, XP award hook
  cache.py                  ResultCache: bounded, URL-hash keyed
  circuit.py                CircuitBreaker: per node and per destination
  fabric.py                 NodeFabric: the facade every consumer calls
  service.py                NodesService(BaseService): reconcile, metrics, prune, relay hook
  adapters/
    base.py                 SearchAdapter (abstract): build_request, parse_response, normalize
    reddit.py  x.py  ...    one class per platform
    direct/                 hackernews.py bluesky.py mastodon.py lobsters.py stackexchange.py

5.2 Routers (devplacepy/routers/nodes/ package, mounted at /nodes in main.py)

Route	Auth	Purpose
GET /nodes/spec.json	public	protocol spec for clients and tests
GET /nodes/client-version	public	latest client versions, min_protocol, sha256 per artifact
GET /nodes/download/{artifact}	public	native binaries, extension zip, systemd unit, container compose (static files from DATA_DIR/node_clients/, sha256 listed)
POST /nodes/devices	require_user + consent	create device, returns the secret ONCE
POST /nodes/devices/{uid}/rotate	owner	new secret, old revoked immediately
POST /nodes/devices/{uid}/delete	owner or admin	soft delete + kick socket
POST /nodes/ticket	device secret	exchange secret for a 30 s single-use WS ticket
WS /nodes/ws	ticket	the node socket; lock-owner gated, 4013 fast retry like /pubsub/ws
POST /nodes/internal/fetch	admin / API key, service accounts	the HTTP face of NodeFabric.fetch for agent systems
GET /nodes/stats	public	aggregate only: total donated bytes, requests, active donors (no per-user data)
GET /nodes/leaderboard	public	top donors, opt-in display
POST /profile/{username}/donation	owner	settings (bandwidth, hours, platforms)
GET /admin/nodes, POST /admin/nodes/...	admin	pool health, flags, suspend/unsuspend, allowlist, kill switch

Rules from the repo applied to every route: respond(request, template, ctx, model=XOut) for the HTML+JSON faces, *Out schemas in schemas/nodes.py (a context key missing from XOut is silently dropped from JSON), Pydantic Form models in models.py, RedirectResponse(..., status_code=302) literal, no respond() ctx key colliding with a Jinja global (use viewer_is_admin), specific paths before catch-alls, is_senior_admin on every per-user admin mutation, delete = owner OR admin.

5.3 The consent choke point (ConsentGate)

One function, ConsentGate.allows(user_uid) -> Verdict(reason), true only when ALL of: bandwidth_sharing consent granted and not withdrawn, account active (not suspended, not in deletion grace), adult age_band (D5), node_settings.enabled, global nodes_enabled. It is called in exactly three places:

1. POST /nodes/devices and POST /nodes/ticket (refuse to mint).
2. WS /nodes/ws accept (refuse to connect).
3. NodeScheduler.eligible() (a node whose owner stopped qualifying is never assigned work, and NodeRegistry kicks the socket on the next tick).

Withdrawing the consent takes effect within one dispatcher tick, not at next reconnect. Consent change is owner-only, exactly like routers/profile/consent.py (reuse _owner_only). Devii may read status and may withdraw, but must not be able to grant (a chat agent granting a bandwidth-sharing consent is not a consent). Verify how the existing set_consent Devii tool treats grants and restrict this kind there.

5.4 WebSocket and multi-worker model

• /nodes/ws is gated to the service-lock owner worker with the 4013 fast-retry close, so all node sockets converge on one process (precedent: /pubsub/ws, /devii/ws). This keeps NodeRegistry a single in-memory truth. Trade-off, stated: one worker carries all sockets, hence the 2000 cap, and a crash drops every node for a few seconds until they reconnect. Acceptable because reconnect is built in.
• Callers on ANY worker enqueue into node_requests (DB). The lock-owner NodeDispatcher ticks fast while at least one node is connected and a request is queued (target 200 ms, indexed partial query; the cost is measured in Phase 0, not assumed), assigns, and writes the result back. Callers await by short-sleep polling the row. A same-process fast path skips the table when the caller is already on the owner. This is the JobService philosophy (DB-backed, durable enough, no inter-worker RPC) with a tighter tick.
• Large bodies never go through SQLite long term: result_body is capped (2 MiB) and hard-GC'd after 10 minutes; the durable copy is the ResultCache entry, bounded in bytes.
• nginx: add an upgrade location = /nodes/ws to nginx/nginx.conf.template (the catch-all location / strips upgrade headers, and the file already has one per WS route). Also verify on the devplace.net front host, whose own proxy and SSH tunnel have separate idle timeouts; test both hostnames per the production-diagnosis procedure, never with --resolve.

5.5 Frame protocol (summary; the authoritative version is spec.py)

Client to server: hello{protocol, client_version, kind, platform, caps, limits}, pong, job_result{job_id, status, headers, body_b64, truncated, timings}, job_error{job_id, code}, tunnel_data{conn_id, b64}, tunnel_close{conn_id}, usage{bytes_up, bytes_down}. Server to client: welcome{device_uid, limits, allowlist_version, settings}, ping, job{job_id, method, url, headers, timeout}, tunnel_open{conn_id, host, port}, tunnel_data, tunnel_close, settings_update, kick{reason}. Every frame is validated against the spec on both ends. An unknown type or an oversize frame closes the socket with a loud reason and a counter (never silently ignored).

5.6 Trust, verification and integrity

Fetch-mode donors can forge content, and forged content flows into search results and into the AI agent. Controls, in order of strength:

1. Prefer tunnel mode whenever a tunnel-capable node is available: TLS makes tampering impossible. Fetch mode is only used for extension donors.
2. Never trust fetch-mode bodies without checks: structural validation by the adapter (must parse as the platform's JSON/HTML shape), size sanity, and sampled redundancy: a configured fraction of fetch-mode jobs (plus 100 % for a node's first N jobs) are also fetched through a second independent node or directly; a content-hash or structural mismatch increments integrity_failures, suspends the node, and writes a node_flags row.
3. Provenance on every result: results carry via=tunnel|fetch|direct and the agent/search layer can down-weight or label fetch-mode content.
4. Canary / liveness proof: on connect, the verifier makes the node fetch GET /nodes/canary/{nonce} (served by us, records the observed source IP). Proves egress really goes out of the donor's own line, detects a node that is itself behind another proxy, and lets us compare the observed IP with the WS peer IP.
5. ASN class: classify residential/mobile/datacenter via an offline ASN database file under DATA_DIR (no third-party lookup, so no donor IP leaves our server). Datacenter donors get asn_class=datacenter and are excluded from the pool by default (D6) with a flag for an admin.
6. Donor client strips cookies and auth: extension uses credentials: "omit" and never attaches the donor's session; native client sends only the headers the server specified.

5.7 Donor-side safety is enforced on the donor, not only on the server

A compromised or buggy server must not be able to point a donor's machine at their LAN. The clients enforce, independent of any server frame:

• allowlisted hosts only (list baked in per release and refreshed from the server, where a refreshed list can only be equal or narrower than the baked-in superset),
• ports 80 and 443 only,
• resolve DNS locally, refuse private/loopback/link-local/CGNAT/multicast addresses, then connect to the resolved IP (defeats DNS rebinding),
• no redirects to a non-allowlisted host,
• the donor's configured bandwidth, concurrency and schedule caps, applied locally even if the server sends larger ones.

6. Consumers: how search gets the data

NodeFabric is the only thing consumers import:

NodeFabric.fetch(url, *, owner, mode="any", cache=True, timeout=...) -> FabricResult
NodeFabric.social_search(query, *, platforms, owner, limit) -> list[SocialHit]
NodeFabric.available() -> FabricStatus   # nodes online, per-platform health

Resolution order (each step loud when it fails, never silent):

1. ResultCache hit.
2. Direct adapters for platforms that need no donors (Hacker News, Bluesky, Mastodon, Lobsters, StackExchange, GitHub, dev.to).
3. Nodes for allowlisted blocked platforms (Reddit first), tunnel mode preferred.
4. Configured outbound_proxy_url if the admin has one.
5. rsearch (existing, includes its own Jina fallback).
6. FabricUnavailable with a reason code; the caller reports "social results unavailable", never invents any.

Integration points, one commit each:

• services/jobs/deepsearch/crawl.py: search_queries gains a social bucket merged by the existing _interleave; HOSTILE_DOMAINS stops being a static skip list and becomes fabric.can_serve(url): a hostile URL is skipped only when the fabric cannot serve it. fetch_page routes those URLs through NodeFabric.fetch. The rsearch NDJSON ledger frame gets a sibling fabric frame so DeepsearchService._run_worker meters it like record_external.
• Devii: tools social_search, fetch_social_page (requires_auth=True, metered, guests refused), plus owner tools node_status, node_settings_get, node_settings_set, node_device_list, node_device_revoke (CONFIRM_REQUIRED + declared confirm param, per the rule). Update SYSTEM_PROMPT so the model prefers platform data, then social_search only when asked about outside discussion, and never claims a platform it cannot reach.
• POST /nodes/internal/fetch: the stable interface for the pravda.education agent system and any external caller. Auth: API key / Bearer, rate-limited, per-owner daily quota, documented.
• Cost and fairness: every fabric call is metered into a fabric_usage view per caller (requests, bytes, cache hit ratio) so a single caller cannot drain the donor pool; caller quota is checked before any donor is touched.

7. Clients

All clients speak the same protocol from spec.json. Conformance is tested against the Python reference client (Phase 10), not by eyeballing.

7.1 Native node (D2: Go)

• Single static binary, linux/amd64, linux/arm64, darwin/arm64, darwin/amd64, windows/amd64. Tunnel mode (TCP relay) and fetch mode.
• Config file + flags; devplace-node login takes the one-time device secret; devplace-node status; devplace-node run.
• Reconnect with full-jitter backoff, 20 s ping, local bandwidth/schedule/concurrency enforcement, battery-aware pause on laptops, quiet by default, structured log with a per-day totals line.
• Packaging: systemd user unit (loginctl enable-linger documented), Windows Service/scheduled task, macOS LaunchAgent, and a Docker/Podman compose snippet (the repo already ships docker-compose.podman.yml, so container users are a known audience).
• Built reproducibly from a nodes/client/ directory with a make node-client target; artifacts + sha256 land in DATA_DIR/node_clients/; the version manifest is generated, not hand-written.

7.2 Browser extension (MV3, Chrome + Firefox)

• Fetch mode only. Service worker holds the WS; 20 s keepalive ping plus a chrome.alarms reconnect backstop; state in chrome.storage.session.
• fetch() with credentials: "omit", header rewriting via declarativeNetRequest for the headers the spec allows, allowlist enforced before every request, response size cap.
• Honest, prominent disclosure on first run: what is shared, what is not, the on/off switch, live counters.
• Distribution per D3.

7.3 Later (explicitly out of this plan's first release)

Tauri tray app, signed auto-updates, mobile. Mobile stays out: iOS forbids the background behaviour and Android is restricted, as the research notes.

8. Frontend

Rules: responsive down to small phones (a wide button group becomes a dropdown), ES6 modules one class per file instantiated on app, tokens only in CSS, no literals, no var() fallbacks, static_url/assetUrl for assets, dates via local_dt, no data-render on server content, presence/pubsub via the existing managers.

1. Profile tab "Donate" (owner-only; others see only a donor badge and lifetime totals):
   • plain-language explainer first ("What you share, what you never share"), the consent toggle second, everything else only after consent. Product stays simple for the member: one switch, three sliders, a download card.
   • bandwidth per day, concurrent connections, active hours, platform checkboxes, pause-on-battery.
   • devices list with last seen, country/ASN class, bytes today, rotate and remove.
   • "Donated by you" card: requests served, data transferred (up/down), first/last donated, this month vs lifetime. Server-counted only.
   • download card: per-OS choice rendered as a dropdown on narrow screens, checksums shown, copy-paste install lines (systemd, docker), extension zip.
   • live numbers via a private pub/sub topic user.{uid}.nodes published by the lock-owner relay (add one row to LiveViewRelayService.VIEWS, re-checking the owner in the compute callable like the workspace topic), with a 30 s HTTP fallback poll.
2. Files: template templates/_node_donation.html partial included by profile.html; static/js/NodeDonation.js (app.nodeDonation); CSS static/css/nodes.css using existing tokens; reuse Http, Poller, Tabs.js, dp-dialog for confirmations.
3. Public surfaces: landing/footer line "The community has donated X GB across Y requests" from GET /nodes/stats (aggregate only), a Top donors section on the leaderboard (opt-in display), donor badge via _award_badge.html / Avatar.badgeElement reuse.
4. Admin: /admin/nodes (templates admin_nodes.html): pool size by country/ASN class, per-platform success rate, queue depth, flags, suspend/unsuspend, caps, allowlist, kill switch; live via admin.nodes topic row in VIEWS; ServiceMonitor already covers the service tab.

9. Gamification, incentives and anti-fraud

• ACHIEVEMENTS: node.first_donation, node.1gb, node.10gb, node.100gb, node.30_days, in BADGE_CATALOG group "Community".
• XP is awarded only for server-verified successful jobs, with a hard daily XP cap per donor (node_donors.xp_awarded_today), so synthetic traffic cannot farm progression. Through award_rewards only (never wrapped in background.submit).
• Counters are written server-side from what the server actually relayed; the donor's self-reported usage frame is advisory and is compared, never trusted.
• Self-dealing guard: a donor's own account cannot be the caller whose jobs are assigned to that donor's nodes for reward purposes.
• No money, no coins, no random rewards (D4).

10. Audit, privacy, safety, compliance

1. Audit events (events.md + category_for + recorder at the mutation point, via the audit-event skill): node.consent.grant, node.consent.withdraw, node.device.create, node.device.rotate, node.device.delete, node.settings.update, node.suspend, node.unsuspend, node.flag.resolve, node.allowlist.update, node.killswitch. Connects/disconnects/jobs are metrics and node_events, not audit rows (volume).
2. Privacy (this is a new category of personal data plus a third-party disclosure, so it ships WITH the feature): update templates/docs/privacy.html and the app-store privacy declarations. State: IP address and approximate location are processed; only a salted hash and coarse country/ASN class are stored; retention periods for node_events/hashes; what the donor can see (destination host, and in fetch mode the URL and body); that the platform's content passes through the donor machine; the explicit statement that the donor's cookies and accounts are never used. Plain-English donor terms page, ISP-terms warning, consent text, and a visible off switch.
3. App-store conditionals (root rule): no payment path, no purchasable randomness, no cross-app tracking introduced. Hide the donate UI inside any app-store shell build. Verify against current store policy before submission rather than assuming.
4. Abuse response: report a node/destination path, admin suspend within one tick, allowlist narrowing, per-destination circuit breaker, the kill switch, and a documented contact. Because the allowlist is destination-restricted there is no path for credential stuffing, ad fraud or arbitrary traffic through a donor.
5. Maturity/moderation: fetched third-party content is untrusted. It goes through render_content (server) or DOMPurify (client) like everything else and through the existing content filter when it is persisted anywhere. Fabric results are never rendered as raw HTML.

11. Phased delivery

Each phase ends with: clean import, per-language checks, em-dash scan (character and entity), pyflakes/ruff on touched files, the four-layer verification where listed, the full make test (all three tiers), then commit (hook auto-bumps the version), deploy with make docker-reload, and verification live on both hostnames. A change that is not running is not done.

Phase 0: Measure before building (no production code). Scratch scripts against temp DB only.

• Real-machine experiment: a prototype WS relay (use pywssocks / the residential-proxy-aggregator design as reference) with one residential node and one datacenter node. Measure Reddit .json and old.reddit.com success rate per IP class, rate-limit thresholds per IP, X permalink/oEmbed/syndication behaviour without login, Instagram/LinkedIn/Facebook/TikTok reality. Output: a measured per-platform viability table committed to the plan. Claims without a number stay labelled "unmeasured".
• Tunnel-mode feasibility: confirm curl_cffi through an HTTP-CONNECT bridge preserves the Chrome fingerprint end to end and that the cleartext-HTTP/2 gotcha in the repo does not apply (https only).
• Capacity: simulated-node harness (our own, outside locust) for 100/500/2000 sockets on one worker: memory, CPU, frame latency, dispatcher tick cost against SQLite. These numbers set or replace the caps in section 3.
• Reconnect/idle behaviour through BOTH edges: pravda.education (molohttp) and devplace.net (front host + SSH tunnel). Measure the actual idle timeout of each.
• Exit criteria: D1, D2, D3, D5, D6 decided; viability table written; caps confirmed or changed.

Phase 1: Specification and contracts. spec.py, /nodes/spec.json, docs protocol page skeleton, contract-test scaffolding in tests/unit/services/nodes/.

Phase 2: Data layer. Tables, indexes, helpers in database/nodes.py, CONSENT_KINDS entry, OWNED_TABLES, UNREPORTABLE_TABLES with reasons, soft-delete registrations, settings seeds, UsageLedger with atomic upserts. Four-layer verification applies here (counters + daily caps are a spendable resource): property checks on cap arithmetic, stateful fuzz of increment/rollup/prune with invariants (totals never decrease, daily never exceeds cap), real multi-process race on the daily-cap UPDATE ... WHERE using COALESCE(col, 0) and a freshly-created unset row (not pre-zeroed).

Phase 3: Registry, session, consent gate, kill switch. NodeRegistry, NodeSession, frames.py, ConsentGate, nodes_enabled kill switch (built and tested first). Routers for devices/ticket/WS handshake only. No job traffic yet. nginx upgrade location. Reuse of the ticket mechanism by adding a scope column to ws_tickets (default messages) instead of a parallel table.

Phase 4: Broker, scheduler, dispatcher, policy, circuit breakers, cache. Job lifecycle queued -> assigned -> done|failed|expired, fairness, per-domain limits, queue cap with 429. Four-layer verification: property tests on scheduler fairness and cap enforcement; stateful fuzz of connect/disconnect/assign/complete/expire with invariants (a job is assigned to at most one node, a node never exceeds its concurrency, a withdrawn consent is never assigned work); real-process concurrency test of two dispatchers racing one slot (must be impossible by construction, since only the lock owner dispatches, and proven anyway).

Phase 5: Python reference client + test harness. A small reference node used by the api/e2e tests and as the protocol conformance oracle. Includes fault injection: dropped frames, slow nodes, oversize bodies, forged bodies, DNS-rebinding attempt, private-IP target.

Phase 6A: Direct adapters (no donors): Hacker News, Bluesky, Mastodon, Lobsters, StackExchange, GitHub, dev.to into NodeFabric.social_search, DeepSearch and the Devii tools. This ships user value before any donor exists. Phase 6B: Node-backed adapters: Reddit first (per the Phase 0 table), then each additional platform only if Phase 0 measured it viable. NodeVerifier redundancy sampling for fetch mode.

Phase 7: Native client (Go) + packaging (systemd user unit, launchd, Windows service, compose). Cross-compile target, checksums, version manifest, conformance run against the reference oracle.

Phase 8: Browser extension. MV3 Chrome + Firefox build, disclosure screen, keepalive/alarms, conformance run, sideload zip served from /nodes/download.

Phase 9: Frontend and admin. Profile Donate tab, donated-by-you card, devices, downloads, live topic, leaderboard section, landing stat, /admin/nodes. Screenshot-verify with the screenshot skill at phone and desktop widths.

Phase 10: Devii, docs, SEO, CLI.

• Devii actions in actions/catalog/nodes.py (owner tools above), dispatcher/registry wiring, CONFIRM_REQUIRED for revoke, requires_auth matched to route guards, tool-group registration so the admin tool-prefs UI lists them.
• docs_api/groups/nodes.py endpoint() entries with sample_response; DOCS_PAGES for nodes.html and nodes-protocol.html; role-gating correct (protocol page admin/developer tier).
• SEO: Donate tab noindex; any public stats page indexable and in the sitemap.
• CLI: devplace nodes list|stats|suspend|unsuspend|prune|clear, destructive commands support --dry-run per the existing accounts prune convention (flag the conflict with the global dry-run-by-default rule to you if you want the default flipped; consistency with the existing CLI is the project rule).
• README, devplacepy/services/nodes/CLAUDE.md, a row in the root CLAUDE.md Subsystem map (a genuinely new architectural area), locustfile.py untouched for the WS (locust-maintainer: the node socket is deliberately excluded and the HTTP read routes get weighted tasks).

Phase 11: Hardening and launch gate.

• Legal review gate: platform ToS exposure per platform, donor terms, ISP warning text, jurisdiction. Launch is blocked on a recorded decision.
• Load and soak: 24 h soak with simulated nodes, reconnect storms (all sockets drop at once, then return), full-disk and DB-lock behaviour, worker restart mid-job. Numbers replace assumptions in CLAUDE.md.
• Security-maintainer pass (allowlist bypass, SSRF on the server's own fetch path, ticket replay, device secret leakage, admin seniority on node moderation).
• Full maintenance fleet (fleet skill): audit, background, devii, docs, dry, fanout, frontend, locust, security, seo, style, test dimensions.
• Rollout: nodes_enabled=0 by default. Stage 1: your own machines only (including a residential line) for a week, measuring success rate, latency, donor-side CPU and bandwidth. Stage 2: invite-only developers. Stage 3: open, still behind per-caller quotas. The kill switch stays one click away.

Phase 12: Documentation of record. devplacepy/services/nodes/CLAUDE.md states what the system does today, with gaps listed next to the code that will close them (no roadmap cosplay). Measured numbers only, each labelled with the machine it came from.

12. Test plan (by tier, mirroring the path)

• tests/unit/services/nodes/: spec invariants and contract (docs, clients, server agree), frame codec, DestinationPolicy (allowlist, ports, private IPs, redirects, DNS rebinding), scheduler (fairness, per-domain interval, concurrency, bandwidth caps), circuit breaker, cache bounds and LRU, ledger math, ASN classification, XP daily cap, adapters (parse recorded fixtures, malformed input, forged-structure rejection).
• tests/api/nodes/ (mirror the URL path): consent gating on every entry point, device create/rotate/revoke, secret shown once, ticket single-use and expiry, WS handshake with the reference client, full job round trip, withdrawal kicks the socket within a tick, kill switch, daily quotas, 429 paths, account deletion cascade, JSON and HTML faces for every route, admin seniority, /nodes/stats leaks no per-user data, /nodes/internal/fetch auth.
• tests/e2e/profile/ (Playwright, wait_until="domcontentloaded"): enable flow with consent, device creation and one-time secret, settings persistence, live counter update, withdraw flow, responsive layout at phone width, owner-only visibility. Any test flipping a global setting restores it in try/finally.
• Rigorous verification scripts (disposable, temp DB via DEVPLACE_DATABASE_URL/DEVPLACE_DATA_DIR, never data/): the four layers from root CLAUDE.md for the ledger, scheduler and job state machine.
• The test hook that guards the production DB applies: no test or script touches data/devplace.db.

13. Risks, stated plainly

Risk	Likelihood	Mitigation	Residual
Platform ToS/legal action against scraping	real	allowlist, legal gate, per-domain rate limits, kill switch	cannot be engineered away
Donors' IPs get rate-limited or banned by platforms	high on Reddit	per-donor-per-domain caps, caching, consent text says so	some donors will be unhappy; transparent counters help
X/Instagram/LinkedIn/TikTok unreachable (login walls)	high	Phase 0 measures; promise only what is measured	those platforms stay limited
Fetch-mode donor forges content	possible	tunnel preferred, structural validation, sampled redundancy, provenance labels, suspension	extension donors are lower trust, labelled as such
Small donor pool means flaky results	high at launch	cache, fallback chain, honest "unavailable"	search stays at today's quality, not worse
Single worker holds all sockets	by design	cap, fast reconnect, measured capacity	brief drop on worker crash
Donor privacy incident	low-impact by design	no cookies, hashed IP, destination allowlist, docs	disclosure must stay accurate
Scope creep into an open proxy	tempting	the "no open proxy" rule is in spec.py and the contract test	needs reviewer discipline

14. Open questions for you

• Q1: Confirm D1 to D6 above (a one-line answer each is enough).
• Q2: Is there any residential line you control for the Phase 0 measurement, or do we recruit three donors for it?
• Q3: What does the pravda.education agent system need from search (volume, latency, which platforms, guest access)? It sets the daily quotas and the /nodes/internal/fetch contract.
• Q4: Should donor identity be public (leaderboard, badge) by default or opt-in? The plan assumes opt-in display.
• Q5: Who owns the legal review, and is a launch without it acceptable for the invite-only stage?