Zo doen wijdat in oost

This commit is contained in:
2026-10-16 07:20:44 +02:00
parent 25d9cb1c7d
commit 6f80a820da
229 changed files with 23894 additions and 74 deletions
+70
View File
@@ -0,0 +1,70 @@
# DevPlace Node, browser extension
A Manifest V3 extension for Chrome and Firefox from one source tree. It lets a DevPlace member donate a little of their connection to the community node network, in **fetch mode only** (an extension has no raw sockets). The protocol is `devplacepy/services/nodes/spec.py`, served at `GET /nodes/spec.json`; this extension never restates a number from it except in `src/spec.js`, which a test compares against `spec.py`.
## Install (sideload)
1. Chrome: `chrome://extensions`, enable Developer mode, Load unpacked, pick this directory. Firefox: `about:debugging#/runtime/this-firefox`, Load Temporary Add-on, pick `manifest.json`. A temporary Firefox add-on is removed on restart.
2. A disclosure page opens on first install. Read it and accept.
3. On your DevPlace profile give consent to bandwidth sharing, register a device and copy the `dpn_...` secret (shown once).
4. Open the popup, enter the server URL and the secret, grant the host permission for that server when asked, then switch sharing on.
Packaged build: `./build.sh <outdir>` writes `devplace-node-extension.zip` and `devplace-node-extension.zip.sha256`. The zip holds `manifest.json`, `src/`, `ui/` and this file; tests are not shipped.
## Chrome and Firefox from one tree
`manifest.json` declares both `background.service_worker` and `background.scripts`, both pointing at the same ES module.
| Key | Used by |
|-----|---------|
| `background.service_worker` | Chrome (service worker). Firefox ignores it with a console warning. |
| `background.scripts` | Firefox (event page). Chrome ignores it when a service worker is present. |
| `browser_specific_settings.gecko` | Firefox only (add-on id, minimum 128). Chrome warns about the unknown key and ignores it. |
Both use the `chrome.*` namespace with promises. Firefox store submission additionally needs a `data_collection_permissions` declaration, not added yet.
## How it works
- The background worker holds the WebSocket. It exchanges the device secret for a 30 second ticket (`POST /nodes/ticket`, `Authorization: Bearer dpn_...`) and opens `WS /nodes/ws?ticket=...`, then sends `hello` with `kind: "extension"` and `modes: ["fetch"]`.
- A `pong` is sent every `ping_seconds` as a keepalive and for every server `ping`. A `chrome.alarms` alarm every minute is the reconnect backstop when the worker has been suspended.
- Reconnects use full-jitter backoff between `reconnect_min_seconds` and `reconnect_max_seconds`. Close `4013` retries at the minimum delay. A kick (`4403`), a consent or device refusal (`1008` with a reason) or a rejected secret (HTTP 401/403) **halts** and shows the reason; nothing hammers the server.
- State (connection, last error) is in `chrome.storage.session`. Config (server URL, secret, limits, disclosure flag) and the usage counters are in `chrome.storage.local`.
- The secret is never logged and never shown after saving; the popup shows its first 8 characters only.
## Safety enforced in the extension, before every request
Independent of anything the server says:
- allowlist: the hosts baked into `src/spec.js` (`reddit.com`, `redd.it` and their subdomains, dot-safe suffix match). A refreshed list from `welcome` is accepted only entry by entry when it is equal or narrower (same or sub-suffix, methods and ports subsets, fetch mode present); a widening entry is dropped and reported in the popup
- https only, port 443 only (the spec's 80 is never used because plain http is refused), no credentials in the URL, method must be permitted (GET)
- `fetch` with `credentials: "omit"`, `redirect: "manual"`, `cache: "no-store"`, no referrer; only `accept` and `accept-language` are forwarded from the job, so no cookie or authorization header can be injected
- each redirect hop is re-checked against the allowlist, at most 5 hops; a redirect that leaves the allowlist is refused with `redirect_not_allowed`
- response body capped at `max_body_bytes` and by the remaining daily budget, with `truncated: true`; `set-cookie` style response headers are dropped
- timeout by `AbortController`, daily bandwidth cap, concurrency cap and active hours from the local settings, each answered with its refusal code (`timeout`, `bandwidth_exhausted`, `concurrency_exceeded`, `outside_hours`). A server value can only lower these, never raise them
Known limit: browsers hide the target of a `redirect: "manual"` response (an opaque redirect), so the extension cannot read where it points. It fails closed with `redirect_not_allowed`. A followable redirect is only handled when the response exposes its `Location`.
## Permissions
| Permission | Why |
|------------|-----|
| `storage` | config, usage counters, connection state |
| `alarms` | reconnect backstop while the worker is suspended |
| `host_permissions`: `reddit.com`, `*.reddit.com`, `redd.it`, `*.redd.it` over https | fetch the allowlisted destinations without CORS. These are exactly the destinations in `spec.py`. |
| `optional_host_permissions`: any https origin, `localhost`, `127.0.0.1` over http | requested at connect time for the one server origin you enter, so the ticket request works. Not granted at install. |
No `tabs`, `cookies`, `history`, `webRequest` or `<all_urls>` permission.
## What is and is not shared
Shared: the URL of the requested public page, the page content that comes back, your public IP address as seen by the destination, and request and byte counts. Not used: your cookies, logins, browsing history, tabs, other sites, any host outside the list, any non-https or non-443 destination. See `ui/disclosure.html`, shown on first install.
## Tests
`node --test nodes/extension/test/*.mjs` (no dependencies). They cover Policy, Limits, Fetcher with a stubbed `fetch`, WsClient with a fake WebSocket, and a drift check of `src/spec.js` against `spec.py`. Not covered by automated tests: `background.js` and the popup, which need a real browser.
## Gaps
- No icons yet; the browser shows a default.
- The `welcome` frame carries no device label, so the label in the popup is local only.
- Not tested in a real Chrome or Firefox session in this change.
+41
View File
@@ -0,0 +1,41 @@
#!/bin/sh
set -eu
if [ "$#" -ne 1 ]; then
echo "usage: build.sh <outdir>" >&2
exit 2
fi
here=$(cd "$(dirname "$0")" && pwd)
mkdir -p "$1"
out=$(cd "$1" && pwd)
name=devplace-node-extension.zip
archive="$out/$name"
rm -f "$archive"
cd "$here"
if command -v zip >/dev/null 2>&1; then
zip -q -X -r "$archive" manifest.json src ui README.md
else
python3 - "$archive" <<'PY'
import sys
import zipfile
from pathlib import Path
archive = sys.argv[1]
paths = [Path("manifest.json"), Path("README.md")]
for folder in ("src", "ui"):
paths.extend(sorted(item for item in Path(folder).rglob("*") if item.is_file()))
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as bundle:
for path in paths:
bundle.write(path, path.as_posix())
PY
fi
cd "$out"
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$name" > "$name.sha256"
else
shasum -a 256 "$name" > "$name.sha256"
fi
cat "$name.sha256"
+33
View File
@@ -0,0 +1,33 @@
{
"manifest_version": 3,
"name": "DevPlace Node",
"version": "0.1.0",
"description": "Donate a little of your connection to the DevPlace community node network. Fetch mode only, allowlisted hosts only, never your cookies.",
"background": {
"service_worker": "src/background.js",
"scripts": ["src/background.js"],
"type": "module"
},
"action": {
"default_title": "DevPlace Node",
"default_popup": "ui/popup.html"
},
"permissions": ["storage", "alarms"],
"host_permissions": [
"https://reddit.com/*",
"https://*.reddit.com/*",
"https://redd.it/*",
"https://*.redd.it/*"
],
"optional_host_permissions": [
"https://*/*",
"http://localhost/*",
"http://127.0.0.1/*"
],
"browser_specific_settings": {
"gecko": {
"id": "node@devplace.net",
"strict_min_version": "128.0"
}
}
}
+5
View File
@@ -0,0 +1,5 @@
{
"name": "devplace-node-extension",
"private": true,
"type": "module"
}
+126
View File
@@ -0,0 +1,126 @@
import { LIMITS } from "./spec.js";
const KEY = "config";
const SECRET_PREFIX = "dpn_";
const SECRET_MIN_CHARS = 16;
const SECRET_MAX_CHARS = 128;
const LABEL_MAX_CHARS = 64;
const SHOWN_SECRET_CHARS = 8;
const LOOPBACK = ["localhost", "127.0.0.1", "[::1]"];
const DEFAULTS = Object.freeze({
serverUrl: "",
secret: "",
label: "",
disclosed: false,
enabled: false,
bandwidthMb: LIMITS.default_donor_mb_per_day,
maxConcurrent: LIMITS.max_concurrent_per_node,
activeFrom: "",
activeTo: "",
});
export class ConfigError extends Error {}
export class Config {
constructor(area) {
this.area = area;
this.data = { ...DEFAULTS };
}
static validateServerUrl(raw) {
let url;
try {
url = new URL(String(raw).trim());
} catch (error) {
throw new ConfigError("The server URL is not a valid URL");
}
const loopback = LOOPBACK.includes(url.hostname);
if (url.protocol !== "https:" && !(url.protocol === "http:" && loopback)) {
throw new ConfigError("The server URL must use https");
}
if (url.username || url.password) {
throw new ConfigError("The server URL must not contain credentials");
}
return url.origin;
}
static validateSecret(raw) {
const secret = String(raw).trim();
if (
!secret.startsWith(SECRET_PREFIX) ||
secret.length < SECRET_MIN_CHARS ||
secret.length > SECRET_MAX_CHARS ||
/\s/.test(secret)
) {
throw new ConfigError("That is not a DevPlace device secret (it starts with dpn_)");
}
return secret;
}
async load() {
const stored = (await this.area.get(KEY))[KEY];
this.data = { ...DEFAULTS, ...(stored && typeof stored === "object" ? stored : {}) };
}
async save() {
await this.area.set({ [KEY]: { ...this.data } });
}
get hasSecret() {
return Boolean(this.data.secret);
}
get secretPrefix() {
return this.data.secret ? this.data.secret.slice(0, SHOWN_SECRET_CHARS) + "..." : "";
}
credentials() {
return { serverUrl: this.data.serverUrl, secret: this.data.secret };
}
view() {
const { secret, ...rest } = this.data;
return { ...rest, hasSecret: this.hasSecret, secretPrefix: this.secretPrefix };
}
async connect({ serverUrl, secret, label }) {
this.data.serverUrl = Config.validateServerUrl(serverUrl);
this.data.secret = Config.validateSecret(secret);
this.data.label = String(label || "").trim().slice(0, LABEL_MAX_CHARS);
await this.save();
}
async acceptDisclosure() {
this.data.disclosed = true;
await this.save();
}
async setEnabled(enabled) {
if (enabled && !this.data.disclosed) {
throw new ConfigError("Read and accept the disclosure first");
}
if (enabled && !this.hasSecret) {
throw new ConfigError("Connect a device secret first");
}
this.data.enabled = Boolean(enabled);
await this.save();
}
async saveSettings(values) {
for (const key of ["bandwidthMb", "maxConcurrent", "activeFrom", "activeTo"]) {
if (key in values) {
this.data[key] = values[key];
}
}
await this.save();
}
async forget() {
this.data.secret = "";
this.data.serverUrl = "";
this.data.label = "";
this.data.enabled = false;
await this.save();
}
}
+193
View File
@@ -0,0 +1,193 @@
import {
DROPPED_RESPONSE_HEADERS,
FETCH_FAILED,
FORWARDED_REQUEST_HEADERS,
MAX_BODY_CHUNKS,
MAX_HEADER_VALUE_CHARS,
MAX_REDIRECTS,
MAX_RESPONSE_HEADERS,
REFUSALS,
} from "./spec.js";
const REDIRECT_STATUSES = [301, 302, 303, 307, 308];
const REQUEST_OVERHEAD_BYTES = 256;
const BASE64_CHUNK = 0x8000;
export function toBase64(bytes) {
let text = "";
for (let index = 0; index < bytes.length; index += BASE64_CHUNK) {
text += String.fromCharCode.apply(null, bytes.subarray(index, index + BASE64_CHUNK));
}
return btoa(text);
}
function concat(parts, total) {
const joined = new Uint8Array(total);
let offset = 0;
for (const part of parts) {
joined.set(part, offset);
offset += part.length;
}
return joined;
}
export class Fetcher {
constructor({ policy, limits, usage, fetchImpl, clock }) {
this.policy = policy;
this.limits = limits;
this.usage = usage;
this.fetchImpl = fetchImpl || ((...args) => globalThis.fetch(...args));
this.clock = clock || (() => Date.now());
}
setPolicy(policy) {
this.policy = policy;
}
failure(jobId, code) {
return { type: "job_error", job_id: jobId, code };
}
requestHeaders(raw) {
const headers = {};
if (!raw || typeof raw !== "object") {
return headers;
}
for (const [name, value] of Object.entries(raw)) {
const key = name.toLowerCase();
if (FORWARDED_REQUEST_HEADERS.includes(key) && typeof value === "string") {
headers[key] = value;
}
}
return headers;
}
responseHeaders(response) {
const headers = {};
let count = 0;
response.headers.forEach((value, name) => {
const key = name.toLowerCase();
if (DROPPED_RESPONSE_HEADERS.includes(key) || count >= MAX_RESPONSE_HEADERS) {
return;
}
headers[key] = String(value).slice(0, MAX_HEADER_VALUE_CHARS);
count += 1;
});
return headers;
}
isRedirect(response) {
return response.type === "opaqueredirect" || REDIRECT_STATUSES.includes(response.status);
}
async readBody(response, cap) {
if (!response.body || typeof response.body.getReader !== "function") {
const whole = new Uint8Array(await response.arrayBuffer());
return whole.length > cap
? { bytes: whole.slice(0, cap), truncated: true }
: { bytes: whole, truncated: false };
}
const reader = response.body.getReader();
const parts = [];
let total = 0;
let truncated = false;
for (let chunks = 0; chunks < MAX_BODY_CHUNKS; chunks += 1) {
const { done, value } = await reader.read();
if (done) {
return { bytes: concat(parts, total), truncated };
}
const room = cap - total;
if (value.length > room) {
parts.push(value.subarray(0, room));
total += room;
truncated = true;
break;
}
parts.push(value);
total += value.length;
}
truncated = true;
await reader.cancel();
return { bytes: concat(parts, total), truncated };
}
async follow(job, verdict, signal) {
let current = verdict.url;
const headers = this.requestHeaders(job.headers);
for (let hop = 0; hop <= MAX_REDIRECTS; hop += 1) {
const response = await this.fetchImpl(current.href, {
method: verdict.method,
headers,
credentials: "omit",
redirect: "manual",
cache: "no-store",
referrerPolicy: "no-referrer",
signal,
});
if (!this.isRedirect(response)) {
return { response, final: current };
}
const location = response.headers.get("location");
if (response.type === "opaqueredirect" || !location || hop === MAX_REDIRECTS) {
if (response.body && typeof response.body.cancel === "function") {
await response.body.cancel();
}
return { refused: REFUSALS.REDIRECT_NOT_ALLOWED };
}
const next = this.policy.checkRedirect(location, current, verdict.method);
if (response.body && typeof response.body.cancel === "function") {
await response.body.cancel();
}
if (!next.ok) {
return { refused: REFUSALS.REDIRECT_NOT_ALLOWED };
}
current = next.url;
}
return { refused: REFUSALS.REDIRECT_NOT_ALLOWED };
}
async execute(job) {
const verdict = this.policy.check(job.method, job.url);
if (!verdict.ok) {
return this.failure(job.job_id, verdict.code);
}
const refusal = this.limits.admit(this.usage.usedBytes);
if (refusal) {
return this.failure(job.job_id, refusal);
}
const started = this.clock();
const up = job.url.length + REQUEST_OVERHEAD_BYTES;
const controller = new AbortController();
const timer = setTimeout(
() => controller.abort(),
this.limits.timeoutSeconds(job.timeout) * 1000,
);
try {
const outcome = await this.follow(job, verdict, controller.signal);
if (outcome.refused) {
await this.usage.record({ ok: false, up, down: 0 });
return this.failure(job.job_id, outcome.refused);
}
const budget = this.limits.remainingBytes(this.usage.usedBytes + up);
const cap = Math.min(this.limits.maxBodyBytes, budget);
const body = await this.readBody(outcome.response, cap);
const down = body.bytes.length;
await this.usage.record({ ok: true, up, down });
return {
type: "job_result",
job_id: job.job_id,
status: outcome.response.status,
headers: this.responseHeaders(outcome.response),
body_b64: toBase64(body.bytes),
truncated: body.truncated,
elapsed_ms: Math.max(0, Math.round(this.clock() - started)),
};
} catch (error) {
await this.usage.record({ ok: false, up, down: 0 });
return this.failure(job.job_id, controller.signal.aborted ? REFUSALS.TIMEOUT : FETCH_FAILED);
} finally {
clearTimeout(timer);
this.limits.release();
}
}
}
+167
View File
@@ -0,0 +1,167 @@
import { LIMITS, REFUSALS } from "./spec.js";
const MB = 1024 * 1024;
const HOURS = /^([01]\d|2[0-3]):([0-5]\d)$/;
const CAP_KEYS = [
"max_frame_bytes",
"max_body_bytes",
"max_concurrent_per_node",
"job_timeout_seconds",
"ping_seconds",
"url_max_chars",
];
function isCount(value) {
return typeof value === "number" && Number.isInteger(value) && value >= 1;
}
function clampInt(value, low, high, fallback) {
if (!isCount(value)) {
return fallback;
}
return Math.min(high, Math.max(low, value));
}
function minutes(text) {
const match = HOURS.exec(text);
return match ? Number(match[1]) * 60 + Number(match[2]) : null;
}
export class Limits {
constructor(options = {}) {
this.baked = options.baked || LIMITS;
this.clock = options.clock || (() => new Date());
this.caps = {};
for (const key of CAP_KEYS) {
this.caps[key] = this.baked[key];
}
this.reconnectMin = this.baked.reconnect_min_seconds;
this.reconnectMax = this.baked.reconnect_max_seconds;
this.serverBandwidthMb = this.baked.max_donor_mb_per_day;
this.serverConcurrent = this.baked.max_concurrent_per_node;
this.local = Limits.normaliseLocal({}, this.baked);
this.inflight = 0;
if (options.local) {
this.setLocal(options.local);
}
}
static normaliseLocal(values, baked = LIMITS) {
const from = minutes(values.activeFrom || "");
const to = minutes(values.activeTo || "");
const both = from !== null && to !== null;
return {
bandwidthMb: clampInt(
values.bandwidthMb,
1,
baked.max_donor_mb_per_day,
baked.default_donor_mb_per_day,
),
maxConcurrent: clampInt(
values.maxConcurrent,
1,
baked.max_concurrent_per_node,
baked.max_concurrent_per_node,
),
activeFrom: both ? values.activeFrom : "",
activeTo: both ? values.activeTo : "",
};
}
setLocal(values) {
this.local = Limits.normaliseLocal({ ...this.local, ...values }, this.baked);
}
applyWelcome(welcomeLimits, serverSettings) {
const incoming = welcomeLimits && typeof welcomeLimits === "object" ? welcomeLimits : {};
for (const key of CAP_KEYS) {
this.caps[key] = Math.min(this.baked[key], isCount(incoming[key]) ? incoming[key] : this.baked[key]);
}
const ceiling = this.baked.reconnect_max_seconds;
this.reconnectMin = clampInt(incoming.reconnect_min_seconds, 1, ceiling, this.baked.reconnect_min_seconds);
this.reconnectMax = clampInt(incoming.reconnect_max_seconds, this.reconnectMin, ceiling, ceiling);
this.applyServerSettings(serverSettings);
}
applyServerSettings(settings) {
const values = settings && typeof settings === "object" ? settings : {};
this.serverBandwidthMb = clampInt(
values.bandwidth_mb_day,
1,
this.baked.max_donor_mb_per_day,
this.baked.max_donor_mb_per_day,
);
this.serverConcurrent = clampInt(
values.max_concurrent,
1,
this.baked.max_concurrent_per_node,
this.baked.max_concurrent_per_node,
);
}
get maxBodyBytes() {
return this.caps.max_body_bytes;
}
get maxFrameBytes() {
return this.caps.max_frame_bytes;
}
get urlMaxChars() {
return this.caps.url_max_chars;
}
get pingSeconds() {
return this.caps.ping_seconds;
}
get concurrency() {
return Math.min(this.caps.max_concurrent_per_node, this.local.maxConcurrent, this.serverConcurrent);
}
get bandwidthBytes() {
return Math.min(this.local.bandwidthMb, this.serverBandwidthMb) * MB;
}
timeoutSeconds(requested) {
const asked = isCount(requested) ? requested : this.caps.job_timeout_seconds;
return Math.min(asked, this.caps.job_timeout_seconds);
}
helloLimits() {
return { max_concurrent: this.concurrency };
}
remainingBytes(usedBytes) {
return Math.max(0, this.bandwidthBytes - usedBytes);
}
inActiveHours() {
const from = minutes(this.local.activeFrom);
const to = minutes(this.local.activeTo);
if (from === null || to === null || from === to) {
return true;
}
const now = this.clock();
const current = now.getHours() * 60 + now.getMinutes();
return from < to ? current >= from && current < to : current >= from || current < to;
}
admit(usedBytes) {
if (!this.inActiveHours()) {
return REFUSALS.OUTSIDE_HOURS;
}
if (this.remainingBytes(usedBytes) <= 0) {
return REFUSALS.BANDWIDTH_EXHAUSTED;
}
if (this.inflight >= this.concurrency) {
return REFUSALS.CONCURRENCY_EXCEEDED;
}
this.inflight += 1;
return null;
}
release() {
this.inflight = Math.max(0, this.inflight - 1);
}
}
+128
View File
@@ -0,0 +1,128 @@
import {
ALLOWED_PORTS,
DESTINATIONS,
HTTPS_PORT,
LIMITS,
MODE_FETCH,
REFUSALS,
} from "./spec.js";
const HTTPS = "https:";
function isSubset(candidate, allowed) {
return Array.isArray(candidate) && candidate.every((item) => allowed.includes(item));
}
function normaliseHost(host) {
return String(host).trim().toLowerCase().replace(/\.+$/, "");
}
export class Policy {
constructor(destinations = DESTINATIONS, limits = LIMITS) {
this.destinations = destinations;
this.urlMaxChars = limits.url_max_chars;
}
static narrow(baked, refreshed) {
const accepted = [];
const rejected = [];
if (!Array.isArray(refreshed)) {
return { destinations: [], rejected: ["destinations is not a list"] };
}
for (const item of refreshed) {
const suffix = item && typeof item.suffix === "string" ? normaliseHost(item.suffix) : "";
const parent = baked.find(
(entry) => suffix === entry.suffix || suffix.endsWith("." + entry.suffix),
);
if (!parent) {
rejected.push(`${suffix || "?"}: not in the baked-in allowlist`);
continue;
}
const methods = Array.isArray(item.methods) ? item.methods.map((m) => String(m).toUpperCase()) : [];
const ports = Array.isArray(item.ports) ? item.ports : [];
const modes = Array.isArray(item.modes) ? item.modes : [];
if (!methods.length || !isSubset(methods, parent.methods)) {
rejected.push(`${suffix}: methods widen the baked-in list`);
continue;
}
if (!ports.length || !isSubset(ports, parent.ports) || !isSubset(ports, ALLOWED_PORTS)) {
rejected.push(`${suffix}: ports widen the baked-in list`);
continue;
}
if (!modes.includes(MODE_FETCH)) {
rejected.push(`${suffix}: not available in fetch mode`);
continue;
}
accepted.push({
suffix,
adapter: String(item.adapter || parent.adapter),
methods,
ports,
modes,
max_rps: typeof item.max_rps === "number" ? item.max_rps : parent.max_rps,
});
}
return { destinations: accepted, rejected };
}
static fromRefreshed(baked, refreshed, limits = LIMITS) {
const narrowed = Policy.narrow(baked, refreshed);
return { policy: new Policy(narrowed.destinations, limits), rejected: narrowed.rejected };
}
destinationFor(host) {
const name = normaliseHost(host);
return (
this.destinations.find((item) => name === item.suffix || name.endsWith("." + item.suffix)) || null
);
}
refuse(code, reason) {
return { ok: false, code, reason };
}
check(method, rawUrl) {
if (typeof rawUrl !== "string" || !rawUrl || rawUrl.length > this.urlMaxChars) {
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, "url is missing or too long");
}
let url;
try {
url = new URL(rawUrl);
} catch (error) {
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, "url does not parse");
}
if (url.protocol !== HTTPS) {
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, "only https is allowed");
}
if (url.username || url.password) {
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, "credentials in the url are refused");
}
const destination = this.destinationFor(url.hostname);
if (!destination) {
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, `${url.hostname} is not on the allowlist`);
}
const port = url.port === "" ? HTTPS_PORT : Number(url.port);
if (port !== HTTPS_PORT || !destination.ports.includes(port) || !ALLOWED_PORTS.includes(port)) {
return this.refuse(REFUSALS.PORT_NOT_ALLOWED, `port ${port} is not allowed over https`);
}
const verb = String(method || "").toUpperCase();
if (!destination.methods.includes(verb)) {
return this.refuse(REFUSALS.METHOD_NOT_ALLOWED, `${verb || "?"} is not permitted`);
}
return { ok: true, url, destination, method: verb };
}
checkRedirect(location, base, method) {
let target;
try {
target = new URL(location, base);
} catch (error) {
return this.refuse(REFUSALS.REDIRECT_NOT_ALLOWED, "redirect target does not parse");
}
const verdict = this.check(method, target.href);
if (!verdict.ok) {
return this.refuse(REFUSALS.REDIRECT_NOT_ALLOWED, verdict.reason);
}
return verdict;
}
}
+37
View File
@@ -0,0 +1,37 @@
const KEY = "state";
export const STATES = Object.freeze({
OFF: "off",
CONNECTING: "connecting",
ONLINE: "online",
BACKOFF: "backoff",
HALTED: "halted",
});
export class State {
constructor(area, clock = () => Date.now()) {
this.area = area;
this.clock = clock;
this.data = { state: STATES.OFF, detail: "", since: clock(), deviceUid: "", lastError: "" };
this.chain = Promise.resolve();
}
async load() {
const stored = (await this.area.get(KEY))[KEY];
if (stored && typeof stored === "object") {
this.data = { ...this.data, ...stored };
}
}
snapshot() {
return { ...this.data };
}
set(patch) {
const changed = patch.state && patch.state !== this.data.state;
this.data = { ...this.data, ...patch, since: changed ? this.clock() : this.data.since };
const payload = { ...this.data };
this.chain = this.chain.then(() => this.area.set({ [KEY]: payload }));
return this.chain;
}
}
+81
View File
@@ -0,0 +1,81 @@
const KEY = "usage";
function dayKey(date) {
const month = String(date.getMonth() + 1).padStart(2, "0");
const day = String(date.getDate()).padStart(2, "0");
return `${date.getFullYear()}-${month}-${day}`;
}
function empty(day) {
return {
day,
requests: 0,
failed: 0,
bytesUp: 0,
bytesDown: 0,
lifetimeRequests: 0,
lifetimeBytes: 0,
};
}
export class Usage {
constructor(area, clock = () => new Date()) {
this.area = area;
this.clock = clock;
this.data = empty(dayKey(clock()));
this.chain = Promise.resolve();
}
async load() {
const stored = (await this.area.get(KEY))[KEY];
if (stored && typeof stored === "object") {
this.data = { ...empty(dayKey(this.clock())), ...stored };
}
this.roll();
}
roll() {
const today = dayKey(this.clock());
if (this.data.day !== today) {
this.data = {
...empty(today),
lifetimeRequests: this.data.lifetimeRequests,
lifetimeBytes: this.data.lifetimeBytes,
};
}
}
snapshot() {
this.roll();
return { ...this.data };
}
get usedBytes() {
this.roll();
return this.data.bytesUp + this.data.bytesDown;
}
record({ ok, up, down }) {
this.roll();
const bytesUp = Math.max(0, Math.floor(up) || 0);
const bytesDown = Math.max(0, Math.floor(down) || 0);
this.data.requests += 1;
this.data.failed += ok ? 0 : 1;
this.data.bytesUp += bytesUp;
this.data.bytesDown += bytesDown;
this.data.lifetimeRequests += 1;
this.data.lifetimeBytes += bytesUp + bytesDown;
return this.persist();
}
persist() {
const payload = { ...this.data };
this.chain = this.chain.then(() => this.area.set({ [KEY]: payload }));
return this.chain;
}
async reset() {
this.data = empty(dayKey(this.clock()));
await this.persist();
}
}
+375
View File
@@ -0,0 +1,375 @@
import { Policy } from "./Policy.js";
import { STATES } from "./State.js";
import {
CLIENT_KIND,
CLOSE_CAPACITY,
CLOSE_KICKED,
CLOSE_NORMAL,
CLOSE_PROTOCOL,
CLOSE_RETRY_OWNER,
DESTINATIONS,
FRAME_CEILING_BYTES,
HALTING_CLOSE_REASONS_EXEMPT,
MAX_BACKOFF_EXPONENT,
MODE_FETCH,
PROTOCOL_VERSION,
REFUSALS,
SERVER_FRAMES,
} from "./spec.js";
const OPEN = 1;
const POLICY_CLOSE = 1008;
const TICKET_PATH = "/nodes/ticket";
const WS_PATH = "/nodes/ws";
const HALT_STATUSES = [401, 403];
class FrameFault extends Error {}
export class WsClient {
constructor(options) {
this.config = options.config;
this.limits = options.limits;
this.fetcher = options.fetcher;
this.state = options.state;
this.fetchImpl = options.fetchImpl || ((...args) => globalThis.fetch(...args));
this.WebSocketImpl = options.WebSocketImpl || globalThis.WebSocket;
this.timers = options.timers || globalThis;
this.random = options.random || Math.random;
this.clientVersion = options.clientVersion || "0.0.0";
this.platform = options.platform || "chrome";
this.baked = options.baked || DESTINATIONS;
this.socket = null;
this.wanted = false;
this.connecting = false;
this.welcomed = false;
this.attempt = 0;
this.retryTimer = null;
this.pingTimer = null;
this.pending = new Set();
this.generation = 0;
}
async start() {
this.wanted = true;
await this.ensure();
}
async stop(detail = "") {
this.wanted = false;
this.clearTimers();
this.generation += 1;
this.closeSocket(CLOSE_NORMAL, "stop");
await this.state.set({ state: STATES.OFF, detail });
}
async ensure() {
if (!this.wanted || this.socket || this.connecting || this.retryTimer) {
return;
}
await this.connectOnce();
}
clearTimers() {
if (this.retryTimer) {
this.timers.clearTimeout(this.retryTimer);
this.retryTimer = null;
}
if (this.pingTimer) {
this.timers.clearInterval(this.pingTimer);
this.pingTimer = null;
}
}
closeSocket(code, reason) {
const socket = this.socket;
this.socket = null;
if (socket && socket.readyState <= OPEN) {
socket.close(code, reason);
}
}
backoffSeconds(fast) {
const low = this.limits.reconnectMin;
if (fast) {
return low;
}
const exponent = Math.min(this.attempt, MAX_BACKOFF_EXPONENT);
const ceiling = Math.min(this.limits.reconnectMax, low * 2 ** exponent);
return Math.max(low, this.random() * ceiling);
}
async scheduleRetry(detail, fast = false) {
if (!this.wanted) {
return;
}
const seconds = this.backoffSeconds(fast);
this.attempt += 1;
this.clearTimers();
this.retryTimer = this.timers.setTimeout(() => {
this.retryTimer = null;
this.ensure();
}, seconds * 1000);
await this.state.set({ state: STATES.BACKOFF, detail, retryInSeconds: Math.round(seconds) });
}
async halt(detail) {
this.wanted = false;
this.clearTimers();
this.generation += 1;
this.closeSocket(CLOSE_NORMAL, "halt");
await this.state.set({ state: STATES.HALTED, detail, lastError: detail });
}
async fetchTicket() {
const { serverUrl, secret } = this.config.credentials();
const response = await this.fetchImpl(serverUrl + TICKET_PATH, {
method: "POST",
headers: { Authorization: `Bearer ${secret}` },
credentials: "omit",
cache: "no-store",
});
if (!response.ok) {
let reason = "";
try {
reason = (await response.json()).reason || "";
} catch (error) {
reason = "";
}
const error = new Error(`ticket refused with ${response.status} ${reason}`.trim());
error.status = response.status;
throw error;
}
const body = await response.json();
if (!body || typeof body.ticket !== "string" || !body.ticket) {
throw new Error("ticket response has no ticket");
}
const target = new URL(typeof body.ws_path === "string" ? body.ws_path : WS_PATH, serverUrl);
if (target.origin !== new URL(serverUrl).origin) {
throw new Error("ticket response points at another origin");
}
target.protocol = target.protocol === "https:" ? "wss:" : "ws:";
target.searchParams.set("ticket", body.ticket);
return target.href;
}
async connectOnce() {
this.connecting = true;
const generation = this.generation;
await this.state.set({ state: STATES.CONNECTING, detail: "" });
let address;
try {
address = await this.fetchTicket();
} catch (error) {
this.connecting = false;
if (generation !== this.generation) {
return;
}
if (HALT_STATUSES.includes(error.status)) {
await this.halt(error.message);
return;
}
await this.state.set({ lastError: error.message });
await this.scheduleRetry(error.message);
return;
}
this.connecting = false;
if (generation !== this.generation) {
return;
}
this.open(address, generation);
}
open(address, generation) {
this.welcomed = false;
const socket = new this.WebSocketImpl(address);
this.socket = socket;
socket.onopen = () => {
if (generation !== this.generation || this.socket !== socket) {
return;
}
this.sendFrame("hello", {
protocol: PROTOCOL_VERSION,
client_version: this.clientVersion,
kind: CLIENT_KIND,
platform: this.platform,
modes: [MODE_FETCH],
limits: this.limits.helloLimits(),
});
};
socket.onmessage = (event) => {
if (this.socket !== socket) {
return;
}
return this.handleMessage(event.data).catch((error) => this.protocolFault(error));
};
socket.onclose = (event) => {
if (this.socket !== socket) {
return;
}
this.socket = null;
this.onClosed(event.code, event.reason || "");
};
socket.onerror = () => {
this.state.set({ lastError: "websocket error" });
};
}
sendFrame(type, fields = {}) {
if (!this.socket || this.socket.readyState !== OPEN) {
throw new Error(`cannot send ${type}, socket is not open`);
}
const text = JSON.stringify({ type, ...fields });
const ceiling = FRAME_CEILING_BYTES[type] || FRAME_CEILING_BYTES.default;
if (new TextEncoder().encode(text).length > ceiling) {
throw new Error(`${type} frame exceeds its size cap`);
}
this.socket.send(text);
}
async protocolFault(error) {
await this.state.set({ lastError: error.message });
this.closeSocket(CLOSE_PROTOCOL, "protocol");
this.clearTimers();
await this.scheduleRetry(error.message);
}
decode(raw) {
if (typeof raw !== "string") {
throw new FrameFault("binary frames are not part of the protocol");
}
if (raw.length > this.limits.maxFrameBytes) {
throw new FrameFault("server frame exceeds the size cap");
}
let frame;
try {
frame = JSON.parse(raw);
} catch (error) {
throw new FrameFault("server frame is not valid JSON");
}
if (!frame || typeof frame !== "object" || Array.isArray(frame)) {
throw new FrameFault("server frame must be a JSON object");
}
const shape = SERVER_FRAMES[frame.type];
if (!shape) {
throw new FrameFault(`unknown server frame ${String(frame.type)}`);
}
for (const [name, kind] of Object.entries(shape)) {
const value = frame[name];
const good = kind === "array" ? Array.isArray(value) : typeof value === kind && value !== null;
if (!good) {
throw new FrameFault(`frame ${frame.type} field ${name} must be ${kind}`);
}
}
return frame;
}
async handleMessage(raw) {
const frame = this.decode(raw);
if (frame.type !== "welcome" && !this.welcomed && frame.type !== "kick") {
throw new FrameFault(`${frame.type} arrived before welcome`);
}
if (frame.type === "welcome") {
await this.onWelcome(frame);
} else if (frame.type === "ping") {
this.sendFrame("pong");
} else if (frame.type === "job") {
this.runJob(frame);
} else if (frame.type === "settings_update") {
this.limits.applyServerSettings(frame.settings);
} else if (frame.type === "kick") {
await this.halt(`kicked by the server: ${frame.reason}`);
} else {
this.sendFrame("tunnel_opened", {
conn_id: frame.conn_id,
ok: false,
code: REFUSALS.HOST_NOT_ALLOWED,
});
throw new FrameFault(`${frame.type} is a tunnel frame and this client is fetch-only`);
}
}
async onWelcome(frame) {
if (this.welcomed) {
throw new FrameFault("a second welcome arrived");
}
this.limits.applyWelcome(frame.limits, frame.settings);
const refreshed = Policy.fromRefreshed(this.baked, frame.destinations, this.limits.baked);
this.fetcher.setPolicy(refreshed.policy);
this.welcomed = true;
this.attempt = 0;
this.startKeepalive();
const notes = refreshed.rejected.length ? `ignored: ${refreshed.rejected.join("; ")}` : "";
await this.state.set({
state: STATES.ONLINE,
detail: notes,
deviceUid: frame.device_uid,
lastError: notes,
});
}
startKeepalive() {
if (this.pingTimer) {
this.timers.clearInterval(this.pingTimer);
}
this.pingTimer = this.timers.setInterval(() => {
try {
this.sendFrame("pong");
} catch (error) {
this.state.set({ lastError: error.message });
}
}, this.limits.pingSeconds * 1000);
}
runJob(frame) {
const task = this.fetcher
.execute(frame)
.then((result) => {
this.sendFrame(result.type, result);
return this.reportUsage(result);
})
.catch((error) => this.state.set({ lastError: error.message }))
.finally(() => this.pending.delete(task));
this.pending.add(task);
}
reportUsage(result) {
const down = result.type === "job_result" ? Math.ceil((result.body_b64.length * 3) / 4) : 0;
this.sendFrame("usage", { bytes_up: 0, bytes_down: down });
}
async idle() {
while (this.pending.size) {
await Promise.all([...this.pending]);
}
}
classifyClose(code, reason) {
if (code === CLOSE_KICKED) {
return { action: "halt", detail: `closed by the server: ${reason || "kicked"}` };
}
if (code === CLOSE_RETRY_OWNER) {
return { action: "retry", fast: true, detail: "server asked for a retry" };
}
if (code === POLICY_CLOSE && !HALTING_CLOSE_REASONS_EXEMPT.includes(reason)) {
return { action: "halt", detail: `refused by the server: ${reason || "policy"}` };
}
if (code === CLOSE_CAPACITY) {
return { action: "retry", fast: false, detail: "server at capacity" };
}
return { action: "retry", fast: false, detail: `closed ${code} ${reason}`.trim() };
}
async onClosed(code, reason) {
this.clearTimers();
this.welcomed = false;
if (!this.wanted) {
return;
}
const verdict = this.classifyClose(code, reason);
if (verdict.action === "halt") {
await this.halt(verdict.detail);
return;
}
await this.scheduleRetry(verdict.detail, verdict.fast);
}
}
+135
View File
@@ -0,0 +1,135 @@
import { Config, ConfigError } from "./Config.js";
import { Fetcher } from "./Fetcher.js";
import { Limits } from "./Limits.js";
import { Policy } from "./Policy.js";
import { State, STATES } from "./State.js";
import { Usage } from "./Usage.js";
import { WsClient } from "./WsClient.js";
const ALARM = "devplace-node-backstop";
const ALARM_PERIOD_MINUTES = 1;
const DISCLOSURE_PAGE = "ui/disclosure.html";
const config = new Config(chrome.storage.local);
const usage = new Usage(chrome.storage.local);
const state = new State(chrome.storage.session);
const limits = new Limits();
const fetcher = new Fetcher({ policy: new Policy(), limits, usage });
const client = new WsClient({
config,
limits,
fetcher,
state,
clientVersion: chrome.runtime.getManifest().version,
platform: navigator.userAgent.includes("Firefox") ? "firefox" : "chrome",
});
let ready = null;
function boot() {
if (!ready) {
ready = (async () => {
await Promise.all([config.load(), usage.load(), state.load()]);
limits.setLocal(config.data);
})();
}
return ready;
}
async function reconcile() {
await boot();
limits.setLocal(config.data);
if (config.data.enabled && config.data.disclosed && config.hasSecret) {
await client.start();
} else if (client.wanted || state.data.state !== STATES.OFF) {
await client.stop();
}
}
async function status() {
await boot();
return {
config: config.view(),
state: state.snapshot(),
usage: usage.snapshot(),
live: { concurrency: limits.concurrency, bandwidthBytes: limits.bandwidthBytes },
};
}
const COMMANDS = {
status: async () => status(),
connect: async (message) => {
await config.connect(message);
await client.stop("credentials changed");
await reconcile();
return status();
},
accept: async () => {
await config.acceptDisclosure();
return status();
},
enable: async (message) => {
await config.setEnabled(Boolean(message.enabled));
await reconcile();
return status();
},
settings: async (message) => {
await config.saveSettings(message.values || {});
limits.setLocal(config.data);
return status();
},
forget: async () => {
await config.forget();
await client.stop("device forgotten");
return status();
},
resetUsage: async () => {
await usage.reset();
return status();
},
};
async function dispatch(message) {
await boot();
const command = COMMANDS[message && message.cmd];
if (!command) {
return { error: `unknown command ${String(message && message.cmd)}` };
}
try {
return await command(message);
} catch (error) {
if (error instanceof ConfigError) {
return { error: error.message };
}
await state.set({ lastError: error.message });
return { error: error.message };
}
}
chrome.runtime.onMessage.addListener((message, sender, respond) => {
if (sender.id !== chrome.runtime.id) {
return false;
}
dispatch(message).then(respond);
return true;
});
chrome.alarms.onAlarm.addListener((alarm) => {
if (alarm.name === ALARM) {
reconcile().catch((error) => state.set({ lastError: error.message }));
}
});
chrome.runtime.onInstalled.addListener((details) => {
chrome.alarms.create(ALARM, { periodInMinutes: ALARM_PERIOD_MINUTES });
if (details.reason === "install") {
chrome.tabs.create({ url: chrome.runtime.getURL(DISCLOSURE_PAGE) });
}
});
chrome.runtime.onStartup.addListener(() => {
chrome.alarms.create(ALARM, { periodInMinutes: ALARM_PERIOD_MINUTES });
reconcile().catch((error) => state.set({ lastError: error.message }));
});
reconcile().catch((error) => state.set({ lastError: error.message }));
+104
View File
@@ -0,0 +1,104 @@
export const PROTOCOL_VERSION = 1;
export const MIN_SUPPORTED_PROTOCOL = 1;
export const CLIENT_KIND = "extension";
export const MODE_FETCH = "fetch";
export const ALLOWED_PORTS = Object.freeze([80, 443]);
export const HTTPS_PORT = 443;
export const CLOSE_NORMAL = 1000;
export const CLOSE_PROTOCOL = 4400;
export const CLOSE_RETRY_OWNER = 4013;
export const CLOSE_CAPACITY = 4029;
export const CLOSE_KICKED = 4403;
export const CLOSE_POLICY = 1008;
export const MAX_REDIRECTS = 5;
export const MAX_RESPONSE_HEADERS = 64;
export const MAX_HEADER_VALUE_CHARS = 512;
export const MAX_BODY_CHUNKS = 65536;
export const MAX_BACKOFF_EXPONENT = 16;
export const LIMITS = Object.freeze({
max_frame_bytes: 256 * 1024,
max_body_bytes: 2 * 1024 * 1024,
max_concurrent_per_node: 2,
job_timeout_seconds: 25,
default_donor_mb_per_day: 500,
max_donor_mb_per_day: 20 * 1024,
ping_seconds: 20,
reconnect_min_seconds: 1,
reconnect_max_seconds: 60,
url_max_chars: 2048,
});
export const FRAME_CEILING_BYTES = Object.freeze({
job_result: 3 * 1024 * 1024,
default: 256 * 1024,
});
export const DESTINATIONS = Object.freeze([
Object.freeze({
suffix: "reddit.com",
adapter: "reddit",
methods: Object.freeze(["GET"]),
ports: Object.freeze([80, 443]),
modes: Object.freeze(["tunnel", "fetch"]),
max_rps: 0.5,
}),
Object.freeze({
suffix: "redd.it",
adapter: "reddit",
methods: Object.freeze(["GET"]),
ports: Object.freeze([80, 443]),
modes: Object.freeze(["tunnel", "fetch"]),
max_rps: 0.5,
}),
]);
export const REFUSALS = Object.freeze({
HOST_NOT_ALLOWED: "host_not_allowed",
PORT_NOT_ALLOWED: "port_not_allowed",
ADDRESS_NOT_PUBLIC: "address_not_public",
REDIRECT_NOT_ALLOWED: "redirect_not_allowed",
METHOD_NOT_ALLOWED: "method_not_allowed",
BANDWIDTH_EXHAUSTED: "bandwidth_exhausted",
CONCURRENCY_EXCEEDED: "concurrency_exceeded",
OUTSIDE_HOURS: "outside_hours",
BODY_TOO_LARGE: "body_too_large",
TIMEOUT: "timeout",
});
export const FETCH_FAILED = "fetch_failed";
export const SERVER_FRAMES = Object.freeze({
welcome: Object.freeze({
device_uid: "string",
limits: "object",
destinations: "array",
settings: "object",
}),
ping: Object.freeze({}),
job: Object.freeze({
job_id: "string",
method: "string",
url: "string",
timeout: "number",
}),
tunnel_open: Object.freeze({ conn_id: "string", host: "string", port: "number" }),
tunnel_data: Object.freeze({ conn_id: "string", b64: "string" }),
tunnel_close: Object.freeze({ conn_id: "string" }),
settings_update: Object.freeze({ settings: "object" }),
kick: Object.freeze({ reason: "string" }),
});
export const DROPPED_RESPONSE_HEADERS = Object.freeze([
"set-cookie",
"set-cookie2",
"authorization",
"proxy-authenticate",
]);
export const FORWARDED_REQUEST_HEADERS = Object.freeze(["accept", "accept-language"]);
export const HALTING_CLOSE_REASONS_EXEMPT = Object.freeze(["ticket", "heartbeat", "error", "capacity"]);
+197
View File
@@ -0,0 +1,197 @@
import test from "node:test";
import assert from "node:assert/strict";
import { Fetcher } from "../src/Fetcher.js";
import { Limits } from "../src/Limits.js";
import { Policy } from "../src/Policy.js";
import { Usage } from "../src/Usage.js";
import { MAX_REDIRECTS, REFUSALS } from "../src/spec.js";
function memoryArea() {
const store = {};
return {
async get(key) {
return key in store ? { [key]: store[key] } : {};
},
async set(values) {
Object.assign(store, values);
},
};
}
function build(fetchImpl, local = {}) {
const limits = new Limits({ local });
const usage = new Usage(memoryArea());
const fetcher = new Fetcher({ policy: new Policy(), limits, usage, fetchImpl });
return { fetcher, limits, usage };
}
const job = (url = "https://www.reddit.com/r/x.json", extra = {}) => ({
job_id: "j1",
method: "GET",
url,
timeout: 5,
...extra,
});
function response(body, init = {}) {
return new Response(body, { status: 200, ...init });
}
test("success returns a job_result with base64 body and no donor credentials", async () => {
const calls = [];
const { fetcher, usage } = build(async (url, options) => {
calls.push({ url, options });
return response("hello", { headers: { "content-type": "text/plain", "set-cookie": "a=b" } });
});
const result = await fetcher.execute(
job(undefined, { headers: { Cookie: "x=1", Authorization: "Bearer z", Accept: "text/html" } }),
);
assert.equal(result.type, "job_result");
assert.equal(result.status, 200);
assert.equal(Buffer.from(result.body_b64, "base64").toString(), "hello");
assert.equal(result.truncated, false);
assert.equal(result.headers["content-type"], "text/plain");
assert.equal("set-cookie" in result.headers, false);
assert.equal(calls[0].options.credentials, "omit");
assert.equal(calls[0].options.redirect, "manual");
assert.deepEqual(calls[0].options.headers, { accept: "text/html" });
assert.equal(usage.snapshot().requests, 1);
assert.equal(usage.snapshot().bytesDown, 5);
});
test("a disallowed url never reaches fetch", async () => {
let called = 0;
const { fetcher, limits } = build(async () => {
called += 1;
return response("x");
});
for (const url of ["https://evilreddit.com/", "http://reddit.com/", "https://reddit.com@evil.net/"]) {
const result = await fetcher.execute(job(url));
assert.equal(result.type, "job_error");
assert.equal(result.code, REFUSALS.HOST_NOT_ALLOWED);
}
assert.equal(called, 0);
assert.equal(limits.inflight, 0);
});
test("body over the cap is truncated and flagged", async () => {
const big = "a".repeat(5000);
const { fetcher, limits } = build(async () => response(big));
limits.caps.max_body_bytes = 1000;
const result = await fetcher.execute(job());
assert.equal(result.type, "job_result");
assert.equal(result.truncated, true);
assert.equal(Buffer.from(result.body_b64, "base64").length, 1000);
});
test("the local bandwidth budget also bounds the body", async () => {
const { fetcher, usage } = build(async () => response("b".repeat(3 * 1024 * 1024)), { bandwidthMb: 1 });
const result = await fetcher.execute(job());
assert.equal(result.truncated, true);
assert.ok(Buffer.from(result.body_b64, "base64").length < 1024 * 1024);
assert.ok(usage.usedBytes <= 1024 * 1024);
});
test("exhausted bandwidth refuses before fetching", async () => {
let called = 0;
const { fetcher, usage } = build(
async () => {
called += 1;
return response("x");
},
{ bandwidthMb: 1 },
);
await usage.record({ ok: true, up: 0, down: 1024 * 1024 });
const result = await fetcher.execute(job());
assert.equal(result.code, REFUSALS.BANDWIDTH_EXHAUSTED);
assert.equal(called, 0);
});
test("concurrency cap refuses the extra job", async () => {
let release;
const gate = new Promise((resolve) => {
release = resolve;
});
const { fetcher } = build(
async () => {
await gate;
return response("x");
},
{ maxConcurrent: 1 },
);
const first = fetcher.execute(job());
const second = await fetcher.execute(job());
assert.equal(second.code, REFUSALS.CONCURRENCY_EXCEEDED);
release();
assert.equal((await first).type, "job_result");
});
test("timeout aborts the request and reports timeout", async () => {
const { fetcher, limits } = build(
(url, options) =>
new Promise((resolve, reject) => {
options.signal.addEventListener("abort", () => reject(new DOMException("aborted", "AbortError")));
}),
);
limits.caps.job_timeout_seconds = 1;
const started = Date.now();
const result = await fetcher.execute(job(undefined, { timeout: 1 }));
assert.equal(result.code, REFUSALS.TIMEOUT);
assert.ok(Date.now() - started < 3000);
assert.equal(limits.inflight, 0);
});
test("a redirect leaving the allowlist is refused and not followed", async () => {
const urls = [];
const { fetcher } = build(async (url) => {
urls.push(url);
return response("", { status: 302, headers: { location: "https://evil.net/steal" } });
});
const result = await fetcher.execute(job());
assert.equal(result.type, "job_error");
assert.equal(result.code, REFUSALS.REDIRECT_NOT_ALLOWED);
assert.equal(urls.length, 1);
});
test("an opaque redirect fails closed", async () => {
const opaque = { type: "opaqueredirect", status: 0, headers: new Headers(), body: null };
const { fetcher } = build(async () => opaque);
const result = await fetcher.execute(job());
assert.equal(result.code, REFUSALS.REDIRECT_NOT_ALLOWED);
});
test("a same-allowlist redirect is followed, with the policy checked on every hop", async () => {
const urls = [];
const { fetcher } = build(async (url) => {
urls.push(url);
if (urls.length === 1) {
return response("", { status: 301, headers: { location: "https://old.reddit.com/r/x" } });
}
return response("final");
});
const result = await fetcher.execute(job());
assert.equal(result.type, "job_result");
assert.equal(Buffer.from(result.body_b64, "base64").toString(), "final");
assert.deepEqual(urls, ["https://www.reddit.com/r/x.json", "https://old.reddit.com/r/x"]);
});
test("redirect loops are bounded", async () => {
let count = 0;
const { fetcher } = build(async () => {
count += 1;
return response("", { status: 302, headers: { location: "https://www.reddit.com/loop" } });
});
const result = await fetcher.execute(job());
assert.equal(result.code, REFUSALS.REDIRECT_NOT_ALLOWED);
assert.equal(count, MAX_REDIRECTS + 1);
});
test("a network failure is a loud job_error", async () => {
const { fetcher, usage } = build(async () => {
throw new TypeError("network down");
});
const result = await fetcher.execute(job());
assert.equal(result.type, "job_error");
assert.equal(result.code, "fetch_failed");
assert.equal(usage.snapshot().failed, 1);
});
+91
View File
@@ -0,0 +1,91 @@
import test from "node:test";
import assert from "node:assert/strict";
import { Limits } from "../src/Limits.js";
import { LIMITS, REFUSALS } from "../src/spec.js";
const MB = 1024 * 1024;
function at(hour, minute = 0) {
return () => new Date(2026, 0, 15, hour, minute);
}
test("defaults come from the baked spec", () => {
const limits = new Limits();
assert.equal(limits.concurrency, LIMITS.max_concurrent_per_node);
assert.equal(limits.bandwidthBytes, LIMITS.default_donor_mb_per_day * MB);
assert.equal(limits.maxBodyBytes, LIMITS.max_body_bytes);
});
test("local settings are clamped", () => {
const limits = new Limits({ local: { bandwidthMb: 10 ** 9, maxConcurrent: 99 } });
assert.equal(limits.local.bandwidthMb, LIMITS.max_donor_mb_per_day);
assert.equal(limits.local.maxConcurrent, LIMITS.max_concurrent_per_node);
limits.setLocal({ bandwidthMb: -5, maxConcurrent: 0 });
assert.equal(limits.local.bandwidthMb, LIMITS.default_donor_mb_per_day);
assert.equal(limits.local.maxConcurrent, LIMITS.max_concurrent_per_node);
limits.setLocal({ activeFrom: "25:99", activeTo: "10:00" });
assert.equal(limits.local.activeFrom, "");
});
test("a welcome can lower caps and never raise them", () => {
const limits = new Limits();
limits.applyWelcome(
{
max_body_bytes: 10 ** 9,
max_concurrent_per_node: 1,
job_timeout_seconds: 5,
ping_seconds: 999,
reconnect_min_seconds: 2,
reconnect_max_seconds: 10 ** 6,
},
{ bandwidth_mb_day: 5, max_concurrent: 99 },
);
assert.equal(limits.maxBodyBytes, LIMITS.max_body_bytes);
assert.equal(limits.concurrency, 1);
assert.equal(limits.timeoutSeconds(25), 5);
assert.equal(limits.pingSeconds, LIMITS.ping_seconds);
assert.equal(limits.bandwidthBytes, 5 * MB);
assert.equal(limits.reconnectMin, 2);
assert.equal(limits.reconnectMax, LIMITS.reconnect_max_seconds);
});
test("concurrency cap admits and releases", () => {
const limits = new Limits({ local: { maxConcurrent: 1 } });
assert.equal(limits.admit(0), null);
assert.equal(limits.admit(0), REFUSALS.CONCURRENCY_EXCEEDED);
limits.release();
assert.equal(limits.admit(0), null);
limits.release();
limits.release();
assert.equal(limits.inflight, 0);
});
test("bandwidth cap refuses when used up", () => {
const limits = new Limits({ local: { bandwidthMb: 1 } });
assert.equal(limits.admit(MB - 1), null);
limits.release();
assert.equal(limits.admit(MB), REFUSALS.BANDWIDTH_EXHAUSTED);
assert.equal(limits.remainingBytes(2 * MB), 0);
});
test("active hours, including a window that wraps midnight", () => {
const day = new Limits({ clock: at(12), local: { activeFrom: "09:00", activeTo: "17:00" } });
assert.equal(day.admit(0), null);
const early = new Limits({ clock: at(8, 59), local: { activeFrom: "09:00", activeTo: "17:00" } });
assert.equal(early.admit(0), REFUSALS.OUTSIDE_HOURS);
const end = new Limits({ clock: at(17), local: { activeFrom: "09:00", activeTo: "17:00" } });
assert.equal(end.admit(0), REFUSALS.OUTSIDE_HOURS);
const wrapIn = new Limits({ clock: at(23, 30), local: { activeFrom: "22:00", activeTo: "06:00" } });
assert.equal(wrapIn.inActiveHours(), true);
const wrapEarly = new Limits({ clock: at(3), local: { activeFrom: "22:00", activeTo: "06:00" } });
assert.equal(wrapEarly.inActiveHours(), true);
const wrapOut = new Limits({ clock: at(12), local: { activeFrom: "22:00", activeTo: "06:00" } });
assert.equal(wrapOut.inActiveHours(), false);
const always = new Limits({ clock: at(12) });
assert.equal(always.inActiveHours(), true);
});
test("hello limits advertise the effective concurrency", () => {
const limits = new Limits({ local: { maxConcurrent: 1 } });
assert.deepEqual(limits.helloLimits(), { max_concurrent: 1 });
});
+114
View File
@@ -0,0 +1,114 @@
import test from "node:test";
import assert from "node:assert/strict";
import { Policy } from "../src/Policy.js";
import { DESTINATIONS, REFUSALS } from "../src/spec.js";
const policy = new Policy();
test("allows https on the allowlisted hosts and subdomains", () => {
for (const url of [
"https://reddit.com/r/python.json",
"https://www.reddit.com/r/python.json",
"https://old.reddit.com/",
"https://redd.it/abc",
"https://REDDIT.COM./x",
]) {
assert.equal(policy.check("GET", url).ok, true, url);
}
});
test("suffix matching is dot safe", () => {
for (const url of [
"https://evilreddit.com/",
"https://reddit.com.evil.net/",
"https://notredd.it/",
"https://reddit.co/",
]) {
const verdict = policy.check("GET", url);
assert.equal(verdict.ok, false, url);
assert.equal(verdict.code, REFUSALS.HOST_NOT_ALLOWED);
}
});
test("userinfo tricks are refused", () => {
for (const url of [
"https://reddit.com@evil.net/",
"https://user:pass@reddit.com/",
"https://evil.net\\@reddit.com/",
]) {
assert.equal(policy.check("GET", url).ok, false, url);
}
});
test("non https schemes are refused", () => {
for (const url of ["http://reddit.com/", "ftp://reddit.com/", "ws://reddit.com/", "file:///etc/passwd"]) {
const verdict = policy.check("GET", url);
assert.equal(verdict.ok, false, url);
}
});
test("only port 443 passes over https", () => {
assert.equal(policy.check("GET", "https://reddit.com:443/").ok, true);
for (const url of ["https://reddit.com:80/", "https://reddit.com:8443/", "https://reddit.com:22/"]) {
const verdict = policy.check("GET", url);
assert.equal(verdict.ok, false, url);
assert.equal(verdict.code, REFUSALS.PORT_NOT_ALLOWED);
}
});
test("method must be permitted", () => {
const verdict = policy.check("POST", "https://reddit.com/");
assert.equal(verdict.code, REFUSALS.METHOD_NOT_ALLOWED);
assert.equal(policy.check("get", "https://reddit.com/").ok, true);
});
test("ip literals, junk and oversize urls are refused", () => {
for (const url of ["https://127.0.0.1/", "https://[::1]/", "https://10.0.0.1/", "nonsense", "", null, 5]) {
assert.equal(policy.check("GET", url).ok, false, String(url));
}
assert.equal(policy.check("GET", "https://reddit.com/" + "a".repeat(3000)).ok, false);
});
test("a redirect leaving the allowlist is refused with redirect_not_allowed", () => {
const base = new URL("https://www.reddit.com/r/x");
const bad = policy.checkRedirect("https://evil.net/landing", base, "GET");
assert.equal(bad.ok, false);
assert.equal(bad.code, REFUSALS.REDIRECT_NOT_ALLOWED);
assert.equal(policy.checkRedirect("http://reddit.com/x", base, "GET").code, REFUSALS.REDIRECT_NOT_ALLOWED);
assert.equal(policy.checkRedirect("//evilreddit.com/x", base, "GET").code, REFUSALS.REDIRECT_NOT_ALLOWED);
assert.equal(policy.checkRedirect("/r/y", base, "GET").ok, true);
assert.equal(policy.checkRedirect("https://old.reddit.com/r/y", base, "GET").ok, true);
});
test("a refreshed allowlist may only be equal or narrower", () => {
const narrower = [{ ...DESTINATIONS[0], suffix: "old.reddit.com" }];
const kept = Policy.narrow(DESTINATIONS, narrower);
assert.equal(kept.destinations.length, 1);
assert.equal(kept.rejected.length, 0);
const equal = Policy.narrow(DESTINATIONS, DESTINATIONS.map((item) => ({ ...item })));
assert.equal(equal.destinations.length, DESTINATIONS.length);
const widened = Policy.narrow(DESTINATIONS, [
{ suffix: "evil.net", adapter: "x", methods: ["GET"], ports: [443], modes: ["fetch"] },
{ suffix: "reddit.com", adapter: "reddit", methods: ["GET", "POST"], ports: [443], modes: ["fetch"] },
{ suffix: "reddit.com", adapter: "reddit", methods: ["GET"], ports: [443, 8080], modes: ["fetch"] },
{ suffix: "reddit.com", adapter: "reddit", methods: ["GET"], ports: [443], modes: ["tunnel"] },
{ suffix: "evilreddit.com", adapter: "reddit", methods: ["GET"], ports: [443], modes: ["fetch"] },
]);
assert.equal(widened.destinations.length, 0);
assert.equal(widened.rejected.length, 5);
const empty = Policy.narrow(DESTINATIONS, []);
assert.equal(empty.destinations.length, 0);
assert.equal(Policy.narrow(DESTINATIONS, "x").destinations.length, 0);
});
test("a narrowed policy refuses what the baked policy would allow", () => {
const { policy: narrow } = Policy.fromRefreshed(DESTINATIONS, [
{ ...DESTINATIONS[0], suffix: "old.reddit.com" },
]);
assert.equal(narrow.check("GET", "https://old.reddit.com/").ok, true);
assert.equal(narrow.check("GET", "https://www.reddit.com/").ok, false);
assert.equal(narrow.check("GET", "https://redd.it/x").ok, false);
});
+38
View File
@@ -0,0 +1,38 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { DESTINATIONS, LIMITS, REFUSALS, ALLOWED_PORTS, PROTOCOL_VERSION } from "../src/spec.js";
const source = readFileSync(
fileURLToPath(new URL("../../../devplacepy/services/nodes/spec.py", import.meta.url)),
"utf8",
);
function evaluate(expression) {
const clean = expression.replaceAll("_", "").trim();
assert.match(clean, /^[\d\s*]+$/, expression);
return clean.split("*").reduce((product, part) => product * Number(part), 1);
}
test("baked allowlist matches the server specification", () => {
const suffixes = [...source.matchAll(/DestinationSpec\("([^"]+)"/g)].map((match) => match[1]);
assert.deepEqual(DESTINATIONS.map((item) => item.suffix), suffixes);
});
test("baked ports, protocol and refusal codes match the server specification", () => {
const ports = /ALLOWED_PORTS = \(([^)]*)\)/.exec(source)[1].split(",").filter(Boolean).map(Number);
assert.deepEqual([...ALLOWED_PORTS], ports);
assert.equal(PROTOCOL_VERSION, Number(/PROTOCOL_VERSION = (\d+)/.exec(source)[1]));
const codes = [...source.matchAll(/RefusalSpec\("([a-z_]+)"/g)].map((match) => match[1]);
assert.deepEqual(Object.values(REFUSALS).sort(), codes.sort());
});
test("baked limits match the server LimitSpec defaults", () => {
for (const [key, value] of Object.entries(LIMITS)) {
const pattern = new RegExp(`^\\s+${key}: int = ([\\d_ *]+)$`, "m");
const match = pattern.exec(source);
assert.ok(match, `${key} missing from spec.py`);
assert.equal(value, evaluate(match[1]), key);
}
});
+323
View File
@@ -0,0 +1,323 @@
import test from "node:test";
import assert from "node:assert/strict";
import { Fetcher } from "../src/Fetcher.js";
import { Limits } from "../src/Limits.js";
import { Policy } from "../src/Policy.js";
import { State, STATES } from "../src/State.js";
import { Usage } from "../src/Usage.js";
import { WsClient } from "../src/WsClient.js";
import { DESTINATIONS } from "../src/spec.js";
function memoryArea() {
const store = {};
return {
async get(key) {
return key in store ? { [key]: store[key] } : {};
},
async set(values) {
Object.assign(store, values);
},
};
}
class FakeSocket {
static instances = [];
constructor(address) {
this.address = address;
this.readyState = 0;
this.sent = [];
this.closed = null;
FakeSocket.instances.push(this);
}
open() {
this.readyState = 1;
this.onopen();
}
send(text) {
this.sent.push(JSON.parse(text));
}
close(code, reason) {
this.readyState = 3;
this.closed = { code, reason };
}
receive(frame) {
return this.onmessage({ data: typeof frame === "string" ? frame : JSON.stringify(frame) });
}
serverClose(code, reason = "") {
this.readyState = 3;
return this.onclose({ code, reason });
}
}
class FakeTimers {
constructor() {
this.timeouts = [];
this.intervals = [];
}
setTimeout(fn, ms) {
const handle = { fn, ms, cleared: false };
this.timeouts.push(handle);
return handle;
}
clearTimeout(handle) {
handle.cleared = true;
}
setInterval(fn, ms) {
const handle = { fn, ms, cleared: false };
this.intervals.push(handle);
return handle;
}
clearInterval(handle) {
handle.cleared = true;
}
}
function build(overrides = {}) {
FakeSocket.instances = [];
const timers = new FakeTimers();
const limits = new Limits();
const usage = new Usage(memoryArea());
const state = new State(memoryArea());
const fetchLog = [];
const jobFetch = overrides.jobFetch || (async () => new Response("page", { status: 200 }));
const fetcher = new Fetcher({ policy: new Policy(), limits, usage, fetchImpl: jobFetch });
const ticketFetch =
overrides.ticketFetch ||
(async (url, options) => {
fetchLog.push({ url, options });
return new Response(JSON.stringify({ ticket: "tk1", expires_in: 30, ws_path: "/nodes/ws" }), {
status: 200,
});
});
const client = new WsClient({
config: { credentials: () => ({ serverUrl: "https://devplace.net", secret: "dpn_secretsecretsecret" }) },
limits,
fetcher,
state,
fetchImpl: ticketFetch,
WebSocketImpl: FakeSocket,
timers,
random: overrides.random || (() => 0.5),
clientVersion: "9.9.9",
platform: "firefox",
});
return { client, timers, state, fetcher, limits, usage, fetchLog };
}
const welcome = (extra = {}) => ({
type: "welcome",
device_uid: "dev1",
limits: { max_concurrent_per_node: 2, ping_seconds: 20 },
destinations: DESTINATIONS.map((item) => ({ ...item })),
settings: { bandwidth_mb_day: 500, max_concurrent: 2 },
...extra,
});
async function online(context) {
await context.client.start();
const socket = FakeSocket.instances.at(-1);
socket.open();
await socket.receive(welcome());
return socket;
}
test("start fetches a ticket with the bearer secret and opens the socket", async () => {
const context = build();
await context.client.start();
assert.equal(context.fetchLog[0].url, "https://devplace.net/nodes/ticket");
assert.equal(context.fetchLog[0].options.headers.Authorization, "Bearer dpn_secretsecretsecret");
assert.equal(context.fetchLog[0].options.credentials, "omit");
const socket = FakeSocket.instances[0];
assert.equal(socket.address, "wss://devplace.net/nodes/ws?ticket=tk1");
});
test("hello is sent on open with extension kind and fetch mode only", async () => {
const context = build();
await context.client.start();
const socket = FakeSocket.instances[0];
socket.open();
const hello = socket.sent[0];
assert.equal(hello.type, "hello");
assert.equal(hello.protocol, 1);
assert.equal(hello.kind, "extension");
assert.deepEqual(hello.modes, ["fetch"]);
assert.equal(hello.client_version, "9.9.9");
assert.equal(hello.platform, "firefox");
assert.deepEqual(hello.limits, { max_concurrent: 2 });
});
test("welcome puts the client online and starts the keepalive", async () => {
const context = build();
await online(context);
assert.equal(context.state.snapshot().state, STATES.ONLINE);
assert.equal(context.state.snapshot().deviceUid, "dev1");
assert.equal(context.timers.intervals[0].ms, 20000);
});
test("keepalive timer sends pong and a server ping is answered with pong", async () => {
const context = build();
const socket = await online(context);
context.timers.intervals[0].fn();
await socket.receive({ type: "ping" });
assert.deepEqual(
socket.sent.filter((frame) => frame.type === "pong"),
[{ type: "pong" }, { type: "pong" }],
);
});
test("a job is executed and answered with job_result then usage", async () => {
const context = build();
const socket = await online(context);
await socket.receive({
type: "job",
job_id: "j9",
method: "GET",
url: "https://www.reddit.com/r/x.json",
timeout: 5,
});
await context.client.idle();
const result = socket.sent.find((frame) => frame.type === "job_result");
assert.equal(result.job_id, "j9");
assert.equal(Buffer.from(result.body_b64, "base64").toString(), "page");
assert.equal(socket.sent.some((frame) => frame.type === "usage"), true);
});
test("a job for a forbidden host is answered with job_error", async () => {
const context = build();
const socket = await online(context);
await socket.receive({ type: "job", job_id: "j2", method: "GET", url: "https://evilreddit.com/", timeout: 5 });
await context.client.idle();
const error = socket.sent.find((frame) => frame.type === "job_error");
assert.deepEqual(error, { type: "job_error", job_id: "j2", code: "host_not_allowed" });
});
test("a welcome that widens the allowlist is ignored for the widened entries", async () => {
const context = build();
await context.client.start();
const socket = FakeSocket.instances[0];
socket.open();
await socket.receive(
welcome({
destinations: [
...DESTINATIONS.map((item) => ({ ...item })),
{ suffix: "evil.net", adapter: "x", methods: ["GET"], ports: [443], modes: ["fetch"] },
],
}),
);
assert.match(context.state.snapshot().lastError, /evil\.net/);
await socket.receive({ type: "job", job_id: "j3", method: "GET", url: "https://evil.net/", timeout: 5 });
await context.client.idle();
assert.equal(socket.sent.find((frame) => frame.type === "job_error").code, "host_not_allowed");
});
test("a job before welcome is a protocol fault and closes with 4400", async () => {
const context = build();
await context.client.start();
const socket = FakeSocket.instances[0];
socket.open();
await socket.receive({ type: "job", job_id: "j", method: "GET", url: "https://reddit.com/", timeout: 5 });
assert.equal(socket.closed.code, 4400);
assert.equal(context.state.snapshot().state, STATES.BACKOFF);
});
test("malformed frames are rejected loudly", async () => {
const context = build();
const socket = await online(context);
for (const bad of ["not json", "[]", JSON.stringify({ type: "nope" }), JSON.stringify({ type: "job", job_id: "x" })]) {
await assert.rejects(context.client.handleMessage(bad));
}
await assert.rejects(context.client.handleMessage(new ArrayBuffer(4)));
assert.equal(socket.sent.filter((frame) => frame.type === "job_result").length, 0);
});
test("kick halts without reconnecting", async () => {
const context = build();
const socket = await online(context);
await socket.receive({ type: "kick", reason: "replaced" });
assert.equal(context.state.snapshot().state, STATES.HALTED);
assert.match(context.state.snapshot().detail, /replaced/);
assert.equal(context.timers.timeouts.filter((handle) => !handle.cleared).length, 0);
await context.client.ensure();
assert.equal(FakeSocket.instances.length, 1);
});
test("a normal close schedules a full jitter reconnect within the bounds", async () => {
const context = build({ random: () => 0.999 });
const socket = await online(context);
await socket.serverClose(1006);
const timer = context.timers.timeouts.at(-1);
assert.ok(timer.ms >= 1000 && timer.ms <= 60000, String(timer.ms));
assert.equal(context.state.snapshot().state, STATES.BACKOFF);
timer.fn();
await new Promise((resolve) => setImmediate(resolve));
assert.equal(FakeSocket.instances.length, 2);
});
test("backoff grows with attempts and never exceeds reconnect_max", async () => {
const context = build({ random: () => 1 });
await context.client.start();
const delays = [];
for (let index = 0; index < 12; index += 1) {
const socket = FakeSocket.instances.at(-1);
await socket.serverClose(1006);
const timer = context.timers.timeouts.at(-1);
delays.push(timer.ms);
timer.fn();
await new Promise((resolve) => setImmediate(resolve));
}
assert.ok(delays[1] > delays[0]);
assert.ok(Math.max(...delays) <= 60000);
assert.equal(delays.at(-1), 60000);
});
test("retry owner close reconnects at the minimum delay", async () => {
const context = build({ random: () => 1 });
const socket = await online(context);
await socket.serverClose(4013);
assert.equal(context.timers.timeouts.at(-1).ms, 1000);
});
test("a consent or device refusal halts instead of hammering the server", async () => {
const context = build();
await context.client.start();
const socket = FakeSocket.instances[0];
await socket.serverClose(1008, "no_consent");
assert.equal(context.state.snapshot().state, STATES.HALTED);
assert.match(context.state.snapshot().detail, /no_consent/);
});
test("a ticket 401 halts, a ticket 500 retries", async () => {
const halted = build({
ticketFetch: async () => new Response(JSON.stringify({ error: "x" }), { status: 401 }),
});
await halted.client.start();
assert.equal(halted.state.snapshot().state, STATES.HALTED);
assert.equal(FakeSocket.instances.length, 0);
const retried = build({ ticketFetch: async () => new Response("boom", { status: 500 }) });
await retried.client.start();
assert.equal(retried.state.snapshot().state, STATES.BACKOFF);
});
test("stop closes the socket and clears timers", async () => {
const context = build();
const socket = await online(context);
await context.client.stop("user");
assert.equal(socket.closed.code, 1000);
assert.equal(context.timers.intervals[0].cleared, true);
assert.equal(context.state.snapshot().state, STATES.OFF);
});
test("settings_update can only lower the effective limits", async () => {
const context = build();
const socket = await online(context);
await socket.receive({ type: "settings_update", settings: { bandwidth_mb_day: 10, max_concurrent: 1 } });
assert.equal(context.limits.concurrency, 1);
assert.equal(context.limits.bandwidthBytes, 10 * 1024 * 1024);
});
+55
View File
@@ -0,0 +1,55 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>DevPlace Node: what is shared</title>
<link rel="stylesheet" href="popup.css">
</head>
<body class="page">
<main>
<h1>Before you share anything</h1>
<p class="muted">DevPlace Node lets the community use a little of your internet connection to read public pages from sites that block data centers. Read this once. Nothing runs until you accept and switch it on.</p>
<section class="card">
<h2>What passes through your browser</h2>
<ul>
<li>The <strong>address of the page</strong> that is requested (host and full URL), for example a public Reddit thread.</li>
<li>The <strong>page content</strong> that comes back. It is read by this extension and sent to the DevPlace server.</li>
<li>Your <strong>public IP address</strong> is what the destination site sees. That is the point of donating.</li>
<li>How many requests and how many bytes you moved, counted on this device and on the server.</li>
</ul>
</section>
<section class="card">
<h2>What is never used</h2>
<ul>
<li>Your <strong>cookies, logins and accounts</strong>. Every request is sent without credentials, so you are never signed in as yourself.</li>
<li>Your browsing history, open tabs and other sites.</li>
<li>Any host outside the built-in list, any port other than the standard secure one, any address inside your own network.</li>
</ul>
</section>
<section class="card">
<h2>You stay in control</h2>
<ul>
<li>One on/off switch in the popup. Off means disconnected.</li>
<li>A daily data cap, a parallel request cap and optional active hours, enforced on this device even if the server asks for more.</li>
<li>Live counters of requests and bytes in the popup.</li>
<li>You can forget the device secret at any time, and revoke the device on your DevPlace profile.</li>
</ul>
<p class="muted">Pages fetched through your connection are public pages. Do not donate from a connection whose address you cannot afford to have associated with that traffic.</p>
</section>
<section class="card">
<div class="row">
<button type="button" class="primary" id="accept">I understand, allow it</button>
<button type="button" id="decline">Not now</button>
</div>
<p class="error" id="error" role="alert"></p>
<p class="muted" id="done"></p>
</section>
</main>
<script type="module" src="disclosure.js"></script>
</body>
</html>
+31
View File
@@ -0,0 +1,31 @@
class Disclosure {
constructor(root) {
this.root = root;
this.$ = (id) => root.getElementById(id);
}
async accept() {
try {
const reply = await chrome.runtime.sendMessage({ cmd: "accept" });
if (!reply || reply.error) {
throw new Error(reply ? reply.error : "The background worker did not answer");
}
this.$("error").textContent = "";
this.$("done").textContent =
"Accepted. Open the extension popup, connect your device secret and switch sharing on.";
} catch (error) {
this.$("error").textContent = error.message;
}
}
decline() {
this.$("done").textContent = "Nothing was enabled. You can close this tab.";
}
start() {
this.$("accept").addEventListener("click", () => this.accept());
this.$("decline").addEventListener("click", () => this.decline());
}
}
new Disclosure(document).start();
+226
View File
@@ -0,0 +1,226 @@
:root {
color-scheme: light dark;
--bg: #f6f7f9;
--card: #ffffff;
--text: #1c2128;
--muted: #5b6672;
--border: #d5dae0;
--accent: #2f6fed;
--accent-text: #ffffff;
--ok: #1a8a4a;
--warn: #b7791f;
--bad: #c0392b;
--radius: 8px;
--gap: 12px;
--font: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
}
@media (prefers-color-scheme: dark) {
:root {
--bg: #14181d;
--card: #1d232a;
--text: #e8ecf0;
--muted: #97a3b0;
--border: #323b45;
--accent: #5b92ff;
--accent-text: #0b1220;
--ok: #4cc27f;
--warn: #e0b04a;
--bad: #ee7b6e;
}
}
* {
box-sizing: border-box;
}
body {
margin: 0;
background: var(--bg);
color: var(--text);
font: 14px/1.45 var(--font);
width: 100%;
min-width: 260px;
max-width: 420px;
}
body.page {
max-width: 640px;
margin: 0 auto;
}
main {
padding: var(--gap);
display: grid;
gap: var(--gap);
}
h1 {
font-size: 1.15rem;
margin: 0;
}
h2 {
font-size: 0.95rem;
margin: 0 0 6px;
}
p {
margin: 0 0 8px;
}
.card {
background: var(--card);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: var(--gap);
}
.muted {
color: var(--muted);
font-size: 0.85rem;
}
.row {
display: flex;
flex-wrap: wrap;
align-items: center;
justify-content: space-between;
gap: 8px;
}
.status {
display: inline-flex;
align-items: center;
gap: 6px;
font-weight: 600;
}
.dot {
width: 10px;
height: 10px;
border-radius: 50%;
background: var(--muted);
}
.dot.online {
background: var(--ok);
}
.dot.connecting,
.dot.backoff {
background: var(--warn);
}
.dot.halted {
background: var(--bad);
}
.counters {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(110px, 1fr));
gap: 8px;
}
.counter {
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 8px;
}
.counter strong {
display: block;
font-size: 1.1rem;
}
label {
display: grid;
gap: 4px;
margin-bottom: 8px;
font-size: 0.85rem;
color: var(--muted);
}
input {
width: 100%;
font: inherit;
color: var(--text);
background: var(--bg);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 7px 9px;
}
button,
.button {
font: inherit;
cursor: pointer;
color: var(--text);
background: var(--card);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 7px 12px;
text-decoration: none;
}
button.primary {
background: var(--accent);
color: var(--accent-text);
border-color: var(--accent);
}
button.danger {
color: var(--bad);
}
button:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.switch {
display: inline-flex;
align-items: center;
gap: 8px;
margin: 0;
color: var(--text);
font-size: 1rem;
font-weight: 600;
}
.switch input {
width: 20px;
height: 20px;
}
.notice {
border-left: 3px solid var(--warn);
padding-left: 10px;
}
.error {
color: var(--bad);
min-height: 1.2em;
word-break: break-word;
}
.pair {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(110px, 1fr));
gap: 8px;
}
details > summary {
cursor: pointer;
font-weight: 600;
margin-bottom: 8px;
}
ul {
margin: 0 0 8px;
padding-left: 20px;
}
li {
margin-bottom: 4px;
}
+77
View File
@@ -0,0 +1,77 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>DevPlace Node</title>
<link rel="stylesheet" href="popup.css">
</head>
<body>
<main>
<div class="row">
<h1>DevPlace Node</h1>
<span class="status"><span class="dot" id="dot"></span><span id="state">off</span></span>
</div>
<section class="card" id="disclosure-card" hidden>
<p class="notice">Before this can run you must read what is shared.</p>
<a class="button primary" id="open-disclosure" href="disclosure.html" target="_blank" rel="noopener">Read the disclosure</a>
</section>
<section class="card">
<div class="row">
<label class="switch"><input type="checkbox" id="toggle"> <span id="toggle-label">Sharing off</span></label>
<span class="muted" id="label"></span>
</div>
<p class="muted" id="detail"></p>
<p class="error" id="error" role="alert"></p>
</section>
<section class="card">
<h2>Today</h2>
<div class="counters">
<div class="counter"><span class="muted">Requests</span><strong id="requests">0</strong></div>
<div class="counter"><span class="muted">Downloaded</span><strong id="down">0 B</strong></div>
<div class="counter"><span class="muted">Uploaded</span><strong id="up">0 B</strong></div>
<div class="counter"><span class="muted">Failed</span><strong id="failed">0</strong></div>
</div>
<p class="muted" id="lifetime"></p>
</section>
<section class="card">
<details id="settings-box">
<summary>Limits on this device</summary>
<form id="settings">
<div class="pair">
<label>Data per day (MB)<input type="number" id="bandwidth" min="1" step="1"></label>
<label>Parallel requests<input type="number" id="concurrency" min="1" step="1"></label>
</div>
<div class="pair">
<label>Active from<input type="time" id="from"></label>
<label>Active to<input type="time" id="to"></label>
</div>
<p class="muted">Leave the hours empty to share at any time. These limits are enforced here, whatever the server asks.</p>
<button type="submit">Save limits</button>
</form>
</details>
</section>
<section class="card">
<details id="connect-box">
<summary id="connect-summary">Connect to a server</summary>
<form id="connect">
<label>Server URL<input type="url" id="server" placeholder="https://devplace.net" required></label>
<label>Device secret<input type="password" id="secret" placeholder="dpn_..." autocomplete="off" required></label>
<label>Device label<input type="text" id="device-label" maxlength="64"></label>
<button type="submit" class="primary">Save and connect</button>
</form>
<p class="muted" id="saved"></p>
<button type="button" class="danger" id="forget" hidden>Forget this device</button>
</details>
</section>
<p class="muted"><a href="disclosure.html" target="_blank" rel="noopener">What is shared</a></p>
</main>
<script type="module" src="popup.js"></script>
</body>
</html>
+155
View File
@@ -0,0 +1,155 @@
const POLL_MS = 2000;
const UNITS = ["B", "KB", "MB", "GB", "TB"];
const STATE_LABELS = {
off: "Off",
connecting: "Connecting",
online: "Online",
backoff: "Reconnecting",
halted: "Stopped",
};
function formatBytes(value) {
let size = Math.max(0, Number(value) || 0);
let unit = 0;
while (size >= 1024 && unit < UNITS.length - 1) {
size /= 1024;
unit += 1;
}
return `${unit === 0 ? size : size.toFixed(1)} ${UNITS[unit]}`;
}
class Popup {
constructor(root) {
this.root = root;
this.timer = null;
this.view = null;
this.$ = (id) => root.getElementById(id);
}
async send(message) {
const reply = await chrome.runtime.sendMessage(message);
if (!reply) {
throw new Error("The background worker did not answer");
}
if (reply.error) {
throw new Error(reply.error);
}
return reply;
}
showError(error) {
this.$("error").textContent = error ? error.message || String(error) : "";
}
async run(message) {
try {
this.render(await this.send(message));
this.showError(null);
} catch (error) {
this.showError(error);
}
}
render(view) {
this.view = view;
const { config, state, usage } = view;
this.$("dot").className = `dot ${state.state}`;
this.$("state").textContent = STATE_LABELS[state.state] || state.state;
this.$("detail").textContent = state.detail || state.lastError || "";
this.$("label").textContent = config.label;
this.$("disclosure-card").hidden = config.disclosed;
const toggle = this.$("toggle");
toggle.checked = config.enabled;
toggle.disabled = !config.disclosed || !config.hasSecret;
this.$("toggle-label").textContent = config.enabled ? "Sharing on" : "Sharing off";
this.$("requests").textContent = usage.requests;
this.$("failed").textContent = usage.failed;
this.$("down").textContent = formatBytes(usage.bytesDown);
this.$("up").textContent = formatBytes(usage.bytesUp);
this.$("lifetime").textContent =
`All time: ${usage.lifetimeRequests} requests, ${formatBytes(usage.lifetimeBytes)}`;
this.renderSettings(config);
this.renderConnection(config);
}
renderSettings(config) {
if (this.root.activeElement && this.$("settings").contains(this.root.activeElement)) {
return;
}
this.$("bandwidth").value = config.bandwidthMb;
this.$("concurrency").value = config.maxConcurrent;
this.$("from").value = config.activeFrom;
this.$("to").value = config.activeTo;
}
renderConnection(config) {
this.$("forget").hidden = !config.hasSecret;
this.$("connect-summary").textContent = config.hasSecret ? "Connection" : "Connect to a server";
this.$("saved").textContent = config.hasSecret
? `Connected to ${config.serverUrl} with secret ${config.secretPrefix}`
: "";
if (!config.hasSecret) {
this.$("connect-box").open = true;
}
}
async requestOrigin(serverUrl) {
const origin = new URL(serverUrl).origin;
const granted = await chrome.permissions.request({ origins: [origin + "/*"] });
if (!granted) {
throw new Error("Permission to talk to that server was not granted");
}
}
async onConnect(event) {
event.preventDefault();
const secretField = this.$("secret");
try {
await this.requestOrigin(this.$("server").value);
this.render(
await this.send({
cmd: "connect",
serverUrl: this.$("server").value,
secret: secretField.value,
label: this.$("device-label").value,
}),
);
secretField.value = "";
this.showError(null);
} catch (error) {
this.showError(error);
}
}
onSettings(event) {
event.preventDefault();
this.run({
cmd: "settings",
values: {
bandwidthMb: Number(this.$("bandwidth").value),
maxConcurrent: Number(this.$("concurrency").value),
activeFrom: this.$("from").value,
activeTo: this.$("to").value,
},
});
}
onForget() {
if (this.root.defaultView.confirm("Forget this device secret on this browser?")) {
this.run({ cmd: "forget" });
}
}
start() {
this.$("connect").addEventListener("submit", (event) => this.onConnect(event));
this.$("settings").addEventListener("submit", (event) => this.onSettings(event));
this.$("forget").addEventListener("click", () => this.onForget());
this.$("toggle").addEventListener("change", (event) =>
this.run({ cmd: "enable", enabled: event.target.checked }),
);
this.run({ cmd: "status" });
this.timer = setInterval(() => this.run({ cmd: "status" }), POLL_MS);
}
}
new Popup(document).start();