forked from retoor/devplacepy
Zo doen wijdat in oost
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
# DevPlace Node, browser extension
|
||||
|
||||
A Manifest V3 extension for Chrome and Firefox from one source tree. It lets a DevPlace member donate a little of their connection to the community node network, in **fetch mode only** (an extension has no raw sockets). The protocol is `devplacepy/services/nodes/spec.py`, served at `GET /nodes/spec.json`; this extension never restates a number from it except in `src/spec.js`, which a test compares against `spec.py`.
|
||||
|
||||
## Install (sideload)
|
||||
|
||||
1. Chrome: `chrome://extensions`, enable Developer mode, Load unpacked, pick this directory. Firefox: `about:debugging#/runtime/this-firefox`, Load Temporary Add-on, pick `manifest.json`. A temporary Firefox add-on is removed on restart.
|
||||
2. A disclosure page opens on first install. Read it and accept.
|
||||
3. On your DevPlace profile give consent to bandwidth sharing, register a device and copy the `dpn_...` secret (shown once).
|
||||
4. Open the popup, enter the server URL and the secret, grant the host permission for that server when asked, then switch sharing on.
|
||||
|
||||
Packaged build: `./build.sh <outdir>` writes `devplace-node-extension.zip` and `devplace-node-extension.zip.sha256`. The zip holds `manifest.json`, `src/`, `ui/` and this file; tests are not shipped.
|
||||
|
||||
## Chrome and Firefox from one tree
|
||||
|
||||
`manifest.json` declares both `background.service_worker` and `background.scripts`, both pointing at the same ES module.
|
||||
|
||||
| Key | Used by |
|
||||
|-----|---------|
|
||||
| `background.service_worker` | Chrome (service worker). Firefox ignores it with a console warning. |
|
||||
| `background.scripts` | Firefox (event page). Chrome ignores it when a service worker is present. |
|
||||
| `browser_specific_settings.gecko` | Firefox only (add-on id, minimum 128). Chrome warns about the unknown key and ignores it. |
|
||||
|
||||
Both use the `chrome.*` namespace with promises. Firefox store submission additionally needs a `data_collection_permissions` declaration, not added yet.
|
||||
|
||||
## How it works
|
||||
|
||||
- The background worker holds the WebSocket. It exchanges the device secret for a 30 second ticket (`POST /nodes/ticket`, `Authorization: Bearer dpn_...`) and opens `WS /nodes/ws?ticket=...`, then sends `hello` with `kind: "extension"` and `modes: ["fetch"]`.
|
||||
- A `pong` is sent every `ping_seconds` as a keepalive and for every server `ping`. A `chrome.alarms` alarm every minute is the reconnect backstop when the worker has been suspended.
|
||||
- Reconnects use full-jitter backoff between `reconnect_min_seconds` and `reconnect_max_seconds`. Close `4013` retries at the minimum delay. A kick (`4403`), a consent or device refusal (`1008` with a reason) or a rejected secret (HTTP 401/403) **halts** and shows the reason; nothing hammers the server.
|
||||
- State (connection, last error) is in `chrome.storage.session`. Config (server URL, secret, limits, disclosure flag) and the usage counters are in `chrome.storage.local`.
|
||||
- The secret is never logged and never shown after saving; the popup shows its first 8 characters only.
|
||||
|
||||
## Safety enforced in the extension, before every request
|
||||
|
||||
Independent of anything the server says:
|
||||
|
||||
- allowlist: the hosts baked into `src/spec.js` (`reddit.com`, `redd.it` and their subdomains, dot-safe suffix match). A refreshed list from `welcome` is accepted only entry by entry when it is equal or narrower (same or sub-suffix, methods and ports subsets, fetch mode present); a widening entry is dropped and reported in the popup
|
||||
- https only, port 443 only (the spec's 80 is never used because plain http is refused), no credentials in the URL, method must be permitted (GET)
|
||||
- `fetch` with `credentials: "omit"`, `redirect: "manual"`, `cache: "no-store"`, no referrer; only `accept` and `accept-language` are forwarded from the job, so no cookie or authorization header can be injected
|
||||
- each redirect hop is re-checked against the allowlist, at most 5 hops; a redirect that leaves the allowlist is refused with `redirect_not_allowed`
|
||||
- response body capped at `max_body_bytes` and by the remaining daily budget, with `truncated: true`; `set-cookie` style response headers are dropped
|
||||
- timeout by `AbortController`, daily bandwidth cap, concurrency cap and active hours from the local settings, each answered with its refusal code (`timeout`, `bandwidth_exhausted`, `concurrency_exceeded`, `outside_hours`). A server value can only lower these, never raise them
|
||||
|
||||
Known limit: browsers hide the target of a `redirect: "manual"` response (an opaque redirect), so the extension cannot read where it points. It fails closed with `redirect_not_allowed`. A followable redirect is only handled when the response exposes its `Location`.
|
||||
|
||||
## Permissions
|
||||
|
||||
| Permission | Why |
|
||||
|------------|-----|
|
||||
| `storage` | config, usage counters, connection state |
|
||||
| `alarms` | reconnect backstop while the worker is suspended |
|
||||
| `host_permissions`: `reddit.com`, `*.reddit.com`, `redd.it`, `*.redd.it` over https | fetch the allowlisted destinations without CORS. These are exactly the destinations in `spec.py`. |
|
||||
| `optional_host_permissions`: any https origin, `localhost`, `127.0.0.1` over http | requested at connect time for the one server origin you enter, so the ticket request works. Not granted at install. |
|
||||
|
||||
No `tabs`, `cookies`, `history`, `webRequest` or `<all_urls>` permission.
|
||||
|
||||
## What is and is not shared
|
||||
|
||||
Shared: the URL of the requested public page, the page content that comes back, your public IP address as seen by the destination, and request and byte counts. Not used: your cookies, logins, browsing history, tabs, other sites, any host outside the list, any non-https or non-443 destination. See `ui/disclosure.html`, shown on first install.
|
||||
|
||||
## Tests
|
||||
|
||||
`node --test nodes/extension/test/*.mjs` (no dependencies). They cover Policy, Limits, Fetcher with a stubbed `fetch`, WsClient with a fake WebSocket, and a drift check of `src/spec.js` against `spec.py`. Not covered by automated tests: `background.js` and the popup, which need a real browser.
|
||||
|
||||
## Gaps
|
||||
|
||||
- No icons yet; the browser shows a default.
|
||||
- The `welcome` frame carries no device label, so the label in the popup is local only.
|
||||
- Not tested in a real Chrome or Firefox session in this change.
|
||||
Executable
+41
@@ -0,0 +1,41 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
if [ "$#" -ne 1 ]; then
|
||||
echo "usage: build.sh <outdir>" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
here=$(cd "$(dirname "$0")" && pwd)
|
||||
mkdir -p "$1"
|
||||
out=$(cd "$1" && pwd)
|
||||
name=devplace-node-extension.zip
|
||||
archive="$out/$name"
|
||||
rm -f "$archive"
|
||||
|
||||
cd "$here"
|
||||
if command -v zip >/dev/null 2>&1; then
|
||||
zip -q -X -r "$archive" manifest.json src ui README.md
|
||||
else
|
||||
python3 - "$archive" <<'PY'
|
||||
import sys
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
archive = sys.argv[1]
|
||||
paths = [Path("manifest.json"), Path("README.md")]
|
||||
for folder in ("src", "ui"):
|
||||
paths.extend(sorted(item for item in Path(folder).rglob("*") if item.is_file()))
|
||||
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as bundle:
|
||||
for path in paths:
|
||||
bundle.write(path, path.as_posix())
|
||||
PY
|
||||
fi
|
||||
|
||||
cd "$out"
|
||||
if command -v sha256sum >/dev/null 2>&1; then
|
||||
sha256sum "$name" > "$name.sha256"
|
||||
else
|
||||
shasum -a 256 "$name" > "$name.sha256"
|
||||
fi
|
||||
cat "$name.sha256"
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"manifest_version": 3,
|
||||
"name": "DevPlace Node",
|
||||
"version": "0.1.0",
|
||||
"description": "Donate a little of your connection to the DevPlace community node network. Fetch mode only, allowlisted hosts only, never your cookies.",
|
||||
"background": {
|
||||
"service_worker": "src/background.js",
|
||||
"scripts": ["src/background.js"],
|
||||
"type": "module"
|
||||
},
|
||||
"action": {
|
||||
"default_title": "DevPlace Node",
|
||||
"default_popup": "ui/popup.html"
|
||||
},
|
||||
"permissions": ["storage", "alarms"],
|
||||
"host_permissions": [
|
||||
"https://reddit.com/*",
|
||||
"https://*.reddit.com/*",
|
||||
"https://redd.it/*",
|
||||
"https://*.redd.it/*"
|
||||
],
|
||||
"optional_host_permissions": [
|
||||
"https://*/*",
|
||||
"http://localhost/*",
|
||||
"http://127.0.0.1/*"
|
||||
],
|
||||
"browser_specific_settings": {
|
||||
"gecko": {
|
||||
"id": "node@devplace.net",
|
||||
"strict_min_version": "128.0"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"name": "devplace-node-extension",
|
||||
"private": true,
|
||||
"type": "module"
|
||||
}
|
||||
@@ -0,0 +1,126 @@
|
||||
import { LIMITS } from "./spec.js";
|
||||
|
||||
const KEY = "config";
|
||||
const SECRET_PREFIX = "dpn_";
|
||||
const SECRET_MIN_CHARS = 16;
|
||||
const SECRET_MAX_CHARS = 128;
|
||||
const LABEL_MAX_CHARS = 64;
|
||||
const SHOWN_SECRET_CHARS = 8;
|
||||
const LOOPBACK = ["localhost", "127.0.0.1", "[::1]"];
|
||||
|
||||
const DEFAULTS = Object.freeze({
|
||||
serverUrl: "",
|
||||
secret: "",
|
||||
label: "",
|
||||
disclosed: false,
|
||||
enabled: false,
|
||||
bandwidthMb: LIMITS.default_donor_mb_per_day,
|
||||
maxConcurrent: LIMITS.max_concurrent_per_node,
|
||||
activeFrom: "",
|
||||
activeTo: "",
|
||||
});
|
||||
|
||||
export class ConfigError extends Error {}
|
||||
|
||||
export class Config {
|
||||
constructor(area) {
|
||||
this.area = area;
|
||||
this.data = { ...DEFAULTS };
|
||||
}
|
||||
|
||||
static validateServerUrl(raw) {
|
||||
let url;
|
||||
try {
|
||||
url = new URL(String(raw).trim());
|
||||
} catch (error) {
|
||||
throw new ConfigError("The server URL is not a valid URL");
|
||||
}
|
||||
const loopback = LOOPBACK.includes(url.hostname);
|
||||
if (url.protocol !== "https:" && !(url.protocol === "http:" && loopback)) {
|
||||
throw new ConfigError("The server URL must use https");
|
||||
}
|
||||
if (url.username || url.password) {
|
||||
throw new ConfigError("The server URL must not contain credentials");
|
||||
}
|
||||
return url.origin;
|
||||
}
|
||||
|
||||
static validateSecret(raw) {
|
||||
const secret = String(raw).trim();
|
||||
if (
|
||||
!secret.startsWith(SECRET_PREFIX) ||
|
||||
secret.length < SECRET_MIN_CHARS ||
|
||||
secret.length > SECRET_MAX_CHARS ||
|
||||
/\s/.test(secret)
|
||||
) {
|
||||
throw new ConfigError("That is not a DevPlace device secret (it starts with dpn_)");
|
||||
}
|
||||
return secret;
|
||||
}
|
||||
|
||||
async load() {
|
||||
const stored = (await this.area.get(KEY))[KEY];
|
||||
this.data = { ...DEFAULTS, ...(stored && typeof stored === "object" ? stored : {}) };
|
||||
}
|
||||
|
||||
async save() {
|
||||
await this.area.set({ [KEY]: { ...this.data } });
|
||||
}
|
||||
|
||||
get hasSecret() {
|
||||
return Boolean(this.data.secret);
|
||||
}
|
||||
|
||||
get secretPrefix() {
|
||||
return this.data.secret ? this.data.secret.slice(0, SHOWN_SECRET_CHARS) + "..." : "";
|
||||
}
|
||||
|
||||
credentials() {
|
||||
return { serverUrl: this.data.serverUrl, secret: this.data.secret };
|
||||
}
|
||||
|
||||
view() {
|
||||
const { secret, ...rest } = this.data;
|
||||
return { ...rest, hasSecret: this.hasSecret, secretPrefix: this.secretPrefix };
|
||||
}
|
||||
|
||||
async connect({ serverUrl, secret, label }) {
|
||||
this.data.serverUrl = Config.validateServerUrl(serverUrl);
|
||||
this.data.secret = Config.validateSecret(secret);
|
||||
this.data.label = String(label || "").trim().slice(0, LABEL_MAX_CHARS);
|
||||
await this.save();
|
||||
}
|
||||
|
||||
async acceptDisclosure() {
|
||||
this.data.disclosed = true;
|
||||
await this.save();
|
||||
}
|
||||
|
||||
async setEnabled(enabled) {
|
||||
if (enabled && !this.data.disclosed) {
|
||||
throw new ConfigError("Read and accept the disclosure first");
|
||||
}
|
||||
if (enabled && !this.hasSecret) {
|
||||
throw new ConfigError("Connect a device secret first");
|
||||
}
|
||||
this.data.enabled = Boolean(enabled);
|
||||
await this.save();
|
||||
}
|
||||
|
||||
async saveSettings(values) {
|
||||
for (const key of ["bandwidthMb", "maxConcurrent", "activeFrom", "activeTo"]) {
|
||||
if (key in values) {
|
||||
this.data[key] = values[key];
|
||||
}
|
||||
}
|
||||
await this.save();
|
||||
}
|
||||
|
||||
async forget() {
|
||||
this.data.secret = "";
|
||||
this.data.serverUrl = "";
|
||||
this.data.label = "";
|
||||
this.data.enabled = false;
|
||||
await this.save();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,193 @@
|
||||
import {
|
||||
DROPPED_RESPONSE_HEADERS,
|
||||
FETCH_FAILED,
|
||||
FORWARDED_REQUEST_HEADERS,
|
||||
MAX_BODY_CHUNKS,
|
||||
MAX_HEADER_VALUE_CHARS,
|
||||
MAX_REDIRECTS,
|
||||
MAX_RESPONSE_HEADERS,
|
||||
REFUSALS,
|
||||
} from "./spec.js";
|
||||
|
||||
const REDIRECT_STATUSES = [301, 302, 303, 307, 308];
|
||||
const REQUEST_OVERHEAD_BYTES = 256;
|
||||
const BASE64_CHUNK = 0x8000;
|
||||
|
||||
export function toBase64(bytes) {
|
||||
let text = "";
|
||||
for (let index = 0; index < bytes.length; index += BASE64_CHUNK) {
|
||||
text += String.fromCharCode.apply(null, bytes.subarray(index, index + BASE64_CHUNK));
|
||||
}
|
||||
return btoa(text);
|
||||
}
|
||||
|
||||
function concat(parts, total) {
|
||||
const joined = new Uint8Array(total);
|
||||
let offset = 0;
|
||||
for (const part of parts) {
|
||||
joined.set(part, offset);
|
||||
offset += part.length;
|
||||
}
|
||||
return joined;
|
||||
}
|
||||
|
||||
export class Fetcher {
|
||||
constructor({ policy, limits, usage, fetchImpl, clock }) {
|
||||
this.policy = policy;
|
||||
this.limits = limits;
|
||||
this.usage = usage;
|
||||
this.fetchImpl = fetchImpl || ((...args) => globalThis.fetch(...args));
|
||||
this.clock = clock || (() => Date.now());
|
||||
}
|
||||
|
||||
setPolicy(policy) {
|
||||
this.policy = policy;
|
||||
}
|
||||
|
||||
failure(jobId, code) {
|
||||
return { type: "job_error", job_id: jobId, code };
|
||||
}
|
||||
|
||||
requestHeaders(raw) {
|
||||
const headers = {};
|
||||
if (!raw || typeof raw !== "object") {
|
||||
return headers;
|
||||
}
|
||||
for (const [name, value] of Object.entries(raw)) {
|
||||
const key = name.toLowerCase();
|
||||
if (FORWARDED_REQUEST_HEADERS.includes(key) && typeof value === "string") {
|
||||
headers[key] = value;
|
||||
}
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
responseHeaders(response) {
|
||||
const headers = {};
|
||||
let count = 0;
|
||||
response.headers.forEach((value, name) => {
|
||||
const key = name.toLowerCase();
|
||||
if (DROPPED_RESPONSE_HEADERS.includes(key) || count >= MAX_RESPONSE_HEADERS) {
|
||||
return;
|
||||
}
|
||||
headers[key] = String(value).slice(0, MAX_HEADER_VALUE_CHARS);
|
||||
count += 1;
|
||||
});
|
||||
return headers;
|
||||
}
|
||||
|
||||
isRedirect(response) {
|
||||
return response.type === "opaqueredirect" || REDIRECT_STATUSES.includes(response.status);
|
||||
}
|
||||
|
||||
async readBody(response, cap) {
|
||||
if (!response.body || typeof response.body.getReader !== "function") {
|
||||
const whole = new Uint8Array(await response.arrayBuffer());
|
||||
return whole.length > cap
|
||||
? { bytes: whole.slice(0, cap), truncated: true }
|
||||
: { bytes: whole, truncated: false };
|
||||
}
|
||||
const reader = response.body.getReader();
|
||||
const parts = [];
|
||||
let total = 0;
|
||||
let truncated = false;
|
||||
for (let chunks = 0; chunks < MAX_BODY_CHUNKS; chunks += 1) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) {
|
||||
return { bytes: concat(parts, total), truncated };
|
||||
}
|
||||
const room = cap - total;
|
||||
if (value.length > room) {
|
||||
parts.push(value.subarray(0, room));
|
||||
total += room;
|
||||
truncated = true;
|
||||
break;
|
||||
}
|
||||
parts.push(value);
|
||||
total += value.length;
|
||||
}
|
||||
truncated = true;
|
||||
await reader.cancel();
|
||||
return { bytes: concat(parts, total), truncated };
|
||||
}
|
||||
|
||||
async follow(job, verdict, signal) {
|
||||
let current = verdict.url;
|
||||
const headers = this.requestHeaders(job.headers);
|
||||
for (let hop = 0; hop <= MAX_REDIRECTS; hop += 1) {
|
||||
const response = await this.fetchImpl(current.href, {
|
||||
method: verdict.method,
|
||||
headers,
|
||||
credentials: "omit",
|
||||
redirect: "manual",
|
||||
cache: "no-store",
|
||||
referrerPolicy: "no-referrer",
|
||||
signal,
|
||||
});
|
||||
if (!this.isRedirect(response)) {
|
||||
return { response, final: current };
|
||||
}
|
||||
const location = response.headers.get("location");
|
||||
if (response.type === "opaqueredirect" || !location || hop === MAX_REDIRECTS) {
|
||||
if (response.body && typeof response.body.cancel === "function") {
|
||||
await response.body.cancel();
|
||||
}
|
||||
return { refused: REFUSALS.REDIRECT_NOT_ALLOWED };
|
||||
}
|
||||
const next = this.policy.checkRedirect(location, current, verdict.method);
|
||||
if (response.body && typeof response.body.cancel === "function") {
|
||||
await response.body.cancel();
|
||||
}
|
||||
if (!next.ok) {
|
||||
return { refused: REFUSALS.REDIRECT_NOT_ALLOWED };
|
||||
}
|
||||
current = next.url;
|
||||
}
|
||||
return { refused: REFUSALS.REDIRECT_NOT_ALLOWED };
|
||||
}
|
||||
|
||||
async execute(job) {
|
||||
const verdict = this.policy.check(job.method, job.url);
|
||||
if (!verdict.ok) {
|
||||
return this.failure(job.job_id, verdict.code);
|
||||
}
|
||||
const refusal = this.limits.admit(this.usage.usedBytes);
|
||||
if (refusal) {
|
||||
return this.failure(job.job_id, refusal);
|
||||
}
|
||||
const started = this.clock();
|
||||
const up = job.url.length + REQUEST_OVERHEAD_BYTES;
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(
|
||||
() => controller.abort(),
|
||||
this.limits.timeoutSeconds(job.timeout) * 1000,
|
||||
);
|
||||
try {
|
||||
const outcome = await this.follow(job, verdict, controller.signal);
|
||||
if (outcome.refused) {
|
||||
await this.usage.record({ ok: false, up, down: 0 });
|
||||
return this.failure(job.job_id, outcome.refused);
|
||||
}
|
||||
const budget = this.limits.remainingBytes(this.usage.usedBytes + up);
|
||||
const cap = Math.min(this.limits.maxBodyBytes, budget);
|
||||
const body = await this.readBody(outcome.response, cap);
|
||||
const down = body.bytes.length;
|
||||
await this.usage.record({ ok: true, up, down });
|
||||
return {
|
||||
type: "job_result",
|
||||
job_id: job.job_id,
|
||||
status: outcome.response.status,
|
||||
headers: this.responseHeaders(outcome.response),
|
||||
body_b64: toBase64(body.bytes),
|
||||
truncated: body.truncated,
|
||||
elapsed_ms: Math.max(0, Math.round(this.clock() - started)),
|
||||
};
|
||||
} catch (error) {
|
||||
await this.usage.record({ ok: false, up, down: 0 });
|
||||
return this.failure(job.job_id, controller.signal.aborted ? REFUSALS.TIMEOUT : FETCH_FAILED);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
this.limits.release();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,167 @@
|
||||
import { LIMITS, REFUSALS } from "./spec.js";
|
||||
|
||||
const MB = 1024 * 1024;
|
||||
const HOURS = /^([01]\d|2[0-3]):([0-5]\d)$/;
|
||||
const CAP_KEYS = [
|
||||
"max_frame_bytes",
|
||||
"max_body_bytes",
|
||||
"max_concurrent_per_node",
|
||||
"job_timeout_seconds",
|
||||
"ping_seconds",
|
||||
"url_max_chars",
|
||||
];
|
||||
|
||||
function isCount(value) {
|
||||
return typeof value === "number" && Number.isInteger(value) && value >= 1;
|
||||
}
|
||||
|
||||
function clampInt(value, low, high, fallback) {
|
||||
if (!isCount(value)) {
|
||||
return fallback;
|
||||
}
|
||||
return Math.min(high, Math.max(low, value));
|
||||
}
|
||||
|
||||
function minutes(text) {
|
||||
const match = HOURS.exec(text);
|
||||
return match ? Number(match[1]) * 60 + Number(match[2]) : null;
|
||||
}
|
||||
|
||||
export class Limits {
|
||||
constructor(options = {}) {
|
||||
this.baked = options.baked || LIMITS;
|
||||
this.clock = options.clock || (() => new Date());
|
||||
this.caps = {};
|
||||
for (const key of CAP_KEYS) {
|
||||
this.caps[key] = this.baked[key];
|
||||
}
|
||||
this.reconnectMin = this.baked.reconnect_min_seconds;
|
||||
this.reconnectMax = this.baked.reconnect_max_seconds;
|
||||
this.serverBandwidthMb = this.baked.max_donor_mb_per_day;
|
||||
this.serverConcurrent = this.baked.max_concurrent_per_node;
|
||||
this.local = Limits.normaliseLocal({}, this.baked);
|
||||
this.inflight = 0;
|
||||
if (options.local) {
|
||||
this.setLocal(options.local);
|
||||
}
|
||||
}
|
||||
|
||||
static normaliseLocal(values, baked = LIMITS) {
|
||||
const from = minutes(values.activeFrom || "");
|
||||
const to = minutes(values.activeTo || "");
|
||||
const both = from !== null && to !== null;
|
||||
return {
|
||||
bandwidthMb: clampInt(
|
||||
values.bandwidthMb,
|
||||
1,
|
||||
baked.max_donor_mb_per_day,
|
||||
baked.default_donor_mb_per_day,
|
||||
),
|
||||
maxConcurrent: clampInt(
|
||||
values.maxConcurrent,
|
||||
1,
|
||||
baked.max_concurrent_per_node,
|
||||
baked.max_concurrent_per_node,
|
||||
),
|
||||
activeFrom: both ? values.activeFrom : "",
|
||||
activeTo: both ? values.activeTo : "",
|
||||
};
|
||||
}
|
||||
|
||||
setLocal(values) {
|
||||
this.local = Limits.normaliseLocal({ ...this.local, ...values }, this.baked);
|
||||
}
|
||||
|
||||
applyWelcome(welcomeLimits, serverSettings) {
|
||||
const incoming = welcomeLimits && typeof welcomeLimits === "object" ? welcomeLimits : {};
|
||||
for (const key of CAP_KEYS) {
|
||||
this.caps[key] = Math.min(this.baked[key], isCount(incoming[key]) ? incoming[key] : this.baked[key]);
|
||||
}
|
||||
const ceiling = this.baked.reconnect_max_seconds;
|
||||
this.reconnectMin = clampInt(incoming.reconnect_min_seconds, 1, ceiling, this.baked.reconnect_min_seconds);
|
||||
this.reconnectMax = clampInt(incoming.reconnect_max_seconds, this.reconnectMin, ceiling, ceiling);
|
||||
this.applyServerSettings(serverSettings);
|
||||
}
|
||||
|
||||
applyServerSettings(settings) {
|
||||
const values = settings && typeof settings === "object" ? settings : {};
|
||||
this.serverBandwidthMb = clampInt(
|
||||
values.bandwidth_mb_day,
|
||||
1,
|
||||
this.baked.max_donor_mb_per_day,
|
||||
this.baked.max_donor_mb_per_day,
|
||||
);
|
||||
this.serverConcurrent = clampInt(
|
||||
values.max_concurrent,
|
||||
1,
|
||||
this.baked.max_concurrent_per_node,
|
||||
this.baked.max_concurrent_per_node,
|
||||
);
|
||||
}
|
||||
|
||||
get maxBodyBytes() {
|
||||
return this.caps.max_body_bytes;
|
||||
}
|
||||
|
||||
get maxFrameBytes() {
|
||||
return this.caps.max_frame_bytes;
|
||||
}
|
||||
|
||||
get urlMaxChars() {
|
||||
return this.caps.url_max_chars;
|
||||
}
|
||||
|
||||
get pingSeconds() {
|
||||
return this.caps.ping_seconds;
|
||||
}
|
||||
|
||||
get concurrency() {
|
||||
return Math.min(this.caps.max_concurrent_per_node, this.local.maxConcurrent, this.serverConcurrent);
|
||||
}
|
||||
|
||||
get bandwidthBytes() {
|
||||
return Math.min(this.local.bandwidthMb, this.serverBandwidthMb) * MB;
|
||||
}
|
||||
|
||||
timeoutSeconds(requested) {
|
||||
const asked = isCount(requested) ? requested : this.caps.job_timeout_seconds;
|
||||
return Math.min(asked, this.caps.job_timeout_seconds);
|
||||
}
|
||||
|
||||
helloLimits() {
|
||||
return { max_concurrent: this.concurrency };
|
||||
}
|
||||
|
||||
remainingBytes(usedBytes) {
|
||||
return Math.max(0, this.bandwidthBytes - usedBytes);
|
||||
}
|
||||
|
||||
inActiveHours() {
|
||||
const from = minutes(this.local.activeFrom);
|
||||
const to = minutes(this.local.activeTo);
|
||||
if (from === null || to === null || from === to) {
|
||||
return true;
|
||||
}
|
||||
const now = this.clock();
|
||||
const current = now.getHours() * 60 + now.getMinutes();
|
||||
return from < to ? current >= from && current < to : current >= from || current < to;
|
||||
}
|
||||
|
||||
admit(usedBytes) {
|
||||
if (!this.inActiveHours()) {
|
||||
return REFUSALS.OUTSIDE_HOURS;
|
||||
}
|
||||
if (this.remainingBytes(usedBytes) <= 0) {
|
||||
return REFUSALS.BANDWIDTH_EXHAUSTED;
|
||||
}
|
||||
if (this.inflight >= this.concurrency) {
|
||||
return REFUSALS.CONCURRENCY_EXCEEDED;
|
||||
}
|
||||
this.inflight += 1;
|
||||
return null;
|
||||
}
|
||||
|
||||
release() {
|
||||
this.inflight = Math.max(0, this.inflight - 1);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,128 @@
|
||||
import {
|
||||
ALLOWED_PORTS,
|
||||
DESTINATIONS,
|
||||
HTTPS_PORT,
|
||||
LIMITS,
|
||||
MODE_FETCH,
|
||||
REFUSALS,
|
||||
} from "./spec.js";
|
||||
|
||||
const HTTPS = "https:";
|
||||
|
||||
function isSubset(candidate, allowed) {
|
||||
return Array.isArray(candidate) && candidate.every((item) => allowed.includes(item));
|
||||
}
|
||||
|
||||
function normaliseHost(host) {
|
||||
return String(host).trim().toLowerCase().replace(/\.+$/, "");
|
||||
}
|
||||
|
||||
export class Policy {
|
||||
constructor(destinations = DESTINATIONS, limits = LIMITS) {
|
||||
this.destinations = destinations;
|
||||
this.urlMaxChars = limits.url_max_chars;
|
||||
}
|
||||
|
||||
static narrow(baked, refreshed) {
|
||||
const accepted = [];
|
||||
const rejected = [];
|
||||
if (!Array.isArray(refreshed)) {
|
||||
return { destinations: [], rejected: ["destinations is not a list"] };
|
||||
}
|
||||
for (const item of refreshed) {
|
||||
const suffix = item && typeof item.suffix === "string" ? normaliseHost(item.suffix) : "";
|
||||
const parent = baked.find(
|
||||
(entry) => suffix === entry.suffix || suffix.endsWith("." + entry.suffix),
|
||||
);
|
||||
if (!parent) {
|
||||
rejected.push(`${suffix || "?"}: not in the baked-in allowlist`);
|
||||
continue;
|
||||
}
|
||||
const methods = Array.isArray(item.methods) ? item.methods.map((m) => String(m).toUpperCase()) : [];
|
||||
const ports = Array.isArray(item.ports) ? item.ports : [];
|
||||
const modes = Array.isArray(item.modes) ? item.modes : [];
|
||||
if (!methods.length || !isSubset(methods, parent.methods)) {
|
||||
rejected.push(`${suffix}: methods widen the baked-in list`);
|
||||
continue;
|
||||
}
|
||||
if (!ports.length || !isSubset(ports, parent.ports) || !isSubset(ports, ALLOWED_PORTS)) {
|
||||
rejected.push(`${suffix}: ports widen the baked-in list`);
|
||||
continue;
|
||||
}
|
||||
if (!modes.includes(MODE_FETCH)) {
|
||||
rejected.push(`${suffix}: not available in fetch mode`);
|
||||
continue;
|
||||
}
|
||||
accepted.push({
|
||||
suffix,
|
||||
adapter: String(item.adapter || parent.adapter),
|
||||
methods,
|
||||
ports,
|
||||
modes,
|
||||
max_rps: typeof item.max_rps === "number" ? item.max_rps : parent.max_rps,
|
||||
});
|
||||
}
|
||||
return { destinations: accepted, rejected };
|
||||
}
|
||||
|
||||
static fromRefreshed(baked, refreshed, limits = LIMITS) {
|
||||
const narrowed = Policy.narrow(baked, refreshed);
|
||||
return { policy: new Policy(narrowed.destinations, limits), rejected: narrowed.rejected };
|
||||
}
|
||||
|
||||
destinationFor(host) {
|
||||
const name = normaliseHost(host);
|
||||
return (
|
||||
this.destinations.find((item) => name === item.suffix || name.endsWith("." + item.suffix)) || null
|
||||
);
|
||||
}
|
||||
|
||||
refuse(code, reason) {
|
||||
return { ok: false, code, reason };
|
||||
}
|
||||
|
||||
check(method, rawUrl) {
|
||||
if (typeof rawUrl !== "string" || !rawUrl || rawUrl.length > this.urlMaxChars) {
|
||||
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, "url is missing or too long");
|
||||
}
|
||||
let url;
|
||||
try {
|
||||
url = new URL(rawUrl);
|
||||
} catch (error) {
|
||||
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, "url does not parse");
|
||||
}
|
||||
if (url.protocol !== HTTPS) {
|
||||
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, "only https is allowed");
|
||||
}
|
||||
if (url.username || url.password) {
|
||||
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, "credentials in the url are refused");
|
||||
}
|
||||
const destination = this.destinationFor(url.hostname);
|
||||
if (!destination) {
|
||||
return this.refuse(REFUSALS.HOST_NOT_ALLOWED, `${url.hostname} is not on the allowlist`);
|
||||
}
|
||||
const port = url.port === "" ? HTTPS_PORT : Number(url.port);
|
||||
if (port !== HTTPS_PORT || !destination.ports.includes(port) || !ALLOWED_PORTS.includes(port)) {
|
||||
return this.refuse(REFUSALS.PORT_NOT_ALLOWED, `port ${port} is not allowed over https`);
|
||||
}
|
||||
const verb = String(method || "").toUpperCase();
|
||||
if (!destination.methods.includes(verb)) {
|
||||
return this.refuse(REFUSALS.METHOD_NOT_ALLOWED, `${verb || "?"} is not permitted`);
|
||||
}
|
||||
return { ok: true, url, destination, method: verb };
|
||||
}
|
||||
|
||||
checkRedirect(location, base, method) {
|
||||
let target;
|
||||
try {
|
||||
target = new URL(location, base);
|
||||
} catch (error) {
|
||||
return this.refuse(REFUSALS.REDIRECT_NOT_ALLOWED, "redirect target does not parse");
|
||||
}
|
||||
const verdict = this.check(method, target.href);
|
||||
if (!verdict.ok) {
|
||||
return this.refuse(REFUSALS.REDIRECT_NOT_ALLOWED, verdict.reason);
|
||||
}
|
||||
return verdict;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
const KEY = "state";
|
||||
|
||||
export const STATES = Object.freeze({
|
||||
OFF: "off",
|
||||
CONNECTING: "connecting",
|
||||
ONLINE: "online",
|
||||
BACKOFF: "backoff",
|
||||
HALTED: "halted",
|
||||
});
|
||||
|
||||
export class State {
|
||||
constructor(area, clock = () => Date.now()) {
|
||||
this.area = area;
|
||||
this.clock = clock;
|
||||
this.data = { state: STATES.OFF, detail: "", since: clock(), deviceUid: "", lastError: "" };
|
||||
this.chain = Promise.resolve();
|
||||
}
|
||||
|
||||
async load() {
|
||||
const stored = (await this.area.get(KEY))[KEY];
|
||||
if (stored && typeof stored === "object") {
|
||||
this.data = { ...this.data, ...stored };
|
||||
}
|
||||
}
|
||||
|
||||
snapshot() {
|
||||
return { ...this.data };
|
||||
}
|
||||
|
||||
set(patch) {
|
||||
const changed = patch.state && patch.state !== this.data.state;
|
||||
this.data = { ...this.data, ...patch, since: changed ? this.clock() : this.data.since };
|
||||
const payload = { ...this.data };
|
||||
this.chain = this.chain.then(() => this.area.set({ [KEY]: payload }));
|
||||
return this.chain;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
const KEY = "usage";
|
||||
|
||||
function dayKey(date) {
|
||||
const month = String(date.getMonth() + 1).padStart(2, "0");
|
||||
const day = String(date.getDate()).padStart(2, "0");
|
||||
return `${date.getFullYear()}-${month}-${day}`;
|
||||
}
|
||||
|
||||
function empty(day) {
|
||||
return {
|
||||
day,
|
||||
requests: 0,
|
||||
failed: 0,
|
||||
bytesUp: 0,
|
||||
bytesDown: 0,
|
||||
lifetimeRequests: 0,
|
||||
lifetimeBytes: 0,
|
||||
};
|
||||
}
|
||||
|
||||
export class Usage {
|
||||
constructor(area, clock = () => new Date()) {
|
||||
this.area = area;
|
||||
this.clock = clock;
|
||||
this.data = empty(dayKey(clock()));
|
||||
this.chain = Promise.resolve();
|
||||
}
|
||||
|
||||
async load() {
|
||||
const stored = (await this.area.get(KEY))[KEY];
|
||||
if (stored && typeof stored === "object") {
|
||||
this.data = { ...empty(dayKey(this.clock())), ...stored };
|
||||
}
|
||||
this.roll();
|
||||
}
|
||||
|
||||
roll() {
|
||||
const today = dayKey(this.clock());
|
||||
if (this.data.day !== today) {
|
||||
this.data = {
|
||||
...empty(today),
|
||||
lifetimeRequests: this.data.lifetimeRequests,
|
||||
lifetimeBytes: this.data.lifetimeBytes,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
snapshot() {
|
||||
this.roll();
|
||||
return { ...this.data };
|
||||
}
|
||||
|
||||
get usedBytes() {
|
||||
this.roll();
|
||||
return this.data.bytesUp + this.data.bytesDown;
|
||||
}
|
||||
|
||||
record({ ok, up, down }) {
|
||||
this.roll();
|
||||
const bytesUp = Math.max(0, Math.floor(up) || 0);
|
||||
const bytesDown = Math.max(0, Math.floor(down) || 0);
|
||||
this.data.requests += 1;
|
||||
this.data.failed += ok ? 0 : 1;
|
||||
this.data.bytesUp += bytesUp;
|
||||
this.data.bytesDown += bytesDown;
|
||||
this.data.lifetimeRequests += 1;
|
||||
this.data.lifetimeBytes += bytesUp + bytesDown;
|
||||
return this.persist();
|
||||
}
|
||||
|
||||
persist() {
|
||||
const payload = { ...this.data };
|
||||
this.chain = this.chain.then(() => this.area.set({ [KEY]: payload }));
|
||||
return this.chain;
|
||||
}
|
||||
|
||||
async reset() {
|
||||
this.data = empty(dayKey(this.clock()));
|
||||
await this.persist();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,375 @@
|
||||
import { Policy } from "./Policy.js";
|
||||
import { STATES } from "./State.js";
|
||||
import {
|
||||
CLIENT_KIND,
|
||||
CLOSE_CAPACITY,
|
||||
CLOSE_KICKED,
|
||||
CLOSE_NORMAL,
|
||||
CLOSE_PROTOCOL,
|
||||
CLOSE_RETRY_OWNER,
|
||||
DESTINATIONS,
|
||||
FRAME_CEILING_BYTES,
|
||||
HALTING_CLOSE_REASONS_EXEMPT,
|
||||
MAX_BACKOFF_EXPONENT,
|
||||
MODE_FETCH,
|
||||
PROTOCOL_VERSION,
|
||||
REFUSALS,
|
||||
SERVER_FRAMES,
|
||||
} from "./spec.js";
|
||||
|
||||
const OPEN = 1;
|
||||
const POLICY_CLOSE = 1008;
|
||||
const TICKET_PATH = "/nodes/ticket";
|
||||
const WS_PATH = "/nodes/ws";
|
||||
const HALT_STATUSES = [401, 403];
|
||||
|
||||
class FrameFault extends Error {}
|
||||
|
||||
export class WsClient {
|
||||
constructor(options) {
|
||||
this.config = options.config;
|
||||
this.limits = options.limits;
|
||||
this.fetcher = options.fetcher;
|
||||
this.state = options.state;
|
||||
this.fetchImpl = options.fetchImpl || ((...args) => globalThis.fetch(...args));
|
||||
this.WebSocketImpl = options.WebSocketImpl || globalThis.WebSocket;
|
||||
this.timers = options.timers || globalThis;
|
||||
this.random = options.random || Math.random;
|
||||
this.clientVersion = options.clientVersion || "0.0.0";
|
||||
this.platform = options.platform || "chrome";
|
||||
this.baked = options.baked || DESTINATIONS;
|
||||
this.socket = null;
|
||||
this.wanted = false;
|
||||
this.connecting = false;
|
||||
this.welcomed = false;
|
||||
this.attempt = 0;
|
||||
this.retryTimer = null;
|
||||
this.pingTimer = null;
|
||||
this.pending = new Set();
|
||||
this.generation = 0;
|
||||
}
|
||||
|
||||
async start() {
|
||||
this.wanted = true;
|
||||
await this.ensure();
|
||||
}
|
||||
|
||||
async stop(detail = "") {
|
||||
this.wanted = false;
|
||||
this.clearTimers();
|
||||
this.generation += 1;
|
||||
this.closeSocket(CLOSE_NORMAL, "stop");
|
||||
await this.state.set({ state: STATES.OFF, detail });
|
||||
}
|
||||
|
||||
async ensure() {
|
||||
if (!this.wanted || this.socket || this.connecting || this.retryTimer) {
|
||||
return;
|
||||
}
|
||||
await this.connectOnce();
|
||||
}
|
||||
|
||||
clearTimers() {
|
||||
if (this.retryTimer) {
|
||||
this.timers.clearTimeout(this.retryTimer);
|
||||
this.retryTimer = null;
|
||||
}
|
||||
if (this.pingTimer) {
|
||||
this.timers.clearInterval(this.pingTimer);
|
||||
this.pingTimer = null;
|
||||
}
|
||||
}
|
||||
|
||||
closeSocket(code, reason) {
|
||||
const socket = this.socket;
|
||||
this.socket = null;
|
||||
if (socket && socket.readyState <= OPEN) {
|
||||
socket.close(code, reason);
|
||||
}
|
||||
}
|
||||
|
||||
backoffSeconds(fast) {
|
||||
const low = this.limits.reconnectMin;
|
||||
if (fast) {
|
||||
return low;
|
||||
}
|
||||
const exponent = Math.min(this.attempt, MAX_BACKOFF_EXPONENT);
|
||||
const ceiling = Math.min(this.limits.reconnectMax, low * 2 ** exponent);
|
||||
return Math.max(low, this.random() * ceiling);
|
||||
}
|
||||
|
||||
async scheduleRetry(detail, fast = false) {
|
||||
if (!this.wanted) {
|
||||
return;
|
||||
}
|
||||
const seconds = this.backoffSeconds(fast);
|
||||
this.attempt += 1;
|
||||
this.clearTimers();
|
||||
this.retryTimer = this.timers.setTimeout(() => {
|
||||
this.retryTimer = null;
|
||||
this.ensure();
|
||||
}, seconds * 1000);
|
||||
await this.state.set({ state: STATES.BACKOFF, detail, retryInSeconds: Math.round(seconds) });
|
||||
}
|
||||
|
||||
async halt(detail) {
|
||||
this.wanted = false;
|
||||
this.clearTimers();
|
||||
this.generation += 1;
|
||||
this.closeSocket(CLOSE_NORMAL, "halt");
|
||||
await this.state.set({ state: STATES.HALTED, detail, lastError: detail });
|
||||
}
|
||||
|
||||
async fetchTicket() {
|
||||
const { serverUrl, secret } = this.config.credentials();
|
||||
const response = await this.fetchImpl(serverUrl + TICKET_PATH, {
|
||||
method: "POST",
|
||||
headers: { Authorization: `Bearer ${secret}` },
|
||||
credentials: "omit",
|
||||
cache: "no-store",
|
||||
});
|
||||
if (!response.ok) {
|
||||
let reason = "";
|
||||
try {
|
||||
reason = (await response.json()).reason || "";
|
||||
} catch (error) {
|
||||
reason = "";
|
||||
}
|
||||
const error = new Error(`ticket refused with ${response.status} ${reason}`.trim());
|
||||
error.status = response.status;
|
||||
throw error;
|
||||
}
|
||||
const body = await response.json();
|
||||
if (!body || typeof body.ticket !== "string" || !body.ticket) {
|
||||
throw new Error("ticket response has no ticket");
|
||||
}
|
||||
const target = new URL(typeof body.ws_path === "string" ? body.ws_path : WS_PATH, serverUrl);
|
||||
if (target.origin !== new URL(serverUrl).origin) {
|
||||
throw new Error("ticket response points at another origin");
|
||||
}
|
||||
target.protocol = target.protocol === "https:" ? "wss:" : "ws:";
|
||||
target.searchParams.set("ticket", body.ticket);
|
||||
return target.href;
|
||||
}
|
||||
|
||||
async connectOnce() {
|
||||
this.connecting = true;
|
||||
const generation = this.generation;
|
||||
await this.state.set({ state: STATES.CONNECTING, detail: "" });
|
||||
let address;
|
||||
try {
|
||||
address = await this.fetchTicket();
|
||||
} catch (error) {
|
||||
this.connecting = false;
|
||||
if (generation !== this.generation) {
|
||||
return;
|
||||
}
|
||||
if (HALT_STATUSES.includes(error.status)) {
|
||||
await this.halt(error.message);
|
||||
return;
|
||||
}
|
||||
await this.state.set({ lastError: error.message });
|
||||
await this.scheduleRetry(error.message);
|
||||
return;
|
||||
}
|
||||
this.connecting = false;
|
||||
if (generation !== this.generation) {
|
||||
return;
|
||||
}
|
||||
this.open(address, generation);
|
||||
}
|
||||
|
||||
open(address, generation) {
|
||||
this.welcomed = false;
|
||||
const socket = new this.WebSocketImpl(address);
|
||||
this.socket = socket;
|
||||
socket.onopen = () => {
|
||||
if (generation !== this.generation || this.socket !== socket) {
|
||||
return;
|
||||
}
|
||||
this.sendFrame("hello", {
|
||||
protocol: PROTOCOL_VERSION,
|
||||
client_version: this.clientVersion,
|
||||
kind: CLIENT_KIND,
|
||||
platform: this.platform,
|
||||
modes: [MODE_FETCH],
|
||||
limits: this.limits.helloLimits(),
|
||||
});
|
||||
};
|
||||
socket.onmessage = (event) => {
|
||||
if (this.socket !== socket) {
|
||||
return;
|
||||
}
|
||||
return this.handleMessage(event.data).catch((error) => this.protocolFault(error));
|
||||
};
|
||||
socket.onclose = (event) => {
|
||||
if (this.socket !== socket) {
|
||||
return;
|
||||
}
|
||||
this.socket = null;
|
||||
this.onClosed(event.code, event.reason || "");
|
||||
};
|
||||
socket.onerror = () => {
|
||||
this.state.set({ lastError: "websocket error" });
|
||||
};
|
||||
}
|
||||
|
||||
sendFrame(type, fields = {}) {
|
||||
if (!this.socket || this.socket.readyState !== OPEN) {
|
||||
throw new Error(`cannot send ${type}, socket is not open`);
|
||||
}
|
||||
const text = JSON.stringify({ type, ...fields });
|
||||
const ceiling = FRAME_CEILING_BYTES[type] || FRAME_CEILING_BYTES.default;
|
||||
if (new TextEncoder().encode(text).length > ceiling) {
|
||||
throw new Error(`${type} frame exceeds its size cap`);
|
||||
}
|
||||
this.socket.send(text);
|
||||
}
|
||||
|
||||
async protocolFault(error) {
|
||||
await this.state.set({ lastError: error.message });
|
||||
this.closeSocket(CLOSE_PROTOCOL, "protocol");
|
||||
this.clearTimers();
|
||||
await this.scheduleRetry(error.message);
|
||||
}
|
||||
|
||||
decode(raw) {
|
||||
if (typeof raw !== "string") {
|
||||
throw new FrameFault("binary frames are not part of the protocol");
|
||||
}
|
||||
if (raw.length > this.limits.maxFrameBytes) {
|
||||
throw new FrameFault("server frame exceeds the size cap");
|
||||
}
|
||||
let frame;
|
||||
try {
|
||||
frame = JSON.parse(raw);
|
||||
} catch (error) {
|
||||
throw new FrameFault("server frame is not valid JSON");
|
||||
}
|
||||
if (!frame || typeof frame !== "object" || Array.isArray(frame)) {
|
||||
throw new FrameFault("server frame must be a JSON object");
|
||||
}
|
||||
const shape = SERVER_FRAMES[frame.type];
|
||||
if (!shape) {
|
||||
throw new FrameFault(`unknown server frame ${String(frame.type)}`);
|
||||
}
|
||||
for (const [name, kind] of Object.entries(shape)) {
|
||||
const value = frame[name];
|
||||
const good = kind === "array" ? Array.isArray(value) : typeof value === kind && value !== null;
|
||||
if (!good) {
|
||||
throw new FrameFault(`frame ${frame.type} field ${name} must be ${kind}`);
|
||||
}
|
||||
}
|
||||
return frame;
|
||||
}
|
||||
|
||||
async handleMessage(raw) {
|
||||
const frame = this.decode(raw);
|
||||
if (frame.type !== "welcome" && !this.welcomed && frame.type !== "kick") {
|
||||
throw new FrameFault(`${frame.type} arrived before welcome`);
|
||||
}
|
||||
if (frame.type === "welcome") {
|
||||
await this.onWelcome(frame);
|
||||
} else if (frame.type === "ping") {
|
||||
this.sendFrame("pong");
|
||||
} else if (frame.type === "job") {
|
||||
this.runJob(frame);
|
||||
} else if (frame.type === "settings_update") {
|
||||
this.limits.applyServerSettings(frame.settings);
|
||||
} else if (frame.type === "kick") {
|
||||
await this.halt(`kicked by the server: ${frame.reason}`);
|
||||
} else {
|
||||
this.sendFrame("tunnel_opened", {
|
||||
conn_id: frame.conn_id,
|
||||
ok: false,
|
||||
code: REFUSALS.HOST_NOT_ALLOWED,
|
||||
});
|
||||
throw new FrameFault(`${frame.type} is a tunnel frame and this client is fetch-only`);
|
||||
}
|
||||
}
|
||||
|
||||
async onWelcome(frame) {
|
||||
if (this.welcomed) {
|
||||
throw new FrameFault("a second welcome arrived");
|
||||
}
|
||||
this.limits.applyWelcome(frame.limits, frame.settings);
|
||||
const refreshed = Policy.fromRefreshed(this.baked, frame.destinations, this.limits.baked);
|
||||
this.fetcher.setPolicy(refreshed.policy);
|
||||
this.welcomed = true;
|
||||
this.attempt = 0;
|
||||
this.startKeepalive();
|
||||
const notes = refreshed.rejected.length ? `ignored: ${refreshed.rejected.join("; ")}` : "";
|
||||
await this.state.set({
|
||||
state: STATES.ONLINE,
|
||||
detail: notes,
|
||||
deviceUid: frame.device_uid,
|
||||
lastError: notes,
|
||||
});
|
||||
}
|
||||
|
||||
startKeepalive() {
|
||||
if (this.pingTimer) {
|
||||
this.timers.clearInterval(this.pingTimer);
|
||||
}
|
||||
this.pingTimer = this.timers.setInterval(() => {
|
||||
try {
|
||||
this.sendFrame("pong");
|
||||
} catch (error) {
|
||||
this.state.set({ lastError: error.message });
|
||||
}
|
||||
}, this.limits.pingSeconds * 1000);
|
||||
}
|
||||
|
||||
runJob(frame) {
|
||||
const task = this.fetcher
|
||||
.execute(frame)
|
||||
.then((result) => {
|
||||
this.sendFrame(result.type, result);
|
||||
return this.reportUsage(result);
|
||||
})
|
||||
.catch((error) => this.state.set({ lastError: error.message }))
|
||||
.finally(() => this.pending.delete(task));
|
||||
this.pending.add(task);
|
||||
}
|
||||
|
||||
reportUsage(result) {
|
||||
const down = result.type === "job_result" ? Math.ceil((result.body_b64.length * 3) / 4) : 0;
|
||||
this.sendFrame("usage", { bytes_up: 0, bytes_down: down });
|
||||
}
|
||||
|
||||
async idle() {
|
||||
while (this.pending.size) {
|
||||
await Promise.all([...this.pending]);
|
||||
}
|
||||
}
|
||||
|
||||
classifyClose(code, reason) {
|
||||
if (code === CLOSE_KICKED) {
|
||||
return { action: "halt", detail: `closed by the server: ${reason || "kicked"}` };
|
||||
}
|
||||
if (code === CLOSE_RETRY_OWNER) {
|
||||
return { action: "retry", fast: true, detail: "server asked for a retry" };
|
||||
}
|
||||
if (code === POLICY_CLOSE && !HALTING_CLOSE_REASONS_EXEMPT.includes(reason)) {
|
||||
return { action: "halt", detail: `refused by the server: ${reason || "policy"}` };
|
||||
}
|
||||
if (code === CLOSE_CAPACITY) {
|
||||
return { action: "retry", fast: false, detail: "server at capacity" };
|
||||
}
|
||||
return { action: "retry", fast: false, detail: `closed ${code} ${reason}`.trim() };
|
||||
}
|
||||
|
||||
async onClosed(code, reason) {
|
||||
this.clearTimers();
|
||||
this.welcomed = false;
|
||||
if (!this.wanted) {
|
||||
return;
|
||||
}
|
||||
const verdict = this.classifyClose(code, reason);
|
||||
if (verdict.action === "halt") {
|
||||
await this.halt(verdict.detail);
|
||||
return;
|
||||
}
|
||||
await this.scheduleRetry(verdict.detail, verdict.fast);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
import { Config, ConfigError } from "./Config.js";
|
||||
import { Fetcher } from "./Fetcher.js";
|
||||
import { Limits } from "./Limits.js";
|
||||
import { Policy } from "./Policy.js";
|
||||
import { State, STATES } from "./State.js";
|
||||
import { Usage } from "./Usage.js";
|
||||
import { WsClient } from "./WsClient.js";
|
||||
|
||||
const ALARM = "devplace-node-backstop";
|
||||
const ALARM_PERIOD_MINUTES = 1;
|
||||
const DISCLOSURE_PAGE = "ui/disclosure.html";
|
||||
|
||||
const config = new Config(chrome.storage.local);
|
||||
const usage = new Usage(chrome.storage.local);
|
||||
const state = new State(chrome.storage.session);
|
||||
const limits = new Limits();
|
||||
const fetcher = new Fetcher({ policy: new Policy(), limits, usage });
|
||||
const client = new WsClient({
|
||||
config,
|
||||
limits,
|
||||
fetcher,
|
||||
state,
|
||||
clientVersion: chrome.runtime.getManifest().version,
|
||||
platform: navigator.userAgent.includes("Firefox") ? "firefox" : "chrome",
|
||||
});
|
||||
|
||||
let ready = null;
|
||||
|
||||
function boot() {
|
||||
if (!ready) {
|
||||
ready = (async () => {
|
||||
await Promise.all([config.load(), usage.load(), state.load()]);
|
||||
limits.setLocal(config.data);
|
||||
})();
|
||||
}
|
||||
return ready;
|
||||
}
|
||||
|
||||
async function reconcile() {
|
||||
await boot();
|
||||
limits.setLocal(config.data);
|
||||
if (config.data.enabled && config.data.disclosed && config.hasSecret) {
|
||||
await client.start();
|
||||
} else if (client.wanted || state.data.state !== STATES.OFF) {
|
||||
await client.stop();
|
||||
}
|
||||
}
|
||||
|
||||
async function status() {
|
||||
await boot();
|
||||
return {
|
||||
config: config.view(),
|
||||
state: state.snapshot(),
|
||||
usage: usage.snapshot(),
|
||||
live: { concurrency: limits.concurrency, bandwidthBytes: limits.bandwidthBytes },
|
||||
};
|
||||
}
|
||||
|
||||
const COMMANDS = {
|
||||
status: async () => status(),
|
||||
connect: async (message) => {
|
||||
await config.connect(message);
|
||||
await client.stop("credentials changed");
|
||||
await reconcile();
|
||||
return status();
|
||||
},
|
||||
accept: async () => {
|
||||
await config.acceptDisclosure();
|
||||
return status();
|
||||
},
|
||||
enable: async (message) => {
|
||||
await config.setEnabled(Boolean(message.enabled));
|
||||
await reconcile();
|
||||
return status();
|
||||
},
|
||||
settings: async (message) => {
|
||||
await config.saveSettings(message.values || {});
|
||||
limits.setLocal(config.data);
|
||||
return status();
|
||||
},
|
||||
forget: async () => {
|
||||
await config.forget();
|
||||
await client.stop("device forgotten");
|
||||
return status();
|
||||
},
|
||||
resetUsage: async () => {
|
||||
await usage.reset();
|
||||
return status();
|
||||
},
|
||||
};
|
||||
|
||||
async function dispatch(message) {
|
||||
await boot();
|
||||
const command = COMMANDS[message && message.cmd];
|
||||
if (!command) {
|
||||
return { error: `unknown command ${String(message && message.cmd)}` };
|
||||
}
|
||||
try {
|
||||
return await command(message);
|
||||
} catch (error) {
|
||||
if (error instanceof ConfigError) {
|
||||
return { error: error.message };
|
||||
}
|
||||
await state.set({ lastError: error.message });
|
||||
return { error: error.message };
|
||||
}
|
||||
}
|
||||
|
||||
chrome.runtime.onMessage.addListener((message, sender, respond) => {
|
||||
if (sender.id !== chrome.runtime.id) {
|
||||
return false;
|
||||
}
|
||||
dispatch(message).then(respond);
|
||||
return true;
|
||||
});
|
||||
|
||||
chrome.alarms.onAlarm.addListener((alarm) => {
|
||||
if (alarm.name === ALARM) {
|
||||
reconcile().catch((error) => state.set({ lastError: error.message }));
|
||||
}
|
||||
});
|
||||
|
||||
chrome.runtime.onInstalled.addListener((details) => {
|
||||
chrome.alarms.create(ALARM, { periodInMinutes: ALARM_PERIOD_MINUTES });
|
||||
if (details.reason === "install") {
|
||||
chrome.tabs.create({ url: chrome.runtime.getURL(DISCLOSURE_PAGE) });
|
||||
}
|
||||
});
|
||||
|
||||
chrome.runtime.onStartup.addListener(() => {
|
||||
chrome.alarms.create(ALARM, { periodInMinutes: ALARM_PERIOD_MINUTES });
|
||||
reconcile().catch((error) => state.set({ lastError: error.message }));
|
||||
});
|
||||
|
||||
reconcile().catch((error) => state.set({ lastError: error.message }));
|
||||
@@ -0,0 +1,104 @@
|
||||
export const PROTOCOL_VERSION = 1;
|
||||
export const MIN_SUPPORTED_PROTOCOL = 1;
|
||||
export const CLIENT_KIND = "extension";
|
||||
export const MODE_FETCH = "fetch";
|
||||
|
||||
export const ALLOWED_PORTS = Object.freeze([80, 443]);
|
||||
export const HTTPS_PORT = 443;
|
||||
|
||||
export const CLOSE_NORMAL = 1000;
|
||||
export const CLOSE_PROTOCOL = 4400;
|
||||
export const CLOSE_RETRY_OWNER = 4013;
|
||||
export const CLOSE_CAPACITY = 4029;
|
||||
export const CLOSE_KICKED = 4403;
|
||||
export const CLOSE_POLICY = 1008;
|
||||
|
||||
export const MAX_REDIRECTS = 5;
|
||||
export const MAX_RESPONSE_HEADERS = 64;
|
||||
export const MAX_HEADER_VALUE_CHARS = 512;
|
||||
export const MAX_BODY_CHUNKS = 65536;
|
||||
export const MAX_BACKOFF_EXPONENT = 16;
|
||||
|
||||
export const LIMITS = Object.freeze({
|
||||
max_frame_bytes: 256 * 1024,
|
||||
max_body_bytes: 2 * 1024 * 1024,
|
||||
max_concurrent_per_node: 2,
|
||||
job_timeout_seconds: 25,
|
||||
default_donor_mb_per_day: 500,
|
||||
max_donor_mb_per_day: 20 * 1024,
|
||||
ping_seconds: 20,
|
||||
reconnect_min_seconds: 1,
|
||||
reconnect_max_seconds: 60,
|
||||
url_max_chars: 2048,
|
||||
});
|
||||
|
||||
export const FRAME_CEILING_BYTES = Object.freeze({
|
||||
job_result: 3 * 1024 * 1024,
|
||||
default: 256 * 1024,
|
||||
});
|
||||
|
||||
export const DESTINATIONS = Object.freeze([
|
||||
Object.freeze({
|
||||
suffix: "reddit.com",
|
||||
adapter: "reddit",
|
||||
methods: Object.freeze(["GET"]),
|
||||
ports: Object.freeze([80, 443]),
|
||||
modes: Object.freeze(["tunnel", "fetch"]),
|
||||
max_rps: 0.5,
|
||||
}),
|
||||
Object.freeze({
|
||||
suffix: "redd.it",
|
||||
adapter: "reddit",
|
||||
methods: Object.freeze(["GET"]),
|
||||
ports: Object.freeze([80, 443]),
|
||||
modes: Object.freeze(["tunnel", "fetch"]),
|
||||
max_rps: 0.5,
|
||||
}),
|
||||
]);
|
||||
|
||||
export const REFUSALS = Object.freeze({
|
||||
HOST_NOT_ALLOWED: "host_not_allowed",
|
||||
PORT_NOT_ALLOWED: "port_not_allowed",
|
||||
ADDRESS_NOT_PUBLIC: "address_not_public",
|
||||
REDIRECT_NOT_ALLOWED: "redirect_not_allowed",
|
||||
METHOD_NOT_ALLOWED: "method_not_allowed",
|
||||
BANDWIDTH_EXHAUSTED: "bandwidth_exhausted",
|
||||
CONCURRENCY_EXCEEDED: "concurrency_exceeded",
|
||||
OUTSIDE_HOURS: "outside_hours",
|
||||
BODY_TOO_LARGE: "body_too_large",
|
||||
TIMEOUT: "timeout",
|
||||
});
|
||||
|
||||
export const FETCH_FAILED = "fetch_failed";
|
||||
|
||||
export const SERVER_FRAMES = Object.freeze({
|
||||
welcome: Object.freeze({
|
||||
device_uid: "string",
|
||||
limits: "object",
|
||||
destinations: "array",
|
||||
settings: "object",
|
||||
}),
|
||||
ping: Object.freeze({}),
|
||||
job: Object.freeze({
|
||||
job_id: "string",
|
||||
method: "string",
|
||||
url: "string",
|
||||
timeout: "number",
|
||||
}),
|
||||
tunnel_open: Object.freeze({ conn_id: "string", host: "string", port: "number" }),
|
||||
tunnel_data: Object.freeze({ conn_id: "string", b64: "string" }),
|
||||
tunnel_close: Object.freeze({ conn_id: "string" }),
|
||||
settings_update: Object.freeze({ settings: "object" }),
|
||||
kick: Object.freeze({ reason: "string" }),
|
||||
});
|
||||
|
||||
export const DROPPED_RESPONSE_HEADERS = Object.freeze([
|
||||
"set-cookie",
|
||||
"set-cookie2",
|
||||
"authorization",
|
||||
"proxy-authenticate",
|
||||
]);
|
||||
|
||||
export const FORWARDED_REQUEST_HEADERS = Object.freeze(["accept", "accept-language"]);
|
||||
|
||||
export const HALTING_CLOSE_REASONS_EXEMPT = Object.freeze(["ticket", "heartbeat", "error", "capacity"]);
|
||||
@@ -0,0 +1,197 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { Fetcher } from "../src/Fetcher.js";
|
||||
import { Limits } from "../src/Limits.js";
|
||||
import { Policy } from "../src/Policy.js";
|
||||
import { Usage } from "../src/Usage.js";
|
||||
import { MAX_REDIRECTS, REFUSALS } from "../src/spec.js";
|
||||
|
||||
function memoryArea() {
|
||||
const store = {};
|
||||
return {
|
||||
async get(key) {
|
||||
return key in store ? { [key]: store[key] } : {};
|
||||
},
|
||||
async set(values) {
|
||||
Object.assign(store, values);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function build(fetchImpl, local = {}) {
|
||||
const limits = new Limits({ local });
|
||||
const usage = new Usage(memoryArea());
|
||||
const fetcher = new Fetcher({ policy: new Policy(), limits, usage, fetchImpl });
|
||||
return { fetcher, limits, usage };
|
||||
}
|
||||
|
||||
const job = (url = "https://www.reddit.com/r/x.json", extra = {}) => ({
|
||||
job_id: "j1",
|
||||
method: "GET",
|
||||
url,
|
||||
timeout: 5,
|
||||
...extra,
|
||||
});
|
||||
|
||||
function response(body, init = {}) {
|
||||
return new Response(body, { status: 200, ...init });
|
||||
}
|
||||
|
||||
test("success returns a job_result with base64 body and no donor credentials", async () => {
|
||||
const calls = [];
|
||||
const { fetcher, usage } = build(async (url, options) => {
|
||||
calls.push({ url, options });
|
||||
return response("hello", { headers: { "content-type": "text/plain", "set-cookie": "a=b" } });
|
||||
});
|
||||
const result = await fetcher.execute(
|
||||
job(undefined, { headers: { Cookie: "x=1", Authorization: "Bearer z", Accept: "text/html" } }),
|
||||
);
|
||||
assert.equal(result.type, "job_result");
|
||||
assert.equal(result.status, 200);
|
||||
assert.equal(Buffer.from(result.body_b64, "base64").toString(), "hello");
|
||||
assert.equal(result.truncated, false);
|
||||
assert.equal(result.headers["content-type"], "text/plain");
|
||||
assert.equal("set-cookie" in result.headers, false);
|
||||
assert.equal(calls[0].options.credentials, "omit");
|
||||
assert.equal(calls[0].options.redirect, "manual");
|
||||
assert.deepEqual(calls[0].options.headers, { accept: "text/html" });
|
||||
assert.equal(usage.snapshot().requests, 1);
|
||||
assert.equal(usage.snapshot().bytesDown, 5);
|
||||
});
|
||||
|
||||
test("a disallowed url never reaches fetch", async () => {
|
||||
let called = 0;
|
||||
const { fetcher, limits } = build(async () => {
|
||||
called += 1;
|
||||
return response("x");
|
||||
});
|
||||
for (const url of ["https://evilreddit.com/", "http://reddit.com/", "https://reddit.com@evil.net/"]) {
|
||||
const result = await fetcher.execute(job(url));
|
||||
assert.equal(result.type, "job_error");
|
||||
assert.equal(result.code, REFUSALS.HOST_NOT_ALLOWED);
|
||||
}
|
||||
assert.equal(called, 0);
|
||||
assert.equal(limits.inflight, 0);
|
||||
});
|
||||
|
||||
test("body over the cap is truncated and flagged", async () => {
|
||||
const big = "a".repeat(5000);
|
||||
const { fetcher, limits } = build(async () => response(big));
|
||||
limits.caps.max_body_bytes = 1000;
|
||||
const result = await fetcher.execute(job());
|
||||
assert.equal(result.type, "job_result");
|
||||
assert.equal(result.truncated, true);
|
||||
assert.equal(Buffer.from(result.body_b64, "base64").length, 1000);
|
||||
});
|
||||
|
||||
test("the local bandwidth budget also bounds the body", async () => {
|
||||
const { fetcher, usage } = build(async () => response("b".repeat(3 * 1024 * 1024)), { bandwidthMb: 1 });
|
||||
const result = await fetcher.execute(job());
|
||||
assert.equal(result.truncated, true);
|
||||
assert.ok(Buffer.from(result.body_b64, "base64").length < 1024 * 1024);
|
||||
assert.ok(usage.usedBytes <= 1024 * 1024);
|
||||
});
|
||||
|
||||
test("exhausted bandwidth refuses before fetching", async () => {
|
||||
let called = 0;
|
||||
const { fetcher, usage } = build(
|
||||
async () => {
|
||||
called += 1;
|
||||
return response("x");
|
||||
},
|
||||
{ bandwidthMb: 1 },
|
||||
);
|
||||
await usage.record({ ok: true, up: 0, down: 1024 * 1024 });
|
||||
const result = await fetcher.execute(job());
|
||||
assert.equal(result.code, REFUSALS.BANDWIDTH_EXHAUSTED);
|
||||
assert.equal(called, 0);
|
||||
});
|
||||
|
||||
test("concurrency cap refuses the extra job", async () => {
|
||||
let release;
|
||||
const gate = new Promise((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
const { fetcher } = build(
|
||||
async () => {
|
||||
await gate;
|
||||
return response("x");
|
||||
},
|
||||
{ maxConcurrent: 1 },
|
||||
);
|
||||
const first = fetcher.execute(job());
|
||||
const second = await fetcher.execute(job());
|
||||
assert.equal(second.code, REFUSALS.CONCURRENCY_EXCEEDED);
|
||||
release();
|
||||
assert.equal((await first).type, "job_result");
|
||||
});
|
||||
|
||||
test("timeout aborts the request and reports timeout", async () => {
|
||||
const { fetcher, limits } = build(
|
||||
(url, options) =>
|
||||
new Promise((resolve, reject) => {
|
||||
options.signal.addEventListener("abort", () => reject(new DOMException("aborted", "AbortError")));
|
||||
}),
|
||||
);
|
||||
limits.caps.job_timeout_seconds = 1;
|
||||
const started = Date.now();
|
||||
const result = await fetcher.execute(job(undefined, { timeout: 1 }));
|
||||
assert.equal(result.code, REFUSALS.TIMEOUT);
|
||||
assert.ok(Date.now() - started < 3000);
|
||||
assert.equal(limits.inflight, 0);
|
||||
});
|
||||
|
||||
test("a redirect leaving the allowlist is refused and not followed", async () => {
|
||||
const urls = [];
|
||||
const { fetcher } = build(async (url) => {
|
||||
urls.push(url);
|
||||
return response("", { status: 302, headers: { location: "https://evil.net/steal" } });
|
||||
});
|
||||
const result = await fetcher.execute(job());
|
||||
assert.equal(result.type, "job_error");
|
||||
assert.equal(result.code, REFUSALS.REDIRECT_NOT_ALLOWED);
|
||||
assert.equal(urls.length, 1);
|
||||
});
|
||||
|
||||
test("an opaque redirect fails closed", async () => {
|
||||
const opaque = { type: "opaqueredirect", status: 0, headers: new Headers(), body: null };
|
||||
const { fetcher } = build(async () => opaque);
|
||||
const result = await fetcher.execute(job());
|
||||
assert.equal(result.code, REFUSALS.REDIRECT_NOT_ALLOWED);
|
||||
});
|
||||
|
||||
test("a same-allowlist redirect is followed, with the policy checked on every hop", async () => {
|
||||
const urls = [];
|
||||
const { fetcher } = build(async (url) => {
|
||||
urls.push(url);
|
||||
if (urls.length === 1) {
|
||||
return response("", { status: 301, headers: { location: "https://old.reddit.com/r/x" } });
|
||||
}
|
||||
return response("final");
|
||||
});
|
||||
const result = await fetcher.execute(job());
|
||||
assert.equal(result.type, "job_result");
|
||||
assert.equal(Buffer.from(result.body_b64, "base64").toString(), "final");
|
||||
assert.deepEqual(urls, ["https://www.reddit.com/r/x.json", "https://old.reddit.com/r/x"]);
|
||||
});
|
||||
|
||||
test("redirect loops are bounded", async () => {
|
||||
let count = 0;
|
||||
const { fetcher } = build(async () => {
|
||||
count += 1;
|
||||
return response("", { status: 302, headers: { location: "https://www.reddit.com/loop" } });
|
||||
});
|
||||
const result = await fetcher.execute(job());
|
||||
assert.equal(result.code, REFUSALS.REDIRECT_NOT_ALLOWED);
|
||||
assert.equal(count, MAX_REDIRECTS + 1);
|
||||
});
|
||||
|
||||
test("a network failure is a loud job_error", async () => {
|
||||
const { fetcher, usage } = build(async () => {
|
||||
throw new TypeError("network down");
|
||||
});
|
||||
const result = await fetcher.execute(job());
|
||||
assert.equal(result.type, "job_error");
|
||||
assert.equal(result.code, "fetch_failed");
|
||||
assert.equal(usage.snapshot().failed, 1);
|
||||
});
|
||||
@@ -0,0 +1,91 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { Limits } from "../src/Limits.js";
|
||||
import { LIMITS, REFUSALS } from "../src/spec.js";
|
||||
|
||||
const MB = 1024 * 1024;
|
||||
|
||||
function at(hour, minute = 0) {
|
||||
return () => new Date(2026, 0, 15, hour, minute);
|
||||
}
|
||||
|
||||
test("defaults come from the baked spec", () => {
|
||||
const limits = new Limits();
|
||||
assert.equal(limits.concurrency, LIMITS.max_concurrent_per_node);
|
||||
assert.equal(limits.bandwidthBytes, LIMITS.default_donor_mb_per_day * MB);
|
||||
assert.equal(limits.maxBodyBytes, LIMITS.max_body_bytes);
|
||||
});
|
||||
|
||||
test("local settings are clamped", () => {
|
||||
const limits = new Limits({ local: { bandwidthMb: 10 ** 9, maxConcurrent: 99 } });
|
||||
assert.equal(limits.local.bandwidthMb, LIMITS.max_donor_mb_per_day);
|
||||
assert.equal(limits.local.maxConcurrent, LIMITS.max_concurrent_per_node);
|
||||
limits.setLocal({ bandwidthMb: -5, maxConcurrent: 0 });
|
||||
assert.equal(limits.local.bandwidthMb, LIMITS.default_donor_mb_per_day);
|
||||
assert.equal(limits.local.maxConcurrent, LIMITS.max_concurrent_per_node);
|
||||
limits.setLocal({ activeFrom: "25:99", activeTo: "10:00" });
|
||||
assert.equal(limits.local.activeFrom, "");
|
||||
});
|
||||
|
||||
test("a welcome can lower caps and never raise them", () => {
|
||||
const limits = new Limits();
|
||||
limits.applyWelcome(
|
||||
{
|
||||
max_body_bytes: 10 ** 9,
|
||||
max_concurrent_per_node: 1,
|
||||
job_timeout_seconds: 5,
|
||||
ping_seconds: 999,
|
||||
reconnect_min_seconds: 2,
|
||||
reconnect_max_seconds: 10 ** 6,
|
||||
},
|
||||
{ bandwidth_mb_day: 5, max_concurrent: 99 },
|
||||
);
|
||||
assert.equal(limits.maxBodyBytes, LIMITS.max_body_bytes);
|
||||
assert.equal(limits.concurrency, 1);
|
||||
assert.equal(limits.timeoutSeconds(25), 5);
|
||||
assert.equal(limits.pingSeconds, LIMITS.ping_seconds);
|
||||
assert.equal(limits.bandwidthBytes, 5 * MB);
|
||||
assert.equal(limits.reconnectMin, 2);
|
||||
assert.equal(limits.reconnectMax, LIMITS.reconnect_max_seconds);
|
||||
});
|
||||
|
||||
test("concurrency cap admits and releases", () => {
|
||||
const limits = new Limits({ local: { maxConcurrent: 1 } });
|
||||
assert.equal(limits.admit(0), null);
|
||||
assert.equal(limits.admit(0), REFUSALS.CONCURRENCY_EXCEEDED);
|
||||
limits.release();
|
||||
assert.equal(limits.admit(0), null);
|
||||
limits.release();
|
||||
limits.release();
|
||||
assert.equal(limits.inflight, 0);
|
||||
});
|
||||
|
||||
test("bandwidth cap refuses when used up", () => {
|
||||
const limits = new Limits({ local: { bandwidthMb: 1 } });
|
||||
assert.equal(limits.admit(MB - 1), null);
|
||||
limits.release();
|
||||
assert.equal(limits.admit(MB), REFUSALS.BANDWIDTH_EXHAUSTED);
|
||||
assert.equal(limits.remainingBytes(2 * MB), 0);
|
||||
});
|
||||
|
||||
test("active hours, including a window that wraps midnight", () => {
|
||||
const day = new Limits({ clock: at(12), local: { activeFrom: "09:00", activeTo: "17:00" } });
|
||||
assert.equal(day.admit(0), null);
|
||||
const early = new Limits({ clock: at(8, 59), local: { activeFrom: "09:00", activeTo: "17:00" } });
|
||||
assert.equal(early.admit(0), REFUSALS.OUTSIDE_HOURS);
|
||||
const end = new Limits({ clock: at(17), local: { activeFrom: "09:00", activeTo: "17:00" } });
|
||||
assert.equal(end.admit(0), REFUSALS.OUTSIDE_HOURS);
|
||||
const wrapIn = new Limits({ clock: at(23, 30), local: { activeFrom: "22:00", activeTo: "06:00" } });
|
||||
assert.equal(wrapIn.inActiveHours(), true);
|
||||
const wrapEarly = new Limits({ clock: at(3), local: { activeFrom: "22:00", activeTo: "06:00" } });
|
||||
assert.equal(wrapEarly.inActiveHours(), true);
|
||||
const wrapOut = new Limits({ clock: at(12), local: { activeFrom: "22:00", activeTo: "06:00" } });
|
||||
assert.equal(wrapOut.inActiveHours(), false);
|
||||
const always = new Limits({ clock: at(12) });
|
||||
assert.equal(always.inActiveHours(), true);
|
||||
});
|
||||
|
||||
test("hello limits advertise the effective concurrency", () => {
|
||||
const limits = new Limits({ local: { maxConcurrent: 1 } });
|
||||
assert.deepEqual(limits.helloLimits(), { max_concurrent: 1 });
|
||||
});
|
||||
@@ -0,0 +1,114 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { Policy } from "../src/Policy.js";
|
||||
import { DESTINATIONS, REFUSALS } from "../src/spec.js";
|
||||
|
||||
const policy = new Policy();
|
||||
|
||||
test("allows https on the allowlisted hosts and subdomains", () => {
|
||||
for (const url of [
|
||||
"https://reddit.com/r/python.json",
|
||||
"https://www.reddit.com/r/python.json",
|
||||
"https://old.reddit.com/",
|
||||
"https://redd.it/abc",
|
||||
"https://REDDIT.COM./x",
|
||||
]) {
|
||||
assert.equal(policy.check("GET", url).ok, true, url);
|
||||
}
|
||||
});
|
||||
|
||||
test("suffix matching is dot safe", () => {
|
||||
for (const url of [
|
||||
"https://evilreddit.com/",
|
||||
"https://reddit.com.evil.net/",
|
||||
"https://notredd.it/",
|
||||
"https://reddit.co/",
|
||||
]) {
|
||||
const verdict = policy.check("GET", url);
|
||||
assert.equal(verdict.ok, false, url);
|
||||
assert.equal(verdict.code, REFUSALS.HOST_NOT_ALLOWED);
|
||||
}
|
||||
});
|
||||
|
||||
test("userinfo tricks are refused", () => {
|
||||
for (const url of [
|
||||
"https://reddit.com@evil.net/",
|
||||
"https://user:pass@reddit.com/",
|
||||
"https://evil.net\\@reddit.com/",
|
||||
]) {
|
||||
assert.equal(policy.check("GET", url).ok, false, url);
|
||||
}
|
||||
});
|
||||
|
||||
test("non https schemes are refused", () => {
|
||||
for (const url of ["http://reddit.com/", "ftp://reddit.com/", "ws://reddit.com/", "file:///etc/passwd"]) {
|
||||
const verdict = policy.check("GET", url);
|
||||
assert.equal(verdict.ok, false, url);
|
||||
}
|
||||
});
|
||||
|
||||
test("only port 443 passes over https", () => {
|
||||
assert.equal(policy.check("GET", "https://reddit.com:443/").ok, true);
|
||||
for (const url of ["https://reddit.com:80/", "https://reddit.com:8443/", "https://reddit.com:22/"]) {
|
||||
const verdict = policy.check("GET", url);
|
||||
assert.equal(verdict.ok, false, url);
|
||||
assert.equal(verdict.code, REFUSALS.PORT_NOT_ALLOWED);
|
||||
}
|
||||
});
|
||||
|
||||
test("method must be permitted", () => {
|
||||
const verdict = policy.check("POST", "https://reddit.com/");
|
||||
assert.equal(verdict.code, REFUSALS.METHOD_NOT_ALLOWED);
|
||||
assert.equal(policy.check("get", "https://reddit.com/").ok, true);
|
||||
});
|
||||
|
||||
test("ip literals, junk and oversize urls are refused", () => {
|
||||
for (const url of ["https://127.0.0.1/", "https://[::1]/", "https://10.0.0.1/", "nonsense", "", null, 5]) {
|
||||
assert.equal(policy.check("GET", url).ok, false, String(url));
|
||||
}
|
||||
assert.equal(policy.check("GET", "https://reddit.com/" + "a".repeat(3000)).ok, false);
|
||||
});
|
||||
|
||||
test("a redirect leaving the allowlist is refused with redirect_not_allowed", () => {
|
||||
const base = new URL("https://www.reddit.com/r/x");
|
||||
const bad = policy.checkRedirect("https://evil.net/landing", base, "GET");
|
||||
assert.equal(bad.ok, false);
|
||||
assert.equal(bad.code, REFUSALS.REDIRECT_NOT_ALLOWED);
|
||||
assert.equal(policy.checkRedirect("http://reddit.com/x", base, "GET").code, REFUSALS.REDIRECT_NOT_ALLOWED);
|
||||
assert.equal(policy.checkRedirect("//evilreddit.com/x", base, "GET").code, REFUSALS.REDIRECT_NOT_ALLOWED);
|
||||
assert.equal(policy.checkRedirect("/r/y", base, "GET").ok, true);
|
||||
assert.equal(policy.checkRedirect("https://old.reddit.com/r/y", base, "GET").ok, true);
|
||||
});
|
||||
|
||||
test("a refreshed allowlist may only be equal or narrower", () => {
|
||||
const narrower = [{ ...DESTINATIONS[0], suffix: "old.reddit.com" }];
|
||||
const kept = Policy.narrow(DESTINATIONS, narrower);
|
||||
assert.equal(kept.destinations.length, 1);
|
||||
assert.equal(kept.rejected.length, 0);
|
||||
|
||||
const equal = Policy.narrow(DESTINATIONS, DESTINATIONS.map((item) => ({ ...item })));
|
||||
assert.equal(equal.destinations.length, DESTINATIONS.length);
|
||||
|
||||
const widened = Policy.narrow(DESTINATIONS, [
|
||||
{ suffix: "evil.net", adapter: "x", methods: ["GET"], ports: [443], modes: ["fetch"] },
|
||||
{ suffix: "reddit.com", adapter: "reddit", methods: ["GET", "POST"], ports: [443], modes: ["fetch"] },
|
||||
{ suffix: "reddit.com", adapter: "reddit", methods: ["GET"], ports: [443, 8080], modes: ["fetch"] },
|
||||
{ suffix: "reddit.com", adapter: "reddit", methods: ["GET"], ports: [443], modes: ["tunnel"] },
|
||||
{ suffix: "evilreddit.com", adapter: "reddit", methods: ["GET"], ports: [443], modes: ["fetch"] },
|
||||
]);
|
||||
assert.equal(widened.destinations.length, 0);
|
||||
assert.equal(widened.rejected.length, 5);
|
||||
|
||||
const empty = Policy.narrow(DESTINATIONS, []);
|
||||
assert.equal(empty.destinations.length, 0);
|
||||
assert.equal(Policy.narrow(DESTINATIONS, "x").destinations.length, 0);
|
||||
});
|
||||
|
||||
test("a narrowed policy refuses what the baked policy would allow", () => {
|
||||
const { policy: narrow } = Policy.fromRefreshed(DESTINATIONS, [
|
||||
{ ...DESTINATIONS[0], suffix: "old.reddit.com" },
|
||||
]);
|
||||
assert.equal(narrow.check("GET", "https://old.reddit.com/").ok, true);
|
||||
assert.equal(narrow.check("GET", "https://www.reddit.com/").ok, false);
|
||||
assert.equal(narrow.check("GET", "https://redd.it/x").ok, false);
|
||||
});
|
||||
@@ -0,0 +1,38 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { DESTINATIONS, LIMITS, REFUSALS, ALLOWED_PORTS, PROTOCOL_VERSION } from "../src/spec.js";
|
||||
|
||||
const source = readFileSync(
|
||||
fileURLToPath(new URL("../../../devplacepy/services/nodes/spec.py", import.meta.url)),
|
||||
"utf8",
|
||||
);
|
||||
|
||||
function evaluate(expression) {
|
||||
const clean = expression.replaceAll("_", "").trim();
|
||||
assert.match(clean, /^[\d\s*]+$/, expression);
|
||||
return clean.split("*").reduce((product, part) => product * Number(part), 1);
|
||||
}
|
||||
|
||||
test("baked allowlist matches the server specification", () => {
|
||||
const suffixes = [...source.matchAll(/DestinationSpec\("([^"]+)"/g)].map((match) => match[1]);
|
||||
assert.deepEqual(DESTINATIONS.map((item) => item.suffix), suffixes);
|
||||
});
|
||||
|
||||
test("baked ports, protocol and refusal codes match the server specification", () => {
|
||||
const ports = /ALLOWED_PORTS = \(([^)]*)\)/.exec(source)[1].split(",").filter(Boolean).map(Number);
|
||||
assert.deepEqual([...ALLOWED_PORTS], ports);
|
||||
assert.equal(PROTOCOL_VERSION, Number(/PROTOCOL_VERSION = (\d+)/.exec(source)[1]));
|
||||
const codes = [...source.matchAll(/RefusalSpec\("([a-z_]+)"/g)].map((match) => match[1]);
|
||||
assert.deepEqual(Object.values(REFUSALS).sort(), codes.sort());
|
||||
});
|
||||
|
||||
test("baked limits match the server LimitSpec defaults", () => {
|
||||
for (const [key, value] of Object.entries(LIMITS)) {
|
||||
const pattern = new RegExp(`^\\s+${key}: int = ([\\d_ *]+)$`, "m");
|
||||
const match = pattern.exec(source);
|
||||
assert.ok(match, `${key} missing from spec.py`);
|
||||
assert.equal(value, evaluate(match[1]), key);
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,323 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { Fetcher } from "../src/Fetcher.js";
|
||||
import { Limits } from "../src/Limits.js";
|
||||
import { Policy } from "../src/Policy.js";
|
||||
import { State, STATES } from "../src/State.js";
|
||||
import { Usage } from "../src/Usage.js";
|
||||
import { WsClient } from "../src/WsClient.js";
|
||||
import { DESTINATIONS } from "../src/spec.js";
|
||||
|
||||
function memoryArea() {
|
||||
const store = {};
|
||||
return {
|
||||
async get(key) {
|
||||
return key in store ? { [key]: store[key] } : {};
|
||||
},
|
||||
async set(values) {
|
||||
Object.assign(store, values);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
class FakeSocket {
|
||||
static instances = [];
|
||||
|
||||
constructor(address) {
|
||||
this.address = address;
|
||||
this.readyState = 0;
|
||||
this.sent = [];
|
||||
this.closed = null;
|
||||
FakeSocket.instances.push(this);
|
||||
}
|
||||
|
||||
open() {
|
||||
this.readyState = 1;
|
||||
this.onopen();
|
||||
}
|
||||
|
||||
send(text) {
|
||||
this.sent.push(JSON.parse(text));
|
||||
}
|
||||
|
||||
close(code, reason) {
|
||||
this.readyState = 3;
|
||||
this.closed = { code, reason };
|
||||
}
|
||||
|
||||
receive(frame) {
|
||||
return this.onmessage({ data: typeof frame === "string" ? frame : JSON.stringify(frame) });
|
||||
}
|
||||
|
||||
serverClose(code, reason = "") {
|
||||
this.readyState = 3;
|
||||
return this.onclose({ code, reason });
|
||||
}
|
||||
}
|
||||
|
||||
class FakeTimers {
|
||||
constructor() {
|
||||
this.timeouts = [];
|
||||
this.intervals = [];
|
||||
}
|
||||
setTimeout(fn, ms) {
|
||||
const handle = { fn, ms, cleared: false };
|
||||
this.timeouts.push(handle);
|
||||
return handle;
|
||||
}
|
||||
clearTimeout(handle) {
|
||||
handle.cleared = true;
|
||||
}
|
||||
setInterval(fn, ms) {
|
||||
const handle = { fn, ms, cleared: false };
|
||||
this.intervals.push(handle);
|
||||
return handle;
|
||||
}
|
||||
clearInterval(handle) {
|
||||
handle.cleared = true;
|
||||
}
|
||||
}
|
||||
|
||||
function build(overrides = {}) {
|
||||
FakeSocket.instances = [];
|
||||
const timers = new FakeTimers();
|
||||
const limits = new Limits();
|
||||
const usage = new Usage(memoryArea());
|
||||
const state = new State(memoryArea());
|
||||
const fetchLog = [];
|
||||
const jobFetch = overrides.jobFetch || (async () => new Response("page", { status: 200 }));
|
||||
const fetcher = new Fetcher({ policy: new Policy(), limits, usage, fetchImpl: jobFetch });
|
||||
const ticketFetch =
|
||||
overrides.ticketFetch ||
|
||||
(async (url, options) => {
|
||||
fetchLog.push({ url, options });
|
||||
return new Response(JSON.stringify({ ticket: "tk1", expires_in: 30, ws_path: "/nodes/ws" }), {
|
||||
status: 200,
|
||||
});
|
||||
});
|
||||
const client = new WsClient({
|
||||
config: { credentials: () => ({ serverUrl: "https://devplace.net", secret: "dpn_secretsecretsecret" }) },
|
||||
limits,
|
||||
fetcher,
|
||||
state,
|
||||
fetchImpl: ticketFetch,
|
||||
WebSocketImpl: FakeSocket,
|
||||
timers,
|
||||
random: overrides.random || (() => 0.5),
|
||||
clientVersion: "9.9.9",
|
||||
platform: "firefox",
|
||||
});
|
||||
return { client, timers, state, fetcher, limits, usage, fetchLog };
|
||||
}
|
||||
|
||||
const welcome = (extra = {}) => ({
|
||||
type: "welcome",
|
||||
device_uid: "dev1",
|
||||
limits: { max_concurrent_per_node: 2, ping_seconds: 20 },
|
||||
destinations: DESTINATIONS.map((item) => ({ ...item })),
|
||||
settings: { bandwidth_mb_day: 500, max_concurrent: 2 },
|
||||
...extra,
|
||||
});
|
||||
|
||||
async function online(context) {
|
||||
await context.client.start();
|
||||
const socket = FakeSocket.instances.at(-1);
|
||||
socket.open();
|
||||
await socket.receive(welcome());
|
||||
return socket;
|
||||
}
|
||||
|
||||
test("start fetches a ticket with the bearer secret and opens the socket", async () => {
|
||||
const context = build();
|
||||
await context.client.start();
|
||||
assert.equal(context.fetchLog[0].url, "https://devplace.net/nodes/ticket");
|
||||
assert.equal(context.fetchLog[0].options.headers.Authorization, "Bearer dpn_secretsecretsecret");
|
||||
assert.equal(context.fetchLog[0].options.credentials, "omit");
|
||||
const socket = FakeSocket.instances[0];
|
||||
assert.equal(socket.address, "wss://devplace.net/nodes/ws?ticket=tk1");
|
||||
});
|
||||
|
||||
test("hello is sent on open with extension kind and fetch mode only", async () => {
|
||||
const context = build();
|
||||
await context.client.start();
|
||||
const socket = FakeSocket.instances[0];
|
||||
socket.open();
|
||||
const hello = socket.sent[0];
|
||||
assert.equal(hello.type, "hello");
|
||||
assert.equal(hello.protocol, 1);
|
||||
assert.equal(hello.kind, "extension");
|
||||
assert.deepEqual(hello.modes, ["fetch"]);
|
||||
assert.equal(hello.client_version, "9.9.9");
|
||||
assert.equal(hello.platform, "firefox");
|
||||
assert.deepEqual(hello.limits, { max_concurrent: 2 });
|
||||
});
|
||||
|
||||
test("welcome puts the client online and starts the keepalive", async () => {
|
||||
const context = build();
|
||||
await online(context);
|
||||
assert.equal(context.state.snapshot().state, STATES.ONLINE);
|
||||
assert.equal(context.state.snapshot().deviceUid, "dev1");
|
||||
assert.equal(context.timers.intervals[0].ms, 20000);
|
||||
});
|
||||
|
||||
test("keepalive timer sends pong and a server ping is answered with pong", async () => {
|
||||
const context = build();
|
||||
const socket = await online(context);
|
||||
context.timers.intervals[0].fn();
|
||||
await socket.receive({ type: "ping" });
|
||||
assert.deepEqual(
|
||||
socket.sent.filter((frame) => frame.type === "pong"),
|
||||
[{ type: "pong" }, { type: "pong" }],
|
||||
);
|
||||
});
|
||||
|
||||
test("a job is executed and answered with job_result then usage", async () => {
|
||||
const context = build();
|
||||
const socket = await online(context);
|
||||
await socket.receive({
|
||||
type: "job",
|
||||
job_id: "j9",
|
||||
method: "GET",
|
||||
url: "https://www.reddit.com/r/x.json",
|
||||
timeout: 5,
|
||||
});
|
||||
await context.client.idle();
|
||||
const result = socket.sent.find((frame) => frame.type === "job_result");
|
||||
assert.equal(result.job_id, "j9");
|
||||
assert.equal(Buffer.from(result.body_b64, "base64").toString(), "page");
|
||||
assert.equal(socket.sent.some((frame) => frame.type === "usage"), true);
|
||||
});
|
||||
|
||||
test("a job for a forbidden host is answered with job_error", async () => {
|
||||
const context = build();
|
||||
const socket = await online(context);
|
||||
await socket.receive({ type: "job", job_id: "j2", method: "GET", url: "https://evilreddit.com/", timeout: 5 });
|
||||
await context.client.idle();
|
||||
const error = socket.sent.find((frame) => frame.type === "job_error");
|
||||
assert.deepEqual(error, { type: "job_error", job_id: "j2", code: "host_not_allowed" });
|
||||
});
|
||||
|
||||
test("a welcome that widens the allowlist is ignored for the widened entries", async () => {
|
||||
const context = build();
|
||||
await context.client.start();
|
||||
const socket = FakeSocket.instances[0];
|
||||
socket.open();
|
||||
await socket.receive(
|
||||
welcome({
|
||||
destinations: [
|
||||
...DESTINATIONS.map((item) => ({ ...item })),
|
||||
{ suffix: "evil.net", adapter: "x", methods: ["GET"], ports: [443], modes: ["fetch"] },
|
||||
],
|
||||
}),
|
||||
);
|
||||
assert.match(context.state.snapshot().lastError, /evil\.net/);
|
||||
await socket.receive({ type: "job", job_id: "j3", method: "GET", url: "https://evil.net/", timeout: 5 });
|
||||
await context.client.idle();
|
||||
assert.equal(socket.sent.find((frame) => frame.type === "job_error").code, "host_not_allowed");
|
||||
});
|
||||
|
||||
test("a job before welcome is a protocol fault and closes with 4400", async () => {
|
||||
const context = build();
|
||||
await context.client.start();
|
||||
const socket = FakeSocket.instances[0];
|
||||
socket.open();
|
||||
await socket.receive({ type: "job", job_id: "j", method: "GET", url: "https://reddit.com/", timeout: 5 });
|
||||
assert.equal(socket.closed.code, 4400);
|
||||
assert.equal(context.state.snapshot().state, STATES.BACKOFF);
|
||||
});
|
||||
|
||||
test("malformed frames are rejected loudly", async () => {
|
||||
const context = build();
|
||||
const socket = await online(context);
|
||||
for (const bad of ["not json", "[]", JSON.stringify({ type: "nope" }), JSON.stringify({ type: "job", job_id: "x" })]) {
|
||||
await assert.rejects(context.client.handleMessage(bad));
|
||||
}
|
||||
await assert.rejects(context.client.handleMessage(new ArrayBuffer(4)));
|
||||
assert.equal(socket.sent.filter((frame) => frame.type === "job_result").length, 0);
|
||||
});
|
||||
|
||||
test("kick halts without reconnecting", async () => {
|
||||
const context = build();
|
||||
const socket = await online(context);
|
||||
await socket.receive({ type: "kick", reason: "replaced" });
|
||||
assert.equal(context.state.snapshot().state, STATES.HALTED);
|
||||
assert.match(context.state.snapshot().detail, /replaced/);
|
||||
assert.equal(context.timers.timeouts.filter((handle) => !handle.cleared).length, 0);
|
||||
await context.client.ensure();
|
||||
assert.equal(FakeSocket.instances.length, 1);
|
||||
});
|
||||
|
||||
test("a normal close schedules a full jitter reconnect within the bounds", async () => {
|
||||
const context = build({ random: () => 0.999 });
|
||||
const socket = await online(context);
|
||||
await socket.serverClose(1006);
|
||||
const timer = context.timers.timeouts.at(-1);
|
||||
assert.ok(timer.ms >= 1000 && timer.ms <= 60000, String(timer.ms));
|
||||
assert.equal(context.state.snapshot().state, STATES.BACKOFF);
|
||||
timer.fn();
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
assert.equal(FakeSocket.instances.length, 2);
|
||||
});
|
||||
|
||||
test("backoff grows with attempts and never exceeds reconnect_max", async () => {
|
||||
const context = build({ random: () => 1 });
|
||||
await context.client.start();
|
||||
const delays = [];
|
||||
for (let index = 0; index < 12; index += 1) {
|
||||
const socket = FakeSocket.instances.at(-1);
|
||||
await socket.serverClose(1006);
|
||||
const timer = context.timers.timeouts.at(-1);
|
||||
delays.push(timer.ms);
|
||||
timer.fn();
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
}
|
||||
assert.ok(delays[1] > delays[0]);
|
||||
assert.ok(Math.max(...delays) <= 60000);
|
||||
assert.equal(delays.at(-1), 60000);
|
||||
});
|
||||
|
||||
test("retry owner close reconnects at the minimum delay", async () => {
|
||||
const context = build({ random: () => 1 });
|
||||
const socket = await online(context);
|
||||
await socket.serverClose(4013);
|
||||
assert.equal(context.timers.timeouts.at(-1).ms, 1000);
|
||||
});
|
||||
|
||||
test("a consent or device refusal halts instead of hammering the server", async () => {
|
||||
const context = build();
|
||||
await context.client.start();
|
||||
const socket = FakeSocket.instances[0];
|
||||
await socket.serverClose(1008, "no_consent");
|
||||
assert.equal(context.state.snapshot().state, STATES.HALTED);
|
||||
assert.match(context.state.snapshot().detail, /no_consent/);
|
||||
});
|
||||
|
||||
test("a ticket 401 halts, a ticket 500 retries", async () => {
|
||||
const halted = build({
|
||||
ticketFetch: async () => new Response(JSON.stringify({ error: "x" }), { status: 401 }),
|
||||
});
|
||||
await halted.client.start();
|
||||
assert.equal(halted.state.snapshot().state, STATES.HALTED);
|
||||
assert.equal(FakeSocket.instances.length, 0);
|
||||
|
||||
const retried = build({ ticketFetch: async () => new Response("boom", { status: 500 }) });
|
||||
await retried.client.start();
|
||||
assert.equal(retried.state.snapshot().state, STATES.BACKOFF);
|
||||
});
|
||||
|
||||
test("stop closes the socket and clears timers", async () => {
|
||||
const context = build();
|
||||
const socket = await online(context);
|
||||
await context.client.stop("user");
|
||||
assert.equal(socket.closed.code, 1000);
|
||||
assert.equal(context.timers.intervals[0].cleared, true);
|
||||
assert.equal(context.state.snapshot().state, STATES.OFF);
|
||||
});
|
||||
|
||||
test("settings_update can only lower the effective limits", async () => {
|
||||
const context = build();
|
||||
const socket = await online(context);
|
||||
await socket.receive({ type: "settings_update", settings: { bandwidth_mb_day: 10, max_concurrent: 1 } });
|
||||
assert.equal(context.limits.concurrency, 1);
|
||||
assert.equal(context.limits.bandwidthBytes, 10 * 1024 * 1024);
|
||||
});
|
||||
@@ -0,0 +1,55 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>DevPlace Node: what is shared</title>
|
||||
<link rel="stylesheet" href="popup.css">
|
||||
</head>
|
||||
<body class="page">
|
||||
<main>
|
||||
<h1>Before you share anything</h1>
|
||||
<p class="muted">DevPlace Node lets the community use a little of your internet connection to read public pages from sites that block data centers. Read this once. Nothing runs until you accept and switch it on.</p>
|
||||
|
||||
<section class="card">
|
||||
<h2>What passes through your browser</h2>
|
||||
<ul>
|
||||
<li>The <strong>address of the page</strong> that is requested (host and full URL), for example a public Reddit thread.</li>
|
||||
<li>The <strong>page content</strong> that comes back. It is read by this extension and sent to the DevPlace server.</li>
|
||||
<li>Your <strong>public IP address</strong> is what the destination site sees. That is the point of donating.</li>
|
||||
<li>How many requests and how many bytes you moved, counted on this device and on the server.</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
<h2>What is never used</h2>
|
||||
<ul>
|
||||
<li>Your <strong>cookies, logins and accounts</strong>. Every request is sent without credentials, so you are never signed in as yourself.</li>
|
||||
<li>Your browsing history, open tabs and other sites.</li>
|
||||
<li>Any host outside the built-in list, any port other than the standard secure one, any address inside your own network.</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
<h2>You stay in control</h2>
|
||||
<ul>
|
||||
<li>One on/off switch in the popup. Off means disconnected.</li>
|
||||
<li>A daily data cap, a parallel request cap and optional active hours, enforced on this device even if the server asks for more.</li>
|
||||
<li>Live counters of requests and bytes in the popup.</li>
|
||||
<li>You can forget the device secret at any time, and revoke the device on your DevPlace profile.</li>
|
||||
</ul>
|
||||
<p class="muted">Pages fetched through your connection are public pages. Do not donate from a connection whose address you cannot afford to have associated with that traffic.</p>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
<div class="row">
|
||||
<button type="button" class="primary" id="accept">I understand, allow it</button>
|
||||
<button type="button" id="decline">Not now</button>
|
||||
</div>
|
||||
<p class="error" id="error" role="alert"></p>
|
||||
<p class="muted" id="done"></p>
|
||||
</section>
|
||||
</main>
|
||||
<script type="module" src="disclosure.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,31 @@
|
||||
class Disclosure {
|
||||
constructor(root) {
|
||||
this.root = root;
|
||||
this.$ = (id) => root.getElementById(id);
|
||||
}
|
||||
|
||||
async accept() {
|
||||
try {
|
||||
const reply = await chrome.runtime.sendMessage({ cmd: "accept" });
|
||||
if (!reply || reply.error) {
|
||||
throw new Error(reply ? reply.error : "The background worker did not answer");
|
||||
}
|
||||
this.$("error").textContent = "";
|
||||
this.$("done").textContent =
|
||||
"Accepted. Open the extension popup, connect your device secret and switch sharing on.";
|
||||
} catch (error) {
|
||||
this.$("error").textContent = error.message;
|
||||
}
|
||||
}
|
||||
|
||||
decline() {
|
||||
this.$("done").textContent = "Nothing was enabled. You can close this tab.";
|
||||
}
|
||||
|
||||
start() {
|
||||
this.$("accept").addEventListener("click", () => this.accept());
|
||||
this.$("decline").addEventListener("click", () => this.decline());
|
||||
}
|
||||
}
|
||||
|
||||
new Disclosure(document).start();
|
||||
@@ -0,0 +1,226 @@
|
||||
:root {
|
||||
color-scheme: light dark;
|
||||
--bg: #f6f7f9;
|
||||
--card: #ffffff;
|
||||
--text: #1c2128;
|
||||
--muted: #5b6672;
|
||||
--border: #d5dae0;
|
||||
--accent: #2f6fed;
|
||||
--accent-text: #ffffff;
|
||||
--ok: #1a8a4a;
|
||||
--warn: #b7791f;
|
||||
--bad: #c0392b;
|
||||
--radius: 8px;
|
||||
--gap: 12px;
|
||||
--font: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
|
||||
}
|
||||
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--bg: #14181d;
|
||||
--card: #1d232a;
|
||||
--text: #e8ecf0;
|
||||
--muted: #97a3b0;
|
||||
--border: #323b45;
|
||||
--accent: #5b92ff;
|
||||
--accent-text: #0b1220;
|
||||
--ok: #4cc27f;
|
||||
--warn: #e0b04a;
|
||||
--bad: #ee7b6e;
|
||||
}
|
||||
}
|
||||
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
font: 14px/1.45 var(--font);
|
||||
width: 100%;
|
||||
min-width: 260px;
|
||||
max-width: 420px;
|
||||
}
|
||||
|
||||
body.page {
|
||||
max-width: 640px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
main {
|
||||
padding: var(--gap);
|
||||
display: grid;
|
||||
gap: var(--gap);
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-size: 1.15rem;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
h2 {
|
||||
font-size: 0.95rem;
|
||||
margin: 0 0 6px;
|
||||
}
|
||||
|
||||
p {
|
||||
margin: 0 0 8px;
|
||||
}
|
||||
|
||||
.card {
|
||||
background: var(--card);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
padding: var(--gap);
|
||||
}
|
||||
|
||||
.muted {
|
||||
color: var(--muted);
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
|
||||
.row {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.status {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.dot {
|
||||
width: 10px;
|
||||
height: 10px;
|
||||
border-radius: 50%;
|
||||
background: var(--muted);
|
||||
}
|
||||
|
||||
.dot.online {
|
||||
background: var(--ok);
|
||||
}
|
||||
|
||||
.dot.connecting,
|
||||
.dot.backoff {
|
||||
background: var(--warn);
|
||||
}
|
||||
|
||||
.dot.halted {
|
||||
background: var(--bad);
|
||||
}
|
||||
|
||||
.counters {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(110px, 1fr));
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.counter {
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
padding: 8px;
|
||||
}
|
||||
|
||||
.counter strong {
|
||||
display: block;
|
||||
font-size: 1.1rem;
|
||||
}
|
||||
|
||||
label {
|
||||
display: grid;
|
||||
gap: 4px;
|
||||
margin-bottom: 8px;
|
||||
font-size: 0.85rem;
|
||||
color: var(--muted);
|
||||
}
|
||||
|
||||
input {
|
||||
width: 100%;
|
||||
font: inherit;
|
||||
color: var(--text);
|
||||
background: var(--bg);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
padding: 7px 9px;
|
||||
}
|
||||
|
||||
button,
|
||||
.button {
|
||||
font: inherit;
|
||||
cursor: pointer;
|
||||
color: var(--text);
|
||||
background: var(--card);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
padding: 7px 12px;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
button.primary {
|
||||
background: var(--accent);
|
||||
color: var(--accent-text);
|
||||
border-color: var(--accent);
|
||||
}
|
||||
|
||||
button.danger {
|
||||
color: var(--bad);
|
||||
}
|
||||
|
||||
button:disabled {
|
||||
opacity: 0.5;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.switch {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
margin: 0;
|
||||
color: var(--text);
|
||||
font-size: 1rem;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.switch input {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
}
|
||||
|
||||
.notice {
|
||||
border-left: 3px solid var(--warn);
|
||||
padding-left: 10px;
|
||||
}
|
||||
|
||||
.error {
|
||||
color: var(--bad);
|
||||
min-height: 1.2em;
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
.pair {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(110px, 1fr));
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
details > summary {
|
||||
cursor: pointer;
|
||||
font-weight: 600;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
ul {
|
||||
margin: 0 0 8px;
|
||||
padding-left: 20px;
|
||||
}
|
||||
|
||||
li {
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>DevPlace Node</title>
|
||||
<link rel="stylesheet" href="popup.css">
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
<div class="row">
|
||||
<h1>DevPlace Node</h1>
|
||||
<span class="status"><span class="dot" id="dot"></span><span id="state">off</span></span>
|
||||
</div>
|
||||
|
||||
<section class="card" id="disclosure-card" hidden>
|
||||
<p class="notice">Before this can run you must read what is shared.</p>
|
||||
<a class="button primary" id="open-disclosure" href="disclosure.html" target="_blank" rel="noopener">Read the disclosure</a>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
<div class="row">
|
||||
<label class="switch"><input type="checkbox" id="toggle"> <span id="toggle-label">Sharing off</span></label>
|
||||
<span class="muted" id="label"></span>
|
||||
</div>
|
||||
<p class="muted" id="detail"></p>
|
||||
<p class="error" id="error" role="alert"></p>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
<h2>Today</h2>
|
||||
<div class="counters">
|
||||
<div class="counter"><span class="muted">Requests</span><strong id="requests">0</strong></div>
|
||||
<div class="counter"><span class="muted">Downloaded</span><strong id="down">0 B</strong></div>
|
||||
<div class="counter"><span class="muted">Uploaded</span><strong id="up">0 B</strong></div>
|
||||
<div class="counter"><span class="muted">Failed</span><strong id="failed">0</strong></div>
|
||||
</div>
|
||||
<p class="muted" id="lifetime"></p>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
<details id="settings-box">
|
||||
<summary>Limits on this device</summary>
|
||||
<form id="settings">
|
||||
<div class="pair">
|
||||
<label>Data per day (MB)<input type="number" id="bandwidth" min="1" step="1"></label>
|
||||
<label>Parallel requests<input type="number" id="concurrency" min="1" step="1"></label>
|
||||
</div>
|
||||
<div class="pair">
|
||||
<label>Active from<input type="time" id="from"></label>
|
||||
<label>Active to<input type="time" id="to"></label>
|
||||
</div>
|
||||
<p class="muted">Leave the hours empty to share at any time. These limits are enforced here, whatever the server asks.</p>
|
||||
<button type="submit">Save limits</button>
|
||||
</form>
|
||||
</details>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
<details id="connect-box">
|
||||
<summary id="connect-summary">Connect to a server</summary>
|
||||
<form id="connect">
|
||||
<label>Server URL<input type="url" id="server" placeholder="https://devplace.net" required></label>
|
||||
<label>Device secret<input type="password" id="secret" placeholder="dpn_..." autocomplete="off" required></label>
|
||||
<label>Device label<input type="text" id="device-label" maxlength="64"></label>
|
||||
<button type="submit" class="primary">Save and connect</button>
|
||||
</form>
|
||||
<p class="muted" id="saved"></p>
|
||||
<button type="button" class="danger" id="forget" hidden>Forget this device</button>
|
||||
</details>
|
||||
</section>
|
||||
|
||||
<p class="muted"><a href="disclosure.html" target="_blank" rel="noopener">What is shared</a></p>
|
||||
</main>
|
||||
<script type="module" src="popup.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,155 @@
|
||||
const POLL_MS = 2000;
|
||||
const UNITS = ["B", "KB", "MB", "GB", "TB"];
|
||||
const STATE_LABELS = {
|
||||
off: "Off",
|
||||
connecting: "Connecting",
|
||||
online: "Online",
|
||||
backoff: "Reconnecting",
|
||||
halted: "Stopped",
|
||||
};
|
||||
|
||||
function formatBytes(value) {
|
||||
let size = Math.max(0, Number(value) || 0);
|
||||
let unit = 0;
|
||||
while (size >= 1024 && unit < UNITS.length - 1) {
|
||||
size /= 1024;
|
||||
unit += 1;
|
||||
}
|
||||
return `${unit === 0 ? size : size.toFixed(1)} ${UNITS[unit]}`;
|
||||
}
|
||||
|
||||
class Popup {
|
||||
constructor(root) {
|
||||
this.root = root;
|
||||
this.timer = null;
|
||||
this.view = null;
|
||||
this.$ = (id) => root.getElementById(id);
|
||||
}
|
||||
|
||||
async send(message) {
|
||||
const reply = await chrome.runtime.sendMessage(message);
|
||||
if (!reply) {
|
||||
throw new Error("The background worker did not answer");
|
||||
}
|
||||
if (reply.error) {
|
||||
throw new Error(reply.error);
|
||||
}
|
||||
return reply;
|
||||
}
|
||||
|
||||
showError(error) {
|
||||
this.$("error").textContent = error ? error.message || String(error) : "";
|
||||
}
|
||||
|
||||
async run(message) {
|
||||
try {
|
||||
this.render(await this.send(message));
|
||||
this.showError(null);
|
||||
} catch (error) {
|
||||
this.showError(error);
|
||||
}
|
||||
}
|
||||
|
||||
render(view) {
|
||||
this.view = view;
|
||||
const { config, state, usage } = view;
|
||||
this.$("dot").className = `dot ${state.state}`;
|
||||
this.$("state").textContent = STATE_LABELS[state.state] || state.state;
|
||||
this.$("detail").textContent = state.detail || state.lastError || "";
|
||||
this.$("label").textContent = config.label;
|
||||
this.$("disclosure-card").hidden = config.disclosed;
|
||||
const toggle = this.$("toggle");
|
||||
toggle.checked = config.enabled;
|
||||
toggle.disabled = !config.disclosed || !config.hasSecret;
|
||||
this.$("toggle-label").textContent = config.enabled ? "Sharing on" : "Sharing off";
|
||||
this.$("requests").textContent = usage.requests;
|
||||
this.$("failed").textContent = usage.failed;
|
||||
this.$("down").textContent = formatBytes(usage.bytesDown);
|
||||
this.$("up").textContent = formatBytes(usage.bytesUp);
|
||||
this.$("lifetime").textContent =
|
||||
`All time: ${usage.lifetimeRequests} requests, ${formatBytes(usage.lifetimeBytes)}`;
|
||||
this.renderSettings(config);
|
||||
this.renderConnection(config);
|
||||
}
|
||||
|
||||
renderSettings(config) {
|
||||
if (this.root.activeElement && this.$("settings").contains(this.root.activeElement)) {
|
||||
return;
|
||||
}
|
||||
this.$("bandwidth").value = config.bandwidthMb;
|
||||
this.$("concurrency").value = config.maxConcurrent;
|
||||
this.$("from").value = config.activeFrom;
|
||||
this.$("to").value = config.activeTo;
|
||||
}
|
||||
|
||||
renderConnection(config) {
|
||||
this.$("forget").hidden = !config.hasSecret;
|
||||
this.$("connect-summary").textContent = config.hasSecret ? "Connection" : "Connect to a server";
|
||||
this.$("saved").textContent = config.hasSecret
|
||||
? `Connected to ${config.serverUrl} with secret ${config.secretPrefix}`
|
||||
: "";
|
||||
if (!config.hasSecret) {
|
||||
this.$("connect-box").open = true;
|
||||
}
|
||||
}
|
||||
|
||||
async requestOrigin(serverUrl) {
|
||||
const origin = new URL(serverUrl).origin;
|
||||
const granted = await chrome.permissions.request({ origins: [origin + "/*"] });
|
||||
if (!granted) {
|
||||
throw new Error("Permission to talk to that server was not granted");
|
||||
}
|
||||
}
|
||||
|
||||
async onConnect(event) {
|
||||
event.preventDefault();
|
||||
const secretField = this.$("secret");
|
||||
try {
|
||||
await this.requestOrigin(this.$("server").value);
|
||||
this.render(
|
||||
await this.send({
|
||||
cmd: "connect",
|
||||
serverUrl: this.$("server").value,
|
||||
secret: secretField.value,
|
||||
label: this.$("device-label").value,
|
||||
}),
|
||||
);
|
||||
secretField.value = "";
|
||||
this.showError(null);
|
||||
} catch (error) {
|
||||
this.showError(error);
|
||||
}
|
||||
}
|
||||
|
||||
onSettings(event) {
|
||||
event.preventDefault();
|
||||
this.run({
|
||||
cmd: "settings",
|
||||
values: {
|
||||
bandwidthMb: Number(this.$("bandwidth").value),
|
||||
maxConcurrent: Number(this.$("concurrency").value),
|
||||
activeFrom: this.$("from").value,
|
||||
activeTo: this.$("to").value,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
onForget() {
|
||||
if (this.root.defaultView.confirm("Forget this device secret on this browser?")) {
|
||||
this.run({ cmd: "forget" });
|
||||
}
|
||||
}
|
||||
|
||||
start() {
|
||||
this.$("connect").addEventListener("submit", (event) => this.onConnect(event));
|
||||
this.$("settings").addEventListener("submit", (event) => this.onSettings(event));
|
||||
this.$("forget").addEventListener("click", () => this.onForget());
|
||||
this.$("toggle").addEventListener("change", (event) =>
|
||||
this.run({ cmd: "enable", enabled: event.target.checked }),
|
||||
);
|
||||
this.run({ cmd: "status" });
|
||||
this.timer = setInterval(() => this.run({ cmd: "status" }), POLL_MS);
|
||||
}
|
||||
}
|
||||
|
||||
new Popup(document).start();
|
||||
Reference in New Issue
Block a user