diff --git a/.gitignore b/.gitignore index 567609b..6dbfd2a 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,33 @@ +# build output build/ +cmake-build-*/ +CMakeCache.txt +CMakeFiles/ +cmake_install.cmake +compile_commands.json +*.o +*.a +*.so +*.d + +# state and scratch created by running or testing doos +.doos/ +*.tmp +*.log +*.pid +*.core +core + +# editors and OS +.vscode/ +.idea/ +*.swp +*~ +.DS_Store + +# local AI tooling state +.claude/ + +# python bytecode from docs/charts/make_charts.py +__pycache__/ +*.pyc diff --git a/AGENTS.md b/AGENTS.md index 2029d29..8066cc5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,7 +25,7 @@ Usage: [README](README.md). History: [CHANGELOG](CHANGELOG.md). - `patch`: bug fix, docs, tests, tooling, no behavior change for users. - `minor`: new or changed user-visible behavior (flag, command, mount, limit). - `major`: breaking change to CLI, state layout, or container compatibility. -4. Run `scripts/check-docs.sh` (the release script does too: every document must link to every other, every relative link must resolve). +4. Run `make docs` (`charts` plus `check-docs`) (the release script does too: every document must link to every other, every relative link must resolve). 5. Run `scripts/release.sh "imperative commit subject"`. It bumps `VERSION`, builds, runs `tests/run.sh`, commits all changes, and creates the annotated tag `vX.Y.Z`. It aborts on any build or test failure. @@ -46,7 +46,8 @@ Never hand-edit `VERSION` or create tags manually. Never skip the tests to relea | A new measurement | `docs/RESEARCH.md` with machine, kernel, and what was run | | A new known gap or a closed gap | `docs/RESEARCH.md` gaps list and `README.md` | | These workflow rules | this file | -| A feature, limit or measurement that appears in the comparison tables | `COMPARISON.md` (tables, the gaps list, the conclusion) | +| A measurement, or anything the charts show | `make bench` then `make charts` (rewrites `docs/charts/*.svg` and every `` number and table in `COMPARISON.md` and `docs/RESEARCH.md`); never type those numbers by hand | +| A feature or limit that appears in the comparison tables | `COMPARISON.md` (tables, the gaps list, the conclusion) | | A new or renamed markdown file | `scripts/check-docs.sh` DOCS list, the nav line (`**Docs:**`) of every document, `README.md` Project docs | | Image catalog, sources, verification, pinned keys | `SPEC.md` Images, `README.md`, `docs/RESEARCH.md` | | Seccomp list, capability set, `/etc` allowlist | `SPEC.md` Hardening and filesystem, `README.md`, `docs/RESEARCH.md` | @@ -81,6 +82,12 @@ freeze (about 4 minutes in total, needs network, `pasta`, `gpg` and cgroup v2 de before release. On Ubuntu 24.04+ the AppArmor profile in `apparmor/doos` must be installed once for the binary being tested (`doos apparmor`; needs sudo, ask the user, do not do it yourself). +## Make targets + +`make help` lists them: `build`, `test`, `check-docs`, `bench` (re-measure into `docs/charts/bench.json`), +`charts` (SVGs and generated numbers from that file), `docs`, `release KIND=patch MSG="..."`, `clean`. +The Makefile is a thin wrapper; the logic lives in `CMakeLists.txt`, `scripts/` and `docs/charts/`. + ## Environment notes - State dir: `$DOOS_HOME`, else `$XDG_DATA_HOME/doos`, else `~/.local/share/doos`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 768c7a4..cccfc8f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,11 @@ **Docs:** [README](README.md) · [SPEC](SPEC.md) · [ARCHITECTURE](docs/ARCHITECTURE.md) · [RESEARCH](docs/RESEARCH.md) · [COMPARISON](COMPARISON.md) · [AGENTS](AGENTS.md) · [CHANGELOG](CHANGELOG.md) +## 1.1.3 - 2026-10-23 +- Charts: startup time, executable size and CPU limit as SVG (light and dark palettes, doos highlighted), embedded in [COMPARISON](COMPARISON.md) and [RESEARCH](docs/RESEARCH.md). +- Reproducible measurements: `scripts/bench.sh` writes `docs/charts/bench.json`; `docs/charts/make_charts.py` builds the charts and rewrites every generated number and table in the documents from it, so text, tables and charts cannot drift apart. +- `Makefile` wrapper (`make help`), expanded `.gitignore`. + ## 1.1.2 - 2026-10-23 - Docs: new [COMPARISON](COMPARISON.md) (doos against Docker, rootless Docker, Podman, bubblewrap, systemd-nspawn, LXC/Incus, Apptainer, Distrobox/Toolbx, gVisor, Kata/Firecracker) with sourced pros and cons, startup and disk measurements against Docker, Podman and bubblewrap on the same machine, a gaps list tied to code, and a conclusion. - Docs: every document now has a navigation line and links to every other document. `scripts/check-docs.sh` enforces it (and that relative links resolve) and runs inside `scripts/release.sh`. diff --git a/COMPARISON.md b/COMPARISON.md index 83089b0..dad8034 100644 --- a/COMPARISON.md +++ b/COMPARISON.md @@ -30,7 +30,7 @@ Versions compared: doos 1.1.1, Docker 29.1.3 (rootful daemon on this machine), P | Default capabilities | Docker's default set [D][M] | that set [S] | that set | a smaller set [S: [dev.to comparison](https://dev.to/mechcloud_academy/docker-vs-podman-an-in-depth-comparison-2025-2eia)] | | Orchestration (Compose, Kubernetes) | none [D] | Compose, Swarm, Kubernetes via runtimes [K] | same | Compose, pods, `kube play`, systemd units [K] | | Background containers, logs, `stop` | none: a container lives while its first `attach` command lives [D] | yes [K] | yes | yes | -| Binary size | 552 KiB, 6 shared libraries, 3,445 lines of C++ [M] | CLI 29.9 MiB, plus `dockerd` and `containerd` [M][K] | same | 41 MiB [M] | +| Binary size | 552 KiB, 6 shared libraries, 3,445 lines of C++ [M] | CLI 29.9 MiB, plus `dockerd` and `containerd` [M][K] | same | 41.2 MiB [M] | | Platforms | Linux only, tested on x86_64 only [D] | Linux, macOS, Windows (via a VM) [K] | Linux | Linux; macOS and Windows via a VM [K] | | Maturity | version 1.1, one test machine | very mature, huge ecosystem | mature | mature | @@ -48,29 +48,45 @@ Versions compared: doos 1.1.1, Docker 29.1.3 (rootful daemon on this machine), P ## 3. Measurements -Method: trivial command (`true`) started 10 times in a row, average wall time per run, warm caches, the Alpine image already present for every tool, one machine, one run set. Scripts were ad-hoc shell loops and are not part of the repository tests, so treat the numbers as indicative, not as a benchmark. +Method: a trivial command (`true`) started 10 times in a row after one warm-up run, average wall time per run, the Alpine image already present for every tool, one machine, one run set on 2026-10-23. Versions: doos 1.1.2, Docker 29.1.3, Podman 5.7.0, bubblewrap 0.11.1. The numbers come from `scripts/bench.sh` (`make bench`); the charts and the tables below are generated from its output (`make charts`), so they cannot drift from each other. -| Setup | Time per run | -|---|---| -| bubblewrap, host `/usr`, all namespaces unshared, no network | 9 ms | -| bubblewrap, host `/usr`, network shared | 8 ms | -| doos host mode, `--net none` | 14 ms | -| doos image Alpine, `--net none` | 16 ms | -| doos host mode, slirp | 54 ms | -| doos image Alpine, slirp | 57 ms | -| doos image Alpine, pasta | 65 ms | -| Podman, Alpine, `--network none` | 214 ms | -| Podman, Alpine, default network | 234 ms | -| Docker (rootful), Alpine, `--network none` | 590 ms | -| Docker (rootful), Alpine, default bridge | 456 ms | +![Horizontal bar chart of the time to start a trivial command for doos, bubblewrap, Podman and Docker, with and without network. doos is highlighted. Exact values are in the table below.](docs/charts/startup.svg) + + +| Setup | Network | Time per run | +|---|---|---| +| bubblewrap (host /usr) | none | 8.4 ms | +| doos Alpine image | none | 13.8 ms | +| doos host mode | none | 18.7 ms | +| Podman Alpine | none | 208 ms | +| Docker Alpine (rootful) | none | 344 ms | +| bubblewrap (host /usr) | default | 7.4 ms | +| doos Alpine image | default | 52.1 ms | +| doos host mode | default | 56.9 ms | +| doos Alpine, pasta | default | 64.6 ms | +| Podman Alpine | default | 241 ms | +| Docker Alpine (rootful) | default | 436 ms | + How to read this honestly: - doos is fast partly because it does less: no image store database, no registry code, no layer unpacking per run, no container metadata service. The extra time over bubblewrap is the id map helpers (`newuidmap`, `newgidmap`), the overlay mounts, a hardening filter and the network helper process. - Docker here is a rootful daemon, so its numbers include a client-to-daemon round trip. Rootless Docker was not measured. -- The Docker row "none" is slower than "default" in this run; the spread between runs was not measured, so do not read a ranking into it. +- Run-to-run spread is real. An earlier ad-hoc run of the same loop measured rootful Docker at 590 ms (no network) and 456 ms (default network); the run charted above measured 344 ms and 436 ms. Read the order of magnitude, not a ranking between close values. - Startup is one number among many. It matters for scripts that start hundreds of short-lived sandboxes. It does not matter for a service that starts once. +![Horizontal bar chart of executable sizes on a logarithmic axis for bubblewrap, crun, doos, the docker CLI and podman. doos is highlighted. Exact values are in the table below.](docs/charts/size.svg) + + +| Executable | Size | +|---|---| +| bubblewrap | 79 KiB (80,424 bytes) | +| crun | 472 KiB (483,392 bytes) | +| doos | 552 KiB (565,680 bytes) | +| docker CLI | 29.9 MiB (31,374,336 bytes) | +| podman | 41.2 MiB (43,180,896 bytes) | + + Disk, measured with `du` (which cannot read overlayfs' mode-000 work directories, so true usage is slightly higher): | Item | Size | @@ -89,8 +105,8 @@ The Podman figure is this machine's accumulated state, not a like-for-like compa **Where doos is better** - No daemon and no root. With Docker's default install, anyone who can talk to the daemon controls the machine [S]. doos has no such trust boundary: it can only do what the invoking user can do, plus what the `newuidmap` helper allows. -- Tiny and auditable: 552 KiB, 3,445 lines [M], one process tree, no background service, nothing listening. -- Instant: 14 to 65 ms against 456 to 590 ms for a rootful Docker run [M]. +- Tiny and auditable: 552 KiB, 3,445 lines of C++ [M], one process tree, no background service, nothing listening. +- Instant: 14 to 65 ms against 344 to 436 ms for a rootful Docker run and 208 to 241 ms for Podman [M]. - Safe defaults a user does not have to remember: Docker's default capability set, a seccomp filter, an allowlist of host `/etc`, a memory and pid cap, a written-data cap [D][M]. - A "host mode" that reuses the host's installed tools (compilers, `apt` state) for throwaway shells with no image download [D]. - A size cap that freezes instead of filling the disk [D][M]. Docker's writable-layer size limit needs the overlay2 driver on XFS with project quotas [S: [Fabian Lee](https://fabianlee.org/2020/01/15/docker-use-overlay2-with-an-xfs-backing-filesystem-to-limit-rootfs-size/)]. @@ -155,7 +171,7 @@ These are the gaps that matter when comparing, each next to where it lives. They ## 8. Conclusion -doos is not a Docker replacement and does not try to be. It is a small, rootless, daemonless tool for **disposable or per-user Linux shells and sandboxes that start in tens of milliseconds, with safe defaults and hard caps already set**. Its strengths are the ones the measurements show: it is a 552 KiB single binary, it starts in 14 to 65 ms [M], it runs with no daemon and no root, it caps memory, processes, CPU and written data by default, and it verifies what it downloads. +doos is not a Docker replacement and does not try to be. It is a small, rootless, daemonless tool for **disposable or per-user Linux shells and sandboxes that start in tens of milliseconds, with safe defaults and hard caps already set**. Its strengths are the ones the measurements show: it is a 552 KiB single binary, it starts in 14 to 65 ms [M], it runs with no daemon and no root, it caps memory, processes, CPU and written data by default, and it verifies what it downloads. Choose by the job: diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..d98aa6f --- /dev/null +++ b/Makefile @@ -0,0 +1,35 @@ +# Thin wrapper over the commands used to build, test and update the project. CMake does the building, +# scripts/ and docs/charts/ do the rest. `make help` lists the targets. +.DEFAULT_GOAL := help +BUILD ?= build +DOOS := $(BUILD)/doos + +.PHONY: help build test check-docs bench charts docs release clean + +help: ## list the targets + @grep -hE '^[a-z-]+:.*##' $(MAKEFILE_LIST) | sed -E 's/:.*## /\t/' | column -t -s "$$(printf '\t')" + +build: ## configure and compile into build/ + cmake -S . -B $(BUILD) >/dev/null + cmake --build $(BUILD) + +test: build ## end-to-end suite (real downloads, needs network, pasta, gpg, cgroup v2) + tests/run.sh $(DOOS) + +check-docs: ## every document links to every other, every relative link resolves + scripts/check-docs.sh + +bench: build ## re-measure startup, binary sizes and cpu limit into docs/charts/bench.json + scripts/bench.sh + +charts: ## rebuild docs/charts/*.svg and the generated numbers in COMPARISON.md from bench.json + python3 docs/charts/make_charts.py + +docs: charts check-docs ## charts plus the docs check + +release: ## KIND=patch|minor|major MSG="subject": bump, build, test, commit, tag + @test -n "$(KIND)" -a -n "$(MSG)" || { echo 'usage: make release KIND=patch MSG="subject"'; exit 2; } + scripts/release.sh $(KIND) "$(MSG)" + +clean: ## remove build/ + rm -rf $(BUILD) diff --git a/README.md b/README.md index b542503..d41fd47 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ doos image list | add [--sha256 HEX] SOURCE [NAME] | delete [--yes] NAME doos apparmor # profile text for this binary (Ubuntu 24.04+) ``` -Version: see `VERSION` (currently 1.1.2). Release history: `CHANGELOG.md`. +Version: see `VERSION` (currently 1.1.3). Release history: `CHANGELOG.md`. User-level containers in C++20: overlayfs over the host's `/usr`, `/etc`, `/opt`, user/mount/pid namespaces, networking through slirp4netns. No root at runtime. @@ -168,6 +168,10 @@ run with Docker's default capability set and a seccomp filter. Memory, pids, cpu `--net host` removes network isolation and is opt-in (`--net pasta` is the safer host-like choice). What is still not covered is listed at the end of `docs/RESEARCH.md`. +## Development + +`make help` lists the targets: `make build`, `make test` (end-to-end, needs network), `make bench` (re-measure startup, sizes and the cpu limit), `make charts` (rebuild the SVG charts and the generated numbers in the comparison), `make docs`, `make release KIND=patch MSG="..."`. Build and test details are in [AGENTS](AGENTS.md); the charts and how they are generated are in [COMPARISON](COMPARISON.md) section 3 and `docs/charts/`. + ## Project docs | File | Content | diff --git a/VERSION b/VERSION index 45a1b3f..781dcb0 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.1.2 +1.1.3 diff --git a/docs/RESEARCH.md b/docs/RESEARCH.md index a7c625e..dfc67a0 100644 --- a/docs/RESEARCH.md +++ b/docs/RESEARCH.md @@ -260,6 +260,10 @@ delegated with `cpuset cpu io memory pids`). The end-to-end suite passed 83 chec | Size accounting | 80 MB written by container uid 5 in a mode-700 directory (invisible to the host user) against `--size 64m`: frozen; cleanup from a second attach thawed it | | Attach time (5 runs) | host or image, slirp: about 40 ms each; pasta: about 65 ms; `--net none`: about 10 ms | +![Horizontal bar chart of CPU-seconds used by four busy loops in one container, without a limit and with --cpus 0.5. The limited container is highlighted. Values are in the sentence below.](charts/cpu.svg) + +The chart above is generated from `docs/charts/bench.json` (`make bench`, then `make charts`); it shows 16.3 CPU-seconds without a limit and 2.3 with `--cpus 0.5`; the startup and size charts live in [COMPARISON](../COMPARISON.md). + ### What changed in behavior (and why it is not a loss) - Hardening can be switched off per container (`--hardening relaxed`), so nothing that needs the denied calls is lost, it is opt-in now. diff --git a/docs/charts/bench.json b/docs/charts/bench.json new file mode 100644 index 0000000..704c67a --- /dev/null +++ b/docs/charts/bench.json @@ -0,0 +1,121 @@ +{ + "meta": { + "date": "2026-10-23", + "kernel": "7.0.0-38-generic", + "cpus": 12, + "runs": 10, + "doos": "1.1.2", + "podman": "podman version 5.7.0", + "docker": "Docker version 29.1.3, build 29.1.3-0ubuntu4.1", + "bubblewrap": "bubblewrap 0.11.1", + "src_lines": 3445, + "shared_libs": 6 + }, + "startup": [ + { + "id": "bwrap_none", + "label": "bubblewrap (host /usr)", + "group": "none", + "kind": "other", + "ms": 8.4 + }, + { + "id": "bwrap_net", + "label": "bubblewrap (host /usr)", + "group": "default", + "kind": "other", + "ms": 7.4 + }, + { + "id": "doos_host_none", + "label": "doos host mode", + "group": "none", + "kind": "doos", + "ms": 18.7 + }, + { + "id": "doos_alpine_none", + "label": "doos Alpine image", + "group": "none", + "kind": "doos", + "ms": 13.8 + }, + { + "id": "doos_host_slirp", + "label": "doos host mode", + "group": "default", + "kind": "doos", + "ms": 56.9 + }, + { + "id": "doos_alpine_slirp", + "label": "doos Alpine image", + "group": "default", + "kind": "doos", + "ms": 52.1 + }, + { + "id": "doos_alpine_pasta", + "label": "doos Alpine, pasta", + "group": "default", + "kind": "doos", + "ms": 64.6 + }, + { + "id": "podman_none", + "label": "Podman Alpine", + "group": "none", + "kind": "other", + "ms": 207.8 + }, + { + "id": "podman_net", + "label": "Podman Alpine", + "group": "default", + "kind": "other", + "ms": 241.2 + }, + { + "id": "docker_none", + "label": "Docker Alpine (rootful)", + "group": "none", + "kind": "other", + "ms": 343.9 + }, + { + "id": "docker_net", + "label": "Docker Alpine (rootful)", + "group": "default", + "kind": "other", + "ms": 435.6 + } + ], + "sizes": [ + { + "id": "bwrap", + "bytes": 80424 + }, + { + "id": "crun", + "bytes": 483392 + }, + { + "id": "doos", + "bytes": 565680 + }, + { + "id": "docker", + "bytes": 31374336 + }, + { + "id": "podman", + "bytes": 43180896 + } + ], + "cpu": { + "wall_seconds": 4.2, + "busy_loops": 4, + "unlimited": 16.26, + "half_cpu": 2.29 + } +} diff --git a/docs/charts/cpu.svg b/docs/charts/cpu.svg new file mode 100644 index 0000000..75d7ff5 --- /dev/null +++ b/docs/charts/cpu.svg @@ -0,0 +1,48 @@ + +CPU time used by four busy loops +Two bars, CPU seconds consumed in 4.2 seconds of wall time. Without a limit 16.3, with --cpus 0.5 2.3. + + +CPU time used by 4 busy loops in one container (CPU-seconds) +4.2 s of wall time. Half a CPU allows about 2.1 CPU-seconds; measured 2.3. + +no limit + +--cpus 0.5 + +0 + +4 + +8 + +12 + +16 + +20 +no limit +no limit: 16.3 CPU-seconds +16.3 CPU-s +--cpus 0.5 +--cpus 0.5: 2.3 CPU-seconds +2.3 CPU-s + + diff --git a/docs/charts/make_charts.py b/docs/charts/make_charts.py new file mode 100644 index 0000000..f8b0cf8 --- /dev/null +++ b/docs/charts/make_charts.py @@ -0,0 +1,292 @@ +#!/usr/bin/env python3 +"""Builds the SVG charts and the generated numbers in COMPARISON.md / RESEARCH.md from bench.json. + + python3 docs/charts/make_charts.py (or: make charts) + +Input : docs/charts/bench.json (written by scripts/bench.sh, i.e. make bench) +Output: docs/charts/startup.svg, size.svg, cpu.svg + the blocks between and in the markdown documents + and inline values written as text + +Design rules (the dataviz method): one form per job, doos in the accent hue and everything else in the neutral +gray (emphasis), bars at most 24 px with a 4 px rounded data end, direct value labels at the bar tip, a legend, +hairline solid grid, text in ink tokens (never the series color), light and dark palettes selected separately. +Colors were checked with the dataviz validator: contrast, CVD separation and the normal-vision floor pass in +both modes; its "chroma floor" flag on the gray is expected, the gray is the de-emphasis neutral by design. +Standard library only. +""" +import json +import math +import re +import sys +from pathlib import Path +from xml.sax.saxutils import escape + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parent.parent +DATA = HERE / "bench.json" + +# ---- palette (dataviz reference instance) --------------------------------------------------------------- +LIGHT = dict(surface="#fcfcfb", ink="#0b0b0b", ink2="#52514e", muted="#898781", grid="#e1e0d9", axis="#c3c2b7", + accent="#2a78d6", other="#898781") +DARK = dict(surface="#1a1a19", ink="#f0efec", ink2="#c3c2b7", muted="#898781", grid="#2c2c2a", axis="#383835", + accent="#3987e5", other="#898781") + +STYLE = f""" + .bg {{ fill: var(--surface); }} + .t1 {{ fill: var(--ink); font: 600 15px system-ui, -apple-system, "Segoe UI", sans-serif; }} + .t2 {{ fill: var(--ink2); font: 400 12px system-ui, -apple-system, "Segoe UI", sans-serif; }} + .lab {{ fill: var(--ink); font: 400 13px system-ui, -apple-system, "Segoe UI", sans-serif; }} + .val {{ fill: var(--ink2); font: 400 12px system-ui, -apple-system, "Segoe UI", sans-serif; }} + .tick {{ fill: var(--muted); font: 400 11px system-ui, -apple-system, "Segoe UI", sans-serif; }} + .grid {{ stroke: var(--grid); stroke-width: 1; }} + .axis {{ stroke: var(--axis); stroke-width: 1; }} + .acc {{ fill: var(--accent); }} + .oth {{ fill: var(--other); }} + :root {{ --surface:{LIGHT['surface']}; --ink:{LIGHT['ink']}; --ink2:{LIGHT['ink2']}; --muted:{LIGHT['muted']}; + --grid:{LIGHT['grid']}; --axis:{LIGHT['axis']}; --accent:{LIGHT['accent']}; --other:{LIGHT['other']}; }} + @media (prefers-color-scheme: dark) {{ + :root {{ --surface:{DARK['surface']}; --ink:{DARK['ink']}; --ink2:{DARK['ink2']}; --muted:{DARK['muted']}; + --grid:{DARK['grid']}; --axis:{DARK['axis']}; --accent:{DARK['accent']}; --other:{DARK['other']}; }} + }} +""" + +BAR_H = 18 # <= 24 px +ROW = 30 +RADIUS = 4 # rounded data end only + + +def bar_path(x, y, w, h, r=RADIUS): + """Bar growing from x to the right: square at the baseline, rounded at the data end.""" + w = max(w, 1.0) + r = min(r, w, h / 2) + return (f"M{x:.1f},{y:.1f} h{w - r:.1f} a{r},{r} 0 0 1 {r},{r} v{h - 2 * r:.1f} " + f"a{r},{r} 0 0 1 {-r},{r} h{-(w - r):.1f} z") + + +def svg_open(w, h, title, desc): + return (f'\n{escape(title)}\n' + f'{escape(desc)}\n\n' + f'\n') + + +def legend(x, y, items): + out, cx = [], x + for cls, text in items: + out.append(f'') + out.append(f'{escape(text)}') + cx += 18 + 7 * len(text) + 22 + return "\n".join(out) + + +def fmt_ms(v): + return f"{v:.0f} ms" if v >= 100 else f"{v:.1f} ms".replace(".0 ms", " ms") + + +def fmt_bytes(b): + if b < 1 << 20: + return f"{b / 1024:.0f} KiB" + return f"{b / (1 << 20):.1f} MiB" + + +# ---- chart 1: startup time ----------------------------------------------------------------------------- +def startup_chart(d): + rows = [r for r in d["startup"] if r["ms"] is not None] + iv = inline_values(d) + groups = [("none", "No network"), ("default", "Default network")] + left, right, top = 200, 90, 78 + width = 760 + plot_w = width - left - right + vmax = max(r["ms"] for r in rows) + axis_max = math.ceil(vmax / 100.0) * 100 + step = 100 if axis_max <= 600 else 200 + scale = plot_w / axis_max + + y = top + body = [] + for key, name in groups: + grp = sorted([r for r in rows if r["group"] == key], key=lambda r: r["ms"]) + if not grp: + continue + body.append(f'{name}') + y += 14 + y0 = y + for t in range(0, axis_max + 1, step): + x = left + t * scale + body.append(f'') + for r in grp: + cls = "acc" if r["kind"] == "doos" else "oth" + w = r["ms"] * scale + by = y + (ROW - BAR_H) / 2 + body.append(f'{escape(r["label"])}') + body.append(f'{escape(r["label"])}: {fmt_ms(r["ms"])}') + body.append(f'{fmt_ms(r["ms"])}') + y += ROW + body.append(f'') + for t in range(0, axis_max + 1, step): + body.append(f'{t}') + y += 38 + height = int(y - 10) + m = d["meta"] + out = svg_open(width, height, "Time to start a trivial command", + f"Horizontal bars, milliseconds, lower is better. doos: {iv['doos_range']}; Podman: {iv['podman_range']}; " + f"Docker: {iv['docker_range']}; bubblewrap: {iv['bwrap_range']}. Exact values are in the table in COMPARISON.md.") + out += 'Time to start a trivial command (ms, lower is better)\n' + out += (f'{m["runs"]} runs each, average, warm cache, {m["date"]}, kernel {escape(m["kernel"])}, ' + f'{m["cpus"]} cores. Docker is a rootful daemon.\n') + out += legend(20, 64, [("acc", "doos"), ("oth", "other tools")]) + "\n" + out += "\n".join(body) + "\n\n" + return out + + +# ---- chart 2: binary size (log scale) -------------------------------------------------------------------- +def size_chart(d): + names = {"bwrap": "bubblewrap", "crun": "crun (OCI runtime)", "doos": "doos", "docker": "docker CLI", + "podman": "podman"} + rows = sorted([r for r in d["sizes"] if r["bytes"] is not None], key=lambda r: r["bytes"]) + left, right, top, width = 170, 100, 96, 760 + plot_w = width - left - right + lo, hi = math.log10(50 * 1024), math.log10(100 * (1 << 20)) + sx = lambda b: (math.log10(b) - lo) / (hi - lo) * plot_w + ticks = [(100 * 1024, "100 KiB"), (1 << 20, "1 MiB"), (10 << 20, "10 MiB"), (100 << 20, "100 MiB")] + y0 = top + 8 + body = [] + for b, lab in ticks: + x = left + sx(b) + body.append(f'') + body.append(f'{lab}') + y = y0 + for r in rows: + cls = "acc" if r["id"] == "doos" else "oth" + w = sx(r["bytes"]) + by = y + (ROW - BAR_H) / 2 + body.append(f'{names.get(r["id"], r["id"])}') + body.append(f'{names.get(r["id"], r["id"])}: {fmt_bytes(r["bytes"])}') + body.append(f'{fmt_bytes(r["bytes"])}') + y += ROW + body.append(f'') + height = int(y + 62) + out = svg_open(width, height, "Size of the main executable", + "Horizontal bars on a logarithmic axis: " + + ", ".join(f"{names.get(r['id'], r['id'])} {fmt_bytes(r['bytes'])}" for r in rows) + ".") + out += 'Size of the main executable (log scale, smaller is smaller)\n' + out += 'The docker CLI excludes dockerd and containerd.\n' + out += 'doos excludes the helpers it calls (slirp4netns, pasta, newuidmap, curl, tar).\n' + out += legend(20, 82, [("acc", "doos"), ("oth", "other tools")]) + "\n" + out += "\n".join(body) + f'\nAxis is logarithmic: each gridline is 10x the previous one.\n\n' + return out + + +# ---- chart 3: cpu limit ---------------------------------------------------------------------------------- +def cpu_chart(d): + c = d["cpu"] + if c["unlimited"] is None or c["half_cpu"] is None: + return None + rows = [("no limit", c["unlimited"], "oth"), ("--cpus 0.5", c["half_cpu"], "acc")] + left, right, top, width = 170, 90, 78, 760 + plot_w = width - left - right + axis_max = math.ceil(c["unlimited"] / 4.0) * 4 + scale = plot_w / axis_max + y0 = top + 8 + body = [] + for t in range(0, axis_max + 1, 4): + x = left + t * scale + body.append(f'') + body.append(f'{t}') + y = y0 + for lab, v, cls in rows: + w = v * scale + by = y + (ROW - BAR_H) / 2 + body.append(f'{lab}') + body.append(f'{lab}: {v:.1f} CPU-seconds') + body.append(f'{v:.1f} CPU-s') + y += ROW + body.append(f'') + ideal = 0.5 * c["wall_seconds"] + height = int(y + 54) + out = svg_open(width, height, "CPU time used by four busy loops", + f"Two bars, CPU seconds consumed in {c['wall_seconds']} seconds of wall time. Without a limit {c['unlimited']:.1f}, " + f"with --cpus 0.5 {c['half_cpu']:.1f}.") + out += 'CPU time used by 4 busy loops in one container (CPU-seconds)\n' + out += (f'{c["wall_seconds"]} s of wall time. Half a CPU allows about {ideal:.1f} CPU-seconds; ' + f'measured {c["half_cpu"]:.1f}.\n') + out += legend(20, 64, [("oth", "no limit"), ("acc", "--cpus 0.5")]) + "\n" + out += "\n".join(body) + "\n\n" + return out + + +# ---- generated markdown --------------------------------------------------------------------------------- +def startup_table(d): + rows = [r for r in d["startup"] if r["ms"] is not None] + lines = ["| Setup | Network | Time per run |", "|---|---|---|"] + order = {"none": 0, "default": 1} + for r in sorted(rows, key=lambda r: (order[r["group"]], r["ms"])): + net = "none" if r["group"] == "none" else "default" + lines.append(f"| {r['label']} | {net} | {fmt_ms(r['ms'])} |") + return "\n".join(lines) + + +def size_table(d): + names = {"bwrap": "bubblewrap", "crun": "crun", "doos": "doos", "docker": "docker CLI", "podman": "podman"} + lines = ["| Executable | Size |", "|---|---|"] + for r in sorted([r for r in d["sizes"] if r["bytes"] is not None], key=lambda r: r["bytes"]): + lines.append(f"| {names.get(r['id'], r['id'])} | {fmt_bytes(r['bytes'])} ({r['bytes']:,} bytes) |") + return "\n".join(lines) + + +def inline_values(d): + ms = lambda ids: [r["ms"] for r in d["startup"] if r["id"].startswith(ids) and r["ms"] is not None] + rng = lambda xs: f"{min(xs):.0f} to {max(xs):.0f} ms" if xs else "n/a" + size = {r["id"]: r["bytes"] for r in d["sizes"] if r["bytes"] is not None} + v = { + "doos_range": rng(ms("doos")), + "podman_range": rng(ms("podman")), + "docker_range": rng(ms("docker")), + "bwrap_range": rng(ms("bwrap")), + "doos_size": fmt_bytes(size["doos"]) if "doos" in size else "n/a", + "docker_size": fmt_bytes(size["docker"]) if "docker" in size else "n/a", + "podman_size": fmt_bytes(size["podman"]) if "podman" in size else "n/a", + "bench_date": d["meta"]["date"], + "src_lines": f'{d["meta"].get("src_lines", 0):,}', + "shared_libs": str(d["meta"].get("shared_libs", "n/a")), + "runs": str(d["meta"]["runs"]), + "versions": f'doos {d["meta"]["doos"]}, {d["meta"]["docker"].replace("Docker version ", "Docker ").split(",")[0]}, ' + f'{d["meta"]["podman"].replace("podman version ", "Podman ")}, {d["meta"]["bubblewrap"].replace("bubblewrap ", "bubblewrap ")}', + "cpu_free": f'{d["cpu"]["unlimited"]:.1f}' if d["cpu"]["unlimited"] is not None else "n/a", + "cpu_half": f'{d["cpu"]["half_cpu"]:.1f}' if d["cpu"]["half_cpu"] is not None else "n/a", + } + return v + + +def patch_markdown(path, blocks, inline): + s = path.read_text() + orig = s + for name, text in blocks.items(): + pat = re.compile(rf"(\n).*?(\n)", re.S) + s = pat.sub(lambda m: m.group(1) + text + m.group(2), s) + s = re.sub(r".*?", + lambda m: f"{inline.get(m.group(1), 'n/a')}", s) + if s != orig: + path.write_text(s) + return True + return False + + +def main(): + if not DATA.exists(): + sys.exit("no bench.json: run `make bench` first (it needs the built binary)") + d = json.loads(DATA.read_text()) + (HERE / "startup.svg").write_text(startup_chart(d)) + (HERE / "size.svg").write_text(size_chart(d)) + cpu = cpu_chart(d) + if cpu: + (HERE / "cpu.svg").write_text(cpu) + blocks = {"startup": startup_table(d), "sizes": size_table(d)} + inline = inline_values(d) + changed = [p.name for p in (ROOT / "COMPARISON.md", ROOT / "docs" / "RESEARCH.md") if patch_markdown(p, blocks, inline)] + print("charts: startup.svg size.svg" + (" cpu.svg" if cpu else "") + (f"; updated {', '.join(changed)}" if changed else "; docs already current")) + + +if __name__ == "__main__": + main() diff --git a/docs/charts/size.svg b/docs/charts/size.svg new file mode 100644 index 0000000..8085341 --- /dev/null +++ b/docs/charts/size.svg @@ -0,0 +1,55 @@ + +Size of the main executable +Horizontal bars on a logarithmic axis: bubblewrap 79 KiB, crun (OCI runtime) 472 KiB, doos 552 KiB, docker CLI 29.9 MiB, podman 41.2 MiB. + + +Size of the main executable (log scale, smaller is smaller) +The docker CLI excludes dockerd and containerd. +doos excludes the helpers it calls (slirp4netns, pasta, newuidmap, curl, tar). + +doos + +other tools + +100 KiB + +1 MiB + +10 MiB + +100 MiB +bubblewrap +bubblewrap: 79 KiB +79 KiB +crun (OCI runtime) +crun (OCI runtime): 472 KiB +472 KiB +doos +doos: 552 KiB +552 KiB +docker CLI +docker CLI: 29.9 MiB +29.9 MiB +podman +podman: 41.2 MiB +41.2 MiB + +Axis is logarithmic: each gridline is 10x the previous one. + diff --git a/docs/charts/startup.svg b/docs/charts/startup.svg new file mode 100644 index 0000000..f07e0a3 --- /dev/null +++ b/docs/charts/startup.svg @@ -0,0 +1,90 @@ + +Time to start a trivial command +Horizontal bars, milliseconds, lower is better. doos: 14 to 65 ms; Podman: 208 to 241 ms; Docker: 344 to 436 ms; bubblewrap: 7 to 8 ms. Exact values are in the table in COMPARISON.md. + + +Time to start a trivial command (ms, lower is better) +10 runs each, average, warm cache, 2026-10-23, kernel 7.0.0-38-generic, 12 cores. Docker is a rootful daemon. + +doos + +other tools +No network + + + + + + +bubblewrap (host /usr) +bubblewrap (host /usr): 8.4 ms +8.4 ms +doos Alpine image +doos Alpine image: 13.8 ms +13.8 ms +doos host mode +doos host mode: 18.7 ms +18.7 ms +Podman Alpine +Podman Alpine: 208 ms +208 ms +Docker Alpine (rootful) +Docker Alpine (rootful): 344 ms +344 ms + +0 +100 +200 +300 +400 +500 +Default network + + + + + + +bubblewrap (host /usr) +bubblewrap (host /usr): 7.4 ms +7.4 ms +doos Alpine image +doos Alpine image: 52.1 ms +52.1 ms +doos host mode +doos host mode: 56.9 ms +56.9 ms +doos Alpine, pasta +doos Alpine, pasta: 64.6 ms +64.6 ms +Podman Alpine +Podman Alpine: 241 ms +241 ms +Docker Alpine (rootful) +Docker Alpine (rootful): 436 ms +436 ms + +0 +100 +200 +300 +400 +500 + diff --git a/scripts/bench.sh b/scripts/bench.sh new file mode 100755 index 0000000..885108e --- /dev/null +++ b/scripts/bench.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env bash +# Reproducible measurements behind COMPARISON.md and the charts. Writes JSON (default docs/charts/bench.json). +# Tools that are not installed or not usable are skipped and recorded as null, never guessed. +# Usage: scripts/bench.sh [output.json] Environment: BENCH_RUNS (default 10) +set -u +cd "$(dirname "$0")/.." +OUT=${1:-docs/charts/bench.json} +RUNS=${BENCH_RUNS:-10} +DOOS=$(readlink -f build/doos) +[ -x "$DOOS" ] || { echo "bench: build first (make build)" >&2; exit 1; } +export DOOS_HOME=$(mktemp -d) +trap '"$DOOS" delete --yes ha >/dev/null 2>&1; rm -rf "$DOOS_HOME"' EXIT + +have() { command -v "$1" >/dev/null 2>&1; } +now() { date +%s%N; } +# avg_ms LABEL CMD... -> prints average ms per run, or "null" when the command fails +avg_ms() { + timeout 180 "$@" >/dev/null 2>&1 || { echo null; return; } # warm-up, may pull an image + local s e i; s=$(now) + for i in $(seq "$RUNS"); do timeout 60 "$@" >/dev/null 2>&1 || { echo null; return; }; done + e=$(now); awk -v s="$s" -v e="$e" -v n="$RUNS" 'BEGIN{printf "%.1f", (e-s)/1e6/n}' +} +note() { echo "bench: $*" >&2; } + +# ---- setup doos containers (host mode needs nothing, image mode downloads Alpine once) +"$DOOS" image add alpine >/dev/null 2>&1 || note "alpine image unavailable, image-mode rows will be null" +"$DOOS" get_or_create --net none hn >/dev/null 2>&1 +"$DOOS" get_or_create hs >/dev/null 2>&1 +"$DOOS" get_or_create --image alpine --net none an >/dev/null 2>&1 +"$DOOS" get_or_create --image alpine as >/dev/null 2>&1 +"$DOOS" get_or_create --image alpine --net pasta ap >/dev/null 2>&1 + +BW_BASE="--unshare-user --unshare-pid --unshare-ipc --unshare-uts --die-with-parent --ro-bind /usr /usr --symlink usr/bin /bin --symlink usr/lib /lib --symlink usr/lib64 /lib64 --proc /proc --dev /dev" +rows=() +add() { rows+=("{\"id\":\"$1\",\"label\":\"$2\",\"group\":\"$3\",\"kind\":\"$4\",\"ms\":$5}"); } + +echo "bench: startup, $RUNS runs each" >&2 +if have bwrap; then + add bwrap_none "bubblewrap (host /usr)" none other "$(avg_ms bwrap --unshare-all $BW_BASE true)" + add bwrap_net "bubblewrap (host /usr)" default other "$(avg_ms bwrap $BW_BASE true)" +fi +add doos_host_none "doos host mode" none doos "$(avg_ms "$DOOS" attach hn -- true)" +add doos_alpine_none "doos Alpine image" none doos "$(avg_ms "$DOOS" attach an -- true)" +add doos_host_slirp "doos host mode" default doos "$(avg_ms "$DOOS" attach hs -- true)" +add doos_alpine_slirp "doos Alpine image" default doos "$(avg_ms "$DOOS" attach as -- true)" +add doos_alpine_pasta "doos Alpine, pasta" default doos "$(avg_ms "$DOOS" attach ap -- true)" +if have podman; then + add podman_none "Podman Alpine" none other "$(avg_ms podman run --rm --network none alpine true)" + add podman_net "Podman Alpine" default other "$(avg_ms podman run --rm alpine true)" +fi +if have docker; then + add docker_none "Docker Alpine (rootful)" none other "$(avg_ms docker run --rm --network none alpine true)" + add docker_net "Docker Alpine (rootful)" default other "$(avg_ms docker run --rm alpine true)" +fi + +# ---- binary sizes +size_of() { local p; p=$(command -v "$1" 2>/dev/null) && stat -L -c %s "$p" || echo null; } +sizes=() +for t in bwrap crun doos docker podman; do + if [ "$t" = doos ]; then b=$(stat -c %s "$DOOS"); else b=$(size_of "$t"); fi + sizes+=("{\"id\":\"$t\",\"bytes\":$b}") +done + +# ---- cpu limit: 4 busy loops for 4 seconds, CPU seconds consumed (user+sys) +cpu_run() { + /usr/bin/time -f "%U %S" "$DOOS" attach "$1" -- sh -c 'for i in 1 2 3 4; do (timeout 4 sh -c "while :; do :; done" &); done; sleep 4.2' 2>&1 >/dev/null | tail -1 | awk '{printf "%.2f", $1+$2}' +} +echo "bench: cpu limit" >&2 +"$DOOS" get_or_create hc >/dev/null 2>&1; "$DOOS" get_or_create --cpus 0.5 hc5 >/dev/null 2>&1 +cpu_free=$(cpu_run hc); cpu_half=$(cpu_run hc5) +"$DOOS" delete --yes hc >/dev/null 2>&1; "$DOOS" delete --yes hc5 >/dev/null 2>&1 + +ver() { "$@" 2>/dev/null | head -1 | tr -d '"\\' ; } +json_join() { local IFS=,; echo "$*"; } +cat > "$OUT" <&2 diff --git a/scripts/release.sh b/scripts/release.sh index 366cc26..414fc17 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -22,6 +22,7 @@ if grep -rl $'\xe2\x80\x94' --exclude-dir=build --exclude-dir=.git . | grep -q . echo "em dash found, remove it first" >&2; exit 1 fi +python3 docs/charts/make_charts.py scripts/check-docs.sh echo "$new" > VERSION cmake -S . -B build >/dev/null