Add COMPARISON.md, cross-link all documents, enforce with check-docs

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
AppArmor profile per binary path with install hint; verified network examples
2026-10-23 22:18:42 +02:00 · 2026-10-23 22:08:37 +02:00
16 changed files with 389 additions and 19 deletions
@@ -1,5 +1,7 @@
 # AGENTS.md
 **Docs:** [README](README.md) · [SPEC](SPEC.md) · [ARCHITECTURE](docs/ARCHITECTURE.md) · [RESEARCH](docs/RESEARCH.md) · [COMPARISON](COMPARISON.md) · [AGENTS](AGENTS.md) · [CHANGELOG](CHANGELOG.md)
 Rules for any AI agent (or human) changing this repository. The user's global rules
 also apply; this file adds the project-specific ones. Keep this file current: it is
 part of every release.
@@ -11,8 +13,9 @@ part of every release.
 extracted image (image mode) with a per-container upper dir, pivots into it and runs a command.
 Networking is slirp4netns or pasta, ids are mapped through newuidmap when possible, cgroup v2 limits
 and a watchdog enforce `--memory --pids --cpus --size`, capabilities and seccomp confine the command.
-No root at runtime. Authoritative behavior: `SPEC.md`. Design notes: `docs/ARCHITECTURE.md`.
+No root at runtime. Authoritative behavior: [SPEC](SPEC.md). Design notes: [ARCHITECTURE](docs/ARCHITECTURE.md).
-Measurements and gaps: `docs/RESEARCH.md`.
+Measurements and gaps: [RESEARCH](docs/RESEARCH.md). Positioning against other tools: [COMPARISON](COMPARISON.md).
 Usage: [README](README.md). History: [CHANGELOG](CHANGELOG.md).
 ## Mandatory release workflow (every change, same session)
@@ -22,11 +25,12 @@ Measurements and gaps: `docs/RESEARCH.md`.
   - `patch`: bug fix, docs, tests, tooling, no behavior change for users.
   - `minor`: new or changed user-visible behavior (flag, command, mount, limit).
   - `major`: breaking change to CLI, state layout, or container compatibility.
-4. Run `scripts/release.sh <patch|minor|major> "imperative commit subject"`.
+4. Run `scripts/check-docs.sh` (the release script does too: every document must link to every other, every relative link must resolve).
 5. Run `scripts/release.sh <patch|minor|major> "imperative commit subject"`.
   It bumps `VERSION`, builds, runs `tests/run.sh`, commits all changes, and creates
   the annotated tag `vX.Y.Z`. It aborts on any build or test failure.
-5. Verify live: run `build/doos` once against the real thing (the e2e suite does this).
+6. Verify live: run `build/doos` once against the real thing (the e2e suite does this).
-6. Report honestly what passed and what did not. Do not push unless the user asks.
+7. Report honestly what passed and what did not. Do not push unless the user asks.
 Never hand-edit `VERSION` or create tags manually. Never skip the tests to release.
@@ -42,6 +46,8 @@ Never hand-edit `VERSION` or create tags manually. Never skip the tests to relea
 | A new measurement | `docs/RESEARCH.md` with machine, kernel, and what was run |
 | A new known gap or a closed gap | `docs/RESEARCH.md` gaps list and `README.md` |
 | These workflow rules | this file |
 | A feature, limit or measurement that appears in the comparison tables | `COMPARISON.md` (tables, the gaps list, the conclusion) |
 | A new or renamed markdown file | `scripts/check-docs.sh` DOCS list, the nav line (`**Docs:**`) of every document, `README.md` Project docs |
 | Image catalog, sources, verification, pinned keys | `SPEC.md` Images, `README.md`, `docs/RESEARCH.md` |
 | Seccomp list, capability set, `/etc` allowlist | `SPEC.md` Hardening and filesystem, `README.md`, `docs/RESEARCH.md` |
 | Limits, cgroup layout | `SPEC.md` Resource limits, `docs/ARCHITECTURE.md` |
@@ -73,7 +79,7 @@ the script.
 slirp4netns, filesystem, real downloads of the Alpine and Ubuntu images, real apt/apk installs, the size
 freeze (about 4 minutes in total, needs network, `pasta`, `gpg` and cgroup v2 delegation). Add a case for every behavior change. A green run is required
 before release. On Ubuntu 24.04+ the AppArmor profile in `apparmor/doos` must be
-installed once (needs sudo, ask the user, do not do it yourself).
+installed once for the binary being tested (`doos apparmor`; needs sudo, ask the user, do not do it yourself).
 ## Environment notes
@@ -1,5 +1,17 @@
 # Changelog
 **Docs:** [README](README.md) · [SPEC](SPEC.md) · [ARCHITECTURE](docs/ARCHITECTURE.md) · [RESEARCH](docs/RESEARCH.md) · [COMPARISON](COMPARISON.md) · [AGENTS](AGENTS.md) · [CHANGELOG](CHANGELOG.md)
 ## 1.1.2 - 2026-10-23
 - Docs: new [COMPARISON](COMPARISON.md) (doos against Docker, rootless Docker, Podman, bubblewrap, systemd-nspawn, LXC/Incus, Apptainer, Distrobox/Toolbx, gVisor, Kata/Firecracker) with sourced pros and cons, startup and disk measurements against Docker, Podman and bubblewrap on the same machine, a gaps list tied to code, and a conclusion.
 - Docs: every document now has a navigation line and links to every other document. `scripts/check-docs.sh` enforces it (and that relative links resolve) and runs inside `scripts/release.sh`.
 ## 1.1.1 - 2026-10-23
 - `doos apparmor` prints a profile for the exact path of the running binary (own profile name per path, so several installs coexist). A start that fails because of AppArmor now prints the install command instead of only `mount bind /: Permission denied`. Verified with a copy in `~/.local/bin`.
 - `apparmor/doos` (packaging) covers more standard install paths.
 - Docs: Networking section with a verified example per mode (slirp, pasta, host, none), the inbound/loopback behavior of each, and the pasta port-detection delay.
 - Tests: 86 checks (adds apparmor output, `none`, slirp has no inbound path, pasta container to container).
 ## 1.1.0 - 2026-10-23
 - Limits: `--memory`, `--pids`, `--cpus` (cgroup v2; whole-container OOM kill; reports OOM kills and pid-limit hits). Later attaches run in their own cgroup leaf with the same limits.
 - Hardening (default): Docker's default capability set, a seccomp filter (arch check, x32 guard, clone3 ENOSYS, no nested user namespaces, no mounts/keyrings/bpf/perf/io_uring/...), host `/etc` allowlist. `--hardening relaxed` turns it off per container.
@@ -0,0 +1,175 @@
 # doos compared with Docker, Podman and others
 **Docs:** [README](README.md) · [SPEC](SPEC.md) · [ARCHITECTURE](docs/ARCHITECTURE.md) · [RESEARCH](docs/RESEARCH.md) · [COMPARISON](COMPARISON.md) · [AGENTS](AGENTS.md) · [CHANGELOG](CHANGELOG.md)
 This page puts doos next to the tools people usually reach for instead. It separates three kinds of claims, and every row says which one it is:
 - **[M] measured** on the development machine, described in "Measurements" below.
 - **[S] sourced** from the linked documentation or articles. Vendor blogs are marked as such and their numbers are not used.
 - **[D] design**: what the doos source and [SPEC](SPEC.md) do today.
 - **[K] general knowledge** that this research did not re-verify.
 Nothing here is a roadmap. Where doos lacks something, the gap is listed with the code that would have to change.
 Versions compared: doos 1.1.1, Docker 29.1.3 (rootful daemon on this machine), Podman 5.7.0 (netavark, crun, overlay), bubblewrap 0.11.1, on Ubuntu 26.04, kernel 7.0.0-38, 12 cores, one machine.
 ## 1. At a glance
 | | doos | Docker (rootful) | Docker rootless | Podman rootless |
 |---|---|---|---|---|
 | Needs a daemon | no [D] | yes, root daemon [S: [Docker security](https://docs.docker.com/engine/security/)] | yes, per-user daemon [K] | no [S: [rootlesscontaine.rs](https://rootlesscontaine.rs/getting-started/podman/)] |
 | Runs without root | yes [D] | no [S] | yes [S] | yes [S] |
 | Who can use it safely | any user | only trusted users: access to the daemon is root-equivalent [S: [Docker security](https://docs.docker.com/engine/security/), [iximiuz](https://labs.iximiuz.com/tutorials/docker-security-introduction-a859718d)] | any user | any user |
 | Image source | tarball, https URL with sha256, small catalog (Alpine, Ubuntu, Debian) [D] | OCI registries, Dockerfile builds [K] | same as Docker | OCI registries, Containerfile builds [K] |
 | Build images | no [D] | yes (BuildKit) [K] | yes | yes (Buildah) [K] |
 | Host dirs into the container (`-v`) | **no** [D] | yes [K] | yes | yes |
 | Published ports | none for `slirp`; automatic for `pasta` [D] | `-p` [K] | `-p` | `-p` [S: [podman-rootless(7)](https://man.archlinux.org/man/podman-rootless.7.en)] |
 | Rootless network backend | slirp4netns or pasta [D] | n/a | RootlessKit based [S: [Podman blog](https://blog.podman.io/2023/07/podman-performance-root-and-rootless/)] | pasta default since 5.0 [S: [podman-rootless(7)](https://man.archlinux.org/man/podman-rootless.7.en)] |
 | Resource limits | memory, pids, cpus, written data [D][M] | memory, cpu, pids, more [K] | needs cgroup v2 + systemd delegation [S: [Docker rootless tips](https://docs.docker.com/engine/security/rootless/tips/)] | needs cgroup v2 [S: [podman-rootless(7)](https://man.archlinux.org/man/podman-rootless.7.en)] |
 | Default syscall filter | denylist: 48 listed calls plus rules for `clone`, `unshare`, `clone3` [D] | allowlist profile [S: [Docker seccomp](https://docs.docker.com/engine/security/seccomp/)] | same | built-in profile [K] |
 | Default capabilities | Docker's default set [D][M] | that set [S] | that set | a smaller set [S: [dev.to comparison](https://dev.to/mechcloud_academy/docker-vs-podman-an-in-depth-comparison-2025-2eia)] |
 | Orchestration (Compose, Kubernetes) | none [D] | Compose, Swarm, Kubernetes via runtimes [K] | same | Compose, pods, `kube play`, systemd units [K] |
 | Background containers, logs, `stop` | none: a container lives while its first `attach` command lives [D] | yes [K] | yes | yes |
 | Binary size | 552 KiB, 6 shared libraries, 3,445 lines of C++ [M] | CLI 29.9 MiB, plus `dockerd` and `containerd` [M][K] | same | 41 MiB [M] |
 | Platforms | Linux only, tested on x86_64 only [D] | Linux, macOS, Windows (via a VM) [K] | Linux | Linux; macOS and Windows via a VM [K] |
 | Maturity | version 1.1, one test machine | very mature, huge ecosystem | mature | mature |
 ## 2. Other tools, one line each
 | Tool | What it is | Versus doos |
 |---|---|---|
 | bubblewrap (`bwrap`) | Low-level unprivileged sandbox builder, the engine behind Flatpak [S: [bubblewrap README](https://cdn.jsdelivr.net/gh/containers/bubblewrap/README.md), [Julia Evans](https://jvns.ca/blog/2022/06/28/some-notes-on-bubblewrap/)] | Smaller and faster to start [M], but you assemble every mount by hand and it has no image handling, no persistent state and no cgroup limits [K]; it maps a single id unless you add a helper [K]. doos is the "batteries included" layer on the same kernel features. |
 | systemd-nspawn | Boots a whole OS tree as a container [S: [forum overview](https://forums.puri.sm/t/linux-containers-comparison-quest-for-the-best/13344)] | Rootless use is limited to a single UID/GID inside [S]. Not installed here, so not measured. |
 | LXC / Incus (LXD fork) | System containers: a full distro with init and several services, close to a light VM [S: [OneUptime](https://oneuptime.com/blog/post/2026-03-18-choose-between-podman-and-lxc-lxd/view)] | Better for long-lived pet systems and multi-service setups, with API, storage pools and profiles. Heavier to set up. `lxc` binaries exist here, Incus does not, neither was run. |
 | Apptainer (Singularity) | Rootless, daemonless, single-file SIF images, built for HPC (MPI, InfiniBand, GPUs) [S: [Alliance Canada](https://docs.alliancecan.ca/wiki/Apptainer), [NYU HPC](https://services.rt.nyu.edu/docs/hpc/containers/intro/)] | Different audience. It also has `--fakeroot` and signed immutable images [S], which doos lacks. Not installed here. |
 | Distrobox / Toolbx | Podman- or Docker-backed "your distro inside your terminal", home directory shared on purpose, not a sandbox [S: [Arch wiki](https://wiki.archlinux.org/title/Distrobox), [comparison](https://computingforgeeks.com/distrobox-vs-toolbox/)] | Opposite goal: doos hides the host's `/home` by default. |
 | gVisor | A user-space kernel that answers most syscalls itself, cutting host-kernel exposure at an I/O cost [S: [Edera](https://edera.dev/stories/kata-vs-firecracker-vs-gvisor-isolation-compared), [Northflank](https://northflank.com/blog/kata-containers-vs-firecracker-vs-gvisor)] | Stronger isolation than doos, needs its own runtime. Vendor blogs; their numbers are not used here. |
 | Kata Containers / Firecracker | Each workload in its own KVM micro-VM with its own kernel [S: same sources, [MASCOTS 2020 paper](https://cristian.kleinlabs.eu/publications/mascots2020_container_runtime_security.pdf)] | The right tool for hostile or multi-tenant code. doos shares the host kernel, so a kernel bug is a doos escape. |
 ## 3. Measurements
 Method: trivial command (`true`) started 10 times in a row, average wall time per run, warm caches, the Alpine image already present for every tool, one machine, one run set. Scripts were ad-hoc shell loops and are not part of the repository tests, so treat the numbers as indicative, not as a benchmark.
 | Setup | Time per run |
 |---|---|
 | bubblewrap, host `/usr`, all namespaces unshared, no network | 9 ms |
 | bubblewrap, host `/usr`, network shared | 8 ms |
 | doos host mode, `--net none` | 14 ms |
 | doos image Alpine, `--net none` | 16 ms |
 | doos host mode, slirp | 54 ms |
 | doos image Alpine, slirp | 57 ms |
 | doos image Alpine, pasta | 65 ms |
 | Podman, Alpine, `--network none` | 214 ms |
 | Podman, Alpine, default network | 234 ms |
 | Docker (rootful), Alpine, `--network none` | 590 ms |
 | Docker (rootful), Alpine, default bridge | 456 ms |
 How to read this honestly:
 - doos is fast partly because it does less: no image store database, no registry code, no layer unpacking per run, no container metadata service. The extra time over bubblewrap is the id map helpers (`newuidmap`, `newgidmap`), the overlay mounts, a hardening filter and the network helper process.
 - Docker here is a rootful daemon, so its numbers include a client-to-daemon round trip. Rootless Docker was not measured.
 - The Docker row "none" is slower than "default" in this run; the spread between runs was not measured, so do not read a ranking into it.
 - Startup is one number among many. It matters for scripts that start hundreds of short-lived sandboxes. It does not matter for a service that starts once.
 Disk, measured with `du` (which cannot read overlayfs' mode-000 work directories, so true usage is slightly higher):
 | Item | Size |
 |---|---|
 | doos Alpine image, extracted once and shared | 8.3 MiB |
 | doos Ubuntu base image, extracted once and shared | 98 MiB |
 | one doos container on top of an image | a few KiB of metadata plus its own writes |
 | four containers (one Alpine, three Ubuntu) on shared images | 20 KiB together |
 | doos host-mode container after its first create (skeleton, `/etc` allowlist, dpkg state) | 4 MiB, created in 0.5 s |
 | Podman storage on this machine (many unrelated images) | 26.3 GB images, 1.8 GB containers |
 The Podman figure is this machine's accumulated state, not a like-for-like comparison. What the comparison does show is the model: doos extracts an image once and every container adds only its own writes.
 ## 4. doos versus Docker
 **Where doos is better**
 - No daemon and no root. With Docker's default install, anyone who can talk to the daemon controls the machine [S]. doos has no such trust boundary: it can only do what the invoking user can do, plus what the `newuidmap` helper allows.
 - Tiny and auditable: 552 KiB, 3,445 lines [M], one process tree, no background service, nothing listening.
 - Instant: 14 to 65 ms against 456 to 590 ms for a rootful Docker run [M].
 - Safe defaults a user does not have to remember: Docker's default capability set, a seccomp filter, an allowlist of host `/etc`, a memory and pid cap, a written-data cap [D][M].
 - A "host mode" that reuses the host's installed tools (compilers, `apt` state) for throwaway shells with no image download [D].
 - A size cap that freezes instead of filling the disk [D][M]. Docker's writable-layer size limit needs the overlay2 driver on XFS with project quotas [S: [Fabian Lee](https://fabianlee.org/2020/01/15/docker-use-overlay2-with-an-xfs-backing-filesystem-to-limit-rootfs-size/)].
 **Where Docker is better**
 - Everything around images: registries, `docker build`, layer caching, multi-arch, signing and scanning ecosystems [K]. doos cannot pull an arbitrary `nginx:1.27` from a registry.
 - Host directories and devices into the container (`-v`, `--device`, GPUs). doos has no bind-mount option [D].
 - Long-running services: background containers, restart policies, healthchecks, logs, `exec`, `stop` [K]. In doos a container exists only while its first `attach` command runs [D].
 - Networking: user-defined networks, DNS between containers, published ports on any interface [K].
 - Compose, Swarm and the Kubernetes ecosystem [K].
 - macOS and Windows support through a VM [K].
 - Strongest default syscall posture: an allowlist, which denies unknown new syscalls. The doos filter is a denylist, so a new kernel syscall is allowed until it is added [D][S].
 ## 5. doos versus Podman
 Podman is the closest comparison, since both are daemonless and rootless.
 **Where doos is better**
 - Smaller, faster to start [M] and one binary with no storage database. Podman's rootless setup brings a store under `~/.local/share/containers` with its own metadata [K].
 - Opinionated limits and hardening out of the box, with a stated, measured effect (OOM kill, pids, cpus, freeze at a size cap) [M].
 - Two network backends with different, documented trade-offs, including a host-like mode that keeps host abstract unix sockets out of the container [M].
 - Image verification is explicit: sha256 plus signatures checked against fingerprints pinned in the source [D][M].
 - Frozen-at-the-cap behaviour with a never-frozen rescue shell [D][M].
 **Where Podman is better**
 - OCI compatibility end to end: pull, push, build with Buildah, pods, `kube play`, Quadlet and systemd integration [K]. It can also run images made for Docker unchanged.
 - Docker-compatible CLI and socket, so existing tooling mostly works [K].
 - Bind mounts, volumes, secrets, `exec`, `logs`, `stop`, restart policies [K].
 - Years of hardening, a security team, and a large user base finding the sharp edges. doos has one test machine.
 - Podman documents rootless caveats: ports below 1024, `/etc/subuid` setup, cgroup v2 only, and a rootless port forwarder that does not preserve client source IPs [S: [podman-rootless(7)](https://man.archlinux.org/man/podman-rootless.7.en)]. doos shares the subuid and cgroup v2 requirements [D]. Binding ports below 1024 in doos was not tested.
 **What both share:** the same kernel features (user, mount, pid, network namespaces, overlayfs, cgroup v2, seccomp), the same pasta/slirp4netns options, and the same weakness: they share the host kernel [S].
 ## 6. Security, side by side
 - **Shared kernel.** doos, Docker, Podman, bubblewrap, nspawn and LXC all run container processes on the host kernel. A kernel vulnerability reachable through allowed syscalls is a host vulnerability. gVisor reduces that exposure at a performance cost; Kata and Firecracker remove it by giving each workload its own kernel [S].
 - **Rootless is the real difference from rootful Docker.** A bug that gives root inside a rootless doos or Podman container yields the invoking user's rights on the host, not root [D][S].
 - **doos filter style.** A denylist built at runtime with an architecture check, x32 guard and `clone3` handling [D]. Docker and Podman ship allowlist profiles [S]. An allowlist is the more conservative design.
 - **Missing layers.** Docker applies an AppArmor profile (`docker-default`) on this machine [M: `docker info` lists apparmor]. doos installs no AppArmor or SELinux profile of its own; its only profile exists to *permit* user namespaces on Ubuntu 24.04+ [D].
 - **Supply chain.** doos pins publisher signing keys for Alpine and Ubuntu and checks digests for Debian [D][M]. Registry-based tools rely on registry trust, content trust or external signing tools [K].
 Full list of what doos does not cover is at the end of [RESEARCH](docs/RESEARCH.md); the exact hardening is in [SPEC](SPEC.md) under Hardening.
 ## 7. What doos lacks, and which code would change
 These are the gaps that matter when comparing, each next to where it lives. They are descriptions, not promises.
 | Gap | Where |
 |---|---|
 | No OCI registry pull, no layers, no build | `src/image.cpp` (`Catalog`, `ImageStore`) |
 | No host directory bind mounts, no devices, no GPU | `setupRoot` in `src/sandbox.cpp`, option parsing in `src/cli.cpp` |
 | No background containers, `stop`, `exec` by name, logs | lifetime model in `Sandbox::start` / `Sandbox::watch` (container lives with its first command) |
 | No user-defined networks, no `-p` | `src/network.cpp` |
 | No Compose or Kubernetes integration | not present |
 | Environment, user, working-directory flags missing | `applyLimitsAndEnv` in `src/sandbox.cpp` |
 | Linux x86_64 tested only | `src/hardening.cpp` guards aarch64, never run there |
 | seccomp is a denylist | `deniedSyscalls` in `src/hardening.cpp` |
 | Test coverage: one machine | `tests/run.sh` |
 ## 8. Conclusion
 doos is not a Docker replacement and does not try to be. It is a small, rootless, daemonless tool for **disposable or per-user Linux shells and sandboxes that start in tens of milliseconds, with safe defaults and hard caps already set**. Its strengths are the ones the measurements show: it is a 552 KiB single binary, it starts in 14 to 65 ms [M], it runs with no daemon and no root, it caps memory, processes, CPU and written data by default, and it verifies what it downloads.
 Choose by the job:
 | If you need | Use |
 |---|---|
 | A quick, isolated shell or script runner for one user, host tools or a distro image, with limits, no daemon | **doos** |
 | OCI images, Dockerfiles or Containerfiles, registries, CI that matches production | **Podman** (rootless, daemonless) or **Docker** |
 | Compose, Kubernetes parity, Docker ecosystem tooling | **Docker** or **Podman** with its Docker compatibility |
 | Sandbox one application with exact control over mounts | **bubblewrap** |
 | A full OS with init and several services, long lived | **Incus / LXC** or **systemd-nspawn** |
 | Shared HPC cluster, MPI, GPUs, signed single-file images | **Apptainer** |
 | A desktop environment tied to your home directory | **Distrobox / Toolbx** |
 | Running code you do not trust at all, or multi-tenant workloads | **Kata / Firecracker** (or gVisor as a middle step), not any shared-kernel container |
 Honest limits of this comparison: one machine, 10-run averages, no rootless Docker, no systemd-nspawn, Incus or Apptainer measurements, and several Docker and Podman capabilities listed as general knowledge rather than re-verified. The conclusion about when to use doos rests on what the doos tests and measurements show ([RESEARCH](docs/RESEARCH.md)), not on a claim that it beats mature tools on features. On features it does not.
 Back to [README](README.md) for usage, [SPEC](SPEC.md) for the exact behavior, [ARCHITECTURE](docs/ARCHITECTURE.md) for how it works, [AGENTS](AGENTS.md) for the rules, and [CHANGELOG](CHANGELOG.md) for history.
@@ -1,5 +1,7 @@
 # doos
 **Docs:** [README](README.md) · [SPEC](SPEC.md) · [ARCHITECTURE](docs/ARCHITECTURE.md) · [RESEARCH](docs/RESEARCH.md) · [COMPARISON](COMPARISON.md) · [AGENTS](AGENTS.md) · [CHANGELOG](CHANGELOG.md)
 User-level containers in C++20: overlayfs, user/mount/pid namespaces, slirp4netns or pasta networking, optional images (Alpine, Ubuntu, Debian, or your own rootfs), memory/cpu/pid/size limits, default hardening (capabilities, seccomp). No root at runtime.
 ```
@@ -8,9 +10,10 @@ doos get_or_create [OPTIONS] NAME
 doos attach        [OPTIONS] NAME [-- CMD...]
 doos delete [--yes] NAME
 doos image list | add [--sha256 HEX] SOURCE [NAME] | delete [--yes] NAME
 doos apparmor        # profile text for this binary (Ubuntu 24.04+)
 ```
-Version: see `VERSION` (currently 1.1.0). Release history: `CHANGELOG.md`.
+Version: see `VERSION` (currently 1.1.2). Release history: `CHANGELOG.md`.
 User-level containers in C++20: overlayfs over the host's `/usr`, `/etc`, `/opt`, user/mount/pid namespaces, networking through slirp4netns. No root at runtime.
@@ -23,12 +26,17 @@ doos delete [--yes] NAME
 Build: `cmake -S . -B build && cmake --build build`. Test: `tests/run.sh`.
-Requirements: Linux >= 5.11, `slirp4netns` (default network mode), `pasta` (only for `--net pasta`), `gnupg` (signature checks; optional), `uidmap` (`newuidmap`/`newgidmap`, for more than one id inside; falls back to one id), `curl`, `tar` and `sha256sum` (images). On Ubuntu 24.04+ install the AppArmor profile once:
+Requirements: Linux >= 5.11, `slirp4netns` (default network mode), `pasta` (only for `--net pasta`), `gnupg` (signature checks; optional), `uidmap` (`newuidmap`/`newgidmap`, for more than one id inside; falls back to one id), `curl`, `tar` and `sha256sum` (images). On Ubuntu 24.04+ (`kernel.apparmor_restrict_unprivileged_userns=1`) AppArmor must allow user namespaces for the
 binary. Generate a profile for the exact path doos is installed at, once:
 ```
-sudo cp apparmor/doos /etc/apparmor.d/doos && sudo apparmor_parser -r /etc/apparmor.d/doos
+doos apparmor | sudo tee /etc/apparmor.d/doos-local >/dev/null && sudo apparmor_parser -r /etc/apparmor.d/doos-local
 ```
 If you skip this, doos tells you the exact command (with a file name unique to the binary's path, so installing doos
 in a second place does not replace the first profile). `apparmor/doos` is a static profile for packaging that covers
 `/usr/local/bin`, `/usr/bin`, `/opt/doos/bin`, `~/.local/bin`, `~/bin` and `~/projects/doos/build`.
 See `SPEC.md` (behavior, limits) and `docs/RESEARCH.md` (what works, measurements, known gaps).
 ## Usage examples
@@ -67,6 +75,41 @@ doos attach --image mybase c1
 Writes always stay in the container (copy-on-write); `/root` and `/var` persist, `/tmp` is memory-backed and lost on exit. A container is `running` while its first `attach` command is alive; more `attach` calls join it. Data persists until `delete --yes`.
 ## Networking
 Every example below was run on the development machine. Pick the mode when you create the container; it cannot change later.
 | Mode | Outbound | Host `127.0.0.1` from inside | Reach a port served inside, from the host | Host abstract unix sockets |
 |---|---|---|---|---|
 | `slirp` (default) | yes | blocked | no | not shared |
 | `pasta` | yes | reachable | yes (forwarded automatically) | not shared |
 | `host` | yes | the same network | yes | **shared** |
 | `none` | no | no | no | not shared |
 ```
 # slirp (default): the internet works, the host's loopback does not
 doos attach web -- curl -sS -o /dev/null -w "%{http_code}\n" https://example.com          # 200
 doos attach web -- curl -sS -m 3 http://127.0.0.1:8099                                      # fails: host loopback is blocked
 doos attach web -- getent hosts example.com                                                 # DNS works (resolver 10.0.2.3)
 # pasta: behaves like the host network, in a private namespace
 doos attach --net pasta dev -- curl -sS http://127.0.0.1:8099                               # reaches a service on the host's loopback
 doos attach dev -- python3 -m http.server 8088 --bind 0.0.0.0                               # then, on the host:  curl http://127.0.0.1:8088/
 doos attach --image alpine --net pasta al -- wget -qO- https://example.com                  # works with images too
 # none: no network at all, only loopback
 doos attach --net none iso -- curl -m 3 https://example.com                                 # curl exits 6, /proc/net/dev lists only lo
 # host: the host's own network namespace, explicit opt-in (prints a note recommending pasta)
 doos attach --net host h -- curl -sS -o /dev/null -w "%{http_code}\n" https://example.com  # 200
 ```
 Notes, all measured:
 - `slirp` has no inbound path: a server started in a `slirp` container is not reachable from the host (connection fails). Use `pasta` for that.
 - `pasta` notices a newly bound port after about 2 seconds, so start the server and give it a moment before connecting from the host.
 - Two `pasta` containers can talk to each other through the host's loopback: start a server in one (`python3 -m http.server 8066 --bind 0.0.0.0`), wait about 5 seconds, then `doos attach cli -- curl http://127.0.0.1:8066/` from the other returns 200. The host's own public address is not forwarded; use `127.0.0.1` or the default gateway address.
 - `pasta` starts in about 65 ms per attach, `slirp` about 40 ms, `none` about 10 ms.
 ## Options
 | Option | Meaning | Default | Later |
@@ -108,6 +151,7 @@ Changeable options apply the next time the container starts. Examples: `doos att
 | `limits are not enforced and the size cap only warns` | No delegated cgroup v2 (no systemd user manager). Limits and cap are advisory. |
 | `was killed by its memory limit` | Raise `--memory`. `/tmp` is memory-backed and counts. |
 | `hit its process limit` | Raise `--pids`; threads count as processes. |
 | `AppArmor restricts user namespaces for this binary` | Run the command doos prints (`doos apparmor \| sudo tee ... && sudo apparmor_parser -r ...`), once per install path. |
 | `Operation not permitted` from `mount`, `unshare`, `keyctl`, io_uring | Hardening blocked it. Use `--hardening relaxed` on that container. |
 | `signature NOT checked (gpg is not installed)` | Install `gnupg` for signature checks; the checksum still applies. |
 | `doos: seeded ...: unreadable skipped: N` | Host entries the user cannot read were left out of the container. Informational. |
@@ -128,8 +172,11 @@ What is still not covered is listed at the end of `docs/RESEARCH.md`.
 | File | Content |
 |---|---|
-| `SPEC.md` | Behavior contract and limits |
+| [`SPEC.md`](SPEC.md) | Behavior contract and limits |
-| `docs/ARCHITECTURE.md` | Code map and flows |
+| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Code map and flows |
-| `docs/RESEARCH.md` | What works unprivileged, measurements, gaps |
+| [`docs/RESEARCH.md`](docs/RESEARCH.md) | What works unprivileged, measurements, remaining limits |
-| `AGENTS.md` | Rules for AI agents, release workflow |
+| [`COMPARISON.md`](COMPARISON.md) | doos next to Docker, Podman, bubblewrap, nspawn, LXC/Incus, Apptainer, Distrobox, gVisor/Kata, with pros, cons, measurements and a conclusion |
-| `CHANGELOG.md` | Release history |
+| [`AGENTS.md`](AGENTS.md) | Rules for AI agents, release workflow |
 | [`CHANGELOG.md`](CHANGELOG.md) | Release history |
 Not sure whether doos is the right tool? Read the conclusion of [COMPARISON](COMPARISON.md) first.
@@ -1,6 +1,8 @@
 # doos specification
-Authoritative description of behavior and limits. Code constants live in `src/spec.hpp`; change both together.
+**Docs:** [README](README.md) · [SPEC](SPEC.md) · [ARCHITECTURE](docs/ARCHITECTURE.md) · [RESEARCH](docs/RESEARCH.md) · [COMPARISON](COMPARISON.md) · [AGENTS](AGENTS.md) · [CHANGELOG](CHANGELOG.md)
 Authoritative description of behavior and limits. Code constants live in `src/spec.hpp`; change both together. How the code implements it: [ARCHITECTURE](docs/ARCHITECTURE.md). Measurements and remaining limits: [RESEARCH](docs/RESEARCH.md). How it differs from other tools: [COMPARISON](COMPARISON.md). Usage: [README](README.md).
 ## CLI
@@ -12,6 +14,7 @@ Authoritative description of behavior and limits. Code constants live in `src/sp
 | `doos delete [--yes] NAME` | Dry run unless `--yes`. Refuses a running container. |
 | `doos image list` | Installed images (SIZE, IDS) plus catalog entries marked `available`. |
 | `doos image add [--sha256 HEX] SOURCE [NAME]` | Installs an image. See Images. |
 | `doos apparmor` | Prints an AppArmor profile (named after the binary's path) that lets this binary use user namespaces. A start that fails with EPERM/EACCES while `apparmor_restrict_unprivileged_userns=1` prints the command to install it. |
 | `doos image delete [--yes] NAME` | Dry run unless `--yes`. Refuses an image used by any container, even with `--yes`. |
 `get-or-create` is accepted as an alias.
@@ -121,7 +124,8 @@ User, mount, pid, uts, ipc namespaces always; network namespace unless `host`. I
 - `slirp`: private netns, `slirp4netns --configure --disable-host-loopback`, DNS 10.0.2.3, outbound only, host 127.0.0.1 unreachable.
 - `none`: private netns with loopback only.
- `pasta`: private netns that behaves like the host network: addresses and routes mirrored, host loopback reachable, ports bound inside are forwarded to the host, DNS from the host's resolver. The host's abstract unix sockets are not shared. Needs `pasta`.
+- `slirp` has no inbound path (nothing outside reaches a port served inside).
 - `pasta`: private netns that behaves like the host network: addresses and routes mirrored, host loopback reachable, ports bound inside are forwarded to the host, DNS from the host's resolver. The host's abstract unix sockets are not shared. Needs `pasta`. Newly bound ports are picked up after about 2 s; only loopback and the default gateway address reach other containers' forwarded ports, not the host's public address.
 - `host`: shares the host network namespace including its abstract unix sockets (explicit opt-in; a note recommends `pasta`).
 ## Safety rules
@@ -1 +1 @@
-1.1.0
+1.1.2
@@ -2,10 +2,11 @@
 # kernel.apparmor_restrict_unprivileged_userns=1 strips capabilities (mount, etc.)
 # from unknown binaries inside a new userns. Same pattern as /etc/apparmor.d/lxc-unshare.
 # The profile is unconfined apart from granting `userns`; confinement comes from doos itself.
 # Fixed paths for packaging. For any other location run:  doos apparmor | sudo tee /etc/apparmor.d/doos
 abi <abi/5.0>,
 include <tunables/global>
-profile doos /{usr/local/bin,home/*/projects/doos/build}/doos flags=(unconfined) {
+profile doos /{usr/local/bin,usr/bin,opt/doos/bin,home/*/.local/bin,home/*/bin,home/*/projects/doos/build}/doos flags=(unconfined) {
  userns,
  @{exec_path} mr,
 }
@@ -1,6 +1,8 @@
 # Architecture
-How doos works internally. Behavior contract is `SPEC.md`; this explains the code.
+**Docs:** [README](../README.md) · [SPEC](../SPEC.md) · [ARCHITECTURE](ARCHITECTURE.md) · [RESEARCH](RESEARCH.md) · [COMPARISON](../COMPARISON.md) · [AGENTS](../AGENTS.md) · [CHANGELOG](../CHANGELOG.md)
 How doos works internally. The behavior contract is [SPEC](../SPEC.md); this explains the code. Why it was built this way and what was measured: [RESEARCH](RESEARCH.md); how it compares with other tools: [COMPARISON](../COMPARISON.md).
 ## File map
@@ -1,5 +1,7 @@
 # Research: user-level containers with overlayfs
 **Docs:** [README](../README.md) · [SPEC](../SPEC.md) · [ARCHITECTURE](ARCHITECTURE.md) · [RESEARCH](RESEARCH.md) · [COMPARISON](../COMPARISON.md) · [AGENTS](../AGENTS.md) · [CHANGELOG](../CHANGELOG.md)
 Measured on this machine: Ubuntu, kernel 7.0.0-38, AppArmor 5.0.2, slirp4netns 1.3.3, uid 1000, no root at runtime.
 ## What works unprivileged
@@ -278,3 +280,5 @@ delegated with `cpuset cpu io memory pids`). The end-to-end suite passed 83 chec
 - Without `gpg` installed, signature checks are skipped with a printed note.
 - pasta is not signalled by doos (it drops uid); it exits by itself within about a second after the container's namespace is gone. Measured: no pasta process remained after 1.5 s.
 - seccomp is a denylist. A new kernel syscall is allowed until it is added to the list.
 For how doos stacks up against Docker, Podman and others, with these measurements next to theirs, see [COMPARISON](../COMPARISON.md). Behavior contract: [SPEC](../SPEC.md). Code map: [ARCHITECTURE](ARCHITECTURE.md). Usage: [README](../README.md).
@@ -0,0 +1,35 @@
 #!/usr/bin/env bash
 # Checks the documentation set: every markdown file links to every other one, and every relative link resolves.
 # Usage: scripts/check-docs.sh   (exit 1 on any problem, lists them all)
 set -u
 cd "$(dirname "$0")/.."
 DOCS=(README.md SPEC.md COMPARISON.md AGENTS.md CHANGELOG.md docs/ARCHITECTURE.md docs/RESEARCH.md)
 bad=0
 fail() { echo "docs: $*"; bad=$((bad+1)); }
 # no markdown file may exist outside the checked set (so a new doc cannot dodge the rule)
 while IFS= read -r f; do
  f=${f#./}
  printf '%s\n' "${DOCS[@]}" | grep -qxF "$f" || fail "$f is not in the DOCS list of scripts/check-docs.sh"
 done < <(find . -name '*.md' -not -path './build/*' -not -path './.git/*' -not -path './prompt.md')  # prompt.md: the original task statement, not documentation
 for f in "${DOCS[@]}"; do
  [ -f "$f" ] || { fail "$f is missing"; continue; }
  dir=$(dirname "$f")
  # every other document must be linked
  for g in "${DOCS[@]}"; do
    [ "$f" = "$g" ] && continue
    rel=$(realpath --relative-to="$dir" "$g")
    grep -qE "\]\(${rel//./\\.}(#[^)]*)?\)" "$f" || fail "$f does not link to $g (expected ]($rel))"
  done
  # every relative link must point at something that exists
  while IFS= read -r target; do
    case "$target" in http://*|https://*|mailto:*|\#*) continue ;; esac
    path=${target%%#*}
    [ -z "$path" ] && continue
    [ -e "$dir/$path" ] || fail "$f links to missing $target"
  done < <(grep -oE '\]\([^)]+\)' "$f" | sed -E 's/^\]\(//; s/\)$//')
 done
 [ "$bad" -eq 0 ] && echo "docs: ${#DOCS[@]} documents, all cross-linked, all links resolve" || { echo "docs: $bad problem(s)"; exit 1; }
@@ -22,6 +22,7 @@ if grep -rl $'\xe2\x80\x94' --exclude-dir=build --exclude-dir=.git . | grep -q .
  echo "em dash found, remove it first" >&2; exit 1
 fi
 scripts/check-docs.sh
 echo "$new" > VERSION
 cmake -S . -B build >/dev/null
 cmake --build build
@@ -112,6 +112,7 @@ void Cli::usage(std::FILE* out) {
        "  doos image list\n"
        "  doos image add [--sha256 HEX] SOURCE [NAME]\n"
        "  doos image delete [--yes] NAME\n"
        "  doos apparmor          print the AppArmor profile this binary needs on Ubuntu 24.04+\n"
        "\n"
        "options:\n"
        "  --image NAME           start from an image instead of the host system (fetched on demand)  [fixed]\n"
@@ -144,6 +145,7 @@ int Cli::run() {
    if (cmd == "attach") return cmdAttach(rest);
    if (cmd == "delete") return cmdDelete(rest);
    if (cmd == "image") return cmdImage(rest);
    if (cmd == "apparmor") return cmdApparmor();
    std::fprintf(stderr, "doos: unknown command '%s'\n\n", args_[0].c_str());
    usage(stderr);
    return 2;
@@ -187,6 +189,11 @@ int Cli::cmdDelete(std::vector<std::string> rest) {
    return 0;
 }
 int Cli::cmdApparmor() {
    std::fputs(Sandbox::apparmorProfile().c_str(), stdout);
    return 0;
 }
 int Cli::cmdImage(std::vector<std::string> rest) {
    if (rest.empty()) throw std::runtime_error("image needs a subcommand: list, add, delete");
    std::string sub = rest[0];
@@ -15,6 +15,7 @@ private:
    int cmdGetOrCreate(std::vector<std::string> rest);
    int cmdAttach(std::vector<std::string> rest);
    int cmdDelete(std::vector<std::string> rest);
    int cmdApparmor();
    int cmdImage(std::vector<std::string> rest);
    int cmdImageList();
    int cmdImageAdd(std::vector<std::string> rest);
@@ -15,6 +15,7 @@
 #include <unistd.h>
 #include <cstring>
 #include <fstream>
 #include <poll.h>
 #include <algorithm>
@@ -293,12 +294,60 @@ int containerInit(void* arg) {
        ::_exit(code);
    } catch (const std::exception& e) {
        ::dprintf(2, "doos: %s\n", e.what());
        auto* se = dynamic_cast<const SysError*>(&e);
        std::ifstream restrictUserns("/proc/sys/kernel/apparmor_restrict_unprivileged_userns");
        int restricted = 0;
        if (restrictUserns) restrictUserns >> restricted;
        if (se && (se->code() == EPERM || se->code() == EACCES) && restricted == 1) {
            ::dprintf(2, "doos: AppArmor restricts user namespaces for this binary. Install a profile once with:\n  %s\n",
                      Sandbox::apparmorInstallHint().c_str());
        }
        ::_exit(125);
    }
 }
 }  // namespace
 namespace {
 fs::path selfPath() {
    std::error_code ec;
    fs::path self = fs::read_symlink("/proc/self/exe", ec);
    if (ec) throw std::runtime_error("cannot resolve the path of this binary: " + ec.message());
    std::string path = self.string();
    if (path.find_first_of("{}[]*?\"\n,") != std::string::npos) {
        throw std::runtime_error("binary path contains characters AppArmor treats as globs: " + path);
    }
    return self;
 }
 // One profile per binary path, so installing doos in a second place does not replace the first profile.
 std::string profileName(const fs::path& self) {
    std::string n = "doos" + self.string();
    for (auto& c : n) {
        if (c == '/') c = '-';
    }
    return n;
 }
 }  // namespace
 std::string Sandbox::apparmorProfile() {
    fs::path self = selfPath();
    return "# Lets doos use unprivileged user namespaces where kernel.apparmor_restrict_unprivileged_userns=1.\n"
           "# Generated by: doos apparmor   (profile is for exactly this path)\n"
           "abi <abi/5.0>,\n"
           "include <tunables/global>\n"
           "\n"
           "profile " + profileName(self) + " " + self.string() + " flags=(unconfined) {\n"
           "  userns,\n"
           "  @{exec_path} mr,\n"
           "}\n";
 }
 std::string Sandbox::apparmorInstallHint() {
    std::string file = "/etc/apparmor.d/" + profileName(selfPath());
    return "doos apparmor | sudo tee " + file + " >/dev/null && sudo apparmor_parser -r " + file;
 }
 int Sandbox::run() {
    if (command_.size() > spec::kMaxCommandArgs) throw std::runtime_error("too many command arguments");
    // Decide start-or-join under the lock so concurrent attaches agree.
@@ -18,6 +18,11 @@ public:
    int run();
    // AppArmor profile (text) that lets this very binary use user namespaces on Ubuntu 24.04+.
    static std::string apparmorProfile();
    // Shell command that installs it; shown when a start fails because of AppArmor.
    static std::string apparmorInstallHint();
 private:
    int start(FileLock& lock);
    int watch(pid_t child, Cgroup* cgroup);
@@ -12,6 +12,7 @@ out()  { "$DOOS" attach t -- sh -c "$1" 2>&1; }
 trap '"$DOOS" delete --yes t >/dev/null 2>&1; rm -rf "$DOOS_HOME"' EXIT
 check "list empty"                  "$DOOS" list
 check "docs cross-linked, links resolve" "$(dirname "$0")/../scripts/check-docs.sh"
 check "get_or_create creates"       "$DOOS" get_or_create t
 [ "$("$DOOS" get_or_create t)" = "exists t" ] && ok "get_or_create idempotent" || bad "get_or_create idempotent"
 check "invalid name rejected"       bash -c "! '$DOOS' get_or_create 'Bad Name'"
@@ -178,6 +179,26 @@ grep -q hello /tmp/doos-in.$$ && ok "pasta: inbound port forwarded" || bad "past
 rm -f /tmp/doos-in.$$; kill $lp 2>/dev/null; wait 2>/dev/null
 "$DOOS" delete --yes pa >/dev/null
 # ---- documented network examples that were not covered yet, and the apparmor command
 echo "--- network examples"
 "$DOOS" apparmor | grep -q "profile doos.* $DOOS flags=(unconfined)" && ok "apparmor: profile names this binary" || bad "apparmor: profile names this binary"
 "$DOOS" attach --net none iso -- sh -c 'curl -sS -m 3 https://example.com >/dev/null 2>&1; [ $? -eq 6 ] && [ "$(tail -n +3 /proc/net/dev | cut -d: -f1 | tr -d " ")" = lo ]' >/dev/null 2>&1 && ok "none: curl exits 6, only lo" || bad "none: curl exits 6, only lo"
 "$DOOS" attach nets -- timeout 5 python3 -m http.server 8077 --bind 0.0.0.0 >/dev/null 2>&1 & sleep 2.5
 curl -sS -m 2 -o /dev/null http://127.0.0.1:8077/ >/dev/null 2>&1 && bad "slirp: no inbound path" || ok "slirp: no inbound path"
 wait %% 2>/dev/null
 "$DOOS" get_or_create --net pasta pa1 >/dev/null 2>&1; "$DOOS" get_or_create --net pasta pa2 >/dev/null 2>&1
 "$DOOS" attach pa1 -- timeout 25 python3 -m http.server 8066 --bind 0.0.0.0 >/dev/null 2>&1 &
 # pasta notices a newly bound port after a few seconds: poll instead of sleeping a fixed time
 c2c=000
 for _ in $(seq 1 18); do
  sleep 1
  c2c=$("$DOOS" attach pa2 -- curl -sS -m 2 -o /dev/null -w '%{http_code}' http://127.0.0.1:8066/ 2>/dev/null)
  [ "$c2c" = "200" ] && break
 done
 [ "$c2c" = "200" ] && ok "pasta: container to container via loopback" || bad "pasta: container to container via loopback"
 wait 2>/dev/null
 "$DOOS" delete --yes pa1 >/dev/null; "$DOOS" delete --yes pa2 >/dev/null; "$DOOS" delete --yes nets >/dev/null; "$DOOS" delete --yes iso >/dev/null
 # ---- signatures and more catalog images
 echo "--- signatures"
 "$DOOS" image add alpine 2>&1 | grep -q "signature verified" && ok "alpine signature verified against pinned key" || bad "alpine signature verified against pinned key"
Author	SHA1	Message	Date
retoorandClaude Sonnet 5.5	4fed40e5cd	Add COMPARISON.md, cross-link all documents, enforce with check-docs Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>	2026-10-23 22:18:42 +02:00
retoorandClaude Sonnet 5.5	c3ebf21e9c	AppArmor profile per binary path with install hint; verified network examples Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>	2026-10-23 22:08:37 +02:00