# doos specification

**Docs:** [README](README.md) · [SPEC](SPEC.md) · [ARCHITECTURE](docs/ARCHITECTURE.md) · [RESEARCH](docs/RESEARCH.md) · [COMPARISON](COMPARISON.md) · [AGENTS](AGENTS.md) · [CHANGELOG](CHANGELOG.md) · [CI](docs/CI.md)

Authoritative description of behavior and limits. Code constants live in `src/spec.hpp`; change both together. How the code implements it: [ARCHITECTURE](docs/ARCHITECTURE.md). Measurements and remaining limits: [RESEARCH](docs/RESEARCH.md). How it differs from other tools: [COMPARISON](COMPARISON.md). Usage: [README](README.md).

## CLI

| Command | Behavior |
|---|---|
| `doos list` | Table of NAME, STATE (running/stopped), NET, IMAGE (`host` when none), SIZE (cap), AGE. |
| `doos get_or_create [OPTIONS] NAME` | Creates the container if missing. Prints `created NAME` or `exists NAME`. |
| `doos attach [OPTIONS] NAME [-- CMD...]` | get_or_create, then run CMD (default `/bin/bash`, else `/bin/sh`). Starts the container if stopped, joins it if running. Exit code is the command's. |
| `doos delete [--yes] NAME` | Dry run unless `--yes`. Refuses a running container. |
| `doos image list` | Installed images (SIZE, IDS) plus catalog entries marked `available`. |
| `doos image add [--sha256 HEX] SOURCE [NAME]` | Installs an image. See Images. |
| `doos apparmor` | Prints an AppArmor profile (named after the binary's path) that lets this binary use user namespaces. A start that fails with EPERM/EACCES while `apparmor_restrict_unprivileged_userns=1` prints the command to install it. |
| `doos image delete [--yes] NAME` | Dry run unless `--yes`. Refuses an image used by any container, even with `--yes`. |
| `doos davmount list` | Table of davmount targets this host currently manages: ID, STATE (mounted/stale), VENDOR, URL. |
| `doos davmount umount [--yes] ID` | Dry run unless `--yes`. Unmounts and removes that managed target; stops storage for any container still using it. |

`get-or-create` is accepted as an alias.

Container options (accepted by `get_or_create` and `attach`):

| Option | Meaning | Default | Changeable later |
|---|---|---|---|
| `--image NAME` | Start from an image instead of the host system. Missing catalog images are fetched on demand. | none (host mode) | no |
| `--net slirp\|pasta\|host\|none` | Network mode, see Networking. | slirp | no |
| `--ids single\|subuid` | Id mapping. | subuid when available, else single | no |
| `--hardening default\|relaxed` | `relaxed` = no capability drop, no seccomp filter, full readable host `/etc` (host mode). `relaxed` and `--net host` also require `--i-understand` on every `get_or_create` and `attach` call, and print a warning once given. | default | no |
| `--i-understand` | Confirmation flag (no value) for the reduced sandboxing described above. Without it, `--hardening relaxed` or `--net host` is an error. | absent | n/a |
| `--size SIZE` | Cap for what the container writes (`512m`, `8g`, `1t`; k/m/g/t). | 4g | yes |
| `--memory SIZE` | Memory cap (cgroup `memory.max`, swap 0, whole-container OOM kill). | 8g | yes |
| `--pids N` | Process and thread cap (`pids.max`). | 4096 | yes |
| `--cpus N` | CPU cap, decimals allowed (`0.5`, `2`); `0` = all cores. | 0 | yes |
| `--mount SRC:DST[:ro]` | Bind-mount host directory `SRC` at `DST` inside the container, repeatable (max 8). Not subject to `--size`: it is host storage, not the container's writable layer. Replaces the whole mount list when given. | none | yes |
| `--mount URL DST[:ro]` / `--davmount URL DST[:ro]` | `URL` starting `dav://`, `davs://`, `http://` or `https://`: davmount, see below. Same list and cap as `--mount`. | none | yes |
| `--dav-user USER`, `--dav-pass PASS` | Credentials for the davmount entry just given (overrides any `user[:pass]@` in the URL). | none | yes |
| `--dav-vendor NAME` | `other`, `nextcloud`, `owncloud`, `sharepoint` or `sharepoint-ntlm` for that entry. | other | yes |
| `--dav-insecure` | Skip TLS certificate verification for that entry. | off | yes |

Changeable options apply at the next start of the container. Naming a fixed option on an existing container with a different value is an error, never a silent change. A plain `--mount`'s `SRC` must exist and be a directory at the time the container starts; a missing `SRC` fails the start loudly. A davmount's URL is resolved and mounted automatically, see below.

## davmount

`--mount`/`--davmount` with a `dav://`, `davs://`, `http://` or `https://` source turns into a managed WebDAV mount instead of a plain bind: doos downloads, verifies and installs a pinned `rclone` build under `$DOOS_HOME/tools/rclone-VERSION/rclone` (never the system, never the caller's `PATH`; same trust model as the catalog images: a fixed version, fetched over https, checked against a GPG-signed `SHA256SUMS` and a key pinned by fingerprint), then drives it to mount the target on the host, before the container's namespaces are created, in `Sandbox::start`.

- **Identity and reuse**: a target is identified by URL + user + pass + vendor + the insecure flag (not by destination), hashed into a 16-hex-char id under `$DOOS_HOME/davmounts/<id>/{mnt,log,pid,meta,lock}`. The same target reused by another `--mount`/`--davmount` entry, in the same or a different container, is mounted once and reused; a dead or unreachable mount is detected (process liveness plus an active directory listing, since a FUSE mount answers `stat` on its root without ever reaching the backend) and remounted automatically.
- **On-the-fly, no config file**: rclone's connection-string syntax (`:webdav,url=...,vendor=...,user=...,pass=...:`), every field quoted; the password is passed through `rclone obscure` first (rclone's own documented requirement), never in plain text to the process, and never on its command line in a form `ps` would show in full (quoted into one argv element doos builds itself).
- **Lifecycle**: the mount runs as a detached, double-forked process (`setsid`, reparented to init) so it outlives the `doos attach` that created it; writes go through rclone's VFS write-back cache (`--vfs-cache-mode=writes`, required for WebDAV, which cannot stream; `--vfs-write-back=1s`, shortened from rclone's 5 s default) and reach the server shortly after a file is closed, independent of whether the container that wrote it has already exited.
- **Readiness and failure**: a fresh mount is given up to 20 s to answer a real directory listing (not just `stat`, which a FUSE root answers without a backend round trip); failure is classified from the log into one of: authentication, unreachable server, TLS certificate, not a WebDAV endpoint, or a generic fallback naming the log file. A failed attempt is unmounted but its log and directory are left for `doos davmount list`/`umount` to inspect or clear.
- **Not covered**: doos never implements WebDAV itself, and does not audit rclone's own implementation (see `audit/CVE-2020-7039` for the equivalent libslirp caveat on the networking side); a self-signed or misconfigured TLS endpoint needs `--dav-insecure` (an explicit, logged choice, not a default).

## Limits (hard caps)

| Item | Cap |
|---|---|
| Containers | 64 |
| Images | 16 |
| Name | containers `[a-z0-9][a-z0-9_-]{0,31}`, images `[a-z0-9][a-z0-9._-]{0,31}` (`:` becomes `-`) |
| Command arguments | 256 |
| Open files inside | 4096 |
| `/tmp`, `/run`, `/dev/shm` | 256m, 64m, 64m (tmpfs) |
| `--size` | 64m to 1t |
| Size scan interval | 2 s floor, and at least 10x the duration of the last scan |
| `--memory` | 64m to 1t |
| `--pids` | 16 to 4,194,304 |
| `--cpus` | 0.1 to 1024 |
| Single file size inside | the `--size` cap (`RLIMIT_FSIZE`) |
| Size scan entries | 5,000,000 |
| Image download | 2 GiB, 900 s |
| Skeleton directories | 200,000 |
| `/etc` copy | 50,000 files, 256 MiB |
| Id range per container | 65536 ids (0 plus up to 65535 from subuid) |
| slirp4netns ready wait | 5 s |
| Lock wait | 5 s |
| Tree recursion depth | 256 |
| `--mount`/`--davmount` entries | 8 per container |
| davmount targets managed at once | 32 |
| davmount ready wait | 20 s |
| rclone download | 128 MiB |

## State layout

`$DOOS_HOME`, else `$XDG_DATA_HOME/doos`, else `~/.local/share/doos`. Path must not contain `,` or `:`.

```
containers/NAME/{meta,pid,lock,seeded.*,rootfs/,layers/<dir>/{upper,work},private/{root,var,etc}}
images/NAME/{meta,rootfs/}
tools/rclone-VERSION/rclone
davmounts/ID/{mnt/,log,pid,meta,lock}
```

Container `meta` keys: `net created version image ids size memory pids cpus hardening`, then `mounts.count` and,
per entry, `mounts.N.kind mounts.N.src mounts.N.dst mounts.N.ro` (bind), plus `mounts.N.user mounts.N.pass
mounts.N.vendor mounts.N.insecure` (webdav; `pass` is plain text, see Safety rules). A container created under
1.5.0's single-line `mounts=` format is still read, bind entries only.

## Container filesystem, host mode (no `--image`)

| Path | Source |
|---|---|
| `/usr /opt` (and `/bin /sbin /lib /lib32 /lib64` when real dirs) | overlayfs, lower = host dir, upper = `layers/<dir>/upper`, with a directory skeleton in the upper |
| `/bin /sbin /lib...` when host symlinks | same symlink recreated |
| `/etc` | private copy, made once at creation: under `--hardening default` only the allowlist in `spec::kEtcAllow` (passwd, group, nsswitch, ld.so*, ssl, ca-certificates, alternatives, apt, dpkg, shell and tool defaults...) minus names that may hold credentials (`private`, `shadow`, `gshadow`, `sudoers*`, `auth.conf*`, `ssh_host_*`, `*.key`); under `relaxed` every readable file |
| `/root`, `/var` | persistent private dirs; `/var` pre-seeded with the apt/dpkg directories |
| `/var/lib/dpkg` | overlay of the host dpkg database (when the host has it), regular files copied into the upper, lock files pre-created |
| `/tmp /run /dev/shm` | tmpfs |
| `/dev` | tmpfs + bound null zero full random urandom tty, new devpts instance |
| `/proc` | own pid namespace; `sys sysrq-trigger irq bus fs` read-only; `kcore keys timer_list` masked |
| `/sys`, `/home`, everything else on host | absent |

Why the skeleton and copies: host root-owned files appear as `nobody` inside the container. Container root cannot write into directories it does not own, nor open files it does not own for writing, nor hard-link them. The skeleton recreates every host directory as a container-owned directory in the upper layer (same mode, owner rwx added); the copies make the few files package managers rewrite container-owned.

## Container filesystem, image mode (`--image`)

`/` is one overlay: lower = `images/NAME/rootfs` (shared, never written), upper = `layers/root/upper`. `/tmp /run /dev /proc` are mounted over it as in host mode. The image is extracted under the same id map the container uses, so image files owned by root are owned by the container's root and nothing needs a skeleton or copies. `LANG` is `C.UTF-8`.

## Images

`SOURCE` for `image add`:

| Source | Resolution | Verification |
|---|---|---|
| `alpine` | latest-stable minirootfs from `dl-cdn.alpinelinux.org`, via `latest-releases.yaml` | sha256 from the yaml, and the detached `.asc` signature checked against Alpine's pinned release key (`0482D840 22F52DF1 C4E7CD43 293ACD09 07D9495A`) |
| `ubuntu`, `ubuntu-VERSION` | newest `ubuntu-base-*-base-ARCH.tar.gz` of that release from `cdimage.ubuntu.com` (default `26.04`) | `SHA256SUMS` checked with `SHA256SUMS.gpg` against Ubuntu's pinned CD image key (`84393 8DF 228D 22F7 B374 2BC0 D94A A3F0 EFE2 1092`), then sha256 |
| `debian`, `debian-RELEASE` | `debuerreotype/docker-debian-artifacts` OCI layer for `RELEASE` (default `trixie`), slim variant | sha256 = the layer digest in the OCI manifest; no publisher signature exists |
| `https://...` | downloaded | `--sha256 HEX` required, otherwise rejected |
| a file path | read locally | verified when `--sha256` is given; otherwise used with a printed note |

`http://` is refused. Signature keys are fetched over https and rejected unless their fingerprint equals the pinned one; a bad, missing or foreign signature rejects the image. Without `gpg` installed the signature step is skipped with a printed note and the checksum still applies. A mismatch deletes everything it downloaded. The tarball is extracted with `tar -p --numeric-owner`, excluding `dev/*`, inside a user namespace with the image's id mode. Architectures: x86_64 and aarch64 for the catalog.

## Size cap

The parent process measures the container's writable data (upper layers and private dirs, hard links counted once) from inside the container's id map, so files and mode-700 directories of other container ids are counted. A scan runs at most every 2 s and never uses more than 1/10 of the time (the wait is at least 10x the last scan). At 80% it warns once on stderr. At 100% it freezes the container's `main` cgroup (`cgroup.freeze`); processes stop, nothing is deleted. It thaws on its own once usage is below 95%. A second `doos attach` runs in the `shell` cgroup and is not frozen, so the user can free space from inside. No file can exceed the cap on its own (`RLIMIT_FSIZE`), which bounds the overshoot between scans. Without cgroup v2 delegation the cap only warns and says so at start.

## Resource limits

Each running container gets `doos/<name>/` below the cgroup delegated to the user's systemd manager, with leaves `main` (init and its descendants) and `shell` (later attaches). Limits sit on `doos/<name>` and cover both leaves: `memory.max` and `memory.swap.max=0` with `memory.oom.group=1` (an OOM kill takes the whole container), `pids.max`, `cpu.max`. After exit doos reports OOM kills and process-limit hits. A missing controller is named on stderr and the other limits still apply. Without delegation no limit is enforced, and doos says so at start. The group is removed when the container stops.

## Hardening (`--hardening default`)

Every process that runs a container command gets, right before exec: `no_new_privs`, core dumps off, the capability bounding set reduced to the Docker default set (`CHOWN DAC_OVERRIDE FOWNER FSETID KILL SETGID SETUID SETPCAP NET_BIND_SERVICE NET_RAW SYS_CHROOT MKNOD AUDIT_WRITE SETFCAP`, mask `a80425fb`), and a seccomp-bpf filter: wrong architecture kills the process, x32 numbers get ENOSYS, `clone3` gets ENOSYS (libc falls back to `clone`), `clone`/`unshare` with `CLONE_NEWUSER` get EPERM, and these get EPERM: the mount family and `pivot_root`, `setns`, keyring calls, `bpf`, `perf_event_open`, `userfaultfd`, `io_uring_*`, `kexec_*`, module loading, `open_by_handle_at`, `process_vm_*`, `kcmp`, `reboot`, `swapon/off`, clock and time setting, `acct`, `quotactl` and old obsolete calls. Supported on x86_64 and aarch64; elsewhere only the capability drop applies, with a printed note. Tools that need the denied calls (container runtimes inside a container, Chromium's sandbox, io_uring users) need `--hardening relaxed`.

## Isolation

User, mount, pid, uts, ipc namespaces always; network namespace unless `host`. Id map: container 0 is the invoking user; with `subuid` ids 1..65535 map to the user's `/etc/subuid` range through `newuidmap`/`newgidmap`; with `single` nothing else is mappable. `no_new_privs`, core dumps off. pid 1 is a reaping init that forwards SIGTERM/SIGINT/SIGHUP/SIGQUIT and kills the container when the command exits. Container lifetime = the first `attach` command; later attaches join via setns.

## Networking

- `slirp`: private netns, `slirp4netns --configure --disable-host-loopback`, DNS 10.0.2.3, outbound only, host 127.0.0.1 unreachable.
- `none`: private netns with loopback only.
- `slirp` has no inbound path (nothing outside reaches a port served inside).
- `pasta`: private netns that behaves like the host network: addresses and routes mirrored, host loopback reachable, ports bound inside are forwarded to the host, DNS from the host's resolver. The host's abstract unix sockets are not shared. Needs `pasta`. Newly bound ports are picked up after about 2 s; only loopback and the default gateway address reach other containers' forwarded ports, not the host's public address.
- `host`: shares the host network namespace including its abstract unix sockets (explicit opt-in; a note recommends `pasta`).

## Safety rules

- `delete` (containers and images) is dry-run by default, scoped to the store, refuses running containers and used images, never follows symlinks. Trees with subuid-owned files are removed from inside the id map.
- Unverifiable input is rejected: URL images without sha256, hash mismatches, bad names, bad state paths, missing slirp4netns or newuidmap when requested.
- Seeding and scans report what they skipped (`unreadable skipped: N`).
- A failed start kills the half-built container (no orphans); a failed image add or container create leaves nothing behind.
- `--dav-pass` (or a password embedded in a davmount URL) is stored in plain text in the container's `meta` file (0700 directory, owner-only, same trust level as everything else under `$DOOS_HOME`); it is never written to a log, never passed to `rclone` on its command line, and is run through `rclone obscure` only immediately before the mount process execs.
- `doos davmount umount` is dry-run by default, scoped to `$DOOS_HOME/davmounts`, never follows symlinks; unmounting a target in use stops storage for any container still mounting it (not prevented, only named in the dry-run output).