retoor/doos
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

doos 1.1.0: limits, hardening, pasta, signatures, Debian image

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
retoor
2026-10-23 21:59:26 +02:00
co-authored by Claude Sonnet 5.5
parent d446c29372
commit 530582e826
24 changed files with 1182 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE.md
+14 -10
View File
@@ -7,11 +7,12 @@ How doos works internally. Behavior contract is `SPEC.md`; this explains the cod
| File | Class / role |
|---|---|
| `src/spec.hpp` | `spec::` constants: caps, mount lists, device list, paths. Single source of truth with `SPEC.md`. |
| `src/util.{hpp,cpp}` | `SysError`, `Fd` (RAII fd), `FileLock` (bounded flock), `parseSize`/`formatSize`, `runProcess` (fork/exec, capped capture), `fsutil` (mkdirs, read/write, bounded `removeTree`, `mirrorDirs` skeleton, `copyTree`, `copyFiles`, `diskUsage` with hardlink dedupe). |
| `src/util.{hpp,cpp}` | `SysError`, `Fd` (RAII fd), `FileLock` (bounded flock), `parseSize`/`formatSize`, `base64Decode`, `runProcess` (fork/exec, capped capture), `fsutil` (mkdirs, read/write, bounded `removeTree`, `mirrorDirs` skeleton, `copyTree`/`copyEntries` (with skip predicate), `copyFiles`, `diskUsage` with hardlink dedupe). |
| `src/idmap.{hpp,cpp}` | `IdMap` (single or subuid mapping via newuidmap/newgidmap, subuid range lookup), `UserNsJob` (run a function as root of a user namespace with the same map: image extraction, deleting subuid-owned trees). |
| `src/image.{hpp,cpp}` | `Catalog` (alpine, ubuntu resolution from publishers), `ImageStore` (add with download, sha256 check, extraction; list; delete). |
| `src/cgroup.{hpp,cpp}` | `Cgroup`: one cgroup v2 leaf per running container under the user's delegated cgroup, used for `cgroup.freeze`. |
| `src/network.{hpp,cpp}` | `NetMode` + `Network` (parse, resolv.conf per mode), `SlirpSession` (owns the slirp4netns process). |
| `src/image.{hpp,cpp}` | `Catalog` (alpine, ubuntu, debian resolution from publishers), `verifySignature` (gpg with a throwaway home, pinned fingerprint), `ImageStore` (add with download, sha256 and signature check, extraction; list; delete). |
| `src/cgroup.{hpp,cpp}` | `Limits`, `Cgroup`: `doos/<name>/{main,shell}` under the user's delegated cgroup; limits on the parent, freeze on `main`, later attaches in `shell`, OOM and pid-limit counters. |
| `src/hardening.{hpp,cpp}` | `Hardening::apply`: capability bounding set to the Docker default, seccomp-bpf denylist built at runtime (arch check, x32 guard, clone3 ENOSYS, CLONE_NEWUSER filter). |
| `src/network.{hpp,cpp}` | `NetMode` (slirp, pasta, host, none) + `Network` (parse, resolv.conf per mode), `SlirpSession`, `PastaSession` (own the helper processes). |
| `src/container.{hpp,cpp}` | `Options` (what the user asked for), `Container` (paths, meta, pid file, layout, one-time seeding, usage), `Store` (list, get_or_create with on-demand image fetch, remove). |
| `src/sandbox.{hpp,cpp}` | `Sandbox`: start a container (clone + id map + mounts + pivot_root + init), watch it (size cap), or join a running one (setns). |
| `src/cli.{hpp,cpp}` | `Cli`: argument parsing, the four container commands and `image list/add/delete`. |
@@ -33,18 +34,21 @@ start():
clone(NEWUSER|NEWNS|NEWPID|NEWUTS|NEWIPC[|NEWNET]) child = container pid 1
IdMap::apply(child, ids) single: write /proc maps; subuid: newuidmap + newgidmap
slirp mode: SlirpSession.start(child) waits for ready-fd
Cgroup::create + add(child) best effort; without it the size cap only warns
slirp/pasta session for the netns of child
Cgroup::create(limits) + addMain(child) best effort; without it nothing is enforced and doos says so
recordPid(child) (pid + starttime, defeats pid reuse)
lock.unlock() joiners must not wait for container lifetime
write 'G' on sync pipe child continues
parent: ignore SIGINT/SIGQUIT, forward SIGTERM/SIGHUP, watch(child)
watch: poll(pidfd, 10 s); each wake scans usage -> warn 80%, freeze 100%, thaw <95%
watch: poll(pidfd, until next scan); scan = usage from inside the id map (UserNsJob) -> warn 80%, freeze 100%,
thaw <95%; next scan after max(2 s, 10 x scan time)
after exit: report OOM kills / pid-limit hits
on exit: pid file cleared, slirp stopped, cgroup removed (RAII)
child (containerInit, pid 1):
PDEATHSIG=SIGKILL, close inherited lock fd, wait for 'G'
setupRoot -> sethostname -> loopback up (isolated netns) -> rlimits/env/no_new_privs
fork worker -> exec command
fork worker -> applyLimitsAndEnv (RLIMIT_FSIZE = size cap, Hardening::apply unless relaxed) -> exec command
pid 1 loop: forward signals, reap all, on worker exit kill(-1, SIGKILL), exit with its code
```
@@ -74,7 +78,7 @@ unmount of the old root.
`seeded.<name>` marker so it runs once and prints what it did and what it skipped:
1. Directory skeleton of every overlaid host dir into its upper layer (`fsutil::mirrorDirs`).
2. Regular files of `/var/lib/dpkg`, `triggers`, `alternatives` copied into the upper (`copyFiles`), then empty lock files.
3. `/etc` copied into `private/etc` (`copyTree`).
3. `/etc` copied into `private/etc`: the allowlist (`copyEntries` with the credential deny predicate) under default hardening, everything readable (`copyTree`) under relaxed.
Reasons are in `SPEC.md` (host root-owned files appear as `nobody`).
@@ -88,8 +92,8 @@ missing, and pins the container's id mode to the image's.
## join flow
Open `/proc/<pid>/ns/{user,mnt,uts,ipc,net,pid}` and `/proc/<pid>/root` first, fork,
`setns` user first (grants capability for the rest), `fchdir`+`chroot` to the container
Open `/proc/<pid>/ns/{user,mnt,uts,ipc,net,pid}` and `/proc/<pid>/root` first, fork, move the child into the
container's `shell` cgroup (same limits, never frozen), `setns` user first (grants capability for the rest), `fchdir`+`chroot` to the container
root, fork again (pid ns applies to children), apply env and limits, exec.
In `host` network mode the net namespace is not joined.
docs/RESEARCH.md
+62 -6
View File
@@ -48,12 +48,12 @@ removed afterwards. Rows marked "unmeasured" have no run behind them.
| Id | Gap | Fix | Status | Code |
|---|---|---|---|---|
| G1 | Memory/cpu/pids limits | Cgroup v2 leaf under the delegated user service | **partly done in 1.0.0**: the leaf exists and is used for `cgroup.freeze`; `memory.max`, `pids.max`, `cpu.max` are not set yet | `cgroup.cpp` |
| G2 | Disk cap on upper dirs | Usage watchdog with freeze | **done in 1.0.0** (`--size`) | `Sandbox::watch` |
| G1 | Memory/cpu/pids limits | Cgroup v2 limits on `doos/<name>` | **done in 1.1.0** (`--memory`, `--pids`, `--cpus`) | `cgroup.cpp` |
| G2 | Disk cap on upper dirs | Usage watchdog with freeze | **done in 1.0.0**, accounting and overshoot improved in 1.1.0 | `Sandbox::watch` |
| G3 | Single uid mapping | `newuidmap`/`newgidmap` with `/etc/subuid`, fallback single | **done in 1.0.0** (`--ids`, default subuid) | `idmap.cpp`, `Store::remove` |
| G4 | Host `/etc` readable | Allowlist-only private `/etc` | **open** for host mode: `/etc` is now a private copy of all readable files (stable, still world-readable secrets); **closed in image mode** (no host files) | `container.cpp` |
| G5 | `--net host` shares abstract sockets | Keep opt-in; evaluate pasta | **open** | `network.cpp` |
| G6 | No seccomp, nested userns allowed | Hand-built seccomp-bpf denylist | **open** | new `Seccomp` class |
| G4 | Host `/etc` readable | Allowlist-only private `/etc` | **done in 1.1.0** for host mode (`--hardening relaxed` restores the full copy); image mode never had host files | `container.cpp` |
| G5 | `--net host` shares abstract sockets | pasta mode | **done in 1.1.0** (`--net pasta`); `host` stays as an explicit, noted opt-in | `network.cpp` |
| G6 | No seccomp, nested userns allowed | Capability drop + seccomp-bpf denylist | **done in 1.1.0** (`--hardening`) | `hardening.cpp` |
| G7 | Host-owned lower dirs unwritable, `chown` of host files fails, empty `/var` | Image layer + subuid; skeleton and seeding for host mode | **done in 1.0.0** | `image.cpp`, `container.cpp` |
### G1: memory, cpu and pids limits (cgroup v2)
@@ -210,7 +210,7 @@ uid 1000, subuid range 100000:65536). The end-to-end suite (`tests/run.sh`) pass
Subuid mapping added about 15 to 25 ms per start compared with 0.1.x (two setuid helper processes). Creating a host-mode container (skeleton, `/etc` copy, dpkg files) took about 1.2 s once. First `--image alpine` use including download and extraction: 0.3 s end to end on this network.
### Still open, stated plainly
### Still open after 1.0.0 (see "Resolution log (1.1.0)" for what changed since)
- G1: `memory.max`, `pids.max`, `cpu.max` are not set. Measured earlier that they work here; not wired in.
- G4: host mode copies every readable file of host `/etc` at creation. Use an image to avoid host files entirely.
@@ -222,3 +222,59 @@ Subuid mapping added about 15 to 25 ms per start compared with 0.1.x (two setuid
- Ubuntu image signature (`SHA256SUMS.gpg`) is not verified; Alpine's yaml is not signature-checked either.
- `image add FILE` without `--sha256` is accepted and prints a note, since the user chose the file.
- Catalog covers x86_64 and aarch64 only; aarch64 is untested.
## Resolution log (1.1.0)
Built and measured on the same machine (Ubuntu 26.04, kernel 7.0.0-38, 12 cores, ext4, uid 1000, cgroup v2
delegated with `cpuset cpu io memory pids`). The end-to-end suite passed 83 checks.
### Research behind each decision
| Question | Finding | Source | Decision |
|---|---|---|---|
| Which capabilities to keep | Docker's default set: CHOWN DAC_OVERRIDE FSETID FOWNER MKNOD NET_RAW SETGID SETUID SETFCAP SETPCAP NET_BIND_SERVICE SYS_CHROOT KILL AUDIT_WRITE | [Docker run reference via OneUptime](https://oneuptime.com/blog/post/2026-01-25-docker-container-capabilities/markdown), [Red Hat container security guide](https://access.redhat.com/documentation/en/red_hat_enterprise_linux_atomic_host/7/html/container_security_guide/linux_capabilities_and_seccomp) | Same set. Measured: `CapEff` inside is `a80425fb`, the Docker default mask. |
| Which syscalls to deny | Docker's profile blocks keyring calls (not namespaced) and io_uring (container escapes); kexec and bpf are also capability-gated | [Docker seccomp docs](https://docs.docker.com/engine/security/seccomp/) | Denylist in `hardening.cpp`; capability drop covers the capability-gated ones twice. |
| x32 and architecture | A denylist keyed on the x86-64 arch can be dodged with `__X32_SYSCALL_BIT`; kernels >= 5.4 answer such numbers with ENOSYS | [seccomp(2) man page patch](https://lkml.iu.edu/hypermail/linux/kernel/2007.1/03947.html) | Arch check first (kill on mismatch), x32 numbers answered ENOSYS. |
| `clone3` | Flags live in a struct seccomp cannot read; ENOSYS (not EPERM) makes glibc fall back to `clone` | [QEMU seccomp patch](https://qemu.googlesource.com/qemu/+/c542b302707628609f2a33bcbd3e442282806b8e), [libc-alpha thread](https://public-inbox.org/libc-alpha/20210727102222.r2hys526mfkpt4xo@senku/) | `clone3` -> ENOSYS, `clone`/`unshare` with `CLONE_NEWUSER` -> EPERM. Measured both. |
| cgroup layout | Delegation covers a subtree; limit files exist only when the parent enables the controller; a cgroup that has children cannot hold processes | [kernel cgroup v2 docs](https://docs.kernel.org/admin-guide/cgroup-v2.html), [rootlesscontaine.rs](https://rootlesscontaine.rs/getting-started/common/cgroup2/) | `doos/` enables `+memory +pids +cpu`; limits on `doos/<name>` (no processes); leaves `main` and `shell`. |
| Host-like networking without sharing the namespace | pasta attaches to an existing namespace by PID and splices local TCP/UDP between namespaces | [passt(1) man page](https://manpages.ubuntu.com/manpages/resolute/man1/passt.1.html) | `--net pasta`. Measured below. |
| Image signatures | Alpine publishes the release key fingerprint on its downloads page; Ubuntu publishes the CD image key fingerprint in its verification tutorial | [alpinelinux.org/downloads](https://alpinelinux.org/downloads/), [ubuntu.com verify tutorial](https://ubuntu.com/tutorials/how-to-verify-ubuntu) | Fingerprints pinned in `spec.hpp`; keys fetched at install and refused unless the fingerprint matches. |
| Debian rootfs | No `rootfs.tar.xz` served; the OCI layer is at `oci/blobs/rootfs.tar.gz` and its digest is in the manifest embedded (base64) in `oci/index.json` | measured with curl against `debuerreotype/docker-debian-artifacts` | `debian`, `debian-RELEASE` in the catalog; the digest is the integrity check. |
### Measurements
| Check | Result |
|---|---|
| Capabilities | `CapEff a80425fb`, `Seccomp: 2` (filter) inside a default container; relaxed: `CapEff 000001ffffffffff`, `Seccomp: 0` |
| seccomp | `keyctl`, `bpf`, `mount`, `unshare(CLONE_NEWUSER)` -> EPERM; `clone3` -> ENOSYS; `unshare -U` and `mount -t tmpfs` fail; relaxed container runs `unshare -U` fine |
| Tools under hardening | gcc, curl https, git, python3 `ssl`, ping, apt and apk installs all still work |
| `/etc` | 50 entries under the allowlist versus 215 in a relaxed copy; no `shadow`, `sudoers`, `ssh_host_*` |
| Memory | `--memory 256m`, touching 400 MB: whole container killed, doos prints `killed by its memory limit` |
| Pids | `--pids 32`, 80 forks: `can't fork`, doos prints `hit its process limit (32)` |
| CPU | 4 busy loops for 4.3 s wall: unlimited 16.2 CPU-seconds, `--cpus 0.5` 2.3 CPU-seconds (0.54 of one core) |
| pasta | host `127.0.0.1` listener reachable, host abstract unix socket blocked (host mode: reachable), outbound TCP, DNS and https 200, inbound port forwarded to the host |
| Signatures | Alpine and Ubuntu 24.04 verified with the pinned keys; a stand-in `gpg` that reports a valid signature by another key, or `BADSIG`, is refused and leaves no image |
| Debian | `debian` (trixie) image installs and `apt-get install cowsay` completes, `dpkg --audit` clean; image 82M |
| Size accounting | 80 MB written by container uid 5 in a mode-700 directory (invisible to the host user) against `--size 64m`: frozen; cleanup from a second attach thawed it |
| Attach time (5 runs) | host or image, slirp: about 40 ms each; pasta: about 65 ms; `--net none`: about 10 ms |
### What changed in behavior (and why it is not a loss)
- Hardening can be switched off per container (`--hardening relaxed`), so nothing that needs the denied calls is lost, it is opt-in now.
- Host `/etc` content outside the allowlist is absent under default hardening. `relaxed` restores the full copy; images never had it.
- The 10 s scan interval is gone: scans start every 2 s at the earliest but are paced by their own cost, so large trees are not scanned in a tight loop.
- pasta replaces the need for `--net host` in almost every case; `host` remains.
### Still open, stated plainly
- Memory limits count `/tmp` and `/dev/shm` (both RAM-backed) and page cache charged to the container; a build that fills `/tmp` can hit `--memory`. Measured only with anonymous memory.
- `--pids` counts threads. Thread-heavy programs need a higher value.
- `memory.high` is not used, so there is no soft throttle before the OOM kill.
- Size accounting is a scan, not a quota: writes can pass the cap by up to one scan interval, bounded per file by `RLIMIT_FSIZE`. A process can still write many small files faster than a scan.
- A later `doos attach` shares the memory, pids and cpu limits and the per-file size limit, but is never frozen. That is the rescue path by design, so it can also fill the disk past the cap.
- Host-mode containers still drift from the host (snapshots of `/etc`, dpkg state, skeleton). Images do not.
- Debian images have no publisher signature to check; trust is HTTPS to the Debian Docker maintainers' repository plus the content digest. The Alpine key is pinned from alpinelinux.org, not from a second out-of-band channel beyond the fingerprint shown on the same site and in public mailing lists.
- aarch64 paths (Alpine `aarch64`, Debian `dist-arm64v8`, seccomp numbers for aarch64) compile and are guarded but were not run: no aarch64 machine here.
- Without `gpg` installed, signature checks are skipped with a printed note.
- pasta is not signalled by doos (it drops uid); it exits by itself within about a second after the container's namespace is gone. Measured: no pasta process remained after 1.5 s.
- seccomp is a denylist. A new kernel syscall is allowed until it is added to the list.