113 lines
7.3 KiB
Markdown
113 lines
7.3 KiB
Markdown
# Architecture
|
|
|
|
**Docs:** [README](../README.md) · [SPEC](../SPEC.md) · [ARCHITECTURE](ARCHITECTURE.md) · [RESEARCH](RESEARCH.md) · [COMPARISON](../COMPARISON.md) · [AGENTS](../AGENTS.md) · [CHANGELOG](../CHANGELOG.md) · [CI](CI.md)
|
|
|
|
How doos works internally. The behavior contract is [SPEC](../SPEC.md); this explains the code. Why it was built this way and what was measured: [RESEARCH](RESEARCH.md); how it compares with other tools: [COMPARISON](../COMPARISON.md).
|
|
|
|
## File map
|
|
|
|
| File | Class / role |
|
|
|---|---|
|
|
| `src/spec.hpp` | `spec::` constants: caps, mount lists, device list, paths. Single source of truth with `SPEC.md`. |
|
|
| `src/util.{hpp,cpp}` | `SysError`, `Fd` (RAII fd), `FileLock` (bounded flock), `parseSize`/`formatSize`, `base64Decode`, `runProcess` (fork/exec, capped capture), `fsutil` (mkdirs, read/write, bounded `removeTree`, `mirrorDirs` skeleton, `copyTree`/`copyEntries` (with skip predicate), `copyFiles`, `diskUsage` with hardlink dedupe). |
|
|
| `src/idmap.{hpp,cpp}` | `IdMap` (single or subuid mapping via newuidmap/newgidmap, subuid range lookup), `UserNsJob` (run a function as root of a user namespace with the same map: image extraction, deleting subuid-owned trees). |
|
|
| `src/image.{hpp,cpp}` | `Catalog` (alpine, ubuntu, debian resolution from publishers), `verifySignature` (gpg with a throwaway home, pinned fingerprint), `ImageStore` (add with download, sha256 and signature check, extraction; list; delete). |
|
|
| `src/cgroup.{hpp,cpp}` | `Limits`, `Cgroup`: `doos/<name>/{main,shell}` under the user's delegated cgroup; limits on the parent, freeze on `main`, later attaches in `shell`, OOM and pid-limit counters. |
|
|
| `src/hardening.{hpp,cpp}` | `Hardening::apply`: capability bounding set to the Docker default, seccomp-bpf denylist built at runtime (arch check, x32 guard, clone3 ENOSYS, CLONE_NEWUSER filter). |
|
|
| `src/network.{hpp,cpp}` | `NetMode` (slirp, pasta, host, none) + `Network` (parse, resolv.conf per mode), `SlirpSession`, `PastaSession` (own the helper processes). |
|
|
| `src/container.{hpp,cpp}` | `Options` (what the user asked for), `Container` (paths, meta, pid file, layout, one-time seeding, usage), `Store` (list, get_or_create with on-demand image fetch, remove). |
|
|
| `src/sandbox.{hpp,cpp}` | `Sandbox`: start a container (clone + id map + mounts + pivot_root + init), watch it (size cap), or join a running one (setns). |
|
|
| `src/cli.{hpp,cpp}` | `Cli`: argument parsing, the four container commands and `image list/add/delete`. |
|
|
| `src/main.cpp` | Catches exceptions, prints `doos: <message>`, exit 1. |
|
|
| `tests/run.sh` | End-to-end suite. |
|
|
| `scripts/release.sh` | Bump, build, test, commit, tag. |
|
|
| `apparmor/doos` | Profile granting `userns` on Ubuntu 24.04+. |
|
|
|
|
## attach flow
|
|
|
|
```
|
|
Cli::cmdAttach
|
|
Store::getOrCreate lock, create layout + meta if missing
|
|
Sandbox::run
|
|
FileLock (start-or-join decision)
|
|
runningPid()? yes -> join() no -> start()
|
|
|
|
start():
|
|
clone(NEWUSER|NEWNS|NEWPID|NEWUTS|NEWIPC[|NEWNET]) child = container pid 1
|
|
IdMap::apply(child, ids) single: write /proc maps; subuid: newuidmap + newgidmap
|
|
slirp mode: SlirpSession.start(child) waits for ready-fd
|
|
slirp/pasta session for the netns of child
|
|
Cgroup::create(limits) + addMain(child) best effort; without it nothing is enforced and doos says so
|
|
recordPid(child) (pid + starttime, defeats pid reuse)
|
|
lock.unlock() joiners must not wait for container lifetime
|
|
write 'G' on sync pipe child continues
|
|
parent: ignore SIGINT/SIGQUIT, forward SIGTERM/SIGHUP, watch(child)
|
|
watch: poll(pidfd, until next scan); scan = usage from inside the id map (UserNsJob) -> warn 80%, freeze 100%,
|
|
thaw <95%; next scan after max(2 s, 10 x scan time)
|
|
after exit: report OOM kills / pid-limit hits
|
|
on exit: pid file cleared, slirp stopped, cgroup removed (RAII)
|
|
|
|
child (containerInit, pid 1):
|
|
PDEATHSIG=SIGKILL, close inherited lock fd, wait for 'G'
|
|
setupRoot -> sethostname -> loopback up (isolated netns) -> rlimits/env/no_new_privs
|
|
fork worker -> applyLimitsAndEnv (RLIMIT_FSIZE = size cap, Hardening::apply unless relaxed) -> exec command
|
|
pid 1 loop: forward signals, reap all, on worker exit kill(-1, SIGKILL), exit with its code
|
|
```
|
|
|
|
## setupRoot order
|
|
|
|
Common start: `MS_REC|MS_PRIVATE` on `/`, then `resolv.conf` is written into the container's `/etc` location
|
|
(`Container::etcDir`: `private/etc` in host mode, the root upper layer in image mode), and on a single-id
|
|
container with apt present an `APT::Sandbox::User "root"` override is written next to it.
|
|
|
|
Host mode:
|
|
1. Bind the container `rootfs/` onto itself (pivot_root needs a mountpoint).
|
|
2. Overlay mounts for `/usr`, `/opt` and real compat dirs (`lowerdir=<host dir>,upperdir,workdir,userxattr`).
|
|
3. Compat symlinks recreated (usr-merge).
|
|
4. Bind the private dirs `/root`, `/var`, `/etc`; then the overlay for `/var/lib/dpkg` inside the private `/var`.
|
|
|
|
Image mode:
|
|
1. One overlay on `rootfs/`: lower = `images/NAME/rootfs`, upper = `layers/root/upper`.
|
|
2. Mountpoints (`/proc /dev /tmp /run ...`) created inside the merged tree.
|
|
|
|
Both modes then: tmpfs `/tmp` and `/run`; `/dev` (tmpfs, bound host nodes, new devpts, `/dev/shm`, symlinks);
|
|
`/proc` with read-only remounts and masks from `spec.hpp`; `chdir(rootfs)`, `pivot_root(".", ".")`, lazy
|
|
unmount of the old root.
|
|
|
|
## Seeding (host mode, once per container)
|
|
|
|
`Container::prepareLayout` runs on create and as a heal on every later load; each step leaves a
|
|
`seeded.<name>` marker so it runs once and prints what it did and what it skipped:
|
|
1. Directory skeleton of every overlaid host dir into its upper layer (`fsutil::mirrorDirs`).
|
|
2. Regular files of `/var/lib/dpkg`, `triggers`, `alternatives` copied into the upper (`copyFiles`), then empty lock files.
|
|
3. `/etc` copied into `private/etc`: the allowlist (`copyEntries` with the credential deny predicate) under default hardening, everything readable (`copyTree`) under relaxed.
|
|
|
|
Reasons are in `SPEC.md` (host root-owned files appear as `nobody`).
|
|
|
|
## Images
|
|
|
|
`ImageStore::add`: lock-free staging dir `images/.tmp-NAME`, download with curl (https only, size and time caps),
|
|
sha256 check, `tar -x -p --numeric-owner` executed by `UserNsJob` with the image's id mode (`--no-same-owner`
|
|
for single), usage measured, `meta` written, atomic `rename` into place. Any failure removes the staging dir
|
|
from inside the id map, so a failed add leaves nothing. `Store::getOrCreate` calls it when a catalog image is
|
|
missing, and pins the container's id mode to the image's.
|
|
|
|
## join flow
|
|
|
|
Open `/proc/<pid>/ns/{user,mnt,uts,ipc,net,pid}` and `/proc/<pid>/root` first, fork, move the child into the
|
|
container's `shell` cgroup (same limits, never frozen), `setns` user first (grants capability for the rest), `fchdir`+`chroot` to the container
|
|
root, fork again (pid ns applies to children), apply env and limits, exec.
|
|
In `host` network mode the net namespace is not joined.
|
|
|
|
## Lifetime model
|
|
|
|
The namespaces live as long as the first `attach` command. When it exits, pid 1 kills
|
|
the rest and the container is `stopped`. State (upper dirs, `/root`, `/var`) persists on
|
|
disk until `delete --yes`. There is no daemon.
|
|
|
|
## Error handling
|
|
|
|
All failures throw (`SysError` or `std::runtime_error`); `main` prints one line.
|
|
Inside the child, errors print `doos: <message>` and exit 125. A parent-side failure
|
|
after `clone` kills the child (`Reaper`), so no half-built container survives.
|