doos/docs/ARCHITECTURE.md

Architecture

Docs: README · SPEC · ARCHITECTURE · RESEARCH · COMPARISON · AGENTS · CHANGELOG · CI

How doos works internally. The behavior contract is SPEC; this explains the code. Why it was built this way and what was measured: RESEARCH; how it compares with other tools: COMPARISON.

File map

File	Class / role
src/spec.hpp	spec:: constants: caps, mount lists, device list, paths. Single source of truth with SPEC.md.
src/util.{hpp,cpp}	SysError, Fd (RAII fd), FileLock (bounded flock), parseSize/formatSize, base64Decode, runProcess (fork/exec, capped capture), fetchText/fetchFile (curl, https only, capped), TempDir (scratch dir under the state dir, removed on scope exit), isHex64, verifyDetachedSignature/verifyClearsignedSignature (gpg with a throwaway home, pinned fingerprint; shared by image.cpp and davmount.cpp), fsutil (mkdirs, read/write, bounded removeTree, mirrorDirs skeleton, copyTree/copyEntries (with skip predicate), copyFiles, diskUsage with hardlink dedupe).
src/idmap.{hpp,cpp}	IdMap (single or subuid mapping via newuidmap/newgidmap, subuid range lookup), UserNsJob (run a function as root of a user namespace with the same map: image extraction, deleting subuid-owned trees).
src/image.{hpp,cpp}	Catalog (alpine, ubuntu, debian resolution from publishers), ImageStore (add with download, sha256 and signature check via util::verifyDetachedSignature, extraction; list; delete).
src/davmount.{hpp,cpp}	DavMount: downloads/verifies/installs a pinned rclone under the state dir, resolves a --mount/--davmount WebDAV entry to a ready, reused, managed FUSE mount (on-the-fly connection string, double-fork daemonize, mountinfo + active-listing readiness, failure classification), list/umount for doos davmount.
src/cgroup.{hpp,cpp}	Limits, Cgroup: doos/<name>/{main,shell} under the user's delegated cgroup; limits on the parent, freeze on main, later attaches in shell, OOM and pid-limit counters.
src/hardening.{hpp,cpp}	Hardening::apply: capability bounding set to the Docker default, seccomp-bpf denylist built at runtime (arch check, x32 guard, clone3 ENOSYS, CLONE_NEWUSER filter).
src/network.{hpp,cpp}	NetMode (slirp, pasta, host, none) + Network (parse, resolv.conf per mode), SlirpSession, PastaSession (own the helper processes).
src/container.{hpp,cpp}	Options (what the user asked for), Container (paths, meta, pid file, layout, one-time seeding, usage), Store (list, get_or_create with on-demand image fetch, remove).
src/sandbox.{hpp,cpp}	Sandbox: resolves davmounts (DavMount::resolveAll), starts a container (clone + id map + mounts + pivot_root + init), watches it (size cap), or joins a running one (setns).
src/cli.{hpp,cpp}	Cli: argument parsing, the four container commands, image list/add/delete and davmount list/umount.
src/main.cpp	Catches exceptions, prints doos: <message>, exit 1.
tests/run.sh	End-to-end suite.
scripts/release.sh	Bump, build, test, commit, tag.
apparmor/doos	Profile granting userns on Ubuntu 24.04+.

attach flow

Cli::cmdAttach
  Store::getOrCreate           lock, create layout + meta if missing
  Sandbox::run
    FileLock (start-or-join decision)
    runningPid()?  yes -> join()   no -> start()

start():
  DavMount::resolveAll(mounts)    before clone: webdav entries become ready, reused Bind entries
  clone(NEWUSER|NEWNS|NEWPID|NEWUTS|NEWIPC[|NEWNET])      child = container pid 1
  IdMap::apply(child, ids)        single: write /proc maps; subuid: newuidmap + newgidmap
  slirp mode: SlirpSession.start(child)                    waits for ready-fd
  slirp/pasta session for the netns of child
  Cgroup::create(limits) + addMain(child)   best effort; without it nothing is enforced and doos says so
  recordPid(child)  (pid + starttime, defeats pid reuse)
  lock.unlock()     joiners must not wait for container lifetime
  write 'G' on sync pipe                                    child continues
  parent: ignore SIGINT/SIGQUIT, forward SIGTERM/SIGHUP, watch(child)
  watch: poll(pidfd, until next scan); scan = usage from inside the id map (UserNsJob) -> warn 80%, freeze 100%,
         thaw <95%; next scan after max(2 s, 10 x scan time)
  after exit: report OOM kills / pid-limit hits
  on exit: pid file cleared, slirp stopped, cgroup removed (RAII)

child (containerInit, pid 1):
  PDEATHSIG=SIGKILL, close inherited lock fd, wait for 'G'
  setupRoot -> sethostname -> loopback up (isolated netns) -> rlimits/env/no_new_privs
  fork worker -> applyLimitsAndEnv (RLIMIT_FSIZE = size cap, Hardening::apply unless relaxed) -> exec command
  pid 1 loop: forward signals, reap all, on worker exit kill(-1, SIGKILL), exit with its code

setupRoot order

Common start: MS_REC|MS_PRIVATE on /, then resolv.conf is written into the container's /etc location (Container::etcDir: private/etc in host mode, the root upper layer in image mode), and on a single-id container with apt present an APT::Sandbox::User "root" override is written next to it.

Host mode:

1. Bind the container rootfs/ onto itself (pivot_root needs a mountpoint).
2. Overlay mounts for /usr, /opt and real compat dirs (lowerdir=<host dir>,upperdir,workdir,userxattr).
3. Compat symlinks recreated (usr-merge).
4. Bind the private dirs /root, /var, /etc; then the overlay for /var/lib/dpkg inside the private /var.

Image mode:

1. One overlay on rootfs/: lower = images/NAME/rootfs, upper = layers/root/upper.
2. Mountpoints (/proc /dev /tmp /run ...) created inside the merged tree.

Both modes then: tmpfs /tmp and /run; /dev (tmpfs, bound host nodes, new devpts, /dev/shm, symlinks); /proc with read-only remounts and masks from spec.hpp; mountExtra binds each Container::mounts() entry (--mount SRC:DST[:ro]) at DST, creating the mountpoint first (fsutil::mkdirs), read-only ones remounted the same way /proc's read-only paths are; chdir(rootfs), pivot_root(".", "."), lazy unmount of the old root.

Seeding (host mode, once per container)

Container::prepareLayout runs on create and as a heal on every later load; each step leaves a seeded.<name> marker so it runs once and prints what it did and what it skipped:

1. Directory skeleton of every overlaid host dir into its upper layer (fsutil::mirrorDirs).
2. Regular files of /var/lib/dpkg, triggers, alternatives copied into the upper (copyFiles), then empty lock files.
3. /etc copied into private/etc: the allowlist (copyEntries with the credential deny predicate) under default hardening, everything readable (copyTree) under relaxed.

Reasons are in SPEC.md (host root-owned files appear as nobody).

davmount flow

DavMount::resolveAll(mounts)                               (Sandbox::start, before clone)
  for each WebDav entry:
    id = fnv1a(url, user, pass, vendor, insecure)          dst excluded: shared across containers
    dir = $DOOS_HOME/davmounts/<id>/, mkdirs mnt/, FileLock dir/lock
    already in mountinfo (fuse.*) and pidfile's pid alive and an active listing succeeds?
        -> reuse mnt/ as is (sub-second)
        -> else unmountQuiet (fusermount3 -u, then -uz if still mounted) and fall through
    ensureRclone(): $DOOS_HOME/tools/rclone-VERSION/rclone; `rclone version` match -> reuse,
        else download+verify (util::fetchFile, util::verifyClearsignedSignature against the
        pinned key, sha256 against the signed SHA256SUMS), unzip, atomic rename into place
    rclone obscure <pass> (only if a password was given)
    connection string ":webdav,url=\"...\",vendor=\"...\"[,user=...][,pass=...][,no_check_certificate=true]:"
    spawnRcloneMount: pipe, fork, fork again, setsid + redirect to dir/log, execve(rclone, mount, ...)
        first fork writes the second fork's pid down the pipe, then exits (reaped immediately);
        the real daemon is now a child of init, outliving this doos process entirely
    poll up to kDavMountReadyTimeoutMs: daemon pid alive? mountinfo shows it AND a real directory
        listing succeeds (stat() alone answers without touching the backend; only listing does)?
        -> ready, write dir/pid + dir/meta (never the password)
        -> daemon died or never answers in time: unmountQuiet, classify dir/log's tail
           (401/Unauthorized, x509/certificate, 501/Unsupported method, dial/host/refused/timeout,
           else "see the log"), throw
  Bind entries pass through unchanged; every resolved entry becomes a plain Bind MountSpec
  pointing at dir/mnt, so mountExtra() (sandbox.cpp) never has to know davmount exists.

doos davmount list/umount walk/remove $DOOS_HOME/davmounts/* directly; umount is dry-run unless --yes, same pattern as delete/image delete.

Images

ImageStore::add: lock-free staging dir images/.tmp-NAME, download with curl (https only, size and time caps), sha256 check, tar -x -p --numeric-owner executed by UserNsJob with the image's id mode (--no-same-owner for single), usage measured, meta written, atomic rename into place. Any failure removes the staging dir from inside the id map, so a failed add leaves nothing. Store::getOrCreate calls it when a catalog image is missing, and pins the container's id mode to the image's.

join flow

Open /proc/<pid>/ns/{user,mnt,uts,ipc,net,pid} and /proc/<pid>/root first, fork, move the child into the container's shell cgroup (same limits, never frozen), setns user first (grants capability for the rest), fchdir+chroot to the container root, fork again (pid ns applies to children), apply env and limits, exec. In host network mode the net namespace is not joined.

Lifetime model

The namespaces live as long as the first attach command. When it exits, pid 1 kills the rest and the container is stopped. State (upper dirs, /root, /var) persists on disk until delete --yes. There is no daemon.

Error handling

All failures throw (SysError or std::runtime_error); main prints one line. Inside the child, errors print doos: <message> and exit 125. A parent-side failure after clone kills the child (Reaper), so no half-built container survives.