doos/docs/ARCHITECTURE.md

Architecture

How doos works internally. Behavior contract is SPEC.md; this explains the code.

File map

File	Class / role
src/spec.hpp	spec:: constants: caps, mount lists, device list, paths. Single source of truth with SPEC.md.
src/util.{hpp,cpp}	SysError, Fd (RAII fd), FileLock (bounded flock), parseSize/formatSize, base64Decode, runProcess (fork/exec, capped capture), fsutil (mkdirs, read/write, bounded removeTree, mirrorDirs skeleton, copyTree/copyEntries (with skip predicate), copyFiles, diskUsage with hardlink dedupe).
src/idmap.{hpp,cpp}	IdMap (single or subuid mapping via newuidmap/newgidmap, subuid range lookup), UserNsJob (run a function as root of a user namespace with the same map: image extraction, deleting subuid-owned trees).
src/image.{hpp,cpp}	Catalog (alpine, ubuntu, debian resolution from publishers), verifySignature (gpg with a throwaway home, pinned fingerprint), ImageStore (add with download, sha256 and signature check, extraction; list; delete).
src/cgroup.{hpp,cpp}	Limits, Cgroup: doos/<name>/{main,shell} under the user's delegated cgroup; limits on the parent, freeze on main, later attaches in shell, OOM and pid-limit counters.
src/hardening.{hpp,cpp}	Hardening::apply: capability bounding set to the Docker default, seccomp-bpf denylist built at runtime (arch check, x32 guard, clone3 ENOSYS, CLONE_NEWUSER filter).
src/network.{hpp,cpp}	NetMode (slirp, pasta, host, none) + Network (parse, resolv.conf per mode), SlirpSession, PastaSession (own the helper processes).
src/container.{hpp,cpp}	Options (what the user asked for), Container (paths, meta, pid file, layout, one-time seeding, usage), Store (list, get_or_create with on-demand image fetch, remove).
src/sandbox.{hpp,cpp}	Sandbox: start a container (clone + id map + mounts + pivot_root + init), watch it (size cap), or join a running one (setns).
src/cli.{hpp,cpp}	Cli: argument parsing, the four container commands and image list/add/delete.
src/main.cpp	Catches exceptions, prints doos: <message>, exit 1.
tests/run.sh	End-to-end suite.
scripts/release.sh	Bump, build, test, commit, tag.
apparmor/doos	Profile granting userns on Ubuntu 24.04+.

attach flow

Cli::cmdAttach
  Store::getOrCreate           lock, create layout + meta if missing
  Sandbox::run
    FileLock (start-or-join decision)
    runningPid()?  yes -> join()   no -> start()

start():
  clone(NEWUSER|NEWNS|NEWPID|NEWUTS|NEWIPC[|NEWNET])      child = container pid 1
  IdMap::apply(child, ids)        single: write /proc maps; subuid: newuidmap + newgidmap
  slirp mode: SlirpSession.start(child)                    waits for ready-fd
  slirp/pasta session for the netns of child
  Cgroup::create(limits) + addMain(child)   best effort; without it nothing is enforced and doos says so
  recordPid(child)  (pid + starttime, defeats pid reuse)
  lock.unlock()     joiners must not wait for container lifetime
  write 'G' on sync pipe                                    child continues
  parent: ignore SIGINT/SIGQUIT, forward SIGTERM/SIGHUP, watch(child)
  watch: poll(pidfd, until next scan); scan = usage from inside the id map (UserNsJob) -> warn 80%, freeze 100%,
         thaw <95%; next scan after max(2 s, 10 x scan time)
  after exit: report OOM kills / pid-limit hits
  on exit: pid file cleared, slirp stopped, cgroup removed (RAII)

child (containerInit, pid 1):
  PDEATHSIG=SIGKILL, close inherited lock fd, wait for 'G'
  setupRoot -> sethostname -> loopback up (isolated netns) -> rlimits/env/no_new_privs
  fork worker -> applyLimitsAndEnv (RLIMIT_FSIZE = size cap, Hardening::apply unless relaxed) -> exec command
  pid 1 loop: forward signals, reap all, on worker exit kill(-1, SIGKILL), exit with its code

setupRoot order

Common start: MS_REC|MS_PRIVATE on /, then resolv.conf is written into the container's /etc location (Container::etcDir: private/etc in host mode, the root upper layer in image mode), and on a single-id container with apt present an APT::Sandbox::User "root" override is written next to it.

Host mode:

1. Bind the container rootfs/ onto itself (pivot_root needs a mountpoint).
2. Overlay mounts for /usr, /opt and real compat dirs (lowerdir=<host dir>,upperdir,workdir,userxattr).
3. Compat symlinks recreated (usr-merge).
4. Bind the private dirs /root, /var, /etc; then the overlay for /var/lib/dpkg inside the private /var.

Image mode:

1. One overlay on rootfs/: lower = images/NAME/rootfs, upper = layers/root/upper.
2. Mountpoints (/proc /dev /tmp /run ...) created inside the merged tree.

Both modes then: tmpfs /tmp and /run; /dev (tmpfs, bound host nodes, new devpts, /dev/shm, symlinks); /proc with read-only remounts and masks from spec.hpp; chdir(rootfs), pivot_root(".", "."), lazy unmount of the old root.

Seeding (host mode, once per container)

Container::prepareLayout runs on create and as a heal on every later load; each step leaves a seeded.<name> marker so it runs once and prints what it did and what it skipped:

1. Directory skeleton of every overlaid host dir into its upper layer (fsutil::mirrorDirs).
2. Regular files of /var/lib/dpkg, triggers, alternatives copied into the upper (copyFiles), then empty lock files.
3. /etc copied into private/etc: the allowlist (copyEntries with the credential deny predicate) under default hardening, everything readable (copyTree) under relaxed.

Reasons are in SPEC.md (host root-owned files appear as nobody).

Images

ImageStore::add: lock-free staging dir images/.tmp-NAME, download with curl (https only, size and time caps), sha256 check, tar -x -p --numeric-owner executed by UserNsJob with the image's id mode (--no-same-owner for single), usage measured, meta written, atomic rename into place. Any failure removes the staging dir from inside the id map, so a failed add leaves nothing. Store::getOrCreate calls it when a catalog image is missing, and pins the container's id mode to the image's.

join flow

Open /proc/<pid>/ns/{user,mnt,uts,ipc,net,pid} and /proc/<pid>/root first, fork, move the child into the container's shell cgroup (same limits, never frozen), setns user first (grants capability for the rest), fchdir+chroot to the container root, fork again (pid ns applies to children), apply env and limits, exec. In host network mode the net namespace is not joined.

Lifetime model

The namespaces live as long as the first attach command. When it exits, pid 1 kills the rest and the container is stopped. State (upper dirs, /root, /var) persists on disk until delete --yes. There is no daemon.

Error handling

All failures throw (SysError or std::runtime_error); main prints one line. Inside the child, errors print doos: <message> and exit 125. A parent-side failure after clone kills the child (Reaper), so no half-built container survives.