diff --git a/AGENTS.md b/AGENTS.md index ba83cc6..5bdcc16 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,9 +7,10 @@ part of every release. ## Project in one paragraph `doos` is a user-level container tool in C++20. `attach` builds a user + mount + pid -+ uts + ipc (+ net) namespace set, overlays the host's `/usr`, `/etc`, `/opt` with a -per-container upper dir, pivots into it and runs a command. Networking is slirp4netns. -No root at runtime. Authoritative behavior: `SPEC.md`. Design notes: `docs/ARCHITECTURE.md`. ++ uts + ipc (+ net) namespace set, overlays either the host's `/usr`, `/opt` (host mode) or an +extracted image (image mode) with a per-container upper dir, pivots into it and runs a command. +Networking is slirp4netns, ids are mapped through newuidmap when possible, a watchdog enforces +`--size`. No root at runtime. Authoritative behavior: `SPEC.md`. Design notes: `docs/ARCHITECTURE.md`. Measurements and gaps: `docs/RESEARCH.md`. ## Mandatory release workflow (every change, same session) @@ -40,6 +41,7 @@ Never hand-edit `VERSION` or create tags manually. Never skip the tests to relea | A new measurement | `docs/RESEARCH.md` with machine, kernel, and what was run | | A new known gap or a closed gap | `docs/RESEARCH.md` gaps list and `README.md` | | These workflow rules | this file | +| Image catalog, sources, verification | `SPEC.md` Images, `README.md`, `docs/RESEARCH.md` | | Anything above, in every release | `README.md` version line and `CHANGELOG.md` entry | Docs describe what the system does today. No roadmap text. A gap is listed next to @@ -65,7 +67,8 @@ the script. ## Testing `tests/run.sh [path/to/doos]` runs the real path: real overlayfs, namespaces, -slirp4netns, filesystem. Add a case for every behavior change. A green run is required +slirp4netns, filesystem, real downloads of the Alpine and Ubuntu images, real apt/apk installs, the size +freeze (about 3 minutes in total, needs network). Add a case for every behavior change. A green run is required before release. On Ubuntu 24.04+ the AppArmor profile in `apparmor/doos` must be installed once (needs sudo, ask the user, do not do it yourself). diff --git a/CHANGELOG.md b/CHANGELOG.md index ef8131f..b7990b0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,15 @@ # Changelog +## 1.0.0 - 2026-10-23 +Breaking: state layout changed (private `/etc`, `meta` keys). Containers from 0.x must be deleted and recreated. +- Images: `doos image list|add|delete`, `--image NAME` on `get_or_create`/`attach` (fetched on demand). Catalog: `alpine`, `ubuntu`, `ubuntu-VERSION`; also https URL with `--sha256`, or a local tarball. sha256 verified before extraction. +- `--size`: cap for what a container writes (default 4g). Warns at 80%, freezes at 100% via cgroup v2, thaws below 95%. +- `--ids`: subuid mapping through newuidmap/newgidmap by default (chown, su and apt's `_apt` work); falls back to one id. +- Host mode fixes: directory skeleton so host-owned directories are writable, private copy of `/etc`, pre-seeded `/var` and dpkg state. `apt-get install` now completes. +- Deleting trees with subuid-owned files works (runs inside the id map). +- Usage measurement counts hard links once. +- Tests: 56 end-to-end checks, including real image downloads, apt/apk installs and the size freeze. + ## 0.1.3 - 2026-10-23 - Docs: field trial of apt and common tools, new gap G7 (host-owned lower layer), README limits section. No code change. diff --git a/CMakeLists.txt b/CMakeLists.txt index 63d1e7c..2c3a53d 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -9,7 +9,7 @@ if(NOT CMAKE_BUILD_TYPE) endif() add_executable(doos - src/main.cpp src/cli.cpp src/container.cpp src/network.cpp src/sandbox.cpp src/util.cpp) + src/main.cpp src/idmap.cpp src/image.cpp src/cgroup.cpp src/cli.cpp src/container.cpp src/network.cpp src/sandbox.cpp src/util.cpp) target_compile_options(doos PRIVATE -Wall -Wextra -Wpedantic) target_compile_definitions(doos PRIVATE DOOS_VERSION="${DOOS_VERSION}") install(TARGETS doos RUNTIME DESTINATION bin) diff --git a/README.md b/README.md index 5e47523..4c745de 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,16 @@ # doos -Version: see `VERSION` (currently 0.1.3). Release history: `CHANGELOG.md`. +User-level containers in C++20: overlayfs, user/mount/pid namespaces, slirp4netns networking, optional images (Alpine, Ubuntu, or your own rootfs), a size cap per container. No root at runtime. + +``` +doos list +doos get_or_create [--image I] [--size S] [--net M] [--ids X] NAME +doos attach [--image I] [--size S] [--net M] [--ids X] NAME [-- CMD...] +doos delete [--yes] NAME +doos image list | add [--sha256 HEX] SOURCE [NAME] | delete [--yes] NAME +``` + +Version: see `VERSION` (currently 1.0.0). Release history: `CHANGELOG.md`. User-level containers in C++20: overlayfs over the host's `/usr`, `/etc`, `/opt`, user/mount/pid namespaces, networking through slirp4netns. No root at runtime. @@ -13,7 +23,7 @@ doos delete [--yes] NAME Build: `cmake -S . -B build && cmake --build build`. Test: `tests/run.sh`. -Requirements: Linux >= 5.11, `slirp4netns` for the default network mode. On Ubuntu 24.04+ install the AppArmor profile once: +Requirements: Linux >= 5.11, `slirp4netns` (default network mode), `uidmap` (`newuidmap`/`newgidmap`, for more than one id inside; falls back to one id), `curl`, `tar` and `sha256sum` (images). On Ubuntu 24.04+ install the AppArmor profile once: ``` sudo cp apparmor/doos /etc/apparmor.d/doos && sudo apparmor_parser -r /etc/apparmor.d/doos @@ -24,27 +34,42 @@ See `SPEC.md` (behavior, limits) and `docs/RESEARCH.md` (what works, measurement ## Usage examples ``` -doos get_or_create dev # created dev -doos attach dev # interactive shell as root inside -doos attach dev -- apt --version # run one command +doos attach dev # host mode: your system's /usr as base, shell as root inside +doos attach --image alpine box # first use downloads and verifies Alpine, then opens a shell +doos attach --image ubuntu web -- apt-get update +doos attach --image ubuntu-24.04 old # a specific Ubuntu release +doos attach --size 8g big # raise the writable-layer cap (default 4g) doos attach --net none sandbox -- sh # no network at all doos list +doos image list # what is installed, what the catalog offers doos delete dev # dry run: shows what would go doos delete --yes dev # really deletes +doos image delete --yes alpine # refused while a container uses it ``` -Container writes to `/usr`, `/etc`, `/opt` stay in the container (copy-on-write); -`/root` and `/var` are private and persistent. `/tmp` is memory-backed and lost on exit. -A container is `running` while its first `attach` command is alive; more `attach` calls -join it. Data persists until `delete --yes`. +Your own image (a root filesystem tarball): -## Current limits you will hit +``` +doos image add https://example.org/debian-rootfs.tar.xz --sha256 debian # URL needs the hash +doos image add ./rootfs.tar.gz mybase # local file +doos attach --image mybase c1 +``` -Tools run fine (curl, git, gcc, python3, pip, ping all verified). Package management does -not: `apt-get update` needs a seeded `/var` and `-o APT::Sandbox::User=root`, and -`apt-get install` cannot finish configuring packages because host-root-owned files are -unwritable and unchownable inside. Details, measurements and the fix are in -`docs/RESEARCH.md` (G7, "Field trial"). +## Host mode or image mode + +| | Host mode (default) | Image mode (`--image`) | +|---|---|---| +| Base system | your `/usr`, `/opt`, copy of `/etc` | the image, extracted once and shared | +| Needs a download | no | first use only | +| `apt-get install` | works when the host is Debian/Ubuntu based | works for the image's own package manager (apt, apk) | +| Sees host files | `/usr`, `/opt`, readable `/etc` at creation | nothing from the host | +| Best for | quick throwaway shells with your tools | reproducible, separate systems | + +Writes always stay in the container (copy-on-write); `/root` and `/var` persist, `/tmp` is memory-backed and lost on exit. A container is `running` while its first `attach` command is alive; more `attach` calls join it. Data persists until `delete --yes`. + +## Size cap + +`--size` limits what a container writes (default 4g, 64m to 1t). The parent checks every 10 s: a warning at 80%, at 100% the container is frozen (nothing is deleted), and it thaws on its own below 95% after you free space with a second `doos attach NAME -- ...`. Needs cgroup v2 delegation (standard on systemd desktops); without it the cap only warns. ## Configuration @@ -59,6 +84,11 @@ unwritable and unchownable inside. Details, measurements and the fix are in |---|---| | `write /proc//uid_map ... blocked` or `mount ...: Permission denied` | AppArmor restricts user namespaces. Install `apparmor/doos` (command above). | | `slirp4netns failed to start` | Install `slirp4netns`, or use `--net none` / `--net host`. | +| `subuid mapping unavailable` | Install `uidmap` and make sure `/etc/subuid` and `/etc/subgid` list your user, or use `--ids single`. | +| `sha256 mismatch` | The download is not what the publisher lists; nothing was installed. Retry or check the source. | +| `a URL image needs --sha256` | Unverifiable input is rejected by design; pass the hash from the publisher. | +| `size cap only warns, cannot freeze` | No delegated cgroup v2 (no systemd user manager). Cap is advisory. | +| `doos: seeded ...: unreadable skipped: N` | Host entries the user cannot read were left out of the container. Informational. | | `state path must not contain ',' or ':'` | Set `DOOS_HOME` to a plain path. | | `container 'x' is running` on delete | Exit the attached command first. | | `container limit reached (64)` | Delete one first. | @@ -66,7 +96,8 @@ unwritable and unchownable inside. Details, measurements and the fix are in ## Safety summary Host files are never written: system dirs are reached only through overlayfs. Host -`/home`, `/sys` and host loopback are not reachable in the default mode. `--net host` +`/home`, `/sys` and host loopback are not reachable in the default mode. Images are +verified against the publisher's checksum before they are extracted. `--net host` removes network isolation and is opt-in. Known gaps (no cgroups, readable host `/etc`) are listed with researched solutions (G1 to G6, not implemented) in `docs/RESEARCH.md`. diff --git a/SPEC.md b/SPEC.md index dde7c47..0741c95 100644 --- a/SPEC.md +++ b/SPEC.md @@ -6,49 +6,99 @@ Authoritative description of behavior and limits. Code constants live in `src/sp | Command | Behavior | |---|---| -| `doos list` | Table of NAME, STATE (running/stopped), NET, AGE. | -| `doos get_or_create [--net M] NAME` | Creates the container if missing. Prints `created NAME` or `exists NAME`. | -| `doos attach [--net M] NAME [-- CMD...]` | get_or_create, then run CMD (default `/bin/bash`, else `/bin/sh`). Starts the container if stopped, joins it if running. Exit code is the command's. | +| `doos list` | Table of NAME, STATE (running/stopped), NET, IMAGE (`host` when none), SIZE (cap), AGE. | +| `doos get_or_create [OPTIONS] NAME` | Creates the container if missing. Prints `created NAME` or `exists NAME`. | +| `doos attach [OPTIONS] NAME [-- CMD...]` | get_or_create, then run CMD (default `/bin/bash`, else `/bin/sh`). Starts the container if stopped, joins it if running. Exit code is the command's. | | `doos delete [--yes] NAME` | Dry run unless `--yes`. Refuses a running container. | +| `doos image list` | Installed images (SIZE, IDS) plus catalog entries marked `available`. | +| `doos image add [--sha256 HEX] SOURCE [NAME]` | Installs an image. See Images. | +| `doos image delete [--yes] NAME` | Dry run unless `--yes`. Refuses an image used by any container, even with `--yes`. | -`get-or-create` is accepted as an alias. Network mode `M` is fixed at creation: `slirp` (default), `host`, `none`. +`get-or-create` is accepted as an alias. + +Container options (all four commands that take NAME-creating options: `get_or_create`, `attach`): + +| Option | Meaning | Default | Changeable later | +|---|---|---|---| +| `--image NAME` | Start from an image instead of the host system. Missing catalog images are fetched on demand. | none (host mode) | no | +| `--size SIZE` | Cap for what the container writes (`512m`, `8g`, `1t`; k/m/g/t). | 4g | yes, on any later get_or_create/attach | +| `--net slirp\|host\|none` | Network mode. | slirp | no | +| `--ids single\|subuid` | Id mapping. | subuid when available, else single | no | + +Naming an option on an existing container with a different fixed value is an error, never a silent change. ## Limits (hard caps) | Item | Cap | |---|---| | Containers | 64 | -| Name | `[a-z0-9][a-z0-9_-]{0,31}` | +| Images | 16 | +| Name | containers `[a-z0-9][a-z0-9_-]{0,31}`, images `[a-z0-9][a-z0-9._-]{0,31}` (`:` becomes `-`) | | Command arguments | 256 | | Open files inside | 4096 | | `/tmp`, `/run`, `/dev/shm` | 256m, 64m, 64m (tmpfs) | +| `--size` | 64m to 1t | +| Size scan interval | 10 s (floor, never faster) | +| Size scan entries | 5,000,000 | +| Image download | 2 GiB, 900 s | +| Skeleton directories | 200,000 | +| `/etc` copy | 50,000 files, 256 MiB | +| Id range per container | 65536 ids (0 plus up to 65535 from subuid) | | slirp4netns ready wait | 5 s | | Lock wait | 5 s | -| Delete recursion depth | 256 | +| Tree recursion depth | 256 | ## State layout `$DOOS_HOME`, else `$XDG_DATA_HOME/doos`, else `~/.local/share/doos`. Path must not contain `,` or `:`. ``` -containers/NAME/{meta,pid,lock,rootfs/,layers//{upper,work},private/{root,var}} +containers/NAME/{meta,pid,lock,seeded.*,rootfs/,layers//{upper,work},private/{root,var,etc}} +images/NAME/{meta,rootfs/} ``` -## Container filesystem +`meta` keys: `net created version image ids size`. + +## Container filesystem, host mode (no `--image`) | Path | Source | |---|---| -| `/usr /etc /opt` (and `/bin /sbin /lib /lib32 /lib64` when real dirs) | overlayfs, lower = host dir, upper = `layers//upper` | +| `/usr /opt` (and `/bin /sbin /lib /lib32 /lib64` when real dirs) | overlayfs, lower = host dir, upper = `layers//upper`, with a directory skeleton in the upper | | `/bin /sbin /lib...` when host symlinks | same symlink recreated | -| `/root /var` | persistent private dirs, no host data | +| `/etc` | private copy of the readable host files, made once at creation | +| `/root`, `/var` | persistent private dirs; `/var` pre-seeded with the apt/dpkg directories | +| `/var/lib/dpkg` | overlay of the host dpkg database (when the host has it), regular files copied into the upper, lock files pre-created | | `/tmp /run /dev/shm` | tmpfs | | `/dev` | tmpfs + bound null zero full random urandom tty, new devpts instance | | `/proc` | own pid namespace; `sys sysrq-trigger irq bus fs` read-only; `kcore keys timer_list` masked | | `/sys`, `/home`, everything else on host | absent | +Why the skeleton and copies: host root-owned files appear as `nobody` inside the container. Container root cannot write into directories it does not own, nor open files it does not own for writing, nor hard-link them. The skeleton recreates every host directory as a container-owned directory in the upper layer (same mode, owner rwx added); the copies make the few files package managers rewrite container-owned. + +## Container filesystem, image mode (`--image`) + +`/` is one overlay: lower = `images/NAME/rootfs` (shared, never written), upper = `layers/root/upper`. `/tmp /run /dev /proc` are mounted over it as in host mode. The image is extracted under the same id map the container uses, so image files owned by root are owned by the container's root and nothing needs a skeleton or copies. `LANG` is `C.UTF-8`. + +## Images + +`SOURCE` for `image add`: + +| Source | Resolution | Verification | +|---|---|---| +| `alpine` | latest-stable minirootfs from `dl-cdn.alpinelinux.org`, via `latest-releases.yaml` | sha256 from the yaml, over https | +| `ubuntu`, `ubuntu-VERSION` | newest `ubuntu-base-*-base-ARCH.tar.gz` of that release from `cdimage.ubuntu.com` (default `26.04`) | sha256 from `SHA256SUMS`, over https | +| `https://...` | downloaded | `--sha256 HEX` required, otherwise rejected | +| a file path | read locally | verified when `--sha256` is given; otherwise used with a printed note | + +`http://` is refused. A mismatch deletes everything it downloaded. The tarball is extracted with `tar -p --numeric-owner`, excluding `dev/*`, inside a user namespace with the image's id mode. Architectures: x86_64 and aarch64 for the catalog. + +## Size cap + +The parent process scans the container's writable data every 10 s (hard floor). At 80% it warns once on stderr. At 100% it freezes the container's cgroup (`cgroup.freeze`); processes stop, nothing is deleted. It thaws on its own once usage is below 95%. A second `doos attach` is not frozen, so the user can free space from inside. Without cgroup v2 delegation the cap only warns and says so at start. + ## Isolation -User, mount, pid, uts, ipc namespaces always; network namespace unless `host`. Single-entry id map (container root = invoking user). `no_new_privs`, core dumps off. pid 1 is a reaping init that forwards SIGTERM/SIGINT/SIGHUP/SIGQUIT and kills the container when the command exits. Container lifetime = the first `attach` command; later attaches join via setns. +User, mount, pid, uts, ipc namespaces always; network namespace unless `host`. Id map: container 0 is the invoking user; with `subuid` ids 1..65535 map to the user's `/etc/subuid` range through `newuidmap`/`newgidmap`; with `single` nothing else is mappable. `no_new_privs`, core dumps off. pid 1 is a reaping init that forwards SIGTERM/SIGINT/SIGHUP/SIGQUIT and kills the container when the command exits. Container lifetime = the first `attach` command; later attaches join via setns. ## Networking @@ -58,6 +108,7 @@ User, mount, pid, uts, ipc namespaces always; network namespace unless `host`. S ## Safety rules -- `delete` is dry-run by default, scoped to `containers/NAME`, refuses running containers, never follows symlinks. -- Invalid names, bad state paths and missing slirp4netns fail loudly. -- A failed start kills the half-built container (no orphans). +- `delete` (containers and images) is dry-run by default, scoped to the store, refuses running containers and used images, never follows symlinks. Trees with subuid-owned files are removed from inside the id map. +- Unverifiable input is rejected: URL images without sha256, hash mismatches, bad names, bad state paths, missing slirp4netns or newuidmap when requested. +- Seeding and scans report what they skipped (`unreadable skipped: N`). +- A failed start kills the half-built container (no orphans); a failed image add leaves no partial image. diff --git a/VERSION b/VERSION index b1e80bb..3eefcb9 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.3 +1.0.0 diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 9f4c5e3..7031389 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -7,11 +7,14 @@ How doos works internally. Behavior contract is `SPEC.md`; this explains the cod | File | Class / role | |---|---| | `src/spec.hpp` | `spec::` constants: caps, mount lists, device list, paths. Single source of truth with `SPEC.md`. | -| `src/util.{hpp,cpp}` | `SysError` (errno-carrying exception), `Fd` (RAII fd), `FileLock` (bounded flock), `fsutil` (mkdirs, read/write, bounded `removeTree`). | +| `src/util.{hpp,cpp}` | `SysError`, `Fd` (RAII fd), `FileLock` (bounded flock), `parseSize`/`formatSize`, `runProcess` (fork/exec, capped capture), `fsutil` (mkdirs, read/write, bounded `removeTree`, `mirrorDirs` skeleton, `copyTree`, `copyFiles`, `diskUsage` with hardlink dedupe). | +| `src/idmap.{hpp,cpp}` | `IdMap` (single or subuid mapping via newuidmap/newgidmap, subuid range lookup), `UserNsJob` (run a function as root of a user namespace with the same map: image extraction, deleting subuid-owned trees). | +| `src/image.{hpp,cpp}` | `Catalog` (alpine, ubuntu resolution from publishers), `ImageStore` (add with download, sha256 check, extraction; list; delete). | +| `src/cgroup.{hpp,cpp}` | `Cgroup`: one cgroup v2 leaf per running container under the user's delegated cgroup, used for `cgroup.freeze`. | | `src/network.{hpp,cpp}` | `NetMode` + `Network` (parse, resolv.conf per mode), `SlirpSession` (owns the slirp4netns process). | -| `src/container.{hpp,cpp}` | `Container` (one container's paths, meta, pid file, layout), `Store` (list, get_or_create, remove). | -| `src/sandbox.{hpp,cpp}` | `Sandbox`: start a container (clone + mounts + pivot_root + init) or join a running one (setns). | -| `src/cli.{hpp,cpp}` | `Cli`: argument parsing and the four commands. | +| `src/container.{hpp,cpp}` | `Options` (what the user asked for), `Container` (paths, meta, pid file, layout, one-time seeding, usage), `Store` (list, get_or_create with on-demand image fetch, remove). | +| `src/sandbox.{hpp,cpp}` | `Sandbox`: start a container (clone + id map + mounts + pivot_root + init), watch it (size cap), or join a running one (setns). | +| `src/cli.{hpp,cpp}` | `Cli`: argument parsing, the four container commands and `image list/add/delete`. | | `src/main.cpp` | Catches exceptions, prints `doos: `, exit 1. | | `tests/run.sh` | End-to-end suite. | | `scripts/release.sh` | Bump, build, test, commit, tag. | @@ -28,13 +31,15 @@ Cli::cmdAttach start(): clone(NEWUSER|NEWNS|NEWPID|NEWUTS|NEWIPC[|NEWNET]) child = container pid 1 - parent writes /proc//{setgroups,uid_map,gid_map} 0 -> invoking uid, size 1 + IdMap::apply(child, ids) single: write /proc maps; subuid: newuidmap + newgidmap slirp mode: SlirpSession.start(child) waits for ready-fd + Cgroup::create + add(child) best effort; without it the size cap only warns recordPid(child) (pid + starttime, defeats pid reuse) lock.unlock() joiners must not wait for container lifetime write 'G' on sync pipe child continues - parent: ignore SIGINT/SIGQUIT, forward SIGTERM/SIGHUP, waitpid(child) - on exit: pid file cleared, slirp stopped (RAII) + parent: ignore SIGINT/SIGQUIT, forward SIGTERM/SIGHUP, watch(child) + watch: poll(pidfd, 10 s); each wake scans usage -> warn 80%, freeze 100%, thaw <95% + on exit: pid file cleared, slirp stopped, cgroup removed (RAII) child (containerInit, pid 1): PDEATHSIG=SIGKILL, close inherited lock fd, wait for 'G' @@ -45,15 +50,41 @@ child (containerInit, pid 1): ## setupRoot order -1. `MS_REC|MS_PRIVATE` on `/` so nothing propagates to the host. -2. Bind the container `rootfs/` onto itself (pivot_root needs a mountpoint). -3. Write `resolv.conf` into the `/etc` upper dir (matches network mode every start). -4. Overlay mounts (`lowerdir=,upperdir,workdir,userxattr`, nosuid, nodev). -5. Compat dirs: recreate host symlinks (usr-merge) or overlay real dirs. -6. Bind private dirs (`/root`, `/var`), mount tmpfs (`/tmp`, `/run`). -7. `/dev`: tmpfs, bind host device nodes, new devpts, `/dev/shm`, symlinks. -8. `/proc`, then read-only remount and masks from `spec.hpp`. -9. `chdir(rootfs)`, `pivot_root(".", ".")`, lazy unmount of old root. +Common start: `MS_REC|MS_PRIVATE` on `/`, then `resolv.conf` is written into the container's `/etc` location +(`Container::etcDir`: `private/etc` in host mode, the root upper layer in image mode), and on a single-id +container with apt present an `APT::Sandbox::User "root"` override is written next to it. + +Host mode: +1. Bind the container `rootfs/` onto itself (pivot_root needs a mountpoint). +2. Overlay mounts for `/usr`, `/opt` and real compat dirs (`lowerdir=,upperdir,workdir,userxattr`). +3. Compat symlinks recreated (usr-merge). +4. Bind the private dirs `/root`, `/var`, `/etc`; then the overlay for `/var/lib/dpkg` inside the private `/var`. + +Image mode: +1. One overlay on `rootfs/`: lower = `images/NAME/rootfs`, upper = `layers/root/upper`. +2. Mountpoints (`/proc /dev /tmp /run ...`) created inside the merged tree. + +Both modes then: tmpfs `/tmp` and `/run`; `/dev` (tmpfs, bound host nodes, new devpts, `/dev/shm`, symlinks); +`/proc` with read-only remounts and masks from `spec.hpp`; `chdir(rootfs)`, `pivot_root(".", ".")`, lazy +unmount of the old root. + +## Seeding (host mode, once per container) + +`Container::prepareLayout` runs on create and as a heal on every later load; each step leaves a +`seeded.` marker so it runs once and prints what it did and what it skipped: +1. Directory skeleton of every overlaid host dir into its upper layer (`fsutil::mirrorDirs`). +2. Regular files of `/var/lib/dpkg`, `triggers`, `alternatives` copied into the upper (`copyFiles`), then empty lock files. +3. `/etc` copied into `private/etc` (`copyTree`). + +Reasons are in `SPEC.md` (host root-owned files appear as `nobody`). + +## Images + +`ImageStore::add`: lock-free staging dir `images/.tmp-NAME`, download with curl (https only, size and time caps), +sha256 check, `tar -x -p --numeric-owner` executed by `UserNsJob` with the image's id mode (`--no-same-owner` +for single), usage measured, `meta` written, atomic `rename` into place. Any failure removes the staging dir +from inside the id map, so a failed add leaves nothing. `Store::getOrCreate` calls it when a catalog image is +missing, and pins the container's id mode to the image's. ## join flow diff --git a/docs/RESEARCH.md b/docs/RESEARCH.md index 900fa01..4603995 100644 --- a/docs/RESEARCH.md +++ b/docs/RESEARCH.md @@ -42,19 +42,19 @@ Single run set, one machine; no other hardware measured. ## Known gaps and researched solutions -Status of everything in this section: **researched, not implemented**. Probes ran on the +Status per gap is in the table below and in "Resolution log (1.0.0)" at the end. Probes ran on the machine above (cgroup v2, systemd user manager, ext4 root, uid 1000). Probe cgroups were removed afterwards. Rows marked "unmeasured" have no run behind them. -| Id | Gap | Proposed fix | Bump | Code that would change | +| Id | Gap | Fix | Status | Code | |---|---|---|---|---| -| G1 | No memory/cpu/pids limits | Per-container cgroup v2 leaf under the delegated user service | minor | `sandbox.cpp` start/join, new `Cgroup` class, `spec.hpp` | -| G2 | No disk cap on upper dirs | Usage watchdog with `cgroup.freeze` at the hard limit; optional tmpfs upper | minor | `Sandbox::start` parent loop, `spec.hpp` | -| G3 | Single uid mapping | `newuidmap`/`newgidmap` with `/etc/subuid`, fallback to single map | minor, delete path changes | `sandbox.cpp` writeMap, `Store::remove` | -| G4 | Host `/etc` readable | Allowlist-only private `/etc` instead of an overlay of host `/etc` | minor | `spec.hpp` lists, `setupRoot` | -| G5 | `--net host` shares abstract sockets | Keep opt-in; evaluate pasta as an isolated replacement | patch or minor | `network.cpp` | -| G6 | No seccomp, nested userns allowed | Hand-built seccomp-bpf denylist after `no_new_privs` | minor | new `Seccomp` class, `sandbox.cpp` | -| G7 | Host-owned lower dirs are not writable; `chown --reference` of host files fails; `/var` is empty so apt has no databases | Image-based lower layer (extracted rootfs, owned by mapped ids) plus G3; short term: pre-seeded directory skeleton | major (changes layout) | `Container::prepareLayout`, `setupRoot`, new image import | +| G1 | Memory/cpu/pids limits | Cgroup v2 leaf under the delegated user service | **partly done in 1.0.0**: the leaf exists and is used for `cgroup.freeze`; `memory.max`, `pids.max`, `cpu.max` are not set yet | `cgroup.cpp` | +| G2 | Disk cap on upper dirs | Usage watchdog with freeze | **done in 1.0.0** (`--size`) | `Sandbox::watch` | +| G3 | Single uid mapping | `newuidmap`/`newgidmap` with `/etc/subuid`, fallback single | **done in 1.0.0** (`--ids`, default subuid) | `idmap.cpp`, `Store::remove` | +| G4 | Host `/etc` readable | Allowlist-only private `/etc` | **open** for host mode: `/etc` is now a private copy of all readable files (stable, still world-readable secrets); **closed in image mode** (no host files) | `container.cpp` | +| G5 | `--net host` shares abstract sockets | Keep opt-in; evaluate pasta | **open** | `network.cpp` | +| G6 | No seccomp, nested userns allowed | Hand-built seccomp-bpf denylist | **open** | new `Seccomp` class | +| G7 | Host-owned lower dirs unwritable, `chown` of host files fails, empty `/var` | Image layer + subuid; skeleton and seeding for host mode | **done in 1.0.0** | `image.cpp`, `container.cpp` | ### G1: memory, cpu and pids limits (cgroup v2) @@ -161,3 +161,64 @@ Researched ways forward, in order of effort: - Short term, writable skeleton: at create time mirror every directory of the lower dirs into the upper dir with the same mode (directories owned by the user are writable; merged view is unchanged). Directories unreadable to the user (for example `/etc/ssl/private`, mode 700, root-owned) are skipped and stay as they are. Measured effect: writes into `/usr/bin` work, attach time unchanged (41 ms). Unmeasured: C++ implementation speed, and it does not fix the `chown` failure. - Proper fix, image layer: use an extracted distro rootfs (for example from `podman export` or `skopeo copy` plus unpack, both installed here; `mmdebstrap` and `debootstrap` are not) as the lower dir, with ownership shifted into the subuid range (G3). Then the tree is owned by mapped ids, `/var/lib/dpkg` exists, `chown` works, `_apt` exists, and host `/etc` stops leaking (G4). This also removes the dependence on the host distro. Cost: image import step, disk per image (shared lower, so once), and a layout change, hence a major bump. - Alternative to evaluate: `fuse-overlayfs` (installed) supports uid mappings between layers. Unmeasured; adds a FUSE process per container. + +## Resolution log (1.0.0) + +Everything below was built and measured on the machine above (Ubuntu 26.04, kernel 7.0.0-38, ext4 root, +uid 1000, subuid range 100000:65536). The end-to-end suite (`tests/run.sh`) passed 56 of 56. + +### What the field trial broke, and what fixed it + +| Symptom (0.1.3) | Root cause | Fix in 1.0.0 | Result | +|---|---|---|---| +| `mkdir/touch` in `/usr/bin` fails | host dirs show as `nobody`; container root has no right over unmapped owners | `fsutil::mirrorDirs`: every host directory recreated in the upper layer, owned by the container | 13,163 dirs for `/usr` in about 1.1 s (the shell loop took 38 s); `touch /usr/bin/x` works, attach time unchanged | +| `/var/lib/apt/lists/partial is missing`, no dpkg lock | `/var` was empty | `/var` pre-seeded with the apt/dpkg dirs (`kVarDirs`); host `/var/lib/dpkg` overlaid | apt starts | +| `open lock-frontend: Permission denied` | host lock files owned by nobody, mode 640 | empty lock files pre-created in the upper (`kUpperLockFiles`) | lock acquired | +| `dpkg: error creating backup file status-old: Operation not permitted` | dpkg hard-links `status`; kernel `protected_hardlinks` refuses links to files you do not own and cannot write | regular files of `/var/lib/dpkg`, `triggers`, `alternatives` copied into the upper (`kUpperCopyDirs`) | dpkg state writable | +| `chown: changing ownership of '/etc/shells.tmp': Invalid argument` | `chown --reference` of a file owned by an unmapped uid | `/etc` is a private copy owned by the container root (`kCopiedDirs`) | `stat -c %u /etc/shells` is 0 | +| `setegid 65534 failed`, `seteuid 42 failed` (apt `_apt` user) | one mappable id | subuid mapping through `newuidmap`/`newgidmap` (ids 1..65535) | `apt-get update` works with no override; with `--ids single` an `APT::Sandbox::User "root"` override is written instead | +| `apt-get install cowsay` not configured | all of the above | all of the above | install completes, `dpkg --audit` clean, `/usr/games/cowsay` runs, host has no `/usr/games/cowsay` | + +### Image layer + +- Why images are the clean fix: the extracted tree is owned by mapped ids, so nothing needs a skeleton, copies or overrides. Measured: `apk add jq` in Alpine and `apt-get install cowsay curl ca-certificates` in Ubuntu 26.04 complete with a clean `dpkg --audit`; `curl https://example.com` returns 200. +- Sources chosen (checked live with curl, 2026-10-23): + - Alpine: `latest-releases.yaml` lists file name and sha256 per flavor (`alpine-minirootfs-3.24.3-x86_64.tar.gz`, 3.7 MB). + - Ubuntu: `cdimage.ubuntu.com/ubuntu-base/releases//release/SHA256SUMS` plus tarballs (26.04.1 and 24.04.5 present). + - Debian: no usable tarball path found (`debuerreotype/docker-debian-artifacts` returned 404 for `rootfs.tar.xz`; only `.sha256` files resolved), so Debian is supported through `image add URL --sha256`, not through the catalog. + - Not used: `podman export` / `skopeo` (installed here, but one more dependency; a registry client is out of scope). +- Trust model: HTTPS to the publisher plus the checksum file from the same publisher. The Ubuntu `SHA256SUMS.gpg` signature is not checked (unimplemented, listed below). +- Extraction runs inside a user namespace with the id map the containers will use (`UserNsJob`): subuid images keep the real ownership; single-id images use `tar --no-same-owner`. A container's id mode is pinned to its image's. +- Cleanup: files owned by subuids cannot be removed by the host user (`Permission denied`). `Store::remove` and `ImageStore::remove` therefore delete from inside the id map. Measured: deleting containers and images with subuid-owned files works, and a failed add leaves no staging directory. +- Image sizes (hardlinks counted once): Alpine 8.3M, Ubuntu base 97M. The first measurement said 1.3G because uutils-coreutils hard-links one 11 MB binary about 100 times; `diskUsage` now deduplicates by (device, inode). + +### Size cap + +- Mechanism: `Sandbox::watch` polls the init's pidfd with a 10 s timeout and scans usage on every wake. 80% warns, 100% freezes the container's cgroup, below 95% it thaws. +- Measured: `--size 64m`, a 100 MB write: `FROZEN` message at 100M of 64M, tick output stopped; `doos attach cap -- rm /root/big` ran (a second attach is not in the frozen cgroup), the container thawed (`thawed (4.0K used)`) and continued. +- Freezing needs no cgroup controller, only the delegated `user@UID.service` subtree. Without it the cap warns and says so. +- Chosen over a size-capped tmpfs because tmpfs is memory-backed and not persistent; chosen over project quota because that needs root. + +### Timings (5 sequential `attach -- true`, this machine, subuid, one run set) + +| Mode | Per attach | +|---|---| +| host, slirp | about 50 ms | +| image alpine, slirp | about 40 ms | +| image ubuntu, slirp | about 40 ms | +| host, `--net none` | about 10 ms | + +Subuid mapping added about 15 to 25 ms per start compared with 0.1.x (two setuid helper processes). Creating a host-mode container (skeleton, `/etc` copy, dpkg files) took about 1.2 s once. First `--image alpine` use including download and extraction: 0.3 s end to end on this network. + +### Still open, stated plainly + +- G1: `memory.max`, `pids.max`, `cpu.max` are not set. Measured earlier that they work here; not wired in. +- G4: host mode copies every readable file of host `/etc` at creation. Use an image to avoid host files entirely. +- G5, G6: unchanged (see above). +- Size accounting sees only what the host user can read. Directories of other container uids with mode 700 inside the writable layer are not counted (reported only as an internal skip count, not shown). Unmeasured. +- Overshoot: up to one scan interval (10 s) of writes can pass the cap before the freeze; measured 100M written against a 64M cap. +- A second `attach` is outside the frozen cgroup and outside any limit by design (it is the rescue path). +- Host-mode containers drift from the host: the `/etc` copy, dpkg state and skeleton are snapshots at creation. A host package upgrade is visible in `/usr` but not in the container's dpkg database. +- Ubuntu image signature (`SHA256SUMS.gpg`) is not verified; Alpine's yaml is not signature-checked either. +- `image add FILE` without `--sha256` is accepted and prints a note, since the user chose the file. +- Catalog covers x86_64 and aarch64 only; aarch64 is untested. diff --git a/src/cgroup.cpp b/src/cgroup.cpp new file mode 100644 index 0000000..2c73e6d --- /dev/null +++ b/src/cgroup.cpp @@ -0,0 +1,66 @@ +#include "cgroup.hpp" + +#include + +#include +#include + +#include "spec.hpp" +#include "util.hpp" + +namespace fs = std::filesystem; + +namespace doos { + +std::unique_ptr Cgroup::create(const std::string& name, std::string& whyNot) { + try { + std::string self = fsutil::readFile("/proc/self/cgroup"); + std::string marker = "user@" + std::to_string(::geteuid()) + ".service"; + auto at = self.find(marker); + if (self.rfind("0::", 0) != 0 || at == std::string::npos) { + whyNot = "no cgroup v2 delegated to the user manager (" + marker + " not found)"; + return nullptr; + } + fs::path base = fs::path("/sys/fs/cgroup") / self.substr(3, at + marker.size() - 3).substr(1); + fs::path dir = base / std::string(spec::kCgroupDir); + std::error_code ec; + fs::create_directory(dir, ec); + if (ec && ec != std::errc::file_exists) { + whyNot = "cannot create " + dir.string() + ": " + ec.message(); + return nullptr; + } + fs::path leaf = dir / name; + fs::create_directory(leaf, ec); + if (ec && ec != std::errc::file_exists) { + whyNot = "cannot create " + leaf.string() + ": " + ec.message(); + return nullptr; + } + if (::access((leaf / "cgroup.freeze").c_str(), W_OK) != 0) { + whyNot = "cgroup.freeze not writable in " + leaf.string(); + fs::remove(leaf, ec); + return nullptr; + } + return std::unique_ptr(new Cgroup(leaf)); + } catch (const std::exception& e) { + whyNot = e.what(); + return nullptr; + } +} + +Cgroup::~Cgroup() { + // Processes die with the container; give the kernel a moment to empty the group, then remove it. + for (int i = 0; i < 20; ++i) { + std::error_code ec; + if (fs::remove(dir_, ec)) return; + std::this_thread::sleep_for(std::chrono::milliseconds(25)); + } +} + +void Cgroup::add(pid_t pid) const { fsutil::writeFile(dir_ / "cgroup.procs", std::to_string(pid)); } + +void Cgroup::freeze(bool on) const { + fsutil::writeFile(dir_ / "cgroup.freeze", on ? "1" : "0"); + frozen_ = on; +} + +} // namespace doos diff --git a/src/cgroup.hpp b/src/cgroup.hpp new file mode 100644 index 0000000..fa1a4ae --- /dev/null +++ b/src/cgroup.hpp @@ -0,0 +1,30 @@ +#pragma once +#include + +#include +#include +#include + +namespace doos { + +// One cgroup v2 leaf per running container, under the cgroup delegated to the user's systemd manager. +// Used to freeze a container that hit its size cap (freezing needs no controller). Best effort: +// create() returns null when delegation is absent, and the caller must say so. +class Cgroup { +public: + static std::unique_ptr create(const std::string& name, std::string& whyNot); + ~Cgroup(); + Cgroup(const Cgroup&) = delete; + Cgroup& operator=(const Cgroup&) = delete; + + void add(pid_t pid) const; + void freeze(bool on) const; + bool frozen() const { return frozen_; } + +private: + explicit Cgroup(std::filesystem::path dir) : dir_(std::move(dir)) {} + std::filesystem::path dir_; + mutable bool frozen_ = false; +}; + +} // namespace doos diff --git a/src/cli.cpp b/src/cli.cpp index 4b92b59..57e58db 100644 --- a/src/cli.cpp +++ b/src/cli.cpp @@ -1,45 +1,59 @@ #include "cli.hpp" +#include #include +#include #include #include #include "container.hpp" #include "sandbox.hpp" +#include "spec.hpp" +#include "util.hpp" namespace doos { namespace { -// Splits leading --options from the first positional. Everything after the positional is returned as-is. +// Leading --options, then the first positional, then everything else untouched. struct Parsed { - std::optional net; + Options opts; + std::optional sha256; bool yes = false; - std::string name; + std::vector positional; // up to maxPositional, the rest lands in tail std::vector tail; }; -Parsed parse(const std::vector& rest, bool allowNet, bool allowYes) { +struct Allowed { + bool container = false; // --net --image --size --ids + bool yes = false; + bool sha256 = false; +}; + +Parsed parse(const std::vector& rest, Allowed allow, std::size_t wantPositional) { Parsed p; + auto value = [&](std::size_t& i, const std::string& flag) -> std::string { + const std::string& a = rest[i]; + if (a.size() > flag.size() && a.compare(0, flag.size() + 1, flag + "=") == 0) return a.substr(flag.size() + 1); + if (i + 1 >= rest.size()) throw std::runtime_error(flag + " needs a value"); + return rest[++i]; + }; + auto is = [&](const std::string& a, const std::string& flag) { return a == flag || a.rfind(flag + "=", 0) == 0; }; std::size_t i = 0; for (; i < rest.size(); ++i) { const std::string& a = rest[i]; if (a == "--") { ++i; break; } - if (allowNet && a == "--net") { - if (i + 1 >= rest.size()) throw std::runtime_error("--net needs a value (slirp, host, none)"); - p.net = rest[++i]; - } else if (allowNet && a.rfind("--net=", 0) == 0) { - p.net = a.substr(6); - } else if (allowYes && a == "--yes") { - p.yes = true; - } else if (a.size() > 1 && a[0] == '-') { - throw std::runtime_error("unknown option " + a); - } else { - break; - } + if (allow.container && is(a, "--net")) p.opts.net = Network::parse(value(i, "--net")); + else if (allow.container && is(a, "--image")) p.opts.image = value(i, "--image"); + else if (allow.container && is(a, "--size")) p.opts.size = parseSize(value(i, "--size")); + else if (allow.container && is(a, "--ids")) p.opts.ids = IdMap::parse(value(i, "--ids")); + else if (allow.yes && a == "--yes") p.yes = true; + else if (allow.sha256 && is(a, "--sha256")) p.sha256 = value(i, "--sha256"); + else if (a.size() > 1 && a[0] == '-') throw std::runtime_error("unknown option " + a); + else if (p.positional.size() < wantPositional) p.positional.push_back(a); + else break; } - if (i >= rest.size()) throw std::runtime_error("missing container name"); - p.name = rest[i++]; + if (p.positional.size() < wantPositional) throw std::runtime_error("missing argument"); if (i < rest.size() && rest[i] == "--") ++i; p.tail.assign(rest.begin() + static_cast(i), rest.end()); return p; @@ -61,10 +75,20 @@ void Cli::usage(std::FILE* out) { "\n" "usage:\n" " doos list\n" - " doos get_or_create [--net slirp|host|none] NAME\n" - " doos attach [--net slirp|host|none] NAME [-- CMD [ARGS...]]\n" + " doos get_or_create [OPTIONS] NAME\n" + " doos attach [OPTIONS] NAME [-- CMD [ARGS...]]\n" " doos delete [--yes] NAME\n" + " doos image list\n" + " doos image add [--sha256 HEX] SOURCE [NAME]\n" + " doos image delete [--yes] NAME\n" "\n" + "options (fixed at creation, except --size):\n" + " --image NAME start from an image instead of the host system (fetched on demand)\n" + " --size SIZE writable-layer cap, 64m..1t, default 4g (e.g. 512m, 8g)\n" + " --net slirp|host|none network mode, default slirp\n" + " --ids single|subuid id mapping, default: subuid when available\n" + "\n" + "SOURCE for image add: alpine, ubuntu, ubuntu-24.04, an https URL (needs --sha256), or a tar file.\n" "attach creates NAME when missing. delete only reports unless --yes is given.\n" "State lives in $DOOS_HOME or ~/.local/share/doos.\n", out); @@ -78,11 +102,12 @@ int Cli::run() { std::string cmd = args_[0]; for (auto& ch : cmd) if (ch == '-') ch = '_'; std::vector rest(args_.begin() + 1, args_.end()); - if (cmd == "--version" || cmd == "version") { std::puts(DOOS_VERSION); return 0; } + if (cmd == "__version" || cmd == "version") { std::puts(DOOS_VERSION); return 0; } if (cmd == "list") return cmdList(); if (cmd == "get_or_create") return cmdGetOrCreate(rest); if (cmd == "attach") return cmdAttach(rest); if (cmd == "delete") return cmdDelete(rest); + if (cmd == "image") return cmdImage(rest); std::fprintf(stderr, "doos: unknown command '%s'\n\n", args_[0].c_str()); usage(stderr); return 2; @@ -92,40 +117,112 @@ int Cli::cmdList() { Store store = Store::fromEnvironment(); auto all = store.list(); if (all.empty()) { std::puts("no containers"); return 0; } - std::printf("%-*s %-8s %-6s %s\n", 20, "NAME", "STATE", "NET", "AGE"); + std::printf("%-*s %-8s %-6s %-12s %-6s %s\n", 16, "NAME", "STATE", "NET", "IMAGE", "SIZE", "AGE"); for (const auto& c : all) { - std::printf("%-*s %-8s %-6s %s\n", 20, c.name().c_str(), c.runningPid() ? "running" : "stopped", - Network::name(c.net()).c_str(), age(c.created()).c_str()); + std::printf("%-*s %-8s %-6s %-12s %-6s %s\n", 16, c.name().c_str(), c.runningPid() ? "running" : "stopped", + Network::name(c.net()).c_str(), c.imageMode() ? c.image().c_str() : "host", + formatSize(c.sizeCap()).c_str(), age(c.created()).c_str()); } return 0; } int Cli::cmdGetOrCreate(std::vector rest) { - Parsed p = parse(rest, true, false); + Parsed p = parse(rest, {true, false, false}, 1); if (!p.tail.empty()) throw std::runtime_error("get_or_create takes only a name"); - std::optional net; - if (p.net) net = Network::parse(*p.net); Store store = Store::fromEnvironment(); - auto [c, created] = store.getOrCreate(p.name, net); + auto [c, created] = store.getOrCreate(p.positional[0], p.opts); std::printf("%s %s\n", created ? "created" : "exists", c.name().c_str()); return 0; } int Cli::cmdAttach(std::vector rest) { - Parsed p = parse(rest, true, false); - std::optional net; - if (p.net) net = Network::parse(*p.net); + Parsed p = parse(rest, {true, false, false}, 1); Store store = Store::fromEnvironment(); - auto [c, created] = store.getOrCreate(p.name, net); + auto [c, created] = store.getOrCreate(p.positional[0], p.opts); (void)created; return Sandbox(std::move(c), std::move(p.tail)).run(); } int Cli::cmdDelete(std::vector rest) { - Parsed p = parse(rest, false, true); + Parsed p = parse(rest, {false, true, false}, 1); if (!p.tail.empty()) throw std::runtime_error("delete takes only a name"); Store store = Store::fromEnvironment(); - std::puts(store.remove(p.name, p.yes).c_str()); + std::puts(store.remove(p.positional[0], p.yes).c_str()); + return 0; +} + +int Cli::cmdImage(std::vector rest) { + if (rest.empty()) throw std::runtime_error("image needs a subcommand: list, add, delete"); + std::string sub = rest[0]; + std::vector tail(rest.begin() + 1, rest.end()); + if (sub == "list") return cmdImageList(); + if (sub == "add") return cmdImageAdd(tail); + if (sub == "delete") return cmdImageDelete(tail); + throw std::runtime_error("unknown image subcommand '" + sub + "' (list, add, delete)"); +} + +int Cli::cmdImageList() { + Store store = Store::fromEnvironment(); + auto installed = store.images().list(); + std::printf("%-*s %-6s %-7s %s\n", 16, "IMAGE", "SIZE", "IDS", "STATE"); + for (const auto& i : installed) { + std::printf("%-*s %-6s %-7s %s\n", 16, i.name.c_str(), formatSize(i.bytes).c_str(), IdMap::name(i.ids).c_str(), + "installed"); + } + for (const auto& n : Catalog::names()) { + bool have = false; + for (const auto& i : installed) have = have || i.name == n; + if (!have) std::printf("%-*s %-6s %-7s %s\n", 16, n.c_str(), "-", "-", "available"); + } + return 0; +} + +int Cli::cmdImageAdd(std::vector rest) { + Parsed p = parse(rest, {false, false, true}, 1); + if (p.tail.size() > 1) throw std::runtime_error("image add takes SOURCE and an optional NAME"); + const std::string& source = p.positional[0]; + ImageSource src; + bool isUrl = source.rfind("https://", 0) == 0; + bool isFile = !isUrl && source.find('/') != std::string::npos; + if (source.rfind("http://", 0) == 0) throw std::runtime_error("plain http is refused; use https"); + if (!isUrl && !isFile && Catalog::knows(ImageStore::normalize(source))) { + src = Catalog::resolve(ImageStore::normalize(source)); + if (p.sha256) src.sha256 = *p.sha256; + } else if (isUrl) { + if (!p.sha256) throw std::runtime_error("a URL image needs --sha256 HEX (unverifiable input is rejected)"); + src.location = source; + src.sha256 = *p.sha256; + } else { + if (!std::filesystem::is_regular_file(source)) { + throw std::runtime_error("'" + source + "' is neither a catalog image (alpine, ubuntu, ubuntu-VERSION), an https URL, nor a file"); + } + src.location = std::filesystem::absolute(source).string(); + src.isFile = true; + if (p.sha256) src.sha256 = *p.sha256; + } + if (!p.tail.empty()) { + src.name = ImageStore::normalize(p.tail[0]); + } else if (src.name.empty()) { + std::string base = std::filesystem::path(src.location).filename().string(); + for (const char* ext : {".tar.gz", ".tar.xz", ".tar.zst", ".tgz", ".tar"}) { + std::string e = ext; + if (base.size() > e.size() && base.compare(base.size() - e.size(), e.size(), e) == 0) base.erase(base.size() - e.size()); + } + for (auto& ch : base) ch = std::isalnum(static_cast(ch)) ? static_cast(std::tolower(ch)) : '-'; + src.name = base.substr(0, spec::kMaxNameLen); + } + Store store = Store::fromEnvironment(); + ImageInfo info = store.images().add(src, IdMap::best()); + std::printf("added image %s (%s, ids %s)\n", info.name.c_str(), formatSize(info.bytes).c_str(), IdMap::name(info.ids).c_str()); + return 0; +} + +int Cli::cmdImageDelete(std::vector rest) { + Parsed p = parse(rest, {false, true, false}, 1); + if (!p.tail.empty()) throw std::runtime_error("image delete takes only a name"); + Store store = Store::fromEnvironment(); + std::string name = ImageStore::normalize(p.positional[0]); + std::puts(store.images().remove(name, p.yes, store.containersUsing(name)).c_str()); return 0; } diff --git a/src/cli.hpp b/src/cli.hpp index 8a48ad0..ee3ea03 100644 --- a/src/cli.hpp +++ b/src/cli.hpp @@ -15,6 +15,10 @@ private: int cmdGetOrCreate(std::vector rest); int cmdAttach(std::vector rest); int cmdDelete(std::vector rest); + int cmdImage(std::vector rest); + int cmdImageList(); + int cmdImageAdd(std::vector rest); + int cmdImageDelete(std::vector rest); static void usage(std::FILE* out); std::vector args_; diff --git a/src/container.cpp b/src/container.cpp index 2603afc..d64b351 100644 --- a/src/container.cpp +++ b/src/container.cpp @@ -1,6 +1,9 @@ #include "container.hpp" +#include + #include +#include #include #include #include @@ -65,39 +68,158 @@ void Container::validateName(const std::string& name) { fs::path Container::layer(const std::string& hostDir) const { return dir_ / "layers" / slug(hostDir); } fs::path Container::privateDir(const std::string& hostDir) const { return dir_ / "private" / slug(hostDir); } +fs::path Container::etcDir() const { + return imageMode() ? layer("/root") / "upper" / "etc" : privateDir("/etc"); +} + +void Container::seed(const std::string& marker, const std::string& label, + const std::function& work) const { + fs::path m = dir_ / ("seeded." + marker); + if (fs::exists(m)) return; + fsutil::SeedStats st = work(); + std::fprintf(stderr, "doos: seeded %s: %zu entries%s%s\n", label.c_str(), st.made, + st.skipped ? ", unreadable skipped: " : "", st.skipped ? std::to_string(st.skipped).c_str() : ""); + fsutil::writeFile(m, "ok\n"); +} + void Container::prepareLayout() const { + fsutil::mkdirs(rootfs()); + if (imageMode()) { + fsutil::mkdirs(layer("/root") / "upper"); + fsutil::mkdirs(layer("/root") / "work"); + return; + } + std::vector overlays; auto addOverlay = [&](std::string_view d) { std::string dir(d); - fsutil::mkdirs(rootfs() / slug(d)); fsutil::mkdirs(layer(dir) / "upper"); fsutil::mkdirs(layer(dir) / "work"); + overlays.push_back(dir); }; - for (auto d : spec::kOverlayDirs) addOverlay(d); + for (auto d : spec::kOverlayDirs) { + fsutil::mkdirs(rootfs() / slug(d)); + addOverlay(d); + } for (auto d : spec::kCompatDirs) { std::error_code ec; - if (fs::is_directory(fs::symlink_status(std::string(d), ec)) && !ec) addOverlay(d); + if (fs::is_directory(fs::symlink_status(std::string(d), ec)) && !ec) { + fsutil::mkdirs(rootfs() / slug(d)); + addOverlay(d); + } } for (auto d : spec::kPrivateDirs) { fsutil::mkdirs(rootfs() / slug(d)); fsutil::mkdirs(privateDir(std::string(d)), 0755); } + for (auto d : spec::kVarDirs) fsutil::mkdirs(privateDir("/var") / d.path, d.mode); + for (auto d : spec::kVarOverlayDirs) { + std::error_code ec; + if (!fs::is_directory(std::string(d), ec)) continue; + fsutil::mkdirs(privateDir("/var") / std::string(d).substr(5)); // below /var + addOverlay(d); + } for (auto d : spec::kEmptyDirs) fsutil::mkdirs(rootfs() / slug(d)); + + // Host dirs are owned by an unmapped uid (nobody) in the container: without a directory skeleton in the + // upper layer nothing could ever be created in them (package installs, tool caches). + for (const auto& o : overlays) { + std::error_code ec; + if (!fs::is_directory(o, ec)) continue; + std::string key = slug(o); + std::replace(key.begin(), key.end(), '/', '_'); + seed("skeleton-" + key, "directory skeleton " + o, [&] { return fsutil::mirrorDirs(o, layer(o) / "upper"); }); + } + for (auto d : spec::kUpperCopyDirs) { + for (const auto& o : overlays) { + if (std::string(d) != o && std::string(d).rfind(o + "/", 0) != 0) continue; + std::string rel = std::string(d).substr(o.size()); + std::string key = slug(d); + std::replace(key.begin(), key.end(), '/', '_'); + fs::path target = layer(o) / "upper" / (rel.empty() ? "" : rel.substr(1)); + std::error_code ec; + if (!fs::is_directory(std::string(d), ec) || !fs::is_directory(target, ec)) continue; + seed("files-" + key, "files of " + std::string(d), [&] { return fsutil::copyFiles(std::string(d), target); }); + } + } + for (auto f : spec::kUpperLockFiles) { + for (const auto& o : overlays) { + if (std::string(f).rfind(o + "/", 0) != 0) continue; + fs::path target = layer(o) / "upper" / std::string(f).substr(o.size() + 1); + std::error_code ec; + if (!fs::is_directory(target.parent_path(), ec) || fs::exists(target, ec)) continue; + fsutil::writeFile(target, ""); + ::chmod(target.c_str(), 0640); + } + } + for (auto d : spec::kCopiedDirs) { + std::error_code ec; + if (!fs::is_directory(std::string(d), ec)) continue; + seed("copy-" + std::string(slug(d)), "copy of " + std::string(d), + [&] { return fsutil::copyTree(std::string(d), privateDir(std::string(d))); }); + } } -void Container::create(NetMode net) { +namespace { +void findUppers(const fs::path& dir, std::size_t depth, std::vector& out) { + std::error_code ec; + if (depth > 6) return; + for (const auto& e : fs::directory_iterator(dir, ec)) { + if (!e.is_directory(ec) || e.is_symlink(ec)) continue; + std::string n = e.path().filename().string(); + if (n == "upper") out.push_back(e.path()); + else if (n != "work") findUppers(e.path(), depth + 1, out); + } +} +} // namespace + +std::uint64_t Container::usedBytes(std::size_t& skipped) const { + std::vector dirs; + std::error_code ec; + if (fs::exists(dir_ / "layers", ec)) findUppers(dir_ / "layers", 0, dirs); + if (fs::exists(dir_ / "private", ec)) dirs.push_back(dir_ / "private"); + std::uint64_t total = 0; + for (const auto& d : dirs) total += fsutil::diskUsage(d, skipped); + return total; +} + +void Container::writeMeta() const { + fsutil::writeFile(metaFile(), "net=" + Network::name(net_) + "\ncreated=" + std::to_string(created_) + + "\nversion=" DOOS_VERSION "\nimage=" + image_ + "\nids=" + IdMap::name(ids_) + + "\nsize=" + std::to_string(size_) + "\n"); +} + +void Container::setSizeCap(std::uint64_t bytes) { + if (bytes < spec::kMinSizeBytes || bytes > spec::kMaxSizeBytes) { + throw std::runtime_error("size must be between " + formatSize(spec::kMinSizeBytes) + " and " + + formatSize(spec::kMaxSizeBytes)); + } + size_ = bytes; + writeMeta(); +} + +void Container::create(const NetMode& net, const std::string& image, IdMode ids, std::uint64_t size) { net_ = net; + image_ = image; + ids_ = ids; + size_ = size; created_ = std::time(nullptr); fsutil::mkdirs(dir_); prepareLayout(); - fsutil::writeFile(metaFile(), "net=" + Network::name(net_) + "\ncreated=" + std::to_string(created_) + - "\nversion=" DOOS_VERSION "\n"); + writeMeta(); } void Container::load() { + readMeta(); + prepareLayout(); +} + +void Container::readMeta() { auto kv = parseKv(fsutil::readFile(metaFile())); net_ = Network::parse(kv.count("net") ? kv["net"] : "slirp"); created_ = kv.count("created") ? std::stol(kv["created"]) : 0; - prepareLayout(); // heals a container created by an older layout + image_ = kv.count("image") ? kv["image"] : ""; + ids_ = IdMap::parse(kv.count("ids") ? kv["ids"] : "single"); + size_ = kv.count("size") ? std::stoull(kv["size"]) : spec::kDefaultSizeBytes; } std::optional Container::runningPid() const { @@ -148,7 +270,7 @@ std::vector Store::list() const { try { Container::validateName(name); Container c(e.path(), name); - c.load(); + c.readMeta(); out.push_back(std::move(c)); } catch (const std::exception& ex) { std::fprintf(stderr, "doos: skipping %s: %s\n", name.c_str(), ex.what()); @@ -159,18 +281,39 @@ std::vector Store::list() const { return out; } -std::pair Store::getOrCreate(const std::string& name, std::optional net) { +std::vector Store::containersUsing(const std::string& image) const { + std::vector out; + for (const auto& c : list()) { + if (c.image() == image) out.push_back(c.name()); + } + return out; +} + +std::pair Store::getOrCreate(const std::string& name, const Options& opts) { Container::validateName(name); fs::path dir = containersDir() / name; Container c(dir, name); std::error_code ec; + std::optional image; + if (opts.image && !opts.image->empty()) image = ImageStore::normalize(*opts.image); + + auto verifyExisting = [&] { + auto fixed = [&](const char* what, const std::string& have, const std::string& want) { + if (have != want) { + throw std::runtime_error("container '" + name + "' exists with " + what + "=" + have + + "; this is fixed at creation"); + } + }; + if (opts.net) fixed("net", Network::name(c.net()), Network::name(*opts.net)); + if (image) fixed("image", c.image().empty() ? "none" : c.image(), *image); + if (opts.ids) fixed("ids", IdMap::name(c.ids()), IdMap::name(*opts.ids)); + if (opts.size && *opts.size != c.sizeCap()) c.setSizeCap(*opts.size); + }; + if (fs::exists(dir / "meta", ec)) { FileLock lock(c.lockFile(), spec::kLockTimeoutMs); c.load(); - if (net && *net != c.net()) { - throw std::runtime_error("container '" + name + "' exists with net=" + Network::name(c.net()) + - "; network mode is fixed at creation"); - } + verifyExisting(); return {c, false}; } std::size_t count = 0; @@ -179,13 +322,38 @@ std::pair Store::getOrCreate(const std::string& name, std::opti throw std::runtime_error("container limit reached (" + std::to_string(spec::kMaxContainers) + "); delete one first"); } + + // Resolve the image first (may download); ids follow the image so the layer ownership matches. + IdMode ids = opts.ids.value_or(IdMap::best()); + if (image) { + ImageStore store = images(); + if (!store.has(*image)) { + if (!Catalog::knows(*image)) { + throw std::runtime_error("unknown image '" + *image + "'. Add it: doos image add SOURCE [NAME] (catalog: alpine, ubuntu)"); + } + std::fprintf(stderr, "doos: image '%s' not present, fetching it\n", image->c_str()); + store.add(Catalog::resolve(*image), ids); + } + IdMode imageIds = store.ids(*image); + if (opts.ids && *opts.ids != imageIds) { + throw std::runtime_error("image '" + *image + "' was extracted with ids=" + IdMap::name(imageIds) + + "; use the same, or add the image again under another name"); + } + ids = imageIds; + } + std::uint64_t size = opts.size.value_or(spec::kDefaultSizeBytes); + if (size < spec::kMinSizeBytes || size > spec::kMaxSizeBytes) { + throw std::runtime_error("size must be between " + formatSize(spec::kMinSizeBytes) + " and " + + formatSize(spec::kMaxSizeBytes)); + } fsutil::mkdirs(dir, 0700); FileLock lock(c.lockFile(), spec::kLockTimeoutMs); if (fs::exists(dir / "meta", ec)) { // lost a creation race c.load(); + verifyExisting(); return {c, false}; } - c.create(net.value_or(NetMode::Slirp)); + c.create(opts.net.value_or(NetMode::Slirp), image.value_or(""), ids, size); return {c, true}; } @@ -207,7 +375,20 @@ std::string Store::remove(const std::string& name, bool confirmed) { std::size_t n = fsutil::countEntries(dir, 1000000); return "dry run: would delete " + dir.string() + " (" + std::to_string(n) + " entries). Add --yes to confirm."; } - fsutil::removeTree(dir); + IdMode ids = IdMode::Single; + try { + c.readMeta(); + ids = c.ids(); + } catch (const std::exception&) { + // damaged or half-created container: remove as plain user, files may be left behind loudly + } + if (ids == IdMode::Subuid) { + // files owned by subuids can only be removed from inside the id map + int rc = UserNsJob::run(ids, [&] { fsutil::removeTree(dir); return 0; }); + if (rc != 0) throw std::runtime_error("could not remove all files of '" + name + "'"); + } else { + fsutil::removeTree(dir); + } return "deleted " + name; } diff --git a/src/container.hpp b/src/container.hpp index ecf384a..9c2a757 100644 --- a/src/container.hpp +++ b/src/container.hpp @@ -1,15 +1,28 @@ #pragma once #include +#include #include +#include #include #include #include +#include "idmap.hpp" +#include "image.hpp" #include "network.hpp" +#include "util.hpp" namespace doos { +// What the user asked for. Unset fields mean "default on create, no constraint on existing". +struct Options { + std::optional net; + std::optional image; + std::optional ids; + std::optional size; +}; + class Container { public: Container(std::filesystem::path dir, std::string name) : dir_(std::move(dir)), name_(std::move(name)) {} @@ -24,26 +37,42 @@ public: std::filesystem::path lockFile() const { return dir_ / "lock"; } std::filesystem::path pidFile() const { return dir_ / "pid"; } std::filesystem::path metaFile() const { return dir_ / "meta"; } + // Where /etc lives on disk for this container (resolv.conf and tool config are written there). + std::filesystem::path etcDir() const; NetMode net() const { return net_; } long created() const { return created_; } + const std::string& image() const { return image_; } + bool imageMode() const { return !image_.empty(); } + IdMode ids() const { return ids_; } + std::uint64_t sizeCap() const { return size_; } + void setSizeCap(std::uint64_t bytes); - void create(NetMode net); // writes layout and meta - void load(); // reads meta + void create(const NetMode& net, const std::string& image, IdMode ids, std::uint64_t size); + void readMeta(); // meta only, no side effects + void load(); // readMeta + heal the layout // Process id (outside view) of the container's init, if it is alive. std::optional runningPid() const; void recordPid(pid_t pid) const; void clearPid() const; - // Creates the (empty) mountpoint skeleton and layer dirs. Idempotent. + // Mountpoints, layer dirs, and one-time seeding (skeleton, /etc copy). Idempotent. void prepareLayout() const; + // Bytes the container wrote (writable layers and private dirs), as the host user can see them. + std::uint64_t usedBytes(std::size_t& skipped) const; private: + void writeMeta() const; + void seed(const std::string& marker, const std::string& label, const std::function& work) const; + std::filesystem::path dir_; std::string name_; NetMode net_ = NetMode::Slirp; long created_ = 0; + std::string image_; + IdMode ids_ = IdMode::Single; + std::uint64_t size_ = 0; }; class Store { @@ -52,11 +81,13 @@ public: static Store fromEnvironment(); std::vector list() const; - // Returns the container and whether it was created by this call. - std::pair getOrCreate(const std::string& name, std::optional net); + // Returns the container and whether it was created by this call. Pulls a catalog image on demand. + std::pair getOrCreate(const std::string& name, const Options& opts); // Dry-run unless confirmed. Returns a human-readable report line. std::string remove(const std::string& name, bool confirmed); + ImageStore images() const { return ImageStore(root_ / "images"); } + std::vector containersUsing(const std::string& image) const; const std::filesystem::path& root() const { return root_; } private: diff --git a/src/idmap.cpp b/src/idmap.cpp new file mode 100644 index 0000000..3eff5fd --- /dev/null +++ b/src/idmap.cpp @@ -0,0 +1,148 @@ +#include "idmap.hpp" + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "spec.hpp" +#include "util.hpp" + +namespace doos { + +namespace { + +void writeProcFile(pid_t pid, const char* file, const std::string& content) { + std::string path = "/proc/" + std::to_string(pid) + "/" + file; + int fd = ::open(path.c_str(), O_WRONLY | O_CLOEXEC); + if (fd < 0) throw SysError("open " + path); + Fd guard(fd); + if (::write(fd, content.data(), content.size()) != static_cast(content.size())) { + throw SysError("write " + path + " (user namespaces blocked? see apparmor/doos and docs/RESEARCH.md)"); + } +} + +bool onPath(std::string_view bin) { + const char* path = std::getenv("PATH"); + std::stringstream ss(path ? path : "/usr/bin:/bin"); + for (std::string dir; std::getline(ss, dir, ':');) { + std::string full = dir + "/" + std::string(bin); + if (::access(full.c_str(), X_OK) == 0) return true; + } + return false; +} + +} // namespace + +IdMode IdMap::parse(const std::string& s) { + if (s == "single") return IdMode::Single; + if (s == "subuid") return IdMode::Subuid; + throw std::runtime_error("unknown id mode '" + s + "' (use single or subuid)"); +} + +std::string IdMap::name(IdMode m) { return m == IdMode::Subuid ? "subuid" : "single"; } + +std::optional IdMap::range(const char* file) { + std::ifstream in(file); + if (!in) return std::nullopt; + std::string user; + if (const passwd* pw = ::getpwuid(::geteuid())) user = pw->pw_name; + std::string uid = std::to_string(::geteuid()); + for (std::string line; std::getline(in, line);) { + std::vector f; + std::stringstream ss(line); + for (std::string t; std::getline(ss, t, ':');) f.push_back(t); + if (f.size() != 3 || (f[0] != user && f[0] != uid)) continue; + try { + SubRange r{std::stoul(f[1]), std::stoul(f[2])}; + if (r.count > 0) return r; + } catch (const std::exception&) { + } + } + return std::nullopt; +} + +bool IdMap::subuidAvailable() { + return range("/etc/subuid") && range("/etc/subgid") && onPath(spec::kNewUidMap) && onPath(spec::kNewGidMap); +} + +IdMode IdMap::best() { return subuidAvailable() ? IdMode::Subuid : IdMode::Single; } + +void IdMap::apply(pid_t pid, IdMode mode) { + std::string uid = std::to_string(::geteuid()), gid = std::to_string(::getegid()); + if (mode == IdMode::Single) { + writeProcFile(pid, "setgroups", "deny"); + writeProcFile(pid, "uid_map", "0 " + uid + " 1\n"); + writeProcFile(pid, "gid_map", "0 " + gid + " 1\n"); + return; + } + auto u = range("/etc/subuid"); + auto g = range("/etc/subgid"); + if (!u || !g || !onPath(spec::kNewUidMap) || !onPath(spec::kNewGidMap)) { + throw std::runtime_error("subuid mapping unavailable: need /etc/subuid, /etc/subgid, newuidmap and newgidmap " + "(install uidmap, or use --ids single)"); + } + auto n = [](const SubRange& r) { return std::to_string(std::min(r.count, spec::kMaxIdRange - 1)); }; + std::string p = std::to_string(pid); + ProcResult a = runProcess({std::string(spec::kNewUidMap), p, "0", uid, "1", "1", std::to_string(u->start), n(*u)}); + if (a.code != 0) throw std::runtime_error("newuidmap failed (exit " + std::to_string(a.code) + ")"); + ProcResult b = runProcess({std::string(spec::kNewGidMap), p, "0", gid, "1", "1", std::to_string(g->start), n(*g)}); + if (b.code != 0) throw std::runtime_error("newgidmap failed (exit " + std::to_string(b.code) + ")"); +} + +namespace { +struct JobCtx { + const std::function* fn; + int syncFd; +}; + +int jobMain(void* arg) { + auto* c = static_cast(arg); + ::prctl(PR_SET_PDEATHSIG, SIGKILL); + char go = 0; + if (::read(c->syncFd, &go, 1) != 1 || go != 'G') ::_exit(125); + ::close(c->syncFd); + int code = 125; + try { + code = (*c->fn)(); + } catch (const std::exception& e) { + ::dprintf(2, "doos: %s\n", e.what()); + } + ::_exit(code); +} +} // namespace + +int UserNsJob::run(IdMode mode, const std::function& fn) { + int sync[2]; + if (::pipe2(sync, O_CLOEXEC) != 0) throw SysError("pipe"); + Fd rd(sync[0]), wr(sync[1]); + std::vector stack(1 << 20); + JobCtx ctx{&fn, rd.get()}; + pid_t child = ::clone(jobMain, stack.data() + stack.size(), CLONE_NEWUSER | SIGCHLD, &ctx); + if (child < 0) throw SysError("clone (user namespaces available?)"); + try { + IdMap::apply(child, mode); + } catch (...) { + ::kill(child, SIGKILL); + ::waitpid(child, nullptr, 0); + throw; + } + char go = 'G'; + if (::write(wr.get(), &go, 1) != 1) throw SysError("sync job"); + wr.reset(); + int st = 0; + while (::waitpid(child, &st, 0) < 0) { + if (errno != EINTR) throw SysError("waitpid"); + } + return WIFEXITED(st) ? WEXITSTATUS(st) : 128 + WTERMSIG(st); +} + +} // namespace doos diff --git a/src/idmap.hpp b/src/idmap.hpp new file mode 100644 index 0000000..034d138 --- /dev/null +++ b/src/idmap.hpp @@ -0,0 +1,41 @@ +#pragma once +#include + +#include +#include +#include + +namespace doos { + +// How container ids map to host ids. +// single: container 0 -> invoking user, nothing else mappable. +// subuid: container 0 -> invoking user, 1..N -> the user's /etc/subuid and /etc/subgid range (newuidmap). +enum class IdMode { Single, Subuid }; + +struct SubRange { + unsigned long start = 0; + unsigned long count = 0; +}; + +class IdMap { +public: + static IdMode parse(const std::string& s); // "single" or "subuid" + static std::string name(IdMode m); + // Subuid when both ranges and both helper binaries exist, else Single. + static IdMode best(); + static bool subuidAvailable(); + // Applies the map to the user namespace of `pid`. Throws with the reason when it cannot. + static void apply(pid_t pid, IdMode mode); + +private: + static std::optional range(const char* file); +}; + +// Runs `fn` as root of a fresh user namespace carrying the same id map as containers of `mode`. +// Needed to touch files owned by subuids (extract images, delete trees). Returns fn's exit code. +class UserNsJob { +public: + static int run(IdMode mode, const std::function& fn); +}; + +} // namespace doos diff --git a/src/image.cpp b/src/image.cpp new file mode 100644 index 0000000..5971edb --- /dev/null +++ b/src/image.cpp @@ -0,0 +1,269 @@ +#include "image.hpp" + +#include +#include + +#include +#include +#include +#include +#include + +#include "spec.hpp" +#include "util.hpp" + +namespace fs = std::filesystem; + +namespace doos { + +namespace { + +std::string cpuArch(bool debianStyle) { + utsname u; + if (::uname(&u) != 0) throw SysError("uname"); + std::string m = u.machine; + if (m == "x86_64") return debianStyle ? "amd64" : "x86_64"; + if (m == "aarch64") return debianStyle ? "arm64" : "aarch64"; + throw std::runtime_error("no catalog images for architecture " + m + "; use a file or URL with --sha256"); +} + +std::string fetchText(const std::string& url) { + ProcResult r = runProcess({"curl", "--fail", "--silent", "--show-error", "--location", "--proto", "=https", + "--max-time", "60", url}, + true); + if (r.code != 0) throw std::runtime_error("download failed: " + url + " (curl exit " + std::to_string(r.code) + ")"); + return r.out; +} + +bool isHex64(const std::string& s) { + return s.size() == 64 && std::all_of(s.begin(), s.end(), [](unsigned char c) { return std::isxdigit(c); }); +} + +std::vector versionParts(const std::string& v) { + std::vector out; + std::stringstream ss(v); + for (std::string t; std::getline(ss, t, '.');) out.push_back(t.empty() ? 0 : std::stoul(t)); + return out; +} + +std::string lowerCase(std::string s) { + for (auto& c : s) c = static_cast(std::tolower(static_cast(c))); + return s; +} + +} // namespace + +std::vector Catalog::names() { return {"alpine", "ubuntu", "ubuntu-24.04"}; } + +bool Catalog::knows(const std::string& n) { + return n == "alpine" || n == "ubuntu" || (n.rfind("ubuntu-", 0) == 0 && n.size() > 7); +} + +ImageSource Catalog::resolve(const std::string& n) { + ImageSource src; + src.name = n; + if (n == "alpine") { + std::string arch = cpuArch(false); + std::string base = std::string(spec::kAlpineBase) + arch + "/"; + std::string yaml = fetchText(base + "latest-releases.yaml"); + std::istringstream in(yaml); + std::string file, sha; + bool inBlock = false; + for (std::string line; std::getline(in, line);) { + if (line.find("flavor: alpine-minirootfs") != std::string::npos) { inBlock = true; continue; } + if (inBlock && line.rfind("-", 0) == 0) break; + auto val = [&](const char* key) { + auto p = line.find(key); + return p == std::string::npos ? std::string() : line.substr(p + std::string(key).size()); + }; + if (inBlock && file.empty()) file = val("file: "); + if (inBlock && sha.empty()) sha = val("sha256: "); + } + if (file.empty() || !isHex64(sha) || file.find('/') != std::string::npos) { + throw std::runtime_error("cannot find alpine-minirootfs in latest-releases.yaml"); + } + src.location = base + file; + src.sha256 = sha; + return src; + } + if (n == "ubuntu" || n.rfind("ubuntu-", 0) == 0) { + std::string ver = n == "ubuntu" ? std::string(spec::kUbuntuDefaultVersion) : n.substr(7); + if (ver.empty() || !std::all_of(ver.begin(), ver.end(), [](unsigned char c) { return std::isdigit(c) || c == '.'; })) { + throw std::runtime_error("invalid ubuntu version '" + ver + "' (example: ubuntu-24.04)"); + } + std::string arch = cpuArch(true); + std::string base = std::string(spec::kUbuntuBase) + ver + "/release/"; + std::string sums = fetchText(base + "SHA256SUMS"); + std::string suffix = "-base-" + arch + ".tar.gz"; + std::string bestFile, bestSha; + std::vector bestVer; + std::istringstream in(sums); + for (std::string line; std::getline(in, line);) { + auto sp = line.find(" *"); + if (sp == std::string::npos) continue; + std::string sha = line.substr(0, sp), file = line.substr(sp + 2); + if (file.rfind("ubuntu-base-", 0) != 0 || file.size() <= suffix.size() || + file.compare(file.size() - suffix.size(), suffix.size(), suffix) != 0 || !isHex64(sha)) { + continue; + } + auto v = versionParts(file.substr(12, file.size() - 12 - suffix.size())); + if (bestFile.empty() || v > bestVer) { bestFile = file; bestSha = sha; bestVer = v; } + } + if (bestFile.empty()) throw std::runtime_error("no ubuntu-base " + ver + " for " + arch + " found at " + base); + src.location = base + bestFile; + src.sha256 = bestSha; + return src; + } + throw std::runtime_error("unknown catalog image '" + n + "' (known: alpine, ubuntu, ubuntu-VERSION)"); +} + +std::string ImageStore::normalize(const std::string& name) { + std::string s = lowerCase(name); + std::replace(s.begin(), s.end(), ':', '-'); + return s; +} + +void ImageStore::validateName(const std::string& name) { + bool ok = !name.empty() && name.size() <= spec::kMaxNameLen && std::isalnum(static_cast(name[0])); + for (char c : name) ok = ok && (std::islower(static_cast(c)) || std::isdigit(static_cast(c)) || + c == '-' || c == '_' || c == '.'); + if (!ok) throw std::runtime_error("invalid image name '" + name + "': use [a-z0-9][a-z0-9._-]{0,31}"); +} + +ImageStore::ImageStore(fs::path root) : root_(std::move(root)) { fsutil::mkdirs(root_, 0700); } + +bool ImageStore::has(const std::string& name) const { return fs::exists(dir(name) / "meta"); } +fs::path ImageStore::rootfs(const std::string& name) const { return dir(name) / "rootfs"; } + +namespace { +std::map readMeta(const fs::path& p) { + std::map kv; + std::istringstream in(fsutil::readFile(p)); + for (std::string line; std::getline(in, line);) { + auto eq = line.find('='); + if (eq != std::string::npos) kv[line.substr(0, eq)] = line.substr(eq + 1); + } + return kv; +} +} // namespace + +IdMode ImageStore::ids(const std::string& name) const { + auto kv = readMeta(dir(name) / "meta"); + return IdMap::parse(kv.count("ids") ? kv["ids"] : "single"); +} + +std::vector ImageStore::list() const { + std::vector out; + for (const auto& e : fs::directory_iterator(root_)) { + std::string name = e.path().filename().string(); + if (!e.is_directory() || name.rfind(".", 0) == 0 || !has(name)) continue; + auto kv = readMeta(e.path() / "meta"); + ImageInfo i; + i.name = name; + i.source = kv["source"]; + i.ids = IdMap::parse(kv.count("ids") ? kv["ids"] : "single"); + i.created = kv.count("created") ? std::stol(kv["created"]) : 0; + i.bytes = kv.count("bytes") ? std::stoull(kv["bytes"]) : 0; + out.push_back(i); + if (out.size() >= spec::kMaxImages) break; + } + std::sort(out.begin(), out.end(), [](const ImageInfo& a, const ImageInfo& b) { return a.name < b.name; }); + return out; +} + +ImageInfo ImageStore::add(const ImageSource& src, IdMode ids) { + validateName(src.name); + if (has(src.name)) throw std::runtime_error("image '" + src.name + "' already exists"); + std::size_t count = 0; + for (const auto& e : fs::directory_iterator(root_)) count += (e.is_directory() && has(e.path().filename().string())) ? 1 : 0; + if (count >= spec::kMaxImages) { + throw std::runtime_error("image limit reached (" + std::to_string(spec::kMaxImages) + "); delete one first"); + } + for (const char* tool : {"curl", "tar", "sha256sum"}) { + if (runProcess({"sh", "-c", std::string("command -v ") + tool}, true, true).code != 0) { + throw std::runtime_error(std::string("required tool not found: ") + tool); + } + } + fs::path tmp = root_ / (".tmp-" + src.name); + { + std::error_code ec; + if (fs::exists(tmp, ec)) { + int rc = UserNsJob::run(ids, [&] { fsutil::removeTree(tmp); return 0; }); + if (rc != 0) throw std::runtime_error("cannot clean stale " + tmp.string()); + } + } + fsutil::mkdirs(tmp, 0700); + auto cleanup = [&] { UserNsJob::run(ids, [&] { try { fsutil::removeTree(tmp); } catch (...) { return 1; } return 0; }); }; + try { + fs::path archive; + if (src.isFile) { + archive = src.location; + if (!fs::is_regular_file(archive)) throw std::runtime_error("not a file: " + archive.string()); + } else { + archive = tmp / "download"; + std::fprintf(stderr, "downloading %s\n", src.location.c_str()); + ProcResult d = runProcess({"curl", "--fail", "--location", "--proto", "=https", "--progress-bar", + "--max-filesize", std::to_string(spec::kMaxImageDownloadBytes), "--max-time", + std::to_string(spec::kDownloadTimeoutSec), "--output", archive.string(), src.location}); + if (d.code != 0) throw std::runtime_error("download failed (curl exit " + std::to_string(d.code) + ")"); + } + if (!src.sha256.empty()) { + ProcResult h = runProcess({"sha256sum", archive.string()}, true); + std::string got = h.out.substr(0, 64); + if (h.code != 0 || got != lowerCase(src.sha256)) { + throw std::runtime_error("sha256 mismatch for " + src.location + ": expected " + src.sha256 + ", got " + got); + } + std::fprintf(stderr, "sha256 verified\n"); + } else { + std::fprintf(stderr, "note: local file used without --sha256, not verified\n"); + } + fs::path root = tmp / "rootfs"; + fsutil::mkdirs(root, 0755); + std::vector tar = {"tar", "-x", "-p", "--numeric-owner", "--exclude=./dev/*", "--exclude=dev/*", + "-f", archive.string(), "-C", root.string()}; + if (ids == IdMode::Single) tar.insert(tar.begin() + 1, "--no-same-owner"); + std::fprintf(stderr, "extracting (ids: %s)\n", IdMap::name(ids).c_str()); + int rc = UserNsJob::run(ids, [&] { + std::vector a; + for (auto& s : tar) a.push_back(s.data()); + a.push_back(nullptr); + ::execvp(a[0], a.data()); + return 127; + }); + if (rc != 0) throw std::runtime_error("tar failed (exit " + std::to_string(rc) + ")"); + std::size_t skipped = 0; + std::uint64_t bytes = fsutil::diskUsage(root, skipped); + ImageInfo info; + info.name = src.name; + info.source = src.location; + info.ids = ids; + info.created = std::time(nullptr); + info.bytes = bytes; + fsutil::writeFile(tmp / "meta", "source=" + src.location + "\nsha256=" + src.sha256 + "\nids=" + IdMap::name(ids) + + "\ncreated=" + std::to_string(info.created) + "\nbytes=" + std::to_string(bytes) + "\n"); + if (!src.isFile) std::remove(archive.c_str()); + if (::rename(tmp.c_str(), dir(src.name).c_str()) != 0) throw SysError("install image " + src.name); + return info; + } catch (...) { + cleanup(); + throw; + } +} + +std::string ImageStore::remove(const std::string& name, bool confirmed, const std::vector& usedBy) { + validateName(name); + if (!has(name)) throw std::runtime_error("no such image '" + name + "'"); + if (!usedBy.empty()) { + std::string list; + for (const auto& u : usedBy) list += (list.empty() ? "" : ", ") + u; + throw std::runtime_error("image '" + name + "' is used by: " + list); + } + if (!confirmed) return "dry run: would delete image " + name + ". Add --yes to confirm."; + IdMode ids = this->ids(name); + fs::path d = dir(name); + int rc = UserNsJob::run(ids, [&] { fsutil::removeTree(d); return 0; }); + if (rc != 0) throw std::runtime_error("could not remove image " + name); + return "deleted image " + name; +} + +} // namespace doos diff --git a/src/image.hpp b/src/image.hpp new file mode 100644 index 0000000..f769468 --- /dev/null +++ b/src/image.hpp @@ -0,0 +1,58 @@ +#pragma once +#include +#include +#include +#include +#include + +#include "idmap.hpp" + +namespace doos { + +// Where an image comes from, after resolving a catalog name, URL or file. +struct ImageSource { + std::string name; // store name + std::string location; // https URL or local file path + std::string sha256; // expected, empty only for local files + bool isFile = false; +}; + +struct ImageInfo { + std::string name; + std::string source; + IdMode ids = IdMode::Single; + long created = 0; + std::uint64_t bytes = 0; +}; + +// Built-in sources: alpine, ubuntu (latest LTS), ubuntu-VERSION. Resolved from the publishers over https. +class Catalog { +public: + static std::vector names(); + static bool knows(const std::string& imageName); + static ImageSource resolve(const std::string& imageName); +}; + +class ImageStore { +public: + explicit ImageStore(std::filesystem::path root); + + static std::string normalize(const std::string& name); // "ubuntu:24.04" -> "ubuntu-24.04" + static void validateName(const std::string& name); + + bool has(const std::string& name) const; + std::filesystem::path rootfs(const std::string& name) const; + IdMode ids(const std::string& name) const; + std::vector list() const; + + // Downloads (catalog/URL) or reads (file), verifies the sha256, extracts under the id map. + ImageInfo add(const ImageSource& src, IdMode ids); + // Dry run unless confirmed. `usedBy` lists containers referencing the image; non-empty refuses. + std::string remove(const std::string& name, bool confirmed, const std::vector& usedBy); + +private: + std::filesystem::path dir(const std::string& name) const { return root_ / name; } + std::filesystem::path root_; +}; + +} // namespace doos diff --git a/src/sandbox.cpp b/src/sandbox.cpp index 09b41df..526c9e2 100644 --- a/src/sandbox.cpp +++ b/src/sandbox.cpp @@ -16,6 +16,11 @@ #include +#include + +#include + +#include "cgroup.hpp" #include "spec.hpp" #include "util.hpp" @@ -26,8 +31,10 @@ namespace doos { namespace { volatile sig_atomic_t g_forwardTo = 0; +volatile sig_atomic_t g_terminating = 0; void forwardSignal(int sig) { + g_terminating = 1; if (g_forwardTo > 0) ::kill(g_forwardTo, sig); } @@ -95,17 +102,24 @@ void bindHostDevice(const fs::path& root, const std::string& dev) { doMount(("/dev/" + dev).c_str(), target, nullptr, MS_BIND); } -void setupRoot(const Container& c) { - fs::path root = c.rootfs(); - doMount(nullptr, "/", nullptr, MS_REC | MS_PRIVATE); - doMount(root.c_str(), root.string(), nullptr, MS_BIND | MS_REC); +// apt drops to its _apt user for downloads; with one mappable id that user does not exist. +void writeAptSandboxOverride(const Container& c, const fs::path& lowerEtc) { + std::error_code ec; + if (c.ids() != IdMode::Single || !fs::is_directory(lowerEtc / "apt", ec)) return; + fs::path dir = c.etcDir() / "apt/apt.conf.d"; + fsutil::mkdirs(dir); + fsutil::writeFile(dir / "99doos-sandbox", "APT::Sandbox::User \"root\";\n"); +} - // resolv.conf lives in the /etc upperdir so it matches the network mode on every start. - fs::path resolv = c.layer("/etc") / "upper" / "resolv.conf"; +void writeResolvConf(const Container& c) { + fs::path resolv = c.etcDir() / "resolv.conf"; + fsutil::mkdirs(resolv.parent_path()); std::error_code ec; fs::remove(resolv, ec); fsutil::writeFile(resolv, Network::resolvConf(c.net())); +} +void mountHostLayers(const Container& c, const fs::path& root) { for (auto d : spec::kOverlayDirs) { std::error_code e; if (fs::is_directory(std::string(d), e)) mountOverlay(c, root, d); @@ -125,6 +139,32 @@ void setupRoot(const Container& c) { } } for (auto d : spec::kPrivateDirs) doMount(c.privateDir(std::string(d)).c_str(), sub(root, d), nullptr, MS_BIND); + // overlays that live inside a private dir must be mounted after it + for (auto d : spec::kVarOverlayDirs) { + std::error_code e; + if (fs::is_directory(std::string(d), e)) mountOverlay(c, root, d); + } +} + +void mountImageRoot(const Container& c, const fs::path& root, const fs::path& imageRoot) { + std::string opts = "lowerdir=" + imageRoot.string() + ",upperdir=" + (c.layer("/root") / "upper").string() + + ",workdir=" + (c.layer("/root") / "work").string() + ",userxattr"; + doMount("overlay", root.string(), "overlay", MS_NODEV | MS_NOSUID, opts); + for (auto d : spec::kEmptyDirs) fsutil::mkdirs(sub(root, d)); +} + +void setupRoot(const Container& c, const fs::path& imageRoot) { + fs::path root = c.rootfs(); + doMount(nullptr, "/", nullptr, MS_REC | MS_PRIVATE); + writeResolvConf(c); + if (c.imageMode()) { + writeAptSandboxOverride(c, imageRoot / "etc"); + mountImageRoot(c, root, imageRoot); + } else { + writeAptSandboxOverride(c, "/etc"); + doMount(root.c_str(), root.string(), nullptr, MS_BIND | MS_REC); + mountHostLayers(c, root); + } // tmpfs mounts doMount("tmpfs", sub(root, "/tmp"), "tmpfs", MS_NOSUID | MS_NODEV, "mode=1777,size=" + std::string(spec::kTmpSize)); @@ -179,7 +219,9 @@ void applyLimitsAndEnv(const Container& c) { ::setenv("HOSTNAME", c.name().c_str(), 1); ::setenv("container", "doos", 1); ::setenv("PS1", ("(doos:" + c.name() + ") \\u@\\h:\\w\\$ ").c_str(), 1); - if (!langv.empty()) ::setenv("LANG", langv.c_str(), 1); + // The host's locale usually does not exist inside an image; C.UTF-8 is built into glibc and musl. + if (c.imageMode()) ::setenv("LANG", "C.UTF-8", 1); + else if (!langv.empty()) ::setenv("LANG", langv.c_str(), 1); } [[noreturn]] void execCommand(std::vector cmd) { @@ -202,6 +244,7 @@ struct ChildCtx { const Container* container; const std::vector* command; int syncFd; + fs::path imageRoot; int lockFd; // inherited copy must be closed or the flock outlives unlock() }; @@ -216,7 +259,7 @@ int containerInit(void* arg) { ::close(ctx->syncFd); const Container& c = *ctx->container; - setupRoot(c); + setupRoot(c, ctx->imageRoot); if (::sethostname(c.name().c_str(), c.name().size()) != 0) throw SysError("sethostname"); if (Network::isolated(c.net())) bringLoopbackUp(); applyLimitsAndEnv(c); @@ -248,17 +291,6 @@ int containerInit(void* arg) { } } -void writeMap(pid_t pid, const char* file, const std::string& content) { - std::string path = "/proc/" + std::to_string(pid) + "/" + file; - int fd = ::open(path.c_str(), O_WRONLY | O_CLOEXEC); - if (fd < 0) throw SysError("open " + path); - Fd guard(fd); - if (::write(fd, content.data(), content.size()) != static_cast(content.size())) { - throw SysError("write " + path + - " (user namespaces blocked? see apparmor/doos and docs/RESEARCH.md)"); - } -} - } // namespace int Sandbox::run() { @@ -273,6 +305,14 @@ int Sandbox::run() { int Sandbox::start(FileLock& lock) { NetMode net = container_.net(); + fs::path imageRoot; + if (container_.imageMode()) { + Store store = Store::fromEnvironment(); + imageRoot = store.images().rootfs(container_.image()); + if (!fs::is_directory(imageRoot)) { + throw std::runtime_error("image '" + container_.image() + "' is missing; add it again with: doos image add"); + } + } int flags = CLONE_NEWUSER | CLONE_NEWNS | CLONE_NEWPID | CLONE_NEWUTS | CLONE_NEWIPC | SIGCHLD; if (Network::isolated(net)) flags |= CLONE_NEWNET; @@ -281,7 +321,7 @@ int Sandbox::start(FileLock& lock) { Fd syncRead(sync[0]), syncWrite(sync[1]); std::vector stack(1 << 20); - ChildCtx ctx{&container_, &command_, syncRead.get(), lock.fd()}; + ChildCtx ctx{&container_, &command_, syncRead.get(), imageRoot, lock.fd()}; pid_t child = ::clone(containerInit, stack.data() + stack.size(), flags, &ctx); if (child < 0) throw SysError("clone (user namespaces available?)"); @@ -297,13 +337,16 @@ int Sandbox::start(FileLock& lock) { } } reaper{child}; - writeMap(child, "setgroups", "deny"); - writeMap(child, "uid_map", "0 " + std::to_string(::geteuid()) + " 1\n"); - writeMap(child, "gid_map", "0 " + std::to_string(::getegid()) + " 1\n"); + IdMap::apply(child, container_.ids()); SlirpSession slirp; if (net == NetMode::Slirp) slirp.start(child); + std::string cgroupWhy; + std::unique_ptr cgroup = Cgroup::create(container_.name(), cgroupWhy); + if (cgroup) cgroup->add(child); + else std::fprintf(stderr, "doos: size cap only warns, cannot freeze: %s\n", cgroupWhy.c_str()); + container_.recordPid(child); struct PidGuard { const Container& c; @@ -324,12 +367,66 @@ int Sandbox::start(FileLock& lock) { ::sigaction(SIGQUIT, &ign, nullptr); installForwarding(child, {SIGTERM, SIGHUP}); + int status = watch(child, cgroup.get()); + reaper.armed = false; + return exitCodeOf(status); +} + +// Waits for the container's init while enforcing the size cap: warn at kSizeWarnPercent, freeze at the cap, +// thaw again once usage drops below kSizeThawPercent. Nothing is ever deleted by this loop. +int Sandbox::watch(pid_t child, Cgroup* cgroup) { + long fd = ::syscall(SYS_pidfd_open, child, 0); + Fd pidfd(static_cast(fd)); + const std::uint64_t cap = container_.sizeCap(); + bool warned = false; + auto lastScan = std::chrono::steady_clock::now(); + for (;;) { + if (pidfd.valid()) { + pollfd p{pidfd.get(), POLLIN, 0}; + int r = ::poll(&p, 1, spec::kSizeCheckIntervalMs); + if (r > 0) break; + if (r < 0 && errno != EINTR) break; + } else { + int st = 0; + if (::waitpid(child, &st, WNOHANG) == child) return st; + ::poll(nullptr, 0, 1000); + } + if (g_terminating && cgroup && cgroup->frozen()) cgroup->freeze(false); // a frozen init cannot see SIGTERM + auto now = std::chrono::steady_clock::now(); + if (now - lastScan < std::chrono::milliseconds(spec::kSizeCheckIntervalMs)) continue; + lastScan = now; + std::size_t skipped = 0; + std::uint64_t used = container_.usedBytes(skipped); + const char* name = container_.name().c_str(); + if (used >= cap) { + if (cgroup && !cgroup->frozen()) { + cgroup->freeze(true); + std::fprintf(stderr, "doos: '%s' reached its size cap (%s of %s): FROZEN. Free space with 'doos attach %s -- ...'; it thaws below %d%%.\n", + name, formatSize(used).c_str(), formatSize(cap).c_str(), name, spec::kSizeThawPercent); + } else if (!cgroup && !warned) { + std::fprintf(stderr, "doos: '%s' is over its size cap (%s of %s), cannot freeze\n", name, + formatSize(used).c_str(), formatSize(cap).c_str()); + warned = true; + } + } else if (used * 100 >= cap * spec::kSizeWarnPercent) { + if (!warned) { + std::fprintf(stderr, "doos: '%s' uses %s of its %s size cap\n", name, formatSize(used).c_str(), + formatSize(cap).c_str()); + warned = true; + } + } else { + warned = false; + } + if (cgroup && cgroup->frozen() && used * 100 < cap * spec::kSizeThawPercent) { + cgroup->freeze(false); + std::fprintf(stderr, "doos: '%s' thawed (%s used)\n", name, formatSize(used).c_str()); + } + } int status = 0; while (::waitpid(child, &status, 0) < 0) { if (errno != EINTR) throw SysError("waitpid"); } - reaper.armed = false; - return exitCodeOf(status); + return status; } int Sandbox::join(pid_t initPid, FileLock& lock) { diff --git a/src/sandbox.hpp b/src/sandbox.hpp index 5791618..7d373cf 100644 --- a/src/sandbox.hpp +++ b/src/sandbox.hpp @@ -7,6 +7,8 @@ namespace doos { +class Cgroup; + // Runs a command inside a container: starts the namespaces if the container is // stopped, joins them if it is already running. Returns the command's exit code. class Sandbox { @@ -18,6 +20,7 @@ public: private: int start(FileLock& lock); + int watch(pid_t child, Cgroup* cgroup); int join(pid_t initPid, FileLock& lock); Container container_; diff --git a/src/spec.hpp b/src/spec.hpp index 7818a33..27d0108 100644 --- a/src/spec.hpp +++ b/src/spec.hpp @@ -2,6 +2,7 @@ #pragma once #include #include +#include #include namespace spec { @@ -18,6 +19,35 @@ constexpr std::string_view kTmpSize = "256m"; constexpr std::string_view kRunSize = "64m"; constexpr std::string_view kShmSize = "64m"; +// Writable-layer size cap per container (--size). Enforced by a watchdog that freezes the container. +constexpr std::uint64_t kDefaultSizeBytes = 4ULL << 30; +constexpr std::uint64_t kMinSizeBytes = 64ULL << 20; +constexpr std::uint64_t kMaxSizeBytes = 1ULL << 40; +constexpr int kSizeWarnPercent = 80; +constexpr int kSizeThawPercent = 95; // thaw again below this share of the cap +constexpr int kSizeCheckIntervalMs = 10000; // floor between usage scans, never faster +constexpr std::size_t kMaxScanEntries = 5000000; // usage scan stops (and reports) beyond this +constexpr std::string_view kCgroupDir = "doos"; // under the delegated user@UID.service cgroup + +// Seeding limits +constexpr std::size_t kMaxSkeletonDirs = 200000; +constexpr std::size_t kMaxEtcCopyFiles = 50000; +constexpr std::uint64_t kMaxEtcCopyBytes = 256ULL << 20; + +// Id mapping: container ids 0..kMaxIdRange. 0 maps to the invoking user, the rest to the subuid range. +constexpr std::uint32_t kMaxIdRange = 65536; +constexpr std::string_view kNewUidMap = "newuidmap"; +constexpr std::string_view kNewGidMap = "newgidmap"; + +// Images +constexpr std::size_t kMaxImages = 16; +constexpr std::uint64_t kMaxImageDownloadBytes = 2ULL << 30; +constexpr int kDownloadTimeoutSec = 900; +constexpr std::size_t kMaxCaptureBytes = 1 << 20; // listing / checksum fetches +constexpr std::string_view kAlpineBase = "https://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/"; +constexpr std::string_view kUbuntuBase = "https://cdimage.ubuntu.com/ubuntu-base/releases/"; +constexpr std::string_view kUbuntuDefaultVersion = "26.04"; + // Networking constexpr std::string_view kSlirpBinary = "slirp4netns"; constexpr std::string_view kSlirpDns = "10.0.2.3"; @@ -29,14 +59,45 @@ constexpr std::string_view kContainerPath = constexpr std::string_view kHome = "/root"; // Host directories exposed as overlay lowerdirs. Writes land in the per-container upperdir. -constexpr std::array kOverlayDirs = {"/usr", "/etc", "/opt"}; +constexpr std::array kOverlayDirs = {"/usr", "/opt"}; + +// Host directories copied once into a private per-container dir (owned by the container root, so +// chown --reference and in-place edits work). Unreadable entries are skipped and counted. +constexpr std::array kCopiedDirs = {"/etc"}; + +// Extra overlay mounted inside the private /var, only when the host has the directory (dpkg database). +constexpr std::array kVarOverlayDirs = {"/var/lib/dpkg"}; + +// Lock files that exist on the host owned by an unmapped uid: an empty copy is created in the upper layer so +// the container can open them for writing. Each must live under one of the overlay dirs above. +constexpr std::array kUpperLockFiles = {"/var/lib/dpkg/lock", "/var/lib/dpkg/lock-frontend", + "/var/lib/dpkg/triggers/Lock"}; + +// Directories whose regular files (not subdirs) are copied into the upper layer at creation, so the container +// owns them: dpkg hard-links `status` to `status-old`, which the kernel refuses for files owned by nobody. +constexpr std::array kUpperCopyDirs = {"/var/lib/dpkg", "/var/lib/dpkg/triggers", + "/var/lib/dpkg/alternatives"}; + +struct DirSpec { + std::string_view path; + unsigned mode; +}; +// Directories created in the private /var (package managers expect them). +constexpr std::array kVarDirs = {{{"lib/apt/lists/partial", 0755}, + {"lib/dpkg", 0755}, + {"cache/apt/archives/partial", 0755}, + {"log/apt", 0755}, + {"tmp", 01777}, + {"lib/apt", 0755}, + {"cache", 0755}, + {"log", 0755}}}; // Host top-level entries: recreated as symlinks when the host has them as symlinks (usr-merge), // overlaid like kOverlayDirs when they are real directories, skipped when absent. constexpr std::array kCompatDirs = {"/bin", "/sbin", "/lib", "/lib32", "/lib64"}; // Persistent per-container private directories (no host lower layer). -constexpr std::array kPrivateDirs = {"/root", "/var"}; +constexpr std::array kPrivateDirs = {"/root", "/var", "/etc"}; // Empty mountpoints / tmpfs constexpr std::array kEmptyDirs = {"/mnt", "/srv", "/tmp", "/run", "/dev", "/proc"}; diff --git a/src/util.cpp b/src/util.cpp index 0506c31..d98ea1a 100644 --- a/src/util.cpp +++ b/src/util.cpp @@ -4,10 +4,15 @@ #include #include #include +#include +#include #include +#include #include +#include #include +#include #include #include "spec.hpp" @@ -46,8 +51,209 @@ FileLock::FileLock(const fs::path& file, int timeoutMs) { } } +std::uint64_t parseSize(const std::string& text) { + std::size_t i = 0; + while (i < text.size() && std::isdigit(static_cast(text[i]))) ++i; + if (i == 0 || i > 15) throw std::runtime_error("invalid size '" + text + "' (examples: 512m, 4g)"); + std::uint64_t v = std::stoull(text.substr(0, i)); + std::string unit = text.substr(i); + for (auto& c : unit) c = static_cast(std::tolower(static_cast(c))); + if (unit == "b" || unit.empty()) { + } else { + if (unit.size() == 2 && unit[1] == 'b') unit.pop_back(); + if (unit == "k") v <<= 10; + else if (unit == "m") v <<= 20; + else if (unit == "g") v <<= 30; + else if (unit == "t") v <<= 40; + else throw std::runtime_error("invalid size unit in '" + text + "' (use k, m, g or t)"); + } + return v; +} + +std::string formatSize(std::uint64_t b) { + const char* u[] = {"B", "K", "M", "G", "T"}; + double v = static_cast(b); + int i = 0; + while (v >= 1024 && i < 4) { v /= 1024; ++i; } + char buf[32]; + if (i == 0) std::snprintf(buf, sizeof buf, "%lluB", static_cast(b)); + else std::snprintf(buf, sizeof buf, v < 10 ? "%.1f%s" : "%.0f%s", v, u[i]); + return buf; +} + +ProcResult runProcess(const std::vector& argv, bool capture, bool quiet) { + int pipefd[2] = {-1, -1}; + if (capture && ::pipe2(pipefd, O_CLOEXEC) != 0) throw SysError("pipe"); + Fd readEnd(pipefd[0]), writeEnd(pipefd[1]); + pid_t pid = ::fork(); + if (pid < 0) throw SysError("fork " + argv.at(0)); + if (pid == 0) { + if (capture) ::dup2(writeEnd.get(), 1); + if (quiet) { + int nul = ::open("/dev/null", O_WRONLY); + if (nul >= 0) { ::dup2(nul, 2); if (!capture) ::dup2(nul, 1); } + } + std::vector a; + for (const auto& x : argv) a.push_back(const_cast(x.c_str())); + a.push_back(nullptr); + ::execvp(a[0], a.data()); + ::dprintf(2, "doos: cannot run %s: %s\n", a[0], std::strerror(errno)); + ::_exit(127); + } + writeEnd.reset(); + ProcResult r; + if (capture) { + char buf[4096]; + ssize_t n; + while ((n = ::read(readEnd.get(), buf, sizeof buf)) != 0) { + if (n < 0) { if (errno == EINTR) continue; break; } + if (r.out.size() + static_cast(n) > spec::kMaxCaptureBytes) { + ::kill(pid, SIGKILL); + ::waitpid(pid, nullptr, 0); + throw std::runtime_error("output of " + argv.at(0) + " exceeds the capture cap"); + } + r.out.append(buf, static_cast(n)); + } + } + int st = 0; + while (::waitpid(pid, &st, 0) < 0) { + if (errno != EINTR) throw SysError("waitpid " + argv.at(0)); + } + r.code = WIFEXITED(st) ? WEXITSTATUS(st) : 128 + WTERMSIG(st); + return r; +} + namespace fsutil { +namespace { +void mirrorRec(const fs::path& lower, const fs::path& upper, SeedStats& st, std::size_t depth) { + if (depth > spec::kMaxTreeDepth) throw std::runtime_error("tree too deep: " + lower.string()); + std::error_code ec; + fs::directory_iterator it(lower, ec), end; + if (ec) { ++st.skipped; return; } + for (; it != end; it.increment(ec)) { + if (ec) { ++st.skipped; return; } + struct stat sb; + if (::lstat(it->path().c_str(), &sb) != 0 || !S_ISDIR(sb.st_mode)) continue; + if (st.made >= spec::kMaxSkeletonDirs) throw std::runtime_error("directory skeleton cap reached"); + fs::path target = upper / it->path().filename(); + if (::mkdir(target.c_str(), (sb.st_mode & 07777) | 0700) != 0 && errno != EEXIST) { + throw SysError("mkdir " + target.string()); + } + ++st.made; + mirrorRec(it->path(), target, st, depth + 1); + } +} + +void copyRec(const fs::path& from, const fs::path& to, SeedStats& st, std::size_t depth) { + if (depth > spec::kMaxTreeDepth) throw std::runtime_error("tree too deep: " + from.string()); + std::error_code ec; + fs::directory_iterator it(from, ec), end; + if (ec) { ++st.skipped; return; } + for (; it != end; it.increment(ec)) { + if (ec) { ++st.skipped; return; } + struct stat sb; + const fs::path& src = it->path(); + fs::path dst = to / src.filename(); + if (::lstat(src.c_str(), &sb) != 0) { ++st.skipped; continue; } + if (st.made >= spec::kMaxEtcCopyFiles) throw std::runtime_error("copy file cap reached: " + from.string()); + if (S_ISDIR(sb.st_mode)) { + if (::mkdir(dst.c_str(), (sb.st_mode & 07777) | 0700) != 0 && errno != EEXIST) { + throw SysError("mkdir " + dst.string()); + } + ++st.made; + copyRec(src, dst, st, depth + 1); + } else if (S_ISLNK(sb.st_mode)) { + std::error_code e2; + fs::remove(dst, e2); + fs::create_symlink(fs::read_symlink(src), dst); + ++st.made; + } else if (S_ISREG(sb.st_mode)) { + if (::access(src.c_str(), R_OK) != 0) { ++st.skipped; continue; } + st.bytes += static_cast(sb.st_size); + if (st.bytes > spec::kMaxEtcCopyBytes) throw std::runtime_error("copy byte cap reached: " + from.string()); + std::error_code e2; + fs::copy_file(src, dst, fs::copy_options::overwrite_existing, e2); + if (e2) { ++st.skipped; continue; } + ::chmod(dst.c_str(), (sb.st_mode & 07777) | 0600); + ++st.made; + } else { + ++st.skipped; + } + } +} + +struct InodeHash { + std::size_t operator()(const std::pair& k) const { return std::hash()(k.first * 1000003UL ^ k.second); } +}; +using InodeSet = std::unordered_set, InodeHash>; + +std::uint64_t usageRec(const fs::path& p, std::size_t& seen, std::size_t& skipped, std::size_t depth, InodeSet& links) { + struct stat sb; + if (::lstat(p.c_str(), &sb) != 0) { ++skipped; return 0; } + std::uint64_t total = static_cast(sb.st_blocks) * 512; + if (!S_ISDIR(sb.st_mode)) { + // hard links share their blocks: count each inode once + if (sb.st_nlink > 1 && !links.insert({sb.st_dev, sb.st_ino}).second) return 0; + return total; + } + if (depth > spec::kMaxTreeDepth || seen >= spec::kMaxScanEntries) { ++skipped; return total; } + std::error_code ec; + fs::directory_iterator it(p, ec), end; + if (ec) { ++skipped; return total; } + for (; it != end; it.increment(ec)) { + if (ec) { ++skipped; break; } + ++seen; + total += usageRec(it->path(), seen, skipped, depth + 1, links); + } + return total; +} +} // namespace + +SeedStats mirrorDirs(const fs::path& lower, const fs::path& upper) { + SeedStats st; + mkdirs(upper); + mirrorRec(lower, upper, st, 0); + return st; +} + +SeedStats copyTree(const fs::path& from, const fs::path& to) { + SeedStats st; + mkdirs(to); + copyRec(from, to, st, 0); + return st; +} + +SeedStats copyFiles(const fs::path& from, const fs::path& to) { + SeedStats st; + std::error_code ec; + fs::directory_iterator it(from, ec), end; + if (ec) { ++st.skipped; return st; } + for (; it != end; it.increment(ec)) { + if (ec) { ++st.skipped; break; } + struct stat sb; + const fs::path& src = it->path(); + if (::lstat(src.c_str(), &sb) != 0 || !S_ISREG(sb.st_mode)) continue; + fs::path dst = to / src.filename(); + std::error_code e2; + if (fs::exists(dst, e2)) continue; + if (::access(src.c_str(), R_OK) != 0) { ++st.skipped; continue; } + st.bytes += static_cast(sb.st_size); + if (st.bytes > spec::kMaxEtcCopyBytes) throw std::runtime_error("copy byte cap reached: " + from.string()); + fs::copy_file(src, dst, fs::copy_options::none, e2); + if (e2) { ++st.skipped; continue; } + ::chmod(dst.c_str(), (sb.st_mode & 07777) | 0600); + ++st.made; + } + return st; +} + +std::uint64_t diskUsage(const fs::path& p, std::size_t& skipped) { + std::size_t seen = 0; + InodeSet links; + return usageRec(p, seen, skipped, 0, links); +} + void mkdirs(const fs::path& p, mode_t mode) { std::error_code ec; fs::create_directories(p, ec); diff --git a/src/util.hpp b/src/util.hpp index a31144d..87e9589 100644 --- a/src/util.hpp +++ b/src/util.hpp @@ -1,8 +1,10 @@ #pragma once #include +#include #include #include #include +#include namespace doos { @@ -45,7 +47,31 @@ private: Fd fd_; }; +// Parses "512m", "2g", "1t" (k/m/g/t, optional trailing b) into bytes; throws on garbage. +std::uint64_t parseSize(const std::string& text); +std::string formatSize(std::uint64_t bytes); + +struct ProcResult { + int code = -1; + std::string out; // stdout, only when captured +}; +// fork+exec without a shell. Captured output is capped; stderr is inherited unless quiet. +ProcResult runProcess(const std::vector& argv, bool capture = false, bool quiet = false); + namespace fsutil { +struct SeedStats { + std::size_t made = 0; + std::size_t skipped = 0; // unreadable or unsupported entries, never silent + std::uint64_t bytes = 0; +}; +// Creates every directory of `lower` in `upper` (owner rwx added) so a unmapped-owner lower stays writable. +SeedStats mirrorDirs(const std::filesystem::path& lower, const std::filesystem::path& upper); +// Copies dirs, regular files and symlinks readable by the caller. Capped by spec.hpp limits. +SeedStats copyTree(const std::filesystem::path& from, const std::filesystem::path& to); +// Copies only the regular files directly inside `from` (readable ones) into `to`; existing targets stay. +SeedStats copyFiles(const std::filesystem::path& from, const std::filesystem::path& to); +// Sum of allocated bytes (st_blocks) below p, lstat only. `skipped` counts unreadable dirs. +std::uint64_t diskUsage(const std::filesystem::path& p, std::size_t& skipped); void mkdirs(const std::filesystem::path& p, mode_t mode = 0755); void writeFile(const std::filesystem::path& p, const std::string& content); std::string readFile(const std::filesystem::path& p); diff --git a/tests/run.sh b/tests/run.sh index e13a2f5..ed4431d 100755 --- a/tests/run.sh +++ b/tests/run.sh @@ -57,5 +57,67 @@ wait; sleep 0.5 "$DOOS" delete t | grep -q "dry run" && [ -d "$DOOS_HOME/containers/t" ] && ok "delete is dry-run" || bad "delete is dry-run" "$DOOS" delete --yes t | grep -q deleted && [ ! -e "$DOOS_HOME/containers/t" ] && ok "delete --yes removes" || bad "delete --yes removes" +# ---- 1.0: skeleton, seeded /etc, uid mapping (host mode, container t was deleted above) +"$DOOS" get_or_create s >/dev/null 2>&1 +outs() { "$DOOS" attach s -- sh -c "$1" 2>&1; } +[ "$(outs 'touch /usr/bin/doos-x && echo y')" = "y" ] && [ ! -e /usr/bin/doos-x ] && ok "skeleton: write into /usr/bin" || bad "skeleton: write into /usr/bin" +[ "$(outs 'stat -c %u /etc/shells')" = "0" ] && ok "/etc owned by container root" || bad "/etc owned by container root" +[ "$(outs 'chown 100:100 /root && stat -c %u:%g /root')" = "100:100" ] && ok "subuid: chown to other ids" || bad "subuid: chown to other ids" +[ "$(outs 'ls /var/lib/dpkg/status >/dev/null && echo y')" = "y" ] && ok "/var/lib/dpkg present" || bad "/var/lib/dpkg present" +"$DOOS" get_or_create --ids single s1 >/dev/null 2>&1 +[ "$("$DOOS" attach s1 -- cat /proc/self/uid_map | wc -l)" = "1" ] && ok "ids single: one mapping" || bad "ids single: one mapping" +"$DOOS" attach s1 -- sh -c 'cat /etc/apt/apt.conf.d/99doos-sandbox' 2>&1 | grep -q 'Sandbox::User "root"' && ok "ids single: apt sandbox override" || bad "ids single: apt sandbox override" +check "ids fixed at creation" bash -c "! '$DOOS' get_or_create --ids subuid s1" +check "bad size rejected" bash -c "! '$DOOS' get_or_create --size 1x z1" +check "size below minimum rejected" bash -c "! '$DOOS' get_or_create --size 1m z2" +"$DOOS" attach s -- sh -c 'apt-get update >/dev/null 2>&1; apt-get install -y --no-install-recommends cowsay >/dev/null 2>&1; dpkg --audit && /usr/games/cowsay ok >/dev/null' && ok "host mode: apt install completes" || bad "host mode: apt install completes" +[ ! -e /usr/games/cowsay ] && ok "host untouched by apt install" || bad "host untouched by apt install" +"$DOOS" delete --yes s >/dev/null; "$DOOS" delete --yes s1 >/dev/null + +# ---- images +echo "--- images (network)" +"$DOOS" image list | grep -q "alpine.*available" && ok "image list shows catalog" || bad "image list shows catalog" +check "http url refused" bash -c "! '$DOOS' image add http://example.com/x.tar.gz" +check "url without sha256 refused" bash -c "! '$DOOS' image add https://example.com/x.tar.gz" +check "unknown image refused" bash -c "! '$DOOS' get_or_create --image nosuch i0" +tmpd=$(mktemp -d); mkdir -p $tmpd/r/bin; echo hi > $tmpd/r/bin/f; tar -C $tmpd/r -czf $tmpd/mini.tar.gz . +check "wrong sha256 rejected" bash -c "! '$DOOS' image add --sha256 $(printf '0%.0s' $(seq 64)) $tmpd/mini.tar.gz mini" +[ ! -e "$DOOS_HOME/images/mini" ] && [ ! -e "$DOOS_HOME/images/.tmp-mini" ] && ok "failed add leaves nothing" || bad "failed add leaves nothing" +check "local tar added" "$DOOS" image add $tmpd/mini.tar.gz mini +rm -rf $tmpd; "$DOOS" image delete --yes mini >/dev/null + +"$DOOS" attach --image alpine a1 -- true >/dev/null 2>&1 +"$DOOS" image list | grep -q "alpine.*installed" && ok "image auto-fetched on demand" || bad "image auto-fetched on demand" +[ "$("$DOOS" attach a1 -- sh -c '. /etc/os-release; echo $ID')" = "alpine" ] && ok "alpine image runs" || bad "alpine image runs" +"$DOOS" attach a1 -- sh -c 'apk add --no-cache jq >/dev/null 2>&1 && jq --version' | grep -q jq && ok "alpine: apk add" || bad "alpine: apk add" +[ "$("$DOOS" attach a1 -- sh -c 'chown 100:100 /root && stat -c %u /root')" = "100" ] && ok "image: subuid chown" || bad "image: subuid chown" +"$DOOS" attach a1 -- sh -c 'exec 3<>/dev/tcp/1.1.1.1/443' >/dev/null 2>&1; "$DOOS" attach a1 -- sh -c 'wget -qO- https://example.com | grep -q html' && ok "image: network + dns" || bad "image: network + dns" +check "image delete refused while used" bash -c "! '$DOOS' image delete --yes alpine" +check "image in use is a dry run too" bash -c "! '$DOOS' image delete alpine" +"$DOOS" get_or_create --image alpine --ids single a2 >/dev/null 2>&1 && bad "image ids mismatch rejected" || ok "image ids mismatch rejected" +"$DOOS" delete --yes a1 >/dev/null +"$DOOS" image delete alpine | grep -q "dry run" && [ -d "$DOOS_HOME/images/alpine" ] && ok "image delete is dry-run" || bad "image delete is dry-run" +"$DOOS" image delete --yes alpine | grep -q deleted && [ ! -e "$DOOS_HOME/images/alpine" ] && ok "image delete --yes removes" || bad "image delete --yes removes" + +"$DOOS" attach --image ubuntu u1 -- sh -c 'apt-get update >/dev/null 2>&1; apt-get install -y --no-install-recommends cowsay >/dev/null 2>&1; dpkg --audit && /usr/games/cowsay ok >/dev/null' && ok "ubuntu image: apt install completes" || bad "ubuntu image: apt install completes" +"$DOOS" delete --yes u1 >/dev/null; "$DOOS" image delete --yes ubuntu >/dev/null + +# ---- size cap (needs cgroup v2 delegation; takes about 40 s because the scan interval has a 10 s floor) +echo "--- size cap" +"$DOOS" get_or_create --image alpine --size 64m cap >/dev/null 2>&1 +"$DOOS" attach cap -- sh -c 'dd if=/dev/zero of=/root/big bs=1M count=100 2>/dev/null; sleep 40' >/tmp/doos-cap.$$ 2>&1 & +sleep 15 +if grep -q "FROZEN" /tmp/doos-cap.$$; then + ok "size cap freezes" + "$DOOS" attach cap -- rm /root/big >/dev/null 2>&1; sleep 12 + grep -q thawed /tmp/doos-cap.$$ && ok "size cap thaws after cleanup" || bad "size cap thaws after cleanup" +elif grep -q "cannot freeze" /tmp/doos-cap.$$; then + echo "skip size cap: no cgroup delegation here (warn-only mode confirmed)" +else + bad "size cap freezes" +fi +kill %% 2>/dev/null; wait 2>/dev/null; rm -f /tmp/doos-cap.$$ +"$DOOS" delete --yes cap >/dev/null 2>&1; "$DOOS" image delete --yes alpine >/dev/null 2>&1 + echo "passed=$pass failed=$fail" [ "$fail" -eq 0 ]