Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c3ebf21e9c | ||
|
|
530582e826 | ||
|
|
d446c29372 | ||
|
|
fdb96c7b1c | ||
|
|
0d18aff519 | ||
|
|
3b5e28e06e |
@@ -0,0 +1,83 @@
|
||||
# AGENTS.md
|
||||
|
||||
Rules for any AI agent (or human) changing this repository. The user's global rules
|
||||
also apply; this file adds the project-specific ones. Keep this file current: it is
|
||||
part of every release.
|
||||
|
||||
## Project in one paragraph
|
||||
|
||||
`doos` is a user-level container tool in C++20. `attach` builds a user + mount + pid
|
||||
+ uts + ipc (+ net) namespace set, overlays either the host's `/usr`, `/opt` (host mode) or an
|
||||
extracted image (image mode) with a per-container upper dir, pivots into it and runs a command.
|
||||
Networking is slirp4netns or pasta, ids are mapped through newuidmap when possible, cgroup v2 limits
|
||||
and a watchdog enforce `--memory --pids --cpus --size`, capabilities and seccomp confine the command.
|
||||
No root at runtime. Authoritative behavior: `SPEC.md`. Design notes: `docs/ARCHITECTURE.md`.
|
||||
Measurements and gaps: `docs/RESEARCH.md`.
|
||||
|
||||
## Mandatory release workflow (every change, same session)
|
||||
|
||||
1. Make the change. Smallest diff that does what was asked.
|
||||
2. Update docs in the same change (see the table below).
|
||||
3. Pick the bump:
|
||||
- `patch`: bug fix, docs, tests, tooling, no behavior change for users.
|
||||
- `minor`: new or changed user-visible behavior (flag, command, mount, limit).
|
||||
- `major`: breaking change to CLI, state layout, or container compatibility.
|
||||
4. Run `scripts/release.sh <patch|minor|major> "imperative commit subject"`.
|
||||
It bumps `VERSION`, builds, runs `tests/run.sh`, commits all changes, and creates
|
||||
the annotated tag `vX.Y.Z`. It aborts on any build or test failure.
|
||||
5. Verify live: run `build/doos` once against the real thing (the e2e suite does this).
|
||||
6. Report honestly what passed and what did not. Do not push unless the user asks.
|
||||
|
||||
Never hand-edit `VERSION` or create tags manually. Never skip the tests to release.
|
||||
|
||||
## Docs that must be updated with the code
|
||||
|
||||
| If you change | Update |
|
||||
|---|---|
|
||||
| CLI commands, flags, output | `README.md`, `SPEC.md` (CLI), `src/cli.cpp` usage text |
|
||||
| Limits or constants in `src/spec.hpp` | `SPEC.md` limits/layout tables (both must match) |
|
||||
| Mounts, namespaces, isolation | `SPEC.md`, `docs/ARCHITECTURE.md`, `docs/RESEARCH.md` host safety section |
|
||||
| Networking | `SPEC.md`, `docs/RESEARCH.md` networking table |
|
||||
| Source files added/removed/renamed | `docs/ARCHITECTURE.md` file map, `CMakeLists.txt`, this file |
|
||||
| A new measurement | `docs/RESEARCH.md` with machine, kernel, and what was run |
|
||||
| A new known gap or a closed gap | `docs/RESEARCH.md` gaps list and `README.md` |
|
||||
| These workflow rules | this file |
|
||||
| Image catalog, sources, verification, pinned keys | `SPEC.md` Images, `README.md`, `docs/RESEARCH.md` |
|
||||
| Seccomp list, capability set, `/etc` allowlist | `SPEC.md` Hardening and filesystem, `README.md`, `docs/RESEARCH.md` |
|
||||
| Limits, cgroup layout | `SPEC.md` Resource limits, `docs/ARCHITECTURE.md` |
|
||||
| Anything above, in every release | `README.md` version line and `CHANGELOG.md` entry |
|
||||
|
||||
Docs describe what the system does today. No roadmap text. A gap is listed next to
|
||||
what would close it. A number in the docs needs a run behind it, else say it is unmeasured.
|
||||
|
||||
## Changelog
|
||||
|
||||
Every release adds a `CHANGELOG.md` entry (newest first): version, date, bullets of
|
||||
what changed for users. The release script does not write it; write it before running
|
||||
the script.
|
||||
|
||||
## Code rules
|
||||
|
||||
- C++20, extremely object oriented: one class per responsibility (see file map).
|
||||
- Every limit is a named constant in `src/spec.hpp` and listed in `SPEC.md`.
|
||||
- No unbounded loops, sizes or recursion. Add a cap when you add a loop.
|
||||
- Destructive operations: dry-run by default, `--yes` to act, scoped to the store.
|
||||
- Fail loudly with a clear message. Never swallow an error that leaves state half-built.
|
||||
- Match surrounding style. No new dependencies without a stated reason.
|
||||
- Never use em dashes (U+2014) anywhere. The release script rejects them.
|
||||
- Output must stay readable on a narrow terminal.
|
||||
|
||||
## Testing
|
||||
|
||||
`tests/run.sh [path/to/doos]` runs the real path: real overlayfs, namespaces,
|
||||
slirp4netns, filesystem, real downloads of the Alpine and Ubuntu images, real apt/apk installs, the size
|
||||
freeze (about 4 minutes in total, needs network, `pasta`, `gpg` and cgroup v2 delegation). Add a case for every behavior change. A green run is required
|
||||
before release. On Ubuntu 24.04+ the AppArmor profile in `apparmor/doos` must be
|
||||
installed once for the binary being tested (`doos apparmor`; needs sudo, ask the user, do not do it yourself).
|
||||
|
||||
## Environment notes
|
||||
|
||||
- State dir: `$DOOS_HOME`, else `$XDG_DATA_HOME/doos`, else `~/.local/share/doos`.
|
||||
Tests use a temp `DOOS_HOME`; never point tests at the user's real store.
|
||||
- Build: `cmake -S . -B build && cmake --build build`.
|
||||
- Requirements: Linux >= 5.11, `slirp4netns` for the default network mode.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Changelog
|
||||
|
||||
## 1.1.1 - 2026-10-23
|
||||
- `doos apparmor` prints a profile for the exact path of the running binary (own profile name per path, so several installs coexist). A start that fails because of AppArmor now prints the install command instead of only `mount bind /: Permission denied`. Verified with a copy in `~/.local/bin`.
|
||||
- `apparmor/doos` (packaging) covers more standard install paths.
|
||||
- Docs: Networking section with a verified example per mode (slirp, pasta, host, none), the inbound/loopback behavior of each, and the pasta port-detection delay.
|
||||
- Tests: 86 checks (adds apparmor output, `none`, slirp has no inbound path, pasta container to container).
|
||||
|
||||
## 1.1.0 - 2026-10-23
|
||||
- Limits: `--memory`, `--pids`, `--cpus` (cgroup v2; whole-container OOM kill; reports OOM kills and pid-limit hits). Later attaches run in their own cgroup leaf with the same limits.
|
||||
- Hardening (default): Docker's default capability set, a seccomp filter (arch check, x32 guard, clone3 ENOSYS, no nested user namespaces, no mounts/keyrings/bpf/perf/io_uring/...), host `/etc` allowlist. `--hardening relaxed` turns it off per container.
|
||||
- Networking: `--net pasta` (host-like reachability, private namespace and abstract sockets). `--net host` now prints a note.
|
||||
- Size cap: measured from inside the id map (counts other uids), paced by scan cost (2 s floor), `RLIMIT_FSIZE` bounds single files.
|
||||
- Images: Debian in the catalog (`debian`, `debian-RELEASE`); Alpine and Ubuntu signatures verified against pinned key fingerprints.
|
||||
- Fix: a failed `get_or_create` no longer leaves an unreadable container directory behind.
|
||||
- Tests: 83 end-to-end checks.
|
||||
|
||||
## 1.0.0 - 2026-10-23
|
||||
Breaking: state layout changed (private `/etc`, `meta` keys). Containers from 0.x must be deleted and recreated.
|
||||
- Images: `doos image list|add|delete`, `--image NAME` on `get_or_create`/`attach` (fetched on demand). Catalog: `alpine`, `ubuntu`, `ubuntu-VERSION`; also https URL with `--sha256`, or a local tarball. sha256 verified before extraction.
|
||||
- `--size`: cap for what a container writes (default 4g). Warns at 80%, freezes at 100% via cgroup v2, thaws below 95%.
|
||||
- `--ids`: subuid mapping through newuidmap/newgidmap by default (chown, su and apt's `_apt` work); falls back to one id.
|
||||
- Host mode fixes: directory skeleton so host-owned directories are writable, private copy of `/etc`, pre-seeded `/var` and dpkg state. `apt-get install` now completes.
|
||||
- Deleting trees with subuid-owned files works (runs inside the id map).
|
||||
- Usage measurement counts hard links once.
|
||||
- Tests: 56 end-to-end checks, including real image downloads, apt/apk installs and the size freeze.
|
||||
|
||||
## 0.1.3 - 2026-10-23
|
||||
- Docs: field trial of apt and common tools, new gap G7 (host-owned lower layer), README limits section. No code change.
|
||||
|
||||
## 0.1.2 - 2026-10-23
|
||||
- Docs: researched solutions for all known gaps (G1 to G6) in `docs/RESEARCH.md`, with measurements. Nothing implemented yet.
|
||||
|
||||
## 0.1.1 - 2026-10-23
|
||||
- Docs: AGENTS.md release workflow, ARCHITECTURE.md, expanded README, this changelog.
|
||||
- Tooling: `scripts/release.sh` (bump, build, test, commit, tag).
|
||||
|
||||
## 0.1.0 - 2026-10-23
|
||||
- First release: `list`, `get_or_create`, `attach`, `delete`.
|
||||
- overlayfs over host `/usr /etc /opt`, user/mount/pid/uts/ipc/net namespaces.
|
||||
- Networking: slirp4netns (default), `host`, `none`.
|
||||
- End-to-end test suite and AppArmor profile for Ubuntu 24.04+.
|
||||
+1
-1
@@ -9,7 +9,7 @@ if(NOT CMAKE_BUILD_TYPE)
|
||||
endif()
|
||||
|
||||
add_executable(doos
|
||||
src/main.cpp src/cli.cpp src/container.cpp src/network.cpp src/sandbox.cpp src/util.cpp)
|
||||
src/main.cpp src/hardening.cpp src/idmap.cpp src/image.cpp src/cgroup.cpp src/cli.cpp src/container.cpp src/network.cpp src/sandbox.cpp src/util.cpp)
|
||||
target_compile_options(doos PRIVATE -Wall -Wextra -Wpedantic)
|
||||
target_compile_definitions(doos PRIVATE DOOS_VERSION="${DOOS_VERSION}")
|
||||
install(TARGETS doos RUNTIME DESTINATION bin)
|
||||
|
||||
@@ -1,5 +1,18 @@
|
||||
# doos
|
||||
|
||||
User-level containers in C++20: overlayfs, user/mount/pid namespaces, slirp4netns or pasta networking, optional images (Alpine, Ubuntu, Debian, or your own rootfs), memory/cpu/pid/size limits, default hardening (capabilities, seccomp). No root at runtime.
|
||||
|
||||
```
|
||||
doos list
|
||||
doos get_or_create [OPTIONS] NAME
|
||||
doos attach [OPTIONS] NAME [-- CMD...]
|
||||
doos delete [--yes] NAME
|
||||
doos image list | add [--sha256 HEX] SOURCE [NAME] | delete [--yes] NAME
|
||||
doos apparmor # profile text for this binary (Ubuntu 24.04+)
|
||||
```
|
||||
|
||||
Version: see `VERSION` (currently 1.1.1). Release history: `CHANGELOG.md`.
|
||||
|
||||
User-level containers in C++20: overlayfs over the host's `/usr`, `/etc`, `/opt`, user/mount/pid namespaces, networking through slirp4netns. No root at runtime.
|
||||
|
||||
```
|
||||
@@ -11,10 +24,154 @@ doos delete [--yes] NAME
|
||||
|
||||
Build: `cmake -S . -B build && cmake --build build`. Test: `tests/run.sh`.
|
||||
|
||||
Requirements: Linux >= 5.11, `slirp4netns` for the default network mode. On Ubuntu 24.04+ install the AppArmor profile once:
|
||||
Requirements: Linux >= 5.11, `slirp4netns` (default network mode), `pasta` (only for `--net pasta`), `gnupg` (signature checks; optional), `uidmap` (`newuidmap`/`newgidmap`, for more than one id inside; falls back to one id), `curl`, `tar` and `sha256sum` (images). On Ubuntu 24.04+ (`kernel.apparmor_restrict_unprivileged_userns=1`) AppArmor must allow user namespaces for the
|
||||
binary. Generate a profile for the exact path doos is installed at, once:
|
||||
|
||||
```
|
||||
sudo cp apparmor/doos /etc/apparmor.d/doos && sudo apparmor_parser -r /etc/apparmor.d/doos
|
||||
doos apparmor | sudo tee /etc/apparmor.d/doos-local >/dev/null && sudo apparmor_parser -r /etc/apparmor.d/doos-local
|
||||
```
|
||||
|
||||
If you skip this, doos tells you the exact command (with a file name unique to the binary's path, so installing doos
|
||||
in a second place does not replace the first profile). `apparmor/doos` is a static profile for packaging that covers
|
||||
`/usr/local/bin`, `/usr/bin`, `/opt/doos/bin`, `~/.local/bin`, `~/bin` and `~/projects/doos/build`.
|
||||
|
||||
See `SPEC.md` (behavior, limits) and `docs/RESEARCH.md` (what works, measurements, known gaps).
|
||||
|
||||
## Usage examples
|
||||
|
||||
```
|
||||
doos attach dev # host mode: your system's /usr as base, shell as root inside
|
||||
doos attach --image alpine box # first use downloads and verifies Alpine, then opens a shell
|
||||
doos attach --image ubuntu web -- apt-get update
|
||||
doos attach --image ubuntu-24.04 old # a specific Ubuntu release
|
||||
doos attach --size 8g big # raise the writable-layer cap (default 4g)
|
||||
doos attach --net none sandbox -- sh # no network at all
|
||||
doos list
|
||||
doos image list # what is installed, what the catalog offers
|
||||
doos delete dev # dry run: shows what would go
|
||||
doos delete --yes dev # really deletes
|
||||
doos image delete --yes alpine # refused while a container uses it
|
||||
```
|
||||
|
||||
Your own image (a root filesystem tarball):
|
||||
|
||||
```
|
||||
doos image add https://example.org/debian-rootfs.tar.xz --sha256 <hex> debian # URL needs the hash
|
||||
doos image add ./rootfs.tar.gz mybase # local file
|
||||
doos attach --image mybase c1
|
||||
```
|
||||
|
||||
## Host mode or image mode
|
||||
|
||||
| | Host mode (default) | Image mode (`--image`) |
|
||||
|---|---|---|
|
||||
| Base system | your `/usr`, `/opt`, copy of `/etc` | the image, extracted once and shared |
|
||||
| Needs a download | no | first use only |
|
||||
| `apt-get install` | works when the host is Debian/Ubuntu based | works for the image's own package manager (apt, apk) |
|
||||
| Sees host files | `/usr`, `/opt`, readable `/etc` at creation | nothing from the host |
|
||||
| Best for | quick throwaway shells with your tools | reproducible, separate systems |
|
||||
|
||||
Writes always stay in the container (copy-on-write); `/root` and `/var` persist, `/tmp` is memory-backed and lost on exit. A container is `running` while its first `attach` command is alive; more `attach` calls join it. Data persists until `delete --yes`.
|
||||
|
||||
## Networking
|
||||
|
||||
Every example below was run on the development machine. Pick the mode when you create the container; it cannot change later.
|
||||
|
||||
| Mode | Outbound | Host `127.0.0.1` from inside | Reach a port served inside, from the host | Host abstract unix sockets |
|
||||
|---|---|---|---|---|
|
||||
| `slirp` (default) | yes | blocked | no | not shared |
|
||||
| `pasta` | yes | reachable | yes (forwarded automatically) | not shared |
|
||||
| `host` | yes | the same network | yes | **shared** |
|
||||
| `none` | no | no | no | not shared |
|
||||
|
||||
```
|
||||
# slirp (default): the internet works, the host's loopback does not
|
||||
doos attach web -- curl -sS -o /dev/null -w "%{http_code}\n" https://example.com # 200
|
||||
doos attach web -- curl -sS -m 3 http://127.0.0.1:8099 # fails: host loopback is blocked
|
||||
doos attach web -- getent hosts example.com # DNS works (resolver 10.0.2.3)
|
||||
|
||||
# pasta: behaves like the host network, in a private namespace
|
||||
doos attach --net pasta dev -- curl -sS http://127.0.0.1:8099 # reaches a service on the host's loopback
|
||||
doos attach dev -- python3 -m http.server 8088 --bind 0.0.0.0 # then, on the host: curl http://127.0.0.1:8088/
|
||||
doos attach --image alpine --net pasta al -- wget -qO- https://example.com # works with images too
|
||||
|
||||
# none: no network at all, only loopback
|
||||
doos attach --net none iso -- curl -m 3 https://example.com # curl exits 6, /proc/net/dev lists only lo
|
||||
|
||||
# host: the host's own network namespace, explicit opt-in (prints a note recommending pasta)
|
||||
doos attach --net host h -- curl -sS -o /dev/null -w "%{http_code}\n" https://example.com # 200
|
||||
```
|
||||
|
||||
Notes, all measured:
|
||||
- `slirp` has no inbound path: a server started in a `slirp` container is not reachable from the host (connection fails). Use `pasta` for that.
|
||||
- `pasta` notices a newly bound port after about 2 seconds, so start the server and give it a moment before connecting from the host.
|
||||
- Two `pasta` containers can talk to each other through the host's loopback: start a server in one (`python3 -m http.server 8066 --bind 0.0.0.0`), wait about 5 seconds, then `doos attach cli -- curl http://127.0.0.1:8066/` from the other returns 200. The host's own public address is not forwarded; use `127.0.0.1` or the default gateway address.
|
||||
- `pasta` starts in about 65 ms per attach, `slirp` about 40 ms, `none` about 10 ms.
|
||||
|
||||
## Options
|
||||
|
||||
| Option | Meaning | Default | Later |
|
||||
|---|---|---|---|
|
||||
| `--image NAME` | alpine, ubuntu, ubuntu-24.04, debian, debian-bookworm, or your own | host system | fixed |
|
||||
| `--net M` | `slirp` (outbound only, host loopback blocked), `pasta` (host-like, private sockets), `host`, `none` | slirp | fixed |
|
||||
| `--ids X` | `single` or `subuid` | subuid if possible | fixed |
|
||||
| `--hardening H` | `default` or `relaxed` | default | fixed |
|
||||
| `--size S` | writable data cap | 4g | changeable |
|
||||
| `--memory S` | memory cap | 8g | changeable |
|
||||
| `--pids N` | process/thread cap | 4096 | changeable |
|
||||
| `--cpus N` | cpu cap, e.g. `0.5`, `2` | all cores | changeable |
|
||||
|
||||
Changeable options apply the next time the container starts. Examples: `doos attach --memory 2g --cpus 1 build`, `doos attach --net pasta --image debian-bookworm web`.
|
||||
|
||||
## Limits, size cap and hardening
|
||||
|
||||
- **Limits** (`--memory`, `--pids`, `--cpus`): cgroup v2, measured here: a 256m container touching 400 MB is killed whole and doos says so; 80 forks against `--pids 32` fail and are reported; `--cpus 0.5` held 4 busy loops to 2.3 CPU-seconds in 4.3 s of wall time.
|
||||
- **Size cap**: `--size` limits what a container writes. Checked from inside the id map (so other users' files count), warns at 80%, freezes at 100% (nothing is deleted), thaws below 95%. A second `doos attach NAME -- ...` is never frozen, so you can clean up. No single file can exceed the cap.
|
||||
- **Hardening** (default): capabilities reduced to Docker's default set, a seccomp filter (no mounts, keyrings, bpf, perf, io_uring, nested user namespaces, kexec, module loading...), and a host `/etc` allowlist instead of a full copy. `--hardening relaxed` switches all three off for tools that need them (container runtimes, Chromium's sandbox).
|
||||
- All limits need cgroup v2 delegation (standard on systemd desktops). Without it doos prints that nothing is enforced.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Variable | Meaning |
|
||||
|---|---|
|
||||
| `DOOS_HOME` | State directory (default `~/.local/share/doos`). No `,` or `:` allowed. |
|
||||
| `XDG_DATA_HOME` | Used as `$XDG_DATA_HOME/doos` when `DOOS_HOME` is unset. |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Message | Cause and fix |
|
||||
|---|---|
|
||||
| `write /proc/<pid>/uid_map ... blocked` or `mount ...: Permission denied` | AppArmor restricts user namespaces. Install `apparmor/doos` (command above). |
|
||||
| `slirp4netns failed to start` | Install `slirp4netns`, or use `--net none` / `--net host`. |
|
||||
| `subuid mapping unavailable` | Install `uidmap` and make sure `/etc/subuid` and `/etc/subgid` list your user, or use `--ids single`. |
|
||||
| `sha256 mismatch` | The download is not what the publisher lists; nothing was installed. Retry or check the source. |
|
||||
| `a URL image needs --sha256` | Unverifiable input is rejected by design; pass the hash from the publisher. |
|
||||
| `limits are not enforced and the size cap only warns` | No delegated cgroup v2 (no systemd user manager). Limits and cap are advisory. |
|
||||
| `was killed by its memory limit` | Raise `--memory`. `/tmp` is memory-backed and counts. |
|
||||
| `hit its process limit` | Raise `--pids`; threads count as processes. |
|
||||
| `AppArmor restricts user namespaces for this binary` | Run the command doos prints (`doos apparmor \| sudo tee ... && sudo apparmor_parser -r ...`), once per install path. |
|
||||
| `Operation not permitted` from `mount`, `unshare`, `keyctl`, io_uring | Hardening blocked it. Use `--hardening relaxed` on that container. |
|
||||
| `signature NOT checked (gpg is not installed)` | Install `gnupg` for signature checks; the checksum still applies. |
|
||||
| `doos: seeded ...: unreadable skipped: N` | Host entries the user cannot read were left out of the container. Informational. |
|
||||
| `state path must not contain ',' or ':'` | Set `DOOS_HOME` to a plain path. |
|
||||
| `container 'x' is running` on delete | Exit the attached command first. |
|
||||
| `container limit reached (64)` | Delete one first. |
|
||||
|
||||
## Safety summary
|
||||
|
||||
Host files are never written: system dirs are reached only through overlayfs. Host `/home`, `/sys` and host
|
||||
loopback are not reachable in the default network mode. Images are verified before they are extracted: sha256
|
||||
from the publisher, and for Alpine and Ubuntu a signature checked against a key pinned by fingerprint. Processes
|
||||
run with Docker's default capability set and a seccomp filter. Memory, pids, cpus and written data are capped.
|
||||
`--net host` removes network isolation and is opt-in (`--net pasta` is the safer host-like choice).
|
||||
What is still not covered is listed at the end of `docs/RESEARCH.md`.
|
||||
|
||||
## Project docs
|
||||
|
||||
| File | Content |
|
||||
|---|---|
|
||||
| `SPEC.md` | Behavior contract and limits |
|
||||
| `docs/ARCHITECTURE.md` | Code map and flows |
|
||||
| `docs/RESEARCH.md` | What works unprivileged, measurements, gaps |
|
||||
| `AGENTS.md` | Rules for AI agents, release workflow |
|
||||
| `CHANGELOG.md` | Release history |
|
||||
|
||||
@@ -6,58 +6,129 @@ Authoritative description of behavior and limits. Code constants live in `src/sp
|
||||
|
||||
| Command | Behavior |
|
||||
|---|---|
|
||||
| `doos list` | Table of NAME, STATE (running/stopped), NET, AGE. |
|
||||
| `doos get_or_create [--net M] NAME` | Creates the container if missing. Prints `created NAME` or `exists NAME`. |
|
||||
| `doos attach [--net M] NAME [-- CMD...]` | get_or_create, then run CMD (default `/bin/bash`, else `/bin/sh`). Starts the container if stopped, joins it if running. Exit code is the command's. |
|
||||
| `doos list` | Table of NAME, STATE (running/stopped), NET, IMAGE (`host` when none), SIZE (cap), AGE. |
|
||||
| `doos get_or_create [OPTIONS] NAME` | Creates the container if missing. Prints `created NAME` or `exists NAME`. |
|
||||
| `doos attach [OPTIONS] NAME [-- CMD...]` | get_or_create, then run CMD (default `/bin/bash`, else `/bin/sh`). Starts the container if stopped, joins it if running. Exit code is the command's. |
|
||||
| `doos delete [--yes] NAME` | Dry run unless `--yes`. Refuses a running container. |
|
||||
| `doos image list` | Installed images (SIZE, IDS) plus catalog entries marked `available`. |
|
||||
| `doos image add [--sha256 HEX] SOURCE [NAME]` | Installs an image. See Images. |
|
||||
| `doos apparmor` | Prints an AppArmor profile (named after the binary's path) that lets this binary use user namespaces. A start that fails with EPERM/EACCES while `apparmor_restrict_unprivileged_userns=1` prints the command to install it. |
|
||||
| `doos image delete [--yes] NAME` | Dry run unless `--yes`. Refuses an image used by any container, even with `--yes`. |
|
||||
|
||||
`get-or-create` is accepted as an alias. Network mode `M` is fixed at creation: `slirp` (default), `host`, `none`.
|
||||
`get-or-create` is accepted as an alias.
|
||||
|
||||
Container options (accepted by `get_or_create` and `attach`):
|
||||
|
||||
| Option | Meaning | Default | Changeable later |
|
||||
|---|---|---|---|
|
||||
| `--image NAME` | Start from an image instead of the host system. Missing catalog images are fetched on demand. | none (host mode) | no |
|
||||
| `--net slirp\|pasta\|host\|none` | Network mode, see Networking. | slirp | no |
|
||||
| `--ids single\|subuid` | Id mapping. | subuid when available, else single | no |
|
||||
| `--hardening default\|relaxed` | `relaxed` = no capability drop, no seccomp filter, full readable host `/etc` (host mode). | default | no |
|
||||
| `--size SIZE` | Cap for what the container writes (`512m`, `8g`, `1t`; k/m/g/t). | 4g | yes |
|
||||
| `--memory SIZE` | Memory cap (cgroup `memory.max`, swap 0, whole-container OOM kill). | 8g | yes |
|
||||
| `--pids N` | Process and thread cap (`pids.max`). | 4096 | yes |
|
||||
| `--cpus N` | CPU cap, decimals allowed (`0.5`, `2`); `0` = all cores. | 0 | yes |
|
||||
|
||||
Changeable options apply at the next start of the container. Naming a fixed option on an existing container with a different value is an error, never a silent change.
|
||||
|
||||
## Limits (hard caps)
|
||||
|
||||
| Item | Cap |
|
||||
|---|---|
|
||||
| Containers | 64 |
|
||||
| Name | `[a-z0-9][a-z0-9_-]{0,31}` |
|
||||
| Images | 16 |
|
||||
| Name | containers `[a-z0-9][a-z0-9_-]{0,31}`, images `[a-z0-9][a-z0-9._-]{0,31}` (`:` becomes `-`) |
|
||||
| Command arguments | 256 |
|
||||
| Open files inside | 4096 |
|
||||
| `/tmp`, `/run`, `/dev/shm` | 256m, 64m, 64m (tmpfs) |
|
||||
| `--size` | 64m to 1t |
|
||||
| Size scan interval | 2 s floor, and at least 10x the duration of the last scan |
|
||||
| `--memory` | 64m to 1t |
|
||||
| `--pids` | 16 to 4,194,304 |
|
||||
| `--cpus` | 0.1 to 1024 |
|
||||
| Single file size inside | the `--size` cap (`RLIMIT_FSIZE`) |
|
||||
| Size scan entries | 5,000,000 |
|
||||
| Image download | 2 GiB, 900 s |
|
||||
| Skeleton directories | 200,000 |
|
||||
| `/etc` copy | 50,000 files, 256 MiB |
|
||||
| Id range per container | 65536 ids (0 plus up to 65535 from subuid) |
|
||||
| slirp4netns ready wait | 5 s |
|
||||
| Lock wait | 5 s |
|
||||
| Delete recursion depth | 256 |
|
||||
| Tree recursion depth | 256 |
|
||||
|
||||
## State layout
|
||||
|
||||
`$DOOS_HOME`, else `$XDG_DATA_HOME/doos`, else `~/.local/share/doos`. Path must not contain `,` or `:`.
|
||||
|
||||
```
|
||||
containers/NAME/{meta,pid,lock,rootfs/,layers/<dir>/{upper,work},private/{root,var}}
|
||||
containers/NAME/{meta,pid,lock,seeded.*,rootfs/,layers/<dir>/{upper,work},private/{root,var,etc}}
|
||||
images/NAME/{meta,rootfs/}
|
||||
```
|
||||
|
||||
## Container filesystem
|
||||
`meta` keys: `net created version image ids size`.
|
||||
|
||||
## Container filesystem, host mode (no `--image`)
|
||||
|
||||
| Path | Source |
|
||||
|---|---|
|
||||
| `/usr /etc /opt` (and `/bin /sbin /lib /lib32 /lib64` when real dirs) | overlayfs, lower = host dir, upper = `layers/<dir>/upper` |
|
||||
| `/usr /opt` (and `/bin /sbin /lib /lib32 /lib64` when real dirs) | overlayfs, lower = host dir, upper = `layers/<dir>/upper`, with a directory skeleton in the upper |
|
||||
| `/bin /sbin /lib...` when host symlinks | same symlink recreated |
|
||||
| `/root /var` | persistent private dirs, no host data |
|
||||
| `/etc` | private copy, made once at creation: under `--hardening default` only the allowlist in `spec::kEtcAllow` (passwd, group, nsswitch, ld.so*, ssl, ca-certificates, alternatives, apt, dpkg, shell and tool defaults...) minus names that may hold credentials (`private`, `shadow`, `gshadow`, `sudoers*`, `auth.conf*`, `ssh_host_*`, `*.key`); under `relaxed` every readable file |
|
||||
| `/root`, `/var` | persistent private dirs; `/var` pre-seeded with the apt/dpkg directories |
|
||||
| `/var/lib/dpkg` | overlay of the host dpkg database (when the host has it), regular files copied into the upper, lock files pre-created |
|
||||
| `/tmp /run /dev/shm` | tmpfs |
|
||||
| `/dev` | tmpfs + bound null zero full random urandom tty, new devpts instance |
|
||||
| `/proc` | own pid namespace; `sys sysrq-trigger irq bus fs` read-only; `kcore keys timer_list` masked |
|
||||
| `/sys`, `/home`, everything else on host | absent |
|
||||
|
||||
Why the skeleton and copies: host root-owned files appear as `nobody` inside the container. Container root cannot write into directories it does not own, nor open files it does not own for writing, nor hard-link them. The skeleton recreates every host directory as a container-owned directory in the upper layer (same mode, owner rwx added); the copies make the few files package managers rewrite container-owned.
|
||||
|
||||
## Container filesystem, image mode (`--image`)
|
||||
|
||||
`/` is one overlay: lower = `images/NAME/rootfs` (shared, never written), upper = `layers/root/upper`. `/tmp /run /dev /proc` are mounted over it as in host mode. The image is extracted under the same id map the container uses, so image files owned by root are owned by the container's root and nothing needs a skeleton or copies. `LANG` is `C.UTF-8`.
|
||||
|
||||
## Images
|
||||
|
||||
`SOURCE` for `image add`:
|
||||
|
||||
| Source | Resolution | Verification |
|
||||
|---|---|---|
|
||||
| `alpine` | latest-stable minirootfs from `dl-cdn.alpinelinux.org`, via `latest-releases.yaml` | sha256 from the yaml, and the detached `.asc` signature checked against Alpine's pinned release key (`0482D840 22F52DF1 C4E7CD43 293ACD09 07D9495A`) |
|
||||
| `ubuntu`, `ubuntu-VERSION` | newest `ubuntu-base-*-base-ARCH.tar.gz` of that release from `cdimage.ubuntu.com` (default `26.04`) | `SHA256SUMS` checked with `SHA256SUMS.gpg` against Ubuntu's pinned CD image key (`84393 8DF 228D 22F7 B374 2BC0 D94A A3F0 EFE2 1092`), then sha256 |
|
||||
| `debian`, `debian-RELEASE` | `debuerreotype/docker-debian-artifacts` OCI layer for `RELEASE` (default `trixie`), slim variant | sha256 = the layer digest in the OCI manifest; no publisher signature exists |
|
||||
| `https://...` | downloaded | `--sha256 HEX` required, otherwise rejected |
|
||||
| a file path | read locally | verified when `--sha256` is given; otherwise used with a printed note |
|
||||
|
||||
`http://` is refused. Signature keys are fetched over https and rejected unless their fingerprint equals the pinned one; a bad, missing or foreign signature rejects the image. Without `gpg` installed the signature step is skipped with a printed note and the checksum still applies. A mismatch deletes everything it downloaded. The tarball is extracted with `tar -p --numeric-owner`, excluding `dev/*`, inside a user namespace with the image's id mode. Architectures: x86_64 and aarch64 for the catalog.
|
||||
|
||||
## Size cap
|
||||
|
||||
The parent process measures the container's writable data (upper layers and private dirs, hard links counted once) from inside the container's id map, so files and mode-700 directories of other container ids are counted. A scan runs at most every 2 s and never uses more than 1/10 of the time (the wait is at least 10x the last scan). At 80% it warns once on stderr. At 100% it freezes the container's `main` cgroup (`cgroup.freeze`); processes stop, nothing is deleted. It thaws on its own once usage is below 95%. A second `doos attach` runs in the `shell` cgroup and is not frozen, so the user can free space from inside. No file can exceed the cap on its own (`RLIMIT_FSIZE`), which bounds the overshoot between scans. Without cgroup v2 delegation the cap only warns and says so at start.
|
||||
|
||||
## Resource limits
|
||||
|
||||
Each running container gets `doos/<name>/` below the cgroup delegated to the user's systemd manager, with leaves `main` (init and its descendants) and `shell` (later attaches). Limits sit on `doos/<name>` and cover both leaves: `memory.max` and `memory.swap.max=0` with `memory.oom.group=1` (an OOM kill takes the whole container), `pids.max`, `cpu.max`. After exit doos reports OOM kills and process-limit hits. A missing controller is named on stderr and the other limits still apply. Without delegation no limit is enforced, and doos says so at start. The group is removed when the container stops.
|
||||
|
||||
## Hardening (`--hardening default`)
|
||||
|
||||
Every process that runs a container command gets, right before exec: `no_new_privs`, core dumps off, the capability bounding set reduced to the Docker default set (`CHOWN DAC_OVERRIDE FOWNER FSETID KILL SETGID SETUID SETPCAP NET_BIND_SERVICE NET_RAW SYS_CHROOT MKNOD AUDIT_WRITE SETFCAP`, mask `a80425fb`), and a seccomp-bpf filter: wrong architecture kills the process, x32 numbers get ENOSYS, `clone3` gets ENOSYS (libc falls back to `clone`), `clone`/`unshare` with `CLONE_NEWUSER` get EPERM, and these get EPERM: the mount family and `pivot_root`, `setns`, keyring calls, `bpf`, `perf_event_open`, `userfaultfd`, `io_uring_*`, `kexec_*`, module loading, `open_by_handle_at`, `process_vm_*`, `kcmp`, `reboot`, `swapon/off`, clock and time setting, `acct`, `quotactl` and old obsolete calls. Supported on x86_64 and aarch64; elsewhere only the capability drop applies, with a printed note. Tools that need the denied calls (container runtimes inside a container, Chromium's sandbox, io_uring users) need `--hardening relaxed`.
|
||||
|
||||
## Isolation
|
||||
|
||||
User, mount, pid, uts, ipc namespaces always; network namespace unless `host`. Single-entry id map (container root = invoking user). `no_new_privs`, core dumps off. pid 1 is a reaping init that forwards SIGTERM/SIGINT/SIGHUP/SIGQUIT and kills the container when the command exits. Container lifetime = the first `attach` command; later attaches join via setns.
|
||||
User, mount, pid, uts, ipc namespaces always; network namespace unless `host`. Id map: container 0 is the invoking user; with `subuid` ids 1..65535 map to the user's `/etc/subuid` range through `newuidmap`/`newgidmap`; with `single` nothing else is mappable. `no_new_privs`, core dumps off. pid 1 is a reaping init that forwards SIGTERM/SIGINT/SIGHUP/SIGQUIT and kills the container when the command exits. Container lifetime = the first `attach` command; later attaches join via setns.
|
||||
|
||||
## Networking
|
||||
|
||||
- `slirp`: private netns, `slirp4netns --configure --disable-host-loopback`, DNS 10.0.2.3, outbound only, host 127.0.0.1 unreachable.
|
||||
- `none`: private netns with loopback only.
|
||||
- `host`: shares the host network namespace (no isolation, explicit opt-in).
|
||||
- `slirp` has no inbound path (nothing outside reaches a port served inside).
|
||||
- `pasta`: private netns that behaves like the host network: addresses and routes mirrored, host loopback reachable, ports bound inside are forwarded to the host, DNS from the host's resolver. The host's abstract unix sockets are not shared. Needs `pasta`. Newly bound ports are picked up after about 2 s; only loopback and the default gateway address reach other containers' forwarded ports, not the host's public address.
|
||||
- `host`: shares the host network namespace including its abstract unix sockets (explicit opt-in; a note recommends `pasta`).
|
||||
|
||||
## Safety rules
|
||||
|
||||
- `delete` is dry-run by default, scoped to `containers/NAME`, refuses running containers, never follows symlinks.
|
||||
- Invalid names, bad state paths and missing slirp4netns fail loudly.
|
||||
- A failed start kills the half-built container (no orphans).
|
||||
- `delete` (containers and images) is dry-run by default, scoped to the store, refuses running containers and used images, never follows symlinks. Trees with subuid-owned files are removed from inside the id map.
|
||||
- Unverifiable input is rejected: URL images without sha256, hash mismatches, bad names, bad state paths, missing slirp4netns or newuidmap when requested.
|
||||
- Seeding and scans report what they skipped (`unreadable skipped: N`).
|
||||
- A failed start kills the half-built container (no orphans); a failed image add or container create leaves nothing behind.
|
||||
|
||||
+2
-1
@@ -2,10 +2,11 @@
|
||||
# kernel.apparmor_restrict_unprivileged_userns=1 strips capabilities (mount, etc.)
|
||||
# from unknown binaries inside a new userns. Same pattern as /etc/apparmor.d/lxc-unshare.
|
||||
# The profile is unconfined apart from granting `userns`; confinement comes from doos itself.
|
||||
# Fixed paths for packaging. For any other location run: doos apparmor | sudo tee /etc/apparmor.d/doos
|
||||
abi <abi/5.0>,
|
||||
include <tunables/global>
|
||||
|
||||
profile doos /{usr/local/bin,home/*/projects/doos/build}/doos flags=(unconfined) {
|
||||
profile doos /{usr/local/bin,usr/bin,opt/doos/bin,home/*/.local/bin,home/*/bin,home/*/projects/doos/build}/doos flags=(unconfined) {
|
||||
userns,
|
||||
@{exec_path} mr,
|
||||
}
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
# Architecture
|
||||
|
||||
How doos works internally. Behavior contract is `SPEC.md`; this explains the code.
|
||||
|
||||
## File map
|
||||
|
||||
| File | Class / role |
|
||||
|---|---|
|
||||
| `src/spec.hpp` | `spec::` constants: caps, mount lists, device list, paths. Single source of truth with `SPEC.md`. |
|
||||
| `src/util.{hpp,cpp}` | `SysError`, `Fd` (RAII fd), `FileLock` (bounded flock), `parseSize`/`formatSize`, `base64Decode`, `runProcess` (fork/exec, capped capture), `fsutil` (mkdirs, read/write, bounded `removeTree`, `mirrorDirs` skeleton, `copyTree`/`copyEntries` (with skip predicate), `copyFiles`, `diskUsage` with hardlink dedupe). |
|
||||
| `src/idmap.{hpp,cpp}` | `IdMap` (single or subuid mapping via newuidmap/newgidmap, subuid range lookup), `UserNsJob` (run a function as root of a user namespace with the same map: image extraction, deleting subuid-owned trees). |
|
||||
| `src/image.{hpp,cpp}` | `Catalog` (alpine, ubuntu, debian resolution from publishers), `verifySignature` (gpg with a throwaway home, pinned fingerprint), `ImageStore` (add with download, sha256 and signature check, extraction; list; delete). |
|
||||
| `src/cgroup.{hpp,cpp}` | `Limits`, `Cgroup`: `doos/<name>/{main,shell}` under the user's delegated cgroup; limits on the parent, freeze on `main`, later attaches in `shell`, OOM and pid-limit counters. |
|
||||
| `src/hardening.{hpp,cpp}` | `Hardening::apply`: capability bounding set to the Docker default, seccomp-bpf denylist built at runtime (arch check, x32 guard, clone3 ENOSYS, CLONE_NEWUSER filter). |
|
||||
| `src/network.{hpp,cpp}` | `NetMode` (slirp, pasta, host, none) + `Network` (parse, resolv.conf per mode), `SlirpSession`, `PastaSession` (own the helper processes). |
|
||||
| `src/container.{hpp,cpp}` | `Options` (what the user asked for), `Container` (paths, meta, pid file, layout, one-time seeding, usage), `Store` (list, get_or_create with on-demand image fetch, remove). |
|
||||
| `src/sandbox.{hpp,cpp}` | `Sandbox`: start a container (clone + id map + mounts + pivot_root + init), watch it (size cap), or join a running one (setns). |
|
||||
| `src/cli.{hpp,cpp}` | `Cli`: argument parsing, the four container commands and `image list/add/delete`. |
|
||||
| `src/main.cpp` | Catches exceptions, prints `doos: <message>`, exit 1. |
|
||||
| `tests/run.sh` | End-to-end suite. |
|
||||
| `scripts/release.sh` | Bump, build, test, commit, tag. |
|
||||
| `apparmor/doos` | Profile granting `userns` on Ubuntu 24.04+. |
|
||||
|
||||
## attach flow
|
||||
|
||||
```
|
||||
Cli::cmdAttach
|
||||
Store::getOrCreate lock, create layout + meta if missing
|
||||
Sandbox::run
|
||||
FileLock (start-or-join decision)
|
||||
runningPid()? yes -> join() no -> start()
|
||||
|
||||
start():
|
||||
clone(NEWUSER|NEWNS|NEWPID|NEWUTS|NEWIPC[|NEWNET]) child = container pid 1
|
||||
IdMap::apply(child, ids) single: write /proc maps; subuid: newuidmap + newgidmap
|
||||
slirp mode: SlirpSession.start(child) waits for ready-fd
|
||||
slirp/pasta session for the netns of child
|
||||
Cgroup::create(limits) + addMain(child) best effort; without it nothing is enforced and doos says so
|
||||
recordPid(child) (pid + starttime, defeats pid reuse)
|
||||
lock.unlock() joiners must not wait for container lifetime
|
||||
write 'G' on sync pipe child continues
|
||||
parent: ignore SIGINT/SIGQUIT, forward SIGTERM/SIGHUP, watch(child)
|
||||
watch: poll(pidfd, until next scan); scan = usage from inside the id map (UserNsJob) -> warn 80%, freeze 100%,
|
||||
thaw <95%; next scan after max(2 s, 10 x scan time)
|
||||
after exit: report OOM kills / pid-limit hits
|
||||
on exit: pid file cleared, slirp stopped, cgroup removed (RAII)
|
||||
|
||||
child (containerInit, pid 1):
|
||||
PDEATHSIG=SIGKILL, close inherited lock fd, wait for 'G'
|
||||
setupRoot -> sethostname -> loopback up (isolated netns) -> rlimits/env/no_new_privs
|
||||
fork worker -> applyLimitsAndEnv (RLIMIT_FSIZE = size cap, Hardening::apply unless relaxed) -> exec command
|
||||
pid 1 loop: forward signals, reap all, on worker exit kill(-1, SIGKILL), exit with its code
|
||||
```
|
||||
|
||||
## setupRoot order
|
||||
|
||||
Common start: `MS_REC|MS_PRIVATE` on `/`, then `resolv.conf` is written into the container's `/etc` location
|
||||
(`Container::etcDir`: `private/etc` in host mode, the root upper layer in image mode), and on a single-id
|
||||
container with apt present an `APT::Sandbox::User "root"` override is written next to it.
|
||||
|
||||
Host mode:
|
||||
1. Bind the container `rootfs/` onto itself (pivot_root needs a mountpoint).
|
||||
2. Overlay mounts for `/usr`, `/opt` and real compat dirs (`lowerdir=<host dir>,upperdir,workdir,userxattr`).
|
||||
3. Compat symlinks recreated (usr-merge).
|
||||
4. Bind the private dirs `/root`, `/var`, `/etc`; then the overlay for `/var/lib/dpkg` inside the private `/var`.
|
||||
|
||||
Image mode:
|
||||
1. One overlay on `rootfs/`: lower = `images/NAME/rootfs`, upper = `layers/root/upper`.
|
||||
2. Mountpoints (`/proc /dev /tmp /run ...`) created inside the merged tree.
|
||||
|
||||
Both modes then: tmpfs `/tmp` and `/run`; `/dev` (tmpfs, bound host nodes, new devpts, `/dev/shm`, symlinks);
|
||||
`/proc` with read-only remounts and masks from `spec.hpp`; `chdir(rootfs)`, `pivot_root(".", ".")`, lazy
|
||||
unmount of the old root.
|
||||
|
||||
## Seeding (host mode, once per container)
|
||||
|
||||
`Container::prepareLayout` runs on create and as a heal on every later load; each step leaves a
|
||||
`seeded.<name>` marker so it runs once and prints what it did and what it skipped:
|
||||
1. Directory skeleton of every overlaid host dir into its upper layer (`fsutil::mirrorDirs`).
|
||||
2. Regular files of `/var/lib/dpkg`, `triggers`, `alternatives` copied into the upper (`copyFiles`), then empty lock files.
|
||||
3. `/etc` copied into `private/etc`: the allowlist (`copyEntries` with the credential deny predicate) under default hardening, everything readable (`copyTree`) under relaxed.
|
||||
|
||||
Reasons are in `SPEC.md` (host root-owned files appear as `nobody`).
|
||||
|
||||
## Images
|
||||
|
||||
`ImageStore::add`: lock-free staging dir `images/.tmp-NAME`, download with curl (https only, size and time caps),
|
||||
sha256 check, `tar -x -p --numeric-owner` executed by `UserNsJob` with the image's id mode (`--no-same-owner`
|
||||
for single), usage measured, `meta` written, atomic `rename` into place. Any failure removes the staging dir
|
||||
from inside the id map, so a failed add leaves nothing. `Store::getOrCreate` calls it when a catalog image is
|
||||
missing, and pins the container's id mode to the image's.
|
||||
|
||||
## join flow
|
||||
|
||||
Open `/proc/<pid>/ns/{user,mnt,uts,ipc,net,pid}` and `/proc/<pid>/root` first, fork, move the child into the
|
||||
container's `shell` cgroup (same limits, never frozen), `setns` user first (grants capability for the rest), `fchdir`+`chroot` to the container
|
||||
root, fork again (pid ns applies to children), apply env and limits, exec.
|
||||
In `host` network mode the net namespace is not joined.
|
||||
|
||||
## Lifetime model
|
||||
|
||||
The namespaces live as long as the first `attach` command. When it exits, pid 1 kills
|
||||
the rest and the container is `stopped`. State (upper dirs, `/root`, `/var`) persists on
|
||||
disk until `delete --yes`. There is no daemon.
|
||||
|
||||
## Error handling
|
||||
|
||||
All failures throw (`SysError` or `std::runtime_error`); `main` prints one line.
|
||||
Inside the child, errors print `doos: <message>` and exit 125. A parent-side failure
|
||||
after `clone` kills the child (`Reaper`), so no half-built container survives.
|
||||
+240
-1
@@ -29,7 +29,7 @@ Measured on this machine: Ubuntu, kernel 7.0.0-38, AppArmor 5.0.2, slirp4netns 1
|
||||
- No host paths writable: overlay lower dirs are only reachable through overlay (writes copy up to the container upper). Verified: write to `/usr` and `/etc` inside leaves host untouched.
|
||||
- Only `/usr /etc /opt` and compat dirs are exposed read-through. `/home`, `/var`, `/sys`, `/root` of the host are not present.
|
||||
- Old root detached after pivot, `no_new_privs`, `/proc` hardened, nosuid/nodev on tmpfs/devpts/overlay.
|
||||
- Residual exposure (gaps): host `/etc` is readable inside (any world-readable secret in it); abstract unix sockets are shared in `--net host`; no resource cgroups, so a container can fill the disk through its upper dir or fork-bomb within the user's `RLIMIT_NPROC`; kernel attack surface of userns is that of the host kernel.
|
||||
- Residual exposure (gaps G1 to G6 below): no resource limits, no disk cap, single uid, host `/etc` readable, abstract unix sockets shared in `--net host`, no seccomp filter.
|
||||
|
||||
## Measurements (this machine, 5 sequential `attach -- true`)
|
||||
|
||||
@@ -39,3 +39,242 @@ Measured on this machine: Ubuntu, kernel 7.0.0-38, AppArmor 5.0.2, slirp4netns 1
|
||||
| none | ~6 ms |
|
||||
|
||||
Single run set, one machine; no other hardware measured.
|
||||
|
||||
## Known gaps and researched solutions
|
||||
|
||||
Status per gap is in the table below and in "Resolution log (1.0.0)" at the end. Probes ran on the
|
||||
machine above (cgroup v2, systemd user manager, ext4 root, uid 1000). Probe cgroups were
|
||||
removed afterwards. Rows marked "unmeasured" have no run behind them.
|
||||
|
||||
| Id | Gap | Fix | Status | Code |
|
||||
|---|---|---|---|---|
|
||||
| G1 | Memory/cpu/pids limits | Cgroup v2 limits on `doos/<name>` | **done in 1.1.0** (`--memory`, `--pids`, `--cpus`) | `cgroup.cpp` |
|
||||
| G2 | Disk cap on upper dirs | Usage watchdog with freeze | **done in 1.0.0**, accounting and overshoot improved in 1.1.0 | `Sandbox::watch` |
|
||||
| G3 | Single uid mapping | `newuidmap`/`newgidmap` with `/etc/subuid`, fallback single | **done in 1.0.0** (`--ids`, default subuid) | `idmap.cpp`, `Store::remove` |
|
||||
| G4 | Host `/etc` readable | Allowlist-only private `/etc` | **done in 1.1.0** for host mode (`--hardening relaxed` restores the full copy); image mode never had host files | `container.cpp` |
|
||||
| G5 | `--net host` shares abstract sockets | pasta mode | **done in 1.1.0** (`--net pasta`); `host` stays as an explicit, noted opt-in | `network.cpp` |
|
||||
| G6 | No seccomp, nested userns allowed | Capability drop + seccomp-bpf denylist | **done in 1.1.0** (`--hardening`) | `hardening.cpp` |
|
||||
| G7 | Host-owned lower dirs unwritable, `chown` of host files fails, empty `/var` | Image layer + subuid; skeleton and seeding for host mode | **done in 1.0.0** | `image.cpp`, `container.cpp` |
|
||||
|
||||
### G1: memory, cpu and pids limits (cgroup v2)
|
||||
|
||||
Findings, measured:
|
||||
- `/sys/fs/cgroup` is cgroup2 with `nsdelegate`. `user@1000.service/cgroup.controllers` lists `cpuset cpu io memory pids`, and its `subtree_control` already enables all five. A plain user can `mkdir` below `/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/`.
|
||||
- A leaf with `pids.max=20`: the 21st fork failed with `Cannot fork`.
|
||||
- A leaf with `memory.max=64m`: touching 200 MiB ended in `Killed`, `memory.events` showed `oom_kill 1`. An untouched `bytearray(200MiB)` is not charged (lazy pages), so the first attempt proved nothing; only the touched run counts.
|
||||
- Gotcha, measured: enabling `+memory +pids` in `subtree_control` of a cgroup makes it a non-leaf; writing a pid into its `cgroup.procs` then fails with `I/O error`. The layout must be `.../doos/` (controllers enabled here, no processes) with one leaf per container `.../doos/<name>/` (limits set here, processes here).
|
||||
- `RLIMIT_NPROC` is not an option: it counts per real uid across the whole host user, not per container.
|
||||
|
||||
Design:
|
||||
1. Discover the base: parse `/proc/self/cgroup`, take the path up to and including `user@<uid>.service`.
|
||||
2. Parent creates `doos/<name>` after `clone`, writes `memory.max`, `memory.swap.max=0`, `pids.max`, `cpu.max`, then writes the child pid to `cgroup.procs` before the sync byte `G` (child never runs outside limits). `join()` children enter via the same `cgroup.procs` write before `setns`.
|
||||
3. On exit: write `1` to `cgroup.kill` (kernel >= 5.14), then `rmdir`.
|
||||
4. Defaults as constants in `spec.hpp` (static ceilings, no autotuning): for example memory 2g, pids 512, cpu unlimited. Flags `--memory`, `--pids`, `--cpus` may lower or raise within a hard cap.
|
||||
5. No delegation (no systemd user manager, cgroup v1): fail loudly with a counter and a clear reason; `--no-limits` is an explicit opt-out, never silent.
|
||||
|
||||
Rejected: `systemd-run --user --scope -p MemoryMax=...` (needs D-Bus, one more moving part). Unmeasured: behavior when doos itself runs inside a scope that is not under `user@.service` (the tmux scope here is, but other launchers may not be).
|
||||
|
||||
Sources: [rootlesscontaine.rs cgroup v2](https://rootlesscontaine.rs/getting-started/common/cgroup2/), [runc cgroup v2](https://github.com/thaJeztah/runc/blob/main/docs/cgroup-v2.md), [Docker rootless tips](https://docs.docker.com/engine/security/rootless/tips/).
|
||||
|
||||
### G2: disk usage of upper dirs
|
||||
|
||||
Findings:
|
||||
- Project quotas (XFS `pquota`, ext4 project quota) need root to set. This machine's root is ext4 on `/dev/md2`. Not usable unprivileged. Docker's overlay2 size limit also only works on XFS with `pquota` ([source](https://fabianlee.org/2020/01/15/docker-use-overlay2-with-an-xfs-backing-filesystem-to-limit-rootfs-size/)).
|
||||
- Loop-mounted ext4 images need root; the kernel does not allow ext4 mounts from a user namespace. `fuse2fs` is not installed here. `fuse-overlayfs` is installed but has no quota either.
|
||||
- A `size=` tmpfs mounted inside the container userns is enforced, measured: `size=1m`, writing 5 MiB ended in `No space left on device`.
|
||||
|
||||
Design (follows the rule "machines hold, humans analyze"):
|
||||
1. Default: the parent process (already alive in `Sandbox::start`) measures upper-dir usage on a fixed interval with a floor (for example every 30 s, never faster). Soft limit: warning on stderr. Hard limit: write `1` to the container's `cgroup.freeze` (needs G1). Freeze stops processes but deletes nothing; the human frees space or raises the limit. Never auto-delete.
|
||||
2. Opt-in `--ephemeral`: upper dirs on a size-capped tmpfs. Hard cap by the kernel, but memory-backed, counted in the memory cgroup, and lost on exit.
|
||||
3. Constants (soft, hard, interval) in `spec.hpp`.
|
||||
|
||||
Unmeasured: cost of scanning a large upper dir (needs a run on a populated container); overshoot between two scans.
|
||||
|
||||
### G3: more than one uid inside
|
||||
|
||||
Findings, measured with `podman unshare` (map `0 -> 1000`, `1..65536 -> 100000..`): overlayfs with `userxattr` mounts fine under a multi-id map. `/etc/subuid` and `/etc/subgid` contain `retoor:100000:65536`, `newuidmap` and `newgidmap` are installed.
|
||||
- Host root-owned files in the lower dirs show as `65534 nobody`. That is already true with the single map, so nothing gets worse.
|
||||
- Gain: `chown`, `su`, `apt` sandbox users, daemons that drop privileges.
|
||||
- Cost, important: files created by container uids 1..65535 are owned by host uids 100000+. The host user cannot delete them. `Store::remove` as written would fail with `Permission denied`. Delete must run inside a user namespace with the same map (re-enter and `rm` there, or `podman unshare`-style helper) before removing the directory.
|
||||
|
||||
Design: after `clone`, parent execs `newuidmap <pid> 0 <uid> 1 1 <sub> 65536` and the gid equivalent, `setgroups` allowed. Fallback to the current single map when the user has no subuid range, with a printed notice. Store the chosen mode in `meta` (`idmap=single|subuid`) so delete knows which path to take.
|
||||
|
||||
Unmeasured: `apt install` end to end.
|
||||
|
||||
### G4: host `/etc` visible inside
|
||||
|
||||
Finding, measured: `find /etc -type f -perm -004` returns 1116 world-readable files on this host, including names such as `/etc/fail2ban/action.d/cloudflare-token.conf` (content not read). Anything world-readable in host `/etc` is readable in every container.
|
||||
|
||||
Design: replace the `/etc` overlay by a private `/etc`. At create time, populate `private/etc` from an allowlist in `spec.hpp` (`kEtcAllow`): `passwd group nsswitch.conf ld.so.cache ld.so.conf ld.so.conf.d ssl/certs ca-certificates alternatives bash.bashrc profile profile.d localtime timezone os-release protocols services hosts hostname`, plus a generated `resolv.conf`. Two ways: copy (stable, can go stale, for example CA certs) or read-only bind of each allowed path (always current, more mounts). Recommendation: read-only bind, since doos already carries the mount machinery.
|
||||
|
||||
Unmeasured: whether this allowlist is enough for `apt`, `curl`, `python3`, `git`; needs a run per tool before fixing the list.
|
||||
|
||||
### G5: `--net host` and abstract unix sockets
|
||||
|
||||
Fact: abstract `AF_UNIX` sockets belong to the network namespace, so `--net host` exposes host abstract sockets (some X11 and D-Bus setups use them) and host loopback services. This is inherent to sharing the namespace.
|
||||
|
||||
Researched alternative, pasta (installed, `pasta 0.0~git20260120`): it attaches to an existing netns by pid (`pasta --config-net <pid>`), gives outbound access, and mirrors host addresses into the namespace. Measured: outbound TCP worked, and an inside connection to `127.0.0.1:22` reached the host (default flags, once). A second run with `--no-splice --no-map-gw --map-host-loopback none` printed `Couldn't get any nameserver address`, and I could not confirm the loopback result from it. Conclusion: pasta is not proven to match slirp4netns' `--disable-host-loopback` guarantee. Slirp stays the default. Next step is a clean test matrix of pasta flags with a listener on a known host port.
|
||||
|
||||
Interim: `host` remains an explicit opt-in documented as unisolated.
|
||||
|
||||
### G6: kernel attack surface (seccomp, nested user namespaces)
|
||||
|
||||
Findings:
|
||||
- A seccomp-bpf filter installs unprivileged once `no_new_privs` is set (already done). Measured with a raw `prctl(PR_SET_SECCOMP)` program: a denied syscall (`keyctl`) returned `-1 EPERM`. No `libseccomp` headers on this machine, and "fewer dependencies" argues for hand-assembled BPF.
|
||||
- `/proc/sys` is read-only in the container, so `user.max_user_namespaces=0` cannot be set from inside; seccomp is the lever.
|
||||
|
||||
Design: denylist modelled on the Docker default profile, applied after `setupRoot` and before `exec`, and also in the `join()` path. Check `AUDIT_ARCH` first and kill on a foreign ABI (blocks the x32 bypass). Deny with `EPERM`: `kexec_load kexec_file_load bpf perf_event_open userfaultfd keyctl add_key request_key open_by_handle_at process_vm_readv process_vm_writev reboot swapon swapoff init_module finit_module delete_module`. Nested userns: deny `unshare` and `setns` with `CLONE_NEWUSER`, deny `clone` with `CLONE_NEWUSER`, and return `ENOSYS` for `clone3` (flags live in a struct, cannot be inspected; libc falls back to `clone`). Trade-off: breaks nested containers and Chromium-style sandboxes; make it a documented constant, not a switch that hides in code.
|
||||
|
||||
Unmeasured: compatibility impact on real workloads.
|
||||
|
||||
### Priority
|
||||
|
||||
1. G1 (limits): cheapest, measured to work, unlocks G2's freeze.
|
||||
2. G6 (seccomp): self-contained, no new dependency.
|
||||
3. G4 (private `/etc`): needs the per-tool allowlist runs first.
|
||||
4. G2, then G3 (G3 changes the delete path, do it deliberately).
|
||||
5. G5: needs a test matrix before any change.
|
||||
|
||||
## Field trial: apt and common tools (v0.1.2, measured)
|
||||
|
||||
Run in a fresh `--net slirp` container on the host above (Ubuntu with apt 3.2.0, dpkg 1.23.7).
|
||||
|
||||
| Step | Result |
|
||||
|---|---|
|
||||
| curl https, git ls-remote, gcc compile+run, python3 (`ssl`, `sqlite3`), pip, ping 1.1.1.1 | all work |
|
||||
| `apt-get update` as shipped | fails: `/var/lib/apt/lists/partial is missing` (`/var` is an empty private dir) |
|
||||
| `apt-get install` as shipped | fails: `/var/lib/dpkg/lock-frontend` missing, no dpkg status database |
|
||||
| after seeding `/var` (dirs plus copy of host `dpkg/status`, `info`, `alternatives`, 40 MB) | apt starts, but the download method dies: `setegid 65534 failed`, `seteuid 42 failed` (sandbox user `_apt` cannot exist with a single id) |
|
||||
| with `-o APT::Sandbox::User=root` | `apt-get update` works: 58.9 MB fetched in 3 s |
|
||||
| `apt-get install cowsay` | download works; unpack fails: `unable to create '/usr/games/cowsay.dpkg-new': Permission denied` |
|
||||
| after pre-creating the host directory tree in the upper dirs (13164 dirs for `/usr`, done with a shell loop, 38 s; not yet implemented in C++) | unpack works, `/usr/games/cowsay hoi` runs; host `/usr/games/cowsay` still absent (isolation holds) |
|
||||
| configure step | fails: `chown: changing ownership of '/etc/shells.tmp': Invalid argument` (debianutils postinst copies owner of `/etc/shells`, which shows as `65534`), so dpkg reports 4 packages not fully installed |
|
||||
|
||||
Root causes:
|
||||
1. Host files owned by host root appear as `nobody` (uid 65534). Container root has `CAP_DAC_OVERRIDE` only over inodes whose owner is mapped, so it cannot write into host-root-owned directories of the lower layer, and cannot `chown` to or from an unmapped owner.
|
||||
2. `/var` has no package database.
|
||||
3. apt wants a second uid (G3).
|
||||
|
||||
Conclusion: the current design (host `/usr`, `/etc`, `/opt` as lower layer) is good for running tools and for isolated experiments, not for package management. Bare tools and compilers work. Honest status: **`apt-get install` does not complete today.**
|
||||
|
||||
Researched ways forward, in order of effort:
|
||||
- Short term, writable skeleton: at create time mirror every directory of the lower dirs into the upper dir with the same mode (directories owned by the user are writable; merged view is unchanged). Directories unreadable to the user (for example `/etc/ssl/private`, mode 700, root-owned) are skipped and stay as they are. Measured effect: writes into `/usr/bin` work, attach time unchanged (41 ms). Unmeasured: C++ implementation speed, and it does not fix the `chown` failure.
|
||||
- Proper fix, image layer: use an extracted distro rootfs (for example from `podman export` or `skopeo copy` plus unpack, both installed here; `mmdebstrap` and `debootstrap` are not) as the lower dir, with ownership shifted into the subuid range (G3). Then the tree is owned by mapped ids, `/var/lib/dpkg` exists, `chown` works, `_apt` exists, and host `/etc` stops leaking (G4). This also removes the dependence on the host distro. Cost: image import step, disk per image (shared lower, so once), and a layout change, hence a major bump.
|
||||
- Alternative to evaluate: `fuse-overlayfs` (installed) supports uid mappings between layers. Unmeasured; adds a FUSE process per container.
|
||||
|
||||
## Resolution log (1.0.0)
|
||||
|
||||
Everything below was built and measured on the machine above (Ubuntu 26.04, kernel 7.0.0-38, ext4 root,
|
||||
uid 1000, subuid range 100000:65536). The end-to-end suite (`tests/run.sh`) passed 56 of 56.
|
||||
|
||||
### What the field trial broke, and what fixed it
|
||||
|
||||
| Symptom (0.1.3) | Root cause | Fix in 1.0.0 | Result |
|
||||
|---|---|---|---|
|
||||
| `mkdir/touch` in `/usr/bin` fails | host dirs show as `nobody`; container root has no right over unmapped owners | `fsutil::mirrorDirs`: every host directory recreated in the upper layer, owned by the container | 13,163 dirs for `/usr` in about 1.1 s (the shell loop took 38 s); `touch /usr/bin/x` works, attach time unchanged |
|
||||
| `/var/lib/apt/lists/partial is missing`, no dpkg lock | `/var` was empty | `/var` pre-seeded with the apt/dpkg dirs (`kVarDirs`); host `/var/lib/dpkg` overlaid | apt starts |
|
||||
| `open lock-frontend: Permission denied` | host lock files owned by nobody, mode 640 | empty lock files pre-created in the upper (`kUpperLockFiles`) | lock acquired |
|
||||
| `dpkg: error creating backup file status-old: Operation not permitted` | dpkg hard-links `status`; kernel `protected_hardlinks` refuses links to files you do not own and cannot write | regular files of `/var/lib/dpkg`, `triggers`, `alternatives` copied into the upper (`kUpperCopyDirs`) | dpkg state writable |
|
||||
| `chown: changing ownership of '/etc/shells.tmp': Invalid argument` | `chown --reference` of a file owned by an unmapped uid | `/etc` is a private copy owned by the container root (`kCopiedDirs`) | `stat -c %u /etc/shells` is 0 |
|
||||
| `setegid 65534 failed`, `seteuid 42 failed` (apt `_apt` user) | one mappable id | subuid mapping through `newuidmap`/`newgidmap` (ids 1..65535) | `apt-get update` works with no override; with `--ids single` an `APT::Sandbox::User "root"` override is written instead |
|
||||
| `apt-get install cowsay` not configured | all of the above | all of the above | install completes, `dpkg --audit` clean, `/usr/games/cowsay` runs, host has no `/usr/games/cowsay` |
|
||||
|
||||
### Image layer
|
||||
|
||||
- Why images are the clean fix: the extracted tree is owned by mapped ids, so nothing needs a skeleton, copies or overrides. Measured: `apk add jq` in Alpine and `apt-get install cowsay curl ca-certificates` in Ubuntu 26.04 complete with a clean `dpkg --audit`; `curl https://example.com` returns 200.
|
||||
- Sources chosen (checked live with curl, 2026-10-23):
|
||||
- Alpine: `latest-releases.yaml` lists file name and sha256 per flavor (`alpine-minirootfs-3.24.3-x86_64.tar.gz`, 3.7 MB).
|
||||
- Ubuntu: `cdimage.ubuntu.com/ubuntu-base/releases/<ver>/release/SHA256SUMS` plus tarballs (26.04.1 and 24.04.5 present).
|
||||
- Debian: no usable tarball path found (`debuerreotype/docker-debian-artifacts` returned 404 for `rootfs.tar.xz`; only `.sha256` files resolved), so Debian is supported through `image add URL --sha256`, not through the catalog.
|
||||
- Not used: `podman export` / `skopeo` (installed here, but one more dependency; a registry client is out of scope).
|
||||
- Trust model: HTTPS to the publisher plus the checksum file from the same publisher. The Ubuntu `SHA256SUMS.gpg` signature is not checked (unimplemented, listed below).
|
||||
- Extraction runs inside a user namespace with the id map the containers will use (`UserNsJob`): subuid images keep the real ownership; single-id images use `tar --no-same-owner`. A container's id mode is pinned to its image's.
|
||||
- Cleanup: files owned by subuids cannot be removed by the host user (`Permission denied`). `Store::remove` and `ImageStore::remove` therefore delete from inside the id map. Measured: deleting containers and images with subuid-owned files works, and a failed add leaves no staging directory.
|
||||
- Image sizes (hardlinks counted once): Alpine 8.3M, Ubuntu base 97M. The first measurement said 1.3G because uutils-coreutils hard-links one 11 MB binary about 100 times; `diskUsage` now deduplicates by (device, inode).
|
||||
|
||||
### Size cap
|
||||
|
||||
- Mechanism: `Sandbox::watch` polls the init's pidfd with a 10 s timeout and scans usage on every wake. 80% warns, 100% freezes the container's cgroup, below 95% it thaws.
|
||||
- Measured: `--size 64m`, a 100 MB write: `FROZEN` message at 100M of 64M, tick output stopped; `doos attach cap -- rm /root/big` ran (a second attach is not in the frozen cgroup), the container thawed (`thawed (4.0K used)`) and continued.
|
||||
- Freezing needs no cgroup controller, only the delegated `user@UID.service` subtree. Without it the cap warns and says so.
|
||||
- Chosen over a size-capped tmpfs because tmpfs is memory-backed and not persistent; chosen over project quota because that needs root.
|
||||
|
||||
### Timings (5 sequential `attach -- true`, this machine, subuid, one run set)
|
||||
|
||||
| Mode | Per attach |
|
||||
|---|---|
|
||||
| host, slirp | about 50 ms |
|
||||
| image alpine, slirp | about 40 ms |
|
||||
| image ubuntu, slirp | about 40 ms |
|
||||
| host, `--net none` | about 10 ms |
|
||||
|
||||
Subuid mapping added about 15 to 25 ms per start compared with 0.1.x (two setuid helper processes). Creating a host-mode container (skeleton, `/etc` copy, dpkg files) took about 1.2 s once. First `--image alpine` use including download and extraction: 0.3 s end to end on this network.
|
||||
|
||||
### Still open after 1.0.0 (see "Resolution log (1.1.0)" for what changed since)
|
||||
|
||||
- G1: `memory.max`, `pids.max`, `cpu.max` are not set. Measured earlier that they work here; not wired in.
|
||||
- G4: host mode copies every readable file of host `/etc` at creation. Use an image to avoid host files entirely.
|
||||
- G5, G6: unchanged (see above).
|
||||
- Size accounting sees only what the host user can read. Directories of other container uids with mode 700 inside the writable layer are not counted (reported only as an internal skip count, not shown). Unmeasured.
|
||||
- Overshoot: up to one scan interval (10 s) of writes can pass the cap before the freeze; measured 100M written against a 64M cap.
|
||||
- A second `attach` is outside the frozen cgroup and outside any limit by design (it is the rescue path).
|
||||
- Host-mode containers drift from the host: the `/etc` copy, dpkg state and skeleton are snapshots at creation. A host package upgrade is visible in `/usr` but not in the container's dpkg database.
|
||||
- Ubuntu image signature (`SHA256SUMS.gpg`) is not verified; Alpine's yaml is not signature-checked either.
|
||||
- `image add FILE` without `--sha256` is accepted and prints a note, since the user chose the file.
|
||||
- Catalog covers x86_64 and aarch64 only; aarch64 is untested.
|
||||
|
||||
## Resolution log (1.1.0)
|
||||
|
||||
Built and measured on the same machine (Ubuntu 26.04, kernel 7.0.0-38, 12 cores, ext4, uid 1000, cgroup v2
|
||||
delegated with `cpuset cpu io memory pids`). The end-to-end suite passed 83 checks.
|
||||
|
||||
### Research behind each decision
|
||||
|
||||
| Question | Finding | Source | Decision |
|
||||
|---|---|---|---|
|
||||
| Which capabilities to keep | Docker's default set: CHOWN DAC_OVERRIDE FSETID FOWNER MKNOD NET_RAW SETGID SETUID SETFCAP SETPCAP NET_BIND_SERVICE SYS_CHROOT KILL AUDIT_WRITE | [Docker run reference via OneUptime](https://oneuptime.com/blog/post/2026-01-25-docker-container-capabilities/markdown), [Red Hat container security guide](https://access.redhat.com/documentation/en/red_hat_enterprise_linux_atomic_host/7/html/container_security_guide/linux_capabilities_and_seccomp) | Same set. Measured: `CapEff` inside is `a80425fb`, the Docker default mask. |
|
||||
| Which syscalls to deny | Docker's profile blocks keyring calls (not namespaced) and io_uring (container escapes); kexec and bpf are also capability-gated | [Docker seccomp docs](https://docs.docker.com/engine/security/seccomp/) | Denylist in `hardening.cpp`; capability drop covers the capability-gated ones twice. |
|
||||
| x32 and architecture | A denylist keyed on the x86-64 arch can be dodged with `__X32_SYSCALL_BIT`; kernels >= 5.4 answer such numbers with ENOSYS | [seccomp(2) man page patch](https://lkml.iu.edu/hypermail/linux/kernel/2007.1/03947.html) | Arch check first (kill on mismatch), x32 numbers answered ENOSYS. |
|
||||
| `clone3` | Flags live in a struct seccomp cannot read; ENOSYS (not EPERM) makes glibc fall back to `clone` | [QEMU seccomp patch](https://qemu.googlesource.com/qemu/+/c542b302707628609f2a33bcbd3e442282806b8e), [libc-alpha thread](https://public-inbox.org/libc-alpha/20210727102222.r2hys526mfkpt4xo@senku/) | `clone3` -> ENOSYS, `clone`/`unshare` with `CLONE_NEWUSER` -> EPERM. Measured both. |
|
||||
| cgroup layout | Delegation covers a subtree; limit files exist only when the parent enables the controller; a cgroup that has children cannot hold processes | [kernel cgroup v2 docs](https://docs.kernel.org/admin-guide/cgroup-v2.html), [rootlesscontaine.rs](https://rootlesscontaine.rs/getting-started/common/cgroup2/) | `doos/` enables `+memory +pids +cpu`; limits on `doos/<name>` (no processes); leaves `main` and `shell`. |
|
||||
| Host-like networking without sharing the namespace | pasta attaches to an existing namespace by PID and splices local TCP/UDP between namespaces | [passt(1) man page](https://manpages.ubuntu.com/manpages/resolute/man1/passt.1.html) | `--net pasta`. Measured below. |
|
||||
| Image signatures | Alpine publishes the release key fingerprint on its downloads page; Ubuntu publishes the CD image key fingerprint in its verification tutorial | [alpinelinux.org/downloads](https://alpinelinux.org/downloads/), [ubuntu.com verify tutorial](https://ubuntu.com/tutorials/how-to-verify-ubuntu) | Fingerprints pinned in `spec.hpp`; keys fetched at install and refused unless the fingerprint matches. |
|
||||
| Debian rootfs | No `rootfs.tar.xz` served; the OCI layer is at `oci/blobs/rootfs.tar.gz` and its digest is in the manifest embedded (base64) in `oci/index.json` | measured with curl against `debuerreotype/docker-debian-artifacts` | `debian`, `debian-RELEASE` in the catalog; the digest is the integrity check. |
|
||||
|
||||
### Measurements
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Capabilities | `CapEff a80425fb`, `Seccomp: 2` (filter) inside a default container; relaxed: `CapEff 000001ffffffffff`, `Seccomp: 0` |
|
||||
| seccomp | `keyctl`, `bpf`, `mount`, `unshare(CLONE_NEWUSER)` -> EPERM; `clone3` -> ENOSYS; `unshare -U` and `mount -t tmpfs` fail; relaxed container runs `unshare -U` fine |
|
||||
| Tools under hardening | gcc, curl https, git, python3 `ssl`, ping, apt and apk installs all still work |
|
||||
| `/etc` | 50 entries under the allowlist versus 215 in a relaxed copy; no `shadow`, `sudoers`, `ssh_host_*` |
|
||||
| Memory | `--memory 256m`, touching 400 MB: whole container killed, doos prints `killed by its memory limit` |
|
||||
| Pids | `--pids 32`, 80 forks: `can't fork`, doos prints `hit its process limit (32)` |
|
||||
| CPU | 4 busy loops for 4.3 s wall: unlimited 16.2 CPU-seconds, `--cpus 0.5` 2.3 CPU-seconds (0.54 of one core) |
|
||||
| pasta | host `127.0.0.1` listener reachable, host abstract unix socket blocked (host mode: reachable), outbound TCP, DNS and https 200, inbound port forwarded to the host |
|
||||
| Signatures | Alpine and Ubuntu 24.04 verified with the pinned keys; a stand-in `gpg` that reports a valid signature by another key, or `BADSIG`, is refused and leaves no image |
|
||||
| Debian | `debian` (trixie) image installs and `apt-get install cowsay` completes, `dpkg --audit` clean; image 82M |
|
||||
| Size accounting | 80 MB written by container uid 5 in a mode-700 directory (invisible to the host user) against `--size 64m`: frozen; cleanup from a second attach thawed it |
|
||||
| Attach time (5 runs) | host or image, slirp: about 40 ms each; pasta: about 65 ms; `--net none`: about 10 ms |
|
||||
|
||||
### What changed in behavior (and why it is not a loss)
|
||||
|
||||
- Hardening can be switched off per container (`--hardening relaxed`), so nothing that needs the denied calls is lost, it is opt-in now.
|
||||
- Host `/etc` content outside the allowlist is absent under default hardening. `relaxed` restores the full copy; images never had it.
|
||||
- The 10 s scan interval is gone: scans start every 2 s at the earliest but are paced by their own cost, so large trees are not scanned in a tight loop.
|
||||
- pasta replaces the need for `--net host` in almost every case; `host` remains.
|
||||
|
||||
### Still open, stated plainly
|
||||
|
||||
- Memory limits count `/tmp` and `/dev/shm` (both RAM-backed) and page cache charged to the container; a build that fills `/tmp` can hit `--memory`. Measured only with anonymous memory.
|
||||
- `--pids` counts threads. Thread-heavy programs need a higher value.
|
||||
- `memory.high` is not used, so there is no soft throttle before the OOM kill.
|
||||
- Size accounting is a scan, not a quota: writes can pass the cap by up to one scan interval, bounded per file by `RLIMIT_FSIZE`. A process can still write many small files faster than a scan.
|
||||
- A later `doos attach` shares the memory, pids and cpu limits and the per-file size limit, but is never frozen. That is the rescue path by design, so it can also fill the disk past the cap.
|
||||
- Host-mode containers still drift from the host (snapshots of `/etc`, dpkg state, skeleton). Images do not.
|
||||
- Debian images have no publisher signature to check; trust is HTTPS to the Debian Docker maintainers' repository plus the content digest. The Alpine key is pinned from alpinelinux.org, not from a second out-of-band channel beyond the fingerprint shown on the same site and in public mailing lists.
|
||||
- aarch64 paths (Alpine `aarch64`, Debian `dist-arm64v8`, seccomp numbers for aarch64) compile and are guarded but were not run: no aarch64 machine here.
|
||||
- Without `gpg` installed, signature checks are skipped with a printed note.
|
||||
- pasta is not signalled by doos (it drops uid); it exits by itself within about a second after the container's namespace is gone. Measured: no pasta process remained after 1.5 s.
|
||||
- seccomp is a denylist. A new kernel syscall is allowed until it is added to the list.
|
||||
|
||||
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env bash
|
||||
# Bump VERSION, build, run the e2e suite, commit everything, tag.
|
||||
# Usage: scripts/release.sh patch|minor|major "commit subject"
|
||||
# Refuses to run when the build or tests fail. Does not push.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
kind=${1:?usage: release.sh patch|minor|major "subject"}
|
||||
subject=${2:?usage: release.sh patch|minor|major "subject"}
|
||||
|
||||
IFS=. read -r maj min pat < VERSION
|
||||
case "$kind" in
|
||||
patch) pat=$((pat+1)) ;;
|
||||
minor) min=$((min+1)); pat=0 ;;
|
||||
major) maj=$((maj+1)); min=0; pat=0 ;;
|
||||
*) echo "kind must be patch, minor or major" >&2; exit 2 ;;
|
||||
esac
|
||||
new="$maj.$min.$pat"
|
||||
|
||||
if git rev-parse "v$new" >/dev/null 2>&1; then echo "tag v$new already exists" >&2; exit 1; fi
|
||||
if grep -rl $'\xe2\x80\x94' --exclude-dir=build --exclude-dir=.git . | grep -q .; then
|
||||
echo "em dash found, remove it first" >&2; exit 1
|
||||
fi
|
||||
|
||||
echo "$new" > VERSION
|
||||
cmake -S . -B build >/dev/null
|
||||
cmake --build build
|
||||
tests/run.sh
|
||||
|
||||
git add -A
|
||||
git commit -m "$subject
|
||||
|
||||
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>"
|
||||
git tag -a "v$new" -m "doos $new"
|
||||
echo "released v$new"
|
||||
+152
@@ -0,0 +1,152 @@
|
||||
#include "cgroup.hpp"
|
||||
|
||||
#include <unistd.h>
|
||||
|
||||
#include <chrono>
|
||||
#include <sstream>
|
||||
#include <thread>
|
||||
|
||||
#include "spec.hpp"
|
||||
#include "util.hpp"
|
||||
|
||||
namespace fs = std::filesystem;
|
||||
|
||||
namespace doos {
|
||||
|
||||
namespace {
|
||||
bool hasController(const fs::path& dir, const std::string& name) {
|
||||
try {
|
||||
std::istringstream in(fsutil::readFile(dir / "cgroup.controllers"));
|
||||
for (std::string c; in >> c;) {
|
||||
if (c == name) return true;
|
||||
}
|
||||
} catch (const std::exception&) {
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
bool tryWrite(const fs::path& file, const std::string& text) {
|
||||
try {
|
||||
fsutil::writeFile(file, text);
|
||||
return true;
|
||||
} catch (const std::exception&) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
} // namespace
|
||||
|
||||
fs::path Cgroup::base(std::string& whyNot) {
|
||||
std::string self;
|
||||
try {
|
||||
self = fsutil::readFile("/proc/self/cgroup");
|
||||
} catch (const std::exception& e) {
|
||||
whyNot = e.what();
|
||||
return {};
|
||||
}
|
||||
std::string marker = "user@" + std::to_string(::geteuid()) + ".service";
|
||||
auto at = self.find(marker);
|
||||
if (self.rfind("0::", 0) != 0 || at == std::string::npos) {
|
||||
whyNot = "no cgroup v2 delegated to the user manager (" + marker + " not found)";
|
||||
return {};
|
||||
}
|
||||
return fs::path("/sys/fs/cgroup") / self.substr(3, at + marker.size() - 3).substr(1);
|
||||
}
|
||||
|
||||
std::unique_ptr<Cgroup> Cgroup::create(const std::string& name, const Limits& limits, std::string& whyNot,
|
||||
std::string& notEnforced) {
|
||||
try {
|
||||
fs::path b = base(whyNot);
|
||||
if (b.empty()) return nullptr;
|
||||
fs::path top = b / std::string(spec::kCgroupDir);
|
||||
std::error_code ec;
|
||||
fs::create_directory(top, ec);
|
||||
if (ec && ec != std::errc::file_exists) {
|
||||
whyNot = "cannot create " + top.string() + ": " + ec.message();
|
||||
return nullptr;
|
||||
}
|
||||
// Controllers must be enabled for children of `doos/` for the limit files to exist in `doos/<name>`.
|
||||
std::string enable;
|
||||
for (const char* c : {"memory", "pids", "cpu"}) {
|
||||
if (hasController(b, c)) enable += std::string(" +") + c;
|
||||
}
|
||||
if (!enable.empty()) tryWrite(top / "cgroup.subtree_control", enable);
|
||||
|
||||
fs::path dir = top / name;
|
||||
fs::create_directory(dir, ec);
|
||||
if (ec && ec != std::errc::file_exists) {
|
||||
whyNot = "cannot create " + dir.string() + ": " + ec.message();
|
||||
return nullptr;
|
||||
}
|
||||
for (const char* leaf : {"main", "shell"}) fs::create_directory(dir / leaf, ec);
|
||||
if (::access((dir / "main" / "cgroup.freeze").c_str(), W_OK) != 0) {
|
||||
whyNot = "cgroup.freeze not writable in " + dir.string();
|
||||
return nullptr;
|
||||
}
|
||||
auto note = [&](const std::string& what) { notEnforced += (notEnforced.empty() ? "" : ", ") + what; };
|
||||
if (limits.memoryBytes) {
|
||||
if (!tryWrite(dir / "memory.max", std::to_string(limits.memoryBytes))) note("memory");
|
||||
else {
|
||||
tryWrite(dir / "memory.swap.max", "0"); // otherwise the cap can be dodged through swap
|
||||
tryWrite(dir / "memory.oom.group", "1"); // an OOM kill takes the whole container, not one process
|
||||
}
|
||||
}
|
||||
if (limits.pids && !tryWrite(dir / "pids.max", std::to_string(limits.pids))) note("pids");
|
||||
if (limits.milliCpus) {
|
||||
long quota = static_cast<long>(limits.milliCpus) * spec::kCpuPeriodUs / 1000;
|
||||
if (!tryWrite(dir / "cpu.max", std::to_string(quota) + " " + std::to_string(spec::kCpuPeriodUs))) note("cpus");
|
||||
}
|
||||
return std::unique_ptr<Cgroup>(new Cgroup(dir, true));
|
||||
} catch (const std::exception& e) {
|
||||
whyNot = e.what();
|
||||
return nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
std::unique_ptr<Cgroup> Cgroup::open(const std::string& name) {
|
||||
std::string why;
|
||||
fs::path b = base(why);
|
||||
if (b.empty()) return nullptr;
|
||||
fs::path dir = b / std::string(spec::kCgroupDir) / name;
|
||||
std::error_code ec;
|
||||
if (!fs::is_directory(dir / "shell", ec)) return nullptr;
|
||||
return std::unique_ptr<Cgroup>(new Cgroup(dir, false));
|
||||
}
|
||||
|
||||
Cgroup::~Cgroup() {
|
||||
if (!owner_) return;
|
||||
// Everything dies with the container's init; kill any straggler, then remove the leaves and the group.
|
||||
tryWrite(dir_ / "cgroup.kill", "1");
|
||||
for (int i = 0; i < 40; ++i) {
|
||||
std::error_code ec;
|
||||
fs::remove(dir_ / "main", ec);
|
||||
fs::remove(dir_ / "shell", ec);
|
||||
if (fs::remove(dir_, ec)) return;
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(25));
|
||||
}
|
||||
}
|
||||
|
||||
void Cgroup::addMain(pid_t pid) const { fsutil::writeFile(dir_ / "main" / "cgroup.procs", std::to_string(pid)); }
|
||||
void Cgroup::addShell(pid_t pid) const { fsutil::writeFile(dir_ / "shell" / "cgroup.procs", std::to_string(pid)); }
|
||||
|
||||
void Cgroup::freeze(bool on) const {
|
||||
fsutil::writeFile(dir_ / "main" / "cgroup.freeze", on ? "1" : "0");
|
||||
frozen_ = on;
|
||||
}
|
||||
|
||||
std::uint64_t Cgroup::counter(const char* file, const char* key) const {
|
||||
try {
|
||||
std::istringstream in(fsutil::readFile(dir_ / file));
|
||||
std::string k;
|
||||
std::uint64_t v;
|
||||
while (in >> k >> v) {
|
||||
if (k == key) return v;
|
||||
}
|
||||
} catch (const std::exception&) {
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
std::uint64_t Cgroup::oomKills() const { return counter("memory.events", "oom_kill"); }
|
||||
std::uint64_t Cgroup::pidFailures() const { return counter("pids.events", "max"); }
|
||||
|
||||
} // namespace doos
|
||||
@@ -0,0 +1,51 @@
|
||||
#pragma once
|
||||
#include <sys/types.h>
|
||||
|
||||
#include <cstdint>
|
||||
#include <filesystem>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
namespace doos {
|
||||
|
||||
struct Limits {
|
||||
std::uint64_t memoryBytes = 0; // 0 = not limited
|
||||
std::uint32_t pids = 0;
|
||||
std::uint32_t milliCpus = 0; // 0 = all cores
|
||||
};
|
||||
|
||||
// cgroup v2 subtree of one running container under the cgroup delegated to the user's systemd manager:
|
||||
//
|
||||
// doos/<name>/ limits live here (memory.max, pids.max, cpu.max), no processes
|
||||
// main/ the container's init and everything it starts; frozen when the size cap is hit
|
||||
// shell/ processes started by later `doos attach` calls; never frozen (rescue path), same limits
|
||||
//
|
||||
// Best effort: create() returns null when delegation is absent, and the caller must say why.
|
||||
class Cgroup {
|
||||
public:
|
||||
static std::unique_ptr<Cgroup> create(const std::string& name, const Limits& limits, std::string& whyNot,
|
||||
std::string& notEnforced);
|
||||
// Existing group of a running container, for joiners. Null when none.
|
||||
static std::unique_ptr<Cgroup> open(const std::string& name);
|
||||
~Cgroup();
|
||||
Cgroup(const Cgroup&) = delete;
|
||||
Cgroup& operator=(const Cgroup&) = delete;
|
||||
|
||||
void addMain(pid_t pid) const;
|
||||
void addShell(pid_t pid) const;
|
||||
void freeze(bool on) const;
|
||||
bool frozen() const { return frozen_; }
|
||||
// Number of OOM kills and of fork failures at the pids limit since creation.
|
||||
std::uint64_t oomKills() const;
|
||||
std::uint64_t pidFailures() const;
|
||||
|
||||
private:
|
||||
Cgroup(std::filesystem::path dir, bool owner) : dir_(std::move(dir)), owner_(owner) {}
|
||||
static std::filesystem::path base(std::string& whyNot);
|
||||
std::uint64_t counter(const char* file, const char* key) const;
|
||||
std::filesystem::path dir_;
|
||||
bool owner_;
|
||||
mutable bool frozen_ = false;
|
||||
};
|
||||
|
||||
} // namespace doos
|
||||
+173
-33
@@ -1,50 +1,95 @@
|
||||
#include "cli.hpp"
|
||||
|
||||
#include <cctype>
|
||||
#include <ctime>
|
||||
#include <filesystem>
|
||||
#include <iostream>
|
||||
#include <optional>
|
||||
|
||||
#include "container.hpp"
|
||||
#include "sandbox.hpp"
|
||||
#include "spec.hpp"
|
||||
#include "util.hpp"
|
||||
|
||||
namespace doos {
|
||||
|
||||
namespace {
|
||||
|
||||
// Splits leading --options from the first positional. Everything after the positional is returned as-is.
|
||||
std::uint32_t parseCount(const std::string& text);
|
||||
std::uint32_t parseCpus(const std::string& text);
|
||||
|
||||
// Leading --options, then the first positional, then everything else untouched.
|
||||
struct Parsed {
|
||||
std::optional<std::string> net;
|
||||
Options opts;
|
||||
std::optional<std::string> sha256;
|
||||
bool yes = false;
|
||||
std::string name;
|
||||
std::vector<std::string> positional; // up to maxPositional, the rest lands in tail
|
||||
std::vector<std::string> tail;
|
||||
};
|
||||
|
||||
Parsed parse(const std::vector<std::string>& rest, bool allowNet, bool allowYes) {
|
||||
struct Allowed {
|
||||
bool container = false; // --net --image --size --ids
|
||||
bool yes = false;
|
||||
bool sha256 = false;
|
||||
};
|
||||
|
||||
Parsed parse(const std::vector<std::string>& rest, Allowed allow, std::size_t wantPositional) {
|
||||
Parsed p;
|
||||
auto value = [&](std::size_t& i, const std::string& flag) -> std::string {
|
||||
const std::string& a = rest[i];
|
||||
if (a.size() > flag.size() && a.compare(0, flag.size() + 1, flag + "=") == 0) return a.substr(flag.size() + 1);
|
||||
if (i + 1 >= rest.size()) throw std::runtime_error(flag + " needs a value");
|
||||
return rest[++i];
|
||||
};
|
||||
auto is = [&](const std::string& a, const std::string& flag) { return a == flag || a.rfind(flag + "=", 0) == 0; };
|
||||
std::size_t i = 0;
|
||||
for (; i < rest.size(); ++i) {
|
||||
const std::string& a = rest[i];
|
||||
if (a == "--") { ++i; break; }
|
||||
if (allowNet && a == "--net") {
|
||||
if (i + 1 >= rest.size()) throw std::runtime_error("--net needs a value (slirp, host, none)");
|
||||
p.net = rest[++i];
|
||||
} else if (allowNet && a.rfind("--net=", 0) == 0) {
|
||||
p.net = a.substr(6);
|
||||
} else if (allowYes && a == "--yes") {
|
||||
p.yes = true;
|
||||
} else if (a.size() > 1 && a[0] == '-') {
|
||||
throw std::runtime_error("unknown option " + a);
|
||||
} else {
|
||||
break;
|
||||
if (allow.container && is(a, "--net")) p.opts.net = Network::parse(value(i, "--net"));
|
||||
else if (allow.container && is(a, "--image")) p.opts.image = value(i, "--image");
|
||||
else if (allow.container && is(a, "--size")) p.opts.size = parseSize(value(i, "--size"));
|
||||
else if (allow.container && is(a, "--ids")) p.opts.ids = IdMap::parse(value(i, "--ids"));
|
||||
else if (allow.container && is(a, "--memory")) p.opts.memory = parseSize(value(i, "--memory"));
|
||||
else if (allow.container && is(a, "--pids")) p.opts.pids = parseCount(value(i, "--pids"));
|
||||
else if (allow.container && is(a, "--cpus")) p.opts.milliCpus = parseCpus(value(i, "--cpus"));
|
||||
else if (allow.container && is(a, "--hardening")) {
|
||||
std::string v = value(i, "--hardening");
|
||||
if (v != "default" && v != "relaxed") throw std::runtime_error("--hardening takes default or relaxed");
|
||||
p.opts.relaxed = (v == "relaxed");
|
||||
}
|
||||
else if (allow.yes && a == "--yes") p.yes = true;
|
||||
else if (allow.sha256 && is(a, "--sha256")) p.sha256 = value(i, "--sha256");
|
||||
else if (a.size() > 1 && a[0] == '-') throw std::runtime_error("unknown option " + a);
|
||||
else if (p.positional.size() < wantPositional) p.positional.push_back(a);
|
||||
else break;
|
||||
}
|
||||
if (i >= rest.size()) throw std::runtime_error("missing container name");
|
||||
p.name = rest[i++];
|
||||
if (p.positional.size() < wantPositional) throw std::runtime_error("missing argument");
|
||||
if (i < rest.size() && rest[i] == "--") ++i;
|
||||
p.tail.assign(rest.begin() + static_cast<long>(i), rest.end());
|
||||
return p;
|
||||
}
|
||||
|
||||
std::uint32_t parseCount(const std::string& text) {
|
||||
if (text.empty() || text.size() > 9 || text.find_first_not_of("0123456789") != std::string::npos) {
|
||||
throw std::runtime_error("invalid number '" + text + "'");
|
||||
}
|
||||
return static_cast<std::uint32_t>(std::stoul(text));
|
||||
}
|
||||
|
||||
// "2", "0.5", "1.25" cpus -> thousandths; "0" means all cores.
|
||||
std::uint32_t parseCpus(const std::string& text) {
|
||||
std::size_t dot = text.find('.');
|
||||
std::string whole = text.substr(0, dot), frac = dot == std::string::npos ? "" : text.substr(dot + 1);
|
||||
if (whole.empty() || whole.size() > 5 || frac.size() > 3 ||
|
||||
whole.find_first_not_of("0123456789") != std::string::npos ||
|
||||
frac.find_first_not_of("0123456789") != std::string::npos) {
|
||||
throw std::runtime_error("invalid cpus '" + text + "' (examples: 2, 0.5, 1.25, 0 for all cores)");
|
||||
}
|
||||
while (frac.size() < 3) frac += '0';
|
||||
return static_cast<std::uint32_t>(std::stoul(whole)) * 1000 + static_cast<std::uint32_t>(std::stoul(frac));
|
||||
}
|
||||
|
||||
std::string age(long created) {
|
||||
long d = std::time(nullptr) - created;
|
||||
if (d < 120) return std::to_string(d) + "s";
|
||||
@@ -61,10 +106,26 @@ void Cli::usage(std::FILE* out) {
|
||||
"\n"
|
||||
"usage:\n"
|
||||
" doos list\n"
|
||||
" doos get_or_create [--net slirp|host|none] NAME\n"
|
||||
" doos attach [--net slirp|host|none] NAME [-- CMD [ARGS...]]\n"
|
||||
" doos get_or_create [OPTIONS] NAME\n"
|
||||
" doos attach [OPTIONS] NAME [-- CMD [ARGS...]]\n"
|
||||
" doos delete [--yes] NAME\n"
|
||||
" doos image list\n"
|
||||
" doos image add [--sha256 HEX] SOURCE [NAME]\n"
|
||||
" doos image delete [--yes] NAME\n"
|
||||
" doos apparmor print the AppArmor profile this binary needs on Ubuntu 24.04+\n"
|
||||
"\n"
|
||||
"options:\n"
|
||||
" --image NAME start from an image instead of the host system (fetched on demand) [fixed]\n"
|
||||
" --net slirp|pasta|host|none network, default slirp (pasta: host-like, private sockets) [fixed]\n"
|
||||
" --ids single|subuid id mapping, default: subuid when available [fixed]\n"
|
||||
" --hardening default|relaxed relaxed = no capability drop, no seccomp, full /etc [fixed]\n"
|
||||
" --size SIZE writable-layer cap, 64m..1t, default 4g\n"
|
||||
" --memory SIZE memory cap, 64m..1t, default 8g\n"
|
||||
" --pids N process cap, 16..4194304, default 4096\n"
|
||||
" --cpus N cpu cap, 0.1..1024, default 0 (all cores)\n"
|
||||
" size, memory, pids and cpus can be changed on any later call; they apply at the next start.\n"
|
||||
"\n"
|
||||
"SOURCE for image add: alpine, ubuntu, ubuntu-24.04, debian, debian-bookworm, an https URL (needs --sha256), or a tar file.\n"
|
||||
"attach creates NAME when missing. delete only reports unless --yes is given.\n"
|
||||
"State lives in $DOOS_HOME or ~/.local/share/doos.\n",
|
||||
out);
|
||||
@@ -78,11 +139,13 @@ int Cli::run() {
|
||||
std::string cmd = args_[0];
|
||||
for (auto& ch : cmd) if (ch == '-') ch = '_';
|
||||
std::vector<std::string> rest(args_.begin() + 1, args_.end());
|
||||
if (cmd == "--version" || cmd == "version") { std::puts(DOOS_VERSION); return 0; }
|
||||
if (cmd == "__version" || cmd == "version") { std::puts(DOOS_VERSION); return 0; }
|
||||
if (cmd == "list") return cmdList();
|
||||
if (cmd == "get_or_create") return cmdGetOrCreate(rest);
|
||||
if (cmd == "attach") return cmdAttach(rest);
|
||||
if (cmd == "delete") return cmdDelete(rest);
|
||||
if (cmd == "image") return cmdImage(rest);
|
||||
if (cmd == "apparmor") return cmdApparmor();
|
||||
std::fprintf(stderr, "doos: unknown command '%s'\n\n", args_[0].c_str());
|
||||
usage(stderr);
|
||||
return 2;
|
||||
@@ -92,40 +155,117 @@ int Cli::cmdList() {
|
||||
Store store = Store::fromEnvironment();
|
||||
auto all = store.list();
|
||||
if (all.empty()) { std::puts("no containers"); return 0; }
|
||||
std::printf("%-*s %-8s %-6s %s\n", 20, "NAME", "STATE", "NET", "AGE");
|
||||
std::printf("%-*s %-8s %-6s %-12s %-6s %s\n", 16, "NAME", "STATE", "NET", "IMAGE", "SIZE", "AGE");
|
||||
for (const auto& c : all) {
|
||||
std::printf("%-*s %-8s %-6s %s\n", 20, c.name().c_str(), c.runningPid() ? "running" : "stopped",
|
||||
Network::name(c.net()).c_str(), age(c.created()).c_str());
|
||||
std::printf("%-*s %-8s %-6s %-12s %-6s %s\n", 16, c.name().c_str(), c.runningPid() ? "running" : "stopped",
|
||||
Network::name(c.net()).c_str(), c.imageMode() ? c.image().c_str() : "host",
|
||||
formatSize(c.sizeCap()).c_str(), age(c.created()).c_str());
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int Cli::cmdGetOrCreate(std::vector<std::string> rest) {
|
||||
Parsed p = parse(rest, true, false);
|
||||
Parsed p = parse(rest, {true, false, false}, 1);
|
||||
if (!p.tail.empty()) throw std::runtime_error("get_or_create takes only a name");
|
||||
std::optional<NetMode> net;
|
||||
if (p.net) net = Network::parse(*p.net);
|
||||
Store store = Store::fromEnvironment();
|
||||
auto [c, created] = store.getOrCreate(p.name, net);
|
||||
auto [c, created] = store.getOrCreate(p.positional[0], p.opts);
|
||||
std::printf("%s %s\n", created ? "created" : "exists", c.name().c_str());
|
||||
return 0;
|
||||
}
|
||||
|
||||
int Cli::cmdAttach(std::vector<std::string> rest) {
|
||||
Parsed p = parse(rest, true, false);
|
||||
std::optional<NetMode> net;
|
||||
if (p.net) net = Network::parse(*p.net);
|
||||
Parsed p = parse(rest, {true, false, false}, 1);
|
||||
Store store = Store::fromEnvironment();
|
||||
auto [c, created] = store.getOrCreate(p.name, net);
|
||||
auto [c, created] = store.getOrCreate(p.positional[0], p.opts);
|
||||
(void)created;
|
||||
return Sandbox(std::move(c), std::move(p.tail)).run();
|
||||
}
|
||||
|
||||
int Cli::cmdDelete(std::vector<std::string> rest) {
|
||||
Parsed p = parse(rest, false, true);
|
||||
Parsed p = parse(rest, {false, true, false}, 1);
|
||||
if (!p.tail.empty()) throw std::runtime_error("delete takes only a name");
|
||||
Store store = Store::fromEnvironment();
|
||||
std::puts(store.remove(p.name, p.yes).c_str());
|
||||
std::puts(store.remove(p.positional[0], p.yes).c_str());
|
||||
return 0;
|
||||
}
|
||||
|
||||
int Cli::cmdApparmor() {
|
||||
std::fputs(Sandbox::apparmorProfile().c_str(), stdout);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int Cli::cmdImage(std::vector<std::string> rest) {
|
||||
if (rest.empty()) throw std::runtime_error("image needs a subcommand: list, add, delete");
|
||||
std::string sub = rest[0];
|
||||
std::vector<std::string> tail(rest.begin() + 1, rest.end());
|
||||
if (sub == "list") return cmdImageList();
|
||||
if (sub == "add") return cmdImageAdd(tail);
|
||||
if (sub == "delete") return cmdImageDelete(tail);
|
||||
throw std::runtime_error("unknown image subcommand '" + sub + "' (list, add, delete)");
|
||||
}
|
||||
|
||||
int Cli::cmdImageList() {
|
||||
Store store = Store::fromEnvironment();
|
||||
auto installed = store.images().list();
|
||||
std::printf("%-*s %-6s %-7s %s\n", 16, "IMAGE", "SIZE", "IDS", "STATE");
|
||||
for (const auto& i : installed) {
|
||||
std::printf("%-*s %-6s %-7s %s\n", 16, i.name.c_str(), formatSize(i.bytes).c_str(), IdMap::name(i.ids).c_str(),
|
||||
"installed");
|
||||
}
|
||||
for (const auto& n : Catalog::names()) {
|
||||
bool have = false;
|
||||
for (const auto& i : installed) have = have || i.name == n;
|
||||
if (!have) std::printf("%-*s %-6s %-7s %s\n", 16, n.c_str(), "-", "-", "available");
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int Cli::cmdImageAdd(std::vector<std::string> rest) {
|
||||
Parsed p = parse(rest, {false, false, true}, 1);
|
||||
if (p.tail.size() > 1) throw std::runtime_error("image add takes SOURCE and an optional NAME");
|
||||
const std::string& source = p.positional[0];
|
||||
ImageSource src;
|
||||
bool isUrl = source.rfind("https://", 0) == 0;
|
||||
bool isFile = !isUrl && source.find('/') != std::string::npos;
|
||||
if (source.rfind("http://", 0) == 0) throw std::runtime_error("plain http is refused; use https");
|
||||
if (!isUrl && !isFile && Catalog::knows(ImageStore::normalize(source))) {
|
||||
src = Catalog::resolve(ImageStore::normalize(source));
|
||||
if (p.sha256) src.sha256 = *p.sha256;
|
||||
} else if (isUrl) {
|
||||
if (!p.sha256) throw std::runtime_error("a URL image needs --sha256 HEX (unverifiable input is rejected)");
|
||||
src.location = source;
|
||||
src.sha256 = *p.sha256;
|
||||
} else {
|
||||
if (!std::filesystem::is_regular_file(source)) {
|
||||
throw std::runtime_error("'" + source + "' is neither a catalog image (alpine, ubuntu, ubuntu-VERSION, debian, debian-RELEASE), an https URL, nor a file");
|
||||
}
|
||||
src.location = std::filesystem::absolute(source).string();
|
||||
src.isFile = true;
|
||||
if (p.sha256) src.sha256 = *p.sha256;
|
||||
}
|
||||
if (!p.tail.empty()) {
|
||||
src.name = ImageStore::normalize(p.tail[0]);
|
||||
} else if (src.name.empty()) {
|
||||
std::string base = std::filesystem::path(src.location).filename().string();
|
||||
for (const char* ext : {".tar.gz", ".tar.xz", ".tar.zst", ".tgz", ".tar"}) {
|
||||
std::string e = ext;
|
||||
if (base.size() > e.size() && base.compare(base.size() - e.size(), e.size(), e) == 0) base.erase(base.size() - e.size());
|
||||
}
|
||||
for (auto& ch : base) ch = std::isalnum(static_cast<unsigned char>(ch)) ? static_cast<char>(std::tolower(ch)) : '-';
|
||||
src.name = base.substr(0, spec::kMaxNameLen);
|
||||
}
|
||||
Store store = Store::fromEnvironment();
|
||||
ImageInfo info = store.images().add(src, IdMap::best());
|
||||
std::printf("added image %s (%s, ids %s)\n", info.name.c_str(), formatSize(info.bytes).c_str(), IdMap::name(info.ids).c_str());
|
||||
return 0;
|
||||
}
|
||||
|
||||
int Cli::cmdImageDelete(std::vector<std::string> rest) {
|
||||
Parsed p = parse(rest, {false, true, false}, 1);
|
||||
if (!p.tail.empty()) throw std::runtime_error("image delete takes only a name");
|
||||
Store store = Store::fromEnvironment();
|
||||
std::string name = ImageStore::normalize(p.positional[0]);
|
||||
std::puts(store.images().remove(name, p.yes, store.containersUsing(name)).c_str());
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
@@ -15,6 +15,11 @@ private:
|
||||
int cmdGetOrCreate(std::vector<std::string> rest);
|
||||
int cmdAttach(std::vector<std::string> rest);
|
||||
int cmdDelete(std::vector<std::string> rest);
|
||||
int cmdApparmor();
|
||||
int cmdImage(std::vector<std::string> rest);
|
||||
int cmdImageList();
|
||||
int cmdImageAdd(std::vector<std::string> rest);
|
||||
int cmdImageDelete(std::vector<std::string> rest);
|
||||
static void usage(std::FILE* out);
|
||||
|
||||
std::vector<std::string> args_;
|
||||
|
||||
+301
-15
@@ -1,6 +1,11 @@
|
||||
#include "container.hpp"
|
||||
|
||||
#include <fcntl.h>
|
||||
#include <sys/stat.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstdio>
|
||||
#include <cctype>
|
||||
#include <ctime>
|
||||
#include <map>
|
||||
@@ -65,39 +70,243 @@ void Container::validateName(const std::string& name) {
|
||||
fs::path Container::layer(const std::string& hostDir) const { return dir_ / "layers" / slug(hostDir); }
|
||||
fs::path Container::privateDir(const std::string& hostDir) const { return dir_ / "private" / slug(hostDir); }
|
||||
|
||||
fs::path Container::etcDir() const {
|
||||
return imageMode() ? layer("/root") / "upper" / "etc" : privateDir("/etc");
|
||||
}
|
||||
|
||||
void Container::seed(const std::string& marker, const std::string& label,
|
||||
const std::function<fsutil::SeedStats()>& work) const {
|
||||
fs::path m = dir_ / ("seeded." + marker);
|
||||
if (fs::exists(m)) return;
|
||||
fsutil::SeedStats st = work();
|
||||
std::fprintf(stderr, "doos: seeded %s: %zu entries%s%s\n", label.c_str(), st.made,
|
||||
st.skipped ? ", unreadable skipped: " : "", st.skipped ? std::to_string(st.skipped).c_str() : "");
|
||||
fsutil::writeFile(m, "ok\n");
|
||||
}
|
||||
|
||||
namespace {
|
||||
// Names that may hold credentials are never copied, even inside allowed directories.
|
||||
bool etcDenied(const fs::path& p) {
|
||||
std::string n = p.filename().string();
|
||||
if (n.size() > 4 && n.compare(n.size() - 4, 4, ".key") == 0) return true;
|
||||
for (auto d : spec::kEtcDenyNames) {
|
||||
std::string pat(d);
|
||||
if (!pat.empty() && pat.back() == '*') {
|
||||
if (n.rfind(pat.substr(0, pat.size() - 1), 0) == 0) return true;
|
||||
} else if (n == pat) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
void Container::prepareLayout() const {
|
||||
fsutil::mkdirs(rootfs());
|
||||
if (imageMode()) {
|
||||
fsutil::mkdirs(layer("/root") / "upper");
|
||||
fsutil::mkdirs(layer("/root") / "work");
|
||||
return;
|
||||
}
|
||||
std::vector<std::string> overlays;
|
||||
auto addOverlay = [&](std::string_view d) {
|
||||
std::string dir(d);
|
||||
fsutil::mkdirs(rootfs() / slug(d));
|
||||
fsutil::mkdirs(layer(dir) / "upper");
|
||||
fsutil::mkdirs(layer(dir) / "work");
|
||||
overlays.push_back(dir);
|
||||
};
|
||||
for (auto d : spec::kOverlayDirs) addOverlay(d);
|
||||
for (auto d : spec::kOverlayDirs) {
|
||||
fsutil::mkdirs(rootfs() / slug(d));
|
||||
addOverlay(d);
|
||||
}
|
||||
for (auto d : spec::kCompatDirs) {
|
||||
std::error_code ec;
|
||||
if (fs::is_directory(fs::symlink_status(std::string(d), ec)) && !ec) addOverlay(d);
|
||||
if (fs::is_directory(fs::symlink_status(std::string(d), ec)) && !ec) {
|
||||
fsutil::mkdirs(rootfs() / slug(d));
|
||||
addOverlay(d);
|
||||
}
|
||||
}
|
||||
for (auto d : spec::kPrivateDirs) {
|
||||
fsutil::mkdirs(rootfs() / slug(d));
|
||||
fsutil::mkdirs(privateDir(std::string(d)), 0755);
|
||||
}
|
||||
for (auto d : spec::kVarDirs) fsutil::mkdirs(privateDir("/var") / d.path, d.mode);
|
||||
for (auto d : spec::kVarOverlayDirs) {
|
||||
std::error_code ec;
|
||||
if (!fs::is_directory(std::string(d), ec)) continue;
|
||||
fsutil::mkdirs(privateDir("/var") / std::string(d).substr(5)); // below /var
|
||||
addOverlay(d);
|
||||
}
|
||||
for (auto d : spec::kEmptyDirs) fsutil::mkdirs(rootfs() / slug(d));
|
||||
|
||||
// Host dirs are owned by an unmapped uid (nobody) in the container: without a directory skeleton in the
|
||||
// upper layer nothing could ever be created in them (package installs, tool caches).
|
||||
for (const auto& o : overlays) {
|
||||
std::error_code ec;
|
||||
if (!fs::is_directory(o, ec)) continue;
|
||||
std::string key = slug(o);
|
||||
std::replace(key.begin(), key.end(), '/', '_');
|
||||
seed("skeleton-" + key, "directory skeleton " + o, [&] { return fsutil::mirrorDirs(o, layer(o) / "upper"); });
|
||||
}
|
||||
for (auto d : spec::kUpperCopyDirs) {
|
||||
for (const auto& o : overlays) {
|
||||
if (std::string(d) != o && std::string(d).rfind(o + "/", 0) != 0) continue;
|
||||
std::string rel = std::string(d).substr(o.size());
|
||||
std::string key = slug(d);
|
||||
std::replace(key.begin(), key.end(), '/', '_');
|
||||
fs::path target = layer(o) / "upper" / (rel.empty() ? "" : rel.substr(1));
|
||||
std::error_code ec;
|
||||
if (!fs::is_directory(std::string(d), ec) || !fs::is_directory(target, ec)) continue;
|
||||
seed("files-" + key, "files of " + std::string(d), [&] { return fsutil::copyFiles(std::string(d), target); });
|
||||
}
|
||||
}
|
||||
for (auto f : spec::kUpperLockFiles) {
|
||||
for (const auto& o : overlays) {
|
||||
if (std::string(f).rfind(o + "/", 0) != 0) continue;
|
||||
fs::path target = layer(o) / "upper" / std::string(f).substr(o.size() + 1);
|
||||
std::error_code ec;
|
||||
if (!fs::is_directory(target.parent_path(), ec) || fs::exists(target, ec)) continue;
|
||||
fsutil::writeFile(target, "");
|
||||
::chmod(target.c_str(), 0640);
|
||||
}
|
||||
}
|
||||
for (auto d : spec::kCopiedDirs) {
|
||||
std::error_code ec;
|
||||
if (!fs::is_directory(std::string(d), ec)) continue;
|
||||
std::string key = slug(d);
|
||||
std::replace(key.begin(), key.end(), '/', '_');
|
||||
if (relaxed_) {
|
||||
seed("copy-" + key, "copy of " + std::string(d) + " (relaxed)",
|
||||
[&] { return fsutil::copyTree(std::string(d), privateDir(std::string(d))); });
|
||||
} else {
|
||||
seed("copy-" + key, "allowlisted " + std::string(d), [&] {
|
||||
std::vector<std::string> names(spec::kEtcAllow.begin(), spec::kEtcAllow.end());
|
||||
return fsutil::copyEntries(std::string(d), privateDir(std::string(d)), names, etcDenied);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void Container::create(NetMode net) {
|
||||
namespace {
|
||||
void findUppers(const fs::path& dir, std::size_t depth, std::vector<fs::path>& out) {
|
||||
std::error_code ec;
|
||||
if (depth > 6) return;
|
||||
for (const auto& e : fs::directory_iterator(dir, ec)) {
|
||||
if (!e.is_directory(ec) || e.is_symlink(ec)) continue;
|
||||
std::string n = e.path().filename().string();
|
||||
if (n == "upper") out.push_back(e.path());
|
||||
else if (n != "work") findUppers(e.path(), depth + 1, out);
|
||||
}
|
||||
}
|
||||
} // namespace
|
||||
|
||||
std::uint64_t Container::usedBytes(std::size_t& skipped) const {
|
||||
std::vector<fs::path> dirs;
|
||||
std::error_code ec;
|
||||
if (fs::exists(dir_ / "layers", ec)) findUppers(dir_ / "layers", 0, dirs);
|
||||
if (fs::exists(dir_ / "private", ec)) dirs.push_back(dir_ / "private");
|
||||
auto scan = [&](std::size_t& sk) {
|
||||
std::uint64_t total = 0;
|
||||
for (const auto& d : dirs) total += fsutil::diskUsage(d, sk);
|
||||
return total;
|
||||
};
|
||||
if (ids_ == IdMode::Single) return scan(skipped);
|
||||
// Files of other container ids (host uids 100000+) and their mode 700 directories are invisible to the host
|
||||
// user; measuring as root of the id map sees them.
|
||||
int p[2];
|
||||
if (::pipe2(p, O_CLOEXEC) != 0) throw SysError("pipe");
|
||||
Fd rd(p[0]), wr(p[1]);
|
||||
int rc = UserNsJob::run(ids_, [&] {
|
||||
std::size_t sk = 0;
|
||||
std::uint64_t t = scan(sk);
|
||||
::dprintf(wr.get(), "%llu %zu\n", static_cast<unsigned long long>(t), sk);
|
||||
return 0;
|
||||
});
|
||||
wr.reset();
|
||||
if (rc != 0) throw std::runtime_error("size scan failed");
|
||||
char buf[64] = {0};
|
||||
if (::read(rd.get(), buf, sizeof buf - 1) <= 0) throw std::runtime_error("size scan returned nothing");
|
||||
unsigned long long t = 0;
|
||||
std::size_t sk = 0;
|
||||
std::sscanf(buf, "%llu %zu", &t, &sk);
|
||||
skipped += sk;
|
||||
return t;
|
||||
}
|
||||
|
||||
void Container::writeMeta() const {
|
||||
fsutil::writeFile(metaFile(), "net=" + Network::name(net_) + "\ncreated=" + std::to_string(created_) +
|
||||
"\nversion=" DOOS_VERSION "\nimage=" + image_ + "\nids=" + IdMap::name(ids_) +
|
||||
"\nsize=" + std::to_string(size_) + "\nmemory=" + std::to_string(memory_) +
|
||||
"\npids=" + std::to_string(pids_) + "\ncpus=" + std::to_string(milliCpus_) +
|
||||
"\nhardening=" + (relaxed_ ? "relaxed" : "default") + "\n");
|
||||
}
|
||||
|
||||
void Container::setSizeCap(std::uint64_t bytes) {
|
||||
if (bytes < spec::kMinSizeBytes || bytes > spec::kMaxSizeBytes) {
|
||||
throw std::runtime_error("size must be between " + formatSize(spec::kMinSizeBytes) + " and " +
|
||||
formatSize(spec::kMaxSizeBytes));
|
||||
}
|
||||
size_ = bytes;
|
||||
writeMeta();
|
||||
}
|
||||
|
||||
void Container::setLimits(std::optional<std::uint64_t> memory, std::optional<std::uint32_t> pids,
|
||||
std::optional<std::uint32_t> milliCpus) {
|
||||
if (memory) {
|
||||
if (*memory < spec::kMinMemoryBytes || *memory > spec::kMaxMemoryBytes) {
|
||||
throw std::runtime_error("memory must be between " + formatSize(spec::kMinMemoryBytes) + " and " +
|
||||
formatSize(spec::kMaxMemoryBytes));
|
||||
}
|
||||
memory_ = *memory;
|
||||
}
|
||||
if (pids) {
|
||||
if (*pids < spec::kMinPids || *pids > spec::kMaxPids) {
|
||||
throw std::runtime_error("pids must be between " + std::to_string(spec::kMinPids) + " and " +
|
||||
std::to_string(spec::kMaxPids));
|
||||
}
|
||||
pids_ = *pids;
|
||||
}
|
||||
if (milliCpus) {
|
||||
if (*milliCpus != 0 && (*milliCpus < spec::kMinMilliCpus || *milliCpus > spec::kMaxMilliCpus)) {
|
||||
throw std::runtime_error("cpus must be between 0.1 and 1024 (0 = all cores)");
|
||||
}
|
||||
milliCpus_ = *milliCpus;
|
||||
}
|
||||
writeMeta();
|
||||
}
|
||||
|
||||
void Container::create(const NetMode& net, const std::string& image, IdMode ids, const Options& o) {
|
||||
net_ = net;
|
||||
image_ = image;
|
||||
ids_ = ids;
|
||||
size_ = o.size.value_or(spec::kDefaultSizeBytes);
|
||||
relaxed_ = o.relaxed.value_or(false);
|
||||
memory_ = spec::kDefaultMemoryBytes;
|
||||
pids_ = spec::kDefaultPids;
|
||||
milliCpus_ = 0;
|
||||
if (o.memory || o.pids || o.milliCpus) setLimits(o.memory, o.pids, o.milliCpus);
|
||||
created_ = std::time(nullptr);
|
||||
fsutil::mkdirs(dir_);
|
||||
prepareLayout();
|
||||
fsutil::writeFile(metaFile(), "net=" + Network::name(net_) + "\ncreated=" + std::to_string(created_) +
|
||||
"\nversion=" DOOS_VERSION "\n");
|
||||
writeMeta();
|
||||
}
|
||||
|
||||
void Container::load() {
|
||||
readMeta();
|
||||
prepareLayout();
|
||||
}
|
||||
|
||||
void Container::readMeta() {
|
||||
auto kv = parseKv(fsutil::readFile(metaFile()));
|
||||
net_ = Network::parse(kv.count("net") ? kv["net"] : "slirp");
|
||||
created_ = kv.count("created") ? std::stol(kv["created"]) : 0;
|
||||
prepareLayout(); // heals a container created by an older layout
|
||||
image_ = kv.count("image") ? kv["image"] : "";
|
||||
ids_ = IdMap::parse(kv.count("ids") ? kv["ids"] : "single");
|
||||
size_ = kv.count("size") ? std::stoull(kv["size"]) : spec::kDefaultSizeBytes;
|
||||
memory_ = kv.count("memory") ? std::stoull(kv["memory"]) : spec::kDefaultMemoryBytes;
|
||||
pids_ = kv.count("pids") ? static_cast<std::uint32_t>(std::stoul(kv["pids"])) : spec::kDefaultPids;
|
||||
milliCpus_ = kv.count("cpus") ? static_cast<std::uint32_t>(std::stoul(kv["cpus"])) : 0;
|
||||
relaxed_ = kv.count("hardening") && kv["hardening"] == "relaxed";
|
||||
}
|
||||
|
||||
std::optional<pid_t> Container::runningPid() const {
|
||||
@@ -148,7 +357,7 @@ std::vector<Container> Store::list() const {
|
||||
try {
|
||||
Container::validateName(name);
|
||||
Container c(e.path(), name);
|
||||
c.load();
|
||||
c.readMeta();
|
||||
out.push_back(std::move(c));
|
||||
} catch (const std::exception& ex) {
|
||||
std::fprintf(stderr, "doos: skipping %s: %s\n", name.c_str(), ex.what());
|
||||
@@ -159,18 +368,41 @@ std::vector<Container> Store::list() const {
|
||||
return out;
|
||||
}
|
||||
|
||||
std::pair<Container, bool> Store::getOrCreate(const std::string& name, std::optional<NetMode> net) {
|
||||
std::vector<std::string> Store::containersUsing(const std::string& image) const {
|
||||
std::vector<std::string> out;
|
||||
for (const auto& c : list()) {
|
||||
if (c.image() == image) out.push_back(c.name());
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::pair<Container, bool> Store::getOrCreate(const std::string& name, const Options& opts) {
|
||||
Container::validateName(name);
|
||||
fs::path dir = containersDir() / name;
|
||||
Container c(dir, name);
|
||||
std::error_code ec;
|
||||
std::optional<std::string> image;
|
||||
if (opts.image && !opts.image->empty()) image = ImageStore::normalize(*opts.image);
|
||||
|
||||
auto verifyExisting = [&] {
|
||||
auto fixed = [&](const char* what, const std::string& have, const std::string& want) {
|
||||
if (have != want) {
|
||||
throw std::runtime_error("container '" + name + "' exists with " + what + "=" + have +
|
||||
"; this is fixed at creation");
|
||||
}
|
||||
};
|
||||
if (opts.net) fixed("net", Network::name(c.net()), Network::name(*opts.net));
|
||||
if (image) fixed("image", c.image().empty() ? "none" : c.image(), *image);
|
||||
if (opts.ids) fixed("ids", IdMap::name(c.ids()), IdMap::name(*opts.ids));
|
||||
if (opts.relaxed) fixed("hardening", c.relaxed() ? "relaxed" : "default", *opts.relaxed ? "relaxed" : "default");
|
||||
if (opts.size && *opts.size != c.sizeCap()) c.setSizeCap(*opts.size);
|
||||
if (opts.memory || opts.pids || opts.milliCpus) c.setLimits(opts.memory, opts.pids, opts.milliCpus);
|
||||
};
|
||||
|
||||
if (fs::exists(dir / "meta", ec)) {
|
||||
FileLock lock(c.lockFile(), spec::kLockTimeoutMs);
|
||||
c.load();
|
||||
if (net && *net != c.net()) {
|
||||
throw std::runtime_error("container '" + name + "' exists with net=" + Network::name(c.net()) +
|
||||
"; network mode is fixed at creation");
|
||||
}
|
||||
verifyExisting();
|
||||
return {c, false};
|
||||
}
|
||||
std::size_t count = 0;
|
||||
@@ -179,13 +411,54 @@ std::pair<Container, bool> Store::getOrCreate(const std::string& name, std::opti
|
||||
throw std::runtime_error("container limit reached (" + std::to_string(spec::kMaxContainers) +
|
||||
"); delete one first");
|
||||
}
|
||||
|
||||
// Resolve the image first (may download); ids follow the image so the layer ownership matches.
|
||||
IdMode ids = opts.ids.value_or(IdMap::best());
|
||||
if (image) {
|
||||
ImageStore store = images();
|
||||
if (!store.has(*image)) {
|
||||
if (!Catalog::knows(*image)) {
|
||||
throw std::runtime_error("unknown image '" + *image + "'. Add it: doos image add SOURCE [NAME] (catalog: alpine, ubuntu, debian)");
|
||||
}
|
||||
std::fprintf(stderr, "doos: image '%s' not present, fetching it\n", image->c_str());
|
||||
store.add(Catalog::resolve(*image), ids);
|
||||
}
|
||||
IdMode imageIds = store.ids(*image);
|
||||
if (opts.ids && *opts.ids != imageIds) {
|
||||
throw std::runtime_error("image '" + *image + "' was extracted with ids=" + IdMap::name(imageIds) +
|
||||
"; use the same, or add the image again under another name");
|
||||
}
|
||||
ids = imageIds;
|
||||
}
|
||||
std::uint64_t size = opts.size.value_or(spec::kDefaultSizeBytes);
|
||||
if (size < spec::kMinSizeBytes || size > spec::kMaxSizeBytes) {
|
||||
throw std::runtime_error("size must be between " + formatSize(spec::kMinSizeBytes) + " and " +
|
||||
formatSize(spec::kMaxSizeBytes));
|
||||
}
|
||||
fsutil::mkdirs(dir, 0700);
|
||||
FileLock lock(c.lockFile(), spec::kLockTimeoutMs);
|
||||
if (fs::exists(dir / "meta", ec)) { // lost a creation race
|
||||
c.load();
|
||||
verifyExisting();
|
||||
return {c, false};
|
||||
}
|
||||
c.create(net.value_or(NetMode::Slirp));
|
||||
Options resolved = opts;
|
||||
resolved.size = size;
|
||||
try {
|
||||
c.create(opts.net.value_or(NetMode::Slirp), image.value_or(""), ids, resolved);
|
||||
} catch (...) {
|
||||
// a container that failed halfway must not stay behind as an unreadable directory
|
||||
try {
|
||||
if (ids == IdMode::Subuid) UserNsJob::run(ids, [&] { fsutil::removeTree(dir); return 0; });
|
||||
else fsutil::removeTree(dir);
|
||||
} catch (...) {
|
||||
}
|
||||
throw;
|
||||
}
|
||||
if (c.net() == NetMode::Host) {
|
||||
std::fprintf(stderr, "doos: note: --net host shares the host's abstract unix sockets and loopback; "
|
||||
"--net pasta gives the same reachability in a private namespace\n");
|
||||
}
|
||||
return {c, true};
|
||||
}
|
||||
|
||||
@@ -207,7 +480,20 @@ std::string Store::remove(const std::string& name, bool confirmed) {
|
||||
std::size_t n = fsutil::countEntries(dir, 1000000);
|
||||
return "dry run: would delete " + dir.string() + " (" + std::to_string(n) + " entries). Add --yes to confirm.";
|
||||
}
|
||||
fsutil::removeTree(dir);
|
||||
IdMode ids = IdMode::Single;
|
||||
try {
|
||||
c.readMeta();
|
||||
ids = c.ids();
|
||||
} catch (const std::exception&) {
|
||||
// damaged or half-created container: remove as plain user, files may be left behind loudly
|
||||
}
|
||||
if (ids == IdMode::Subuid) {
|
||||
// files owned by subuids can only be removed from inside the id map
|
||||
int rc = UserNsJob::run(ids, [&] { fsutil::removeTree(dir); return 0; });
|
||||
if (rc != 0) throw std::runtime_error("could not remove all files of '" + name + "'");
|
||||
} else {
|
||||
fsutil::removeTree(dir);
|
||||
}
|
||||
return "deleted " + name;
|
||||
}
|
||||
|
||||
|
||||
+52
-5
@@ -1,15 +1,33 @@
|
||||
#pragma once
|
||||
#include <sys/types.h>
|
||||
|
||||
#include <cstdint>
|
||||
#include <filesystem>
|
||||
#include <functional>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "idmap.hpp"
|
||||
#include "image.hpp"
|
||||
#include "network.hpp"
|
||||
#include "util.hpp"
|
||||
|
||||
namespace doos {
|
||||
|
||||
// What the user asked for. Unset fields mean "default on create, no constraint on existing".
|
||||
struct Options {
|
||||
std::optional<NetMode> net;
|
||||
std::optional<std::string> image;
|
||||
std::optional<IdMode> ids;
|
||||
std::optional<std::uint64_t> size;
|
||||
std::optional<std::uint64_t> memory;
|
||||
std::optional<std::uint32_t> pids;
|
||||
std::optional<std::uint32_t> milliCpus; // 0 = all cores
|
||||
std::optional<bool> relaxed; // --hardening relaxed
|
||||
};
|
||||
|
||||
|
||||
class Container {
|
||||
public:
|
||||
Container(std::filesystem::path dir, std::string name) : dir_(std::move(dir)), name_(std::move(name)) {}
|
||||
@@ -24,26 +42,53 @@ public:
|
||||
std::filesystem::path lockFile() const { return dir_ / "lock"; }
|
||||
std::filesystem::path pidFile() const { return dir_ / "pid"; }
|
||||
std::filesystem::path metaFile() const { return dir_ / "meta"; }
|
||||
// Where /etc lives on disk for this container (resolv.conf and tool config are written there).
|
||||
std::filesystem::path etcDir() const;
|
||||
|
||||
NetMode net() const { return net_; }
|
||||
long created() const { return created_; }
|
||||
const std::string& image() const { return image_; }
|
||||
bool imageMode() const { return !image_.empty(); }
|
||||
IdMode ids() const { return ids_; }
|
||||
std::uint64_t sizeCap() const { return size_; }
|
||||
void setSizeCap(std::uint64_t bytes);
|
||||
std::uint64_t memoryCap() const { return memory_; }
|
||||
std::uint32_t pidsCap() const { return pids_; }
|
||||
std::uint32_t milliCpus() const { return milliCpus_; }
|
||||
bool relaxed() const { return relaxed_; }
|
||||
// Limits take effect the next time the container starts.
|
||||
void setLimits(std::optional<std::uint64_t> memory, std::optional<std::uint32_t> pids, std::optional<std::uint32_t> milliCpus);
|
||||
|
||||
void create(NetMode net); // writes layout and meta
|
||||
void load(); // reads meta
|
||||
void create(const NetMode& net, const std::string& image, IdMode ids, const Options& resolved);
|
||||
void readMeta(); // meta only, no side effects
|
||||
void load(); // readMeta + heal the layout
|
||||
|
||||
// Process id (outside view) of the container's init, if it is alive.
|
||||
std::optional<pid_t> runningPid() const;
|
||||
void recordPid(pid_t pid) const;
|
||||
void clearPid() const;
|
||||
|
||||
// Creates the (empty) mountpoint skeleton and layer dirs. Idempotent.
|
||||
// Mountpoints, layer dirs, and one-time seeding (skeleton, /etc copy). Idempotent.
|
||||
void prepareLayout() const;
|
||||
// Bytes the container wrote (writable layers and private dirs), as the host user can see them.
|
||||
// Measured from inside the id map, so directories of other container ids are counted too.
|
||||
std::uint64_t usedBytes(std::size_t& skipped) const;
|
||||
|
||||
private:
|
||||
void writeMeta() const;
|
||||
void seed(const std::string& marker, const std::string& label, const std::function<fsutil::SeedStats()>& work) const;
|
||||
|
||||
std::filesystem::path dir_;
|
||||
std::string name_;
|
||||
NetMode net_ = NetMode::Slirp;
|
||||
long created_ = 0;
|
||||
std::string image_;
|
||||
IdMode ids_ = IdMode::Single;
|
||||
std::uint64_t size_ = 0;
|
||||
std::uint64_t memory_ = 0;
|
||||
std::uint32_t pids_ = 0;
|
||||
std::uint32_t milliCpus_ = 0;
|
||||
bool relaxed_ = false;
|
||||
};
|
||||
|
||||
class Store {
|
||||
@@ -52,11 +97,13 @@ public:
|
||||
static Store fromEnvironment();
|
||||
|
||||
std::vector<Container> list() const;
|
||||
// Returns the container and whether it was created by this call.
|
||||
std::pair<Container, bool> getOrCreate(const std::string& name, std::optional<NetMode> net);
|
||||
// Returns the container and whether it was created by this call. Pulls a catalog image on demand.
|
||||
std::pair<Container, bool> getOrCreate(const std::string& name, const Options& opts);
|
||||
// Dry-run unless confirmed. Returns a human-readable report line.
|
||||
std::string remove(const std::string& name, bool confirmed);
|
||||
|
||||
ImageStore images() const { return ImageStore(root_ / "images"); }
|
||||
std::vector<std::string> containersUsing(const std::string& image) const;
|
||||
const std::filesystem::path& root() const { return root_; }
|
||||
|
||||
private:
|
||||
|
||||
@@ -0,0 +1,262 @@
|
||||
#include "hardening.hpp"
|
||||
|
||||
#include <linux/audit.h>
|
||||
#include <linux/filter.h>
|
||||
#include <linux/seccomp.h>
|
||||
#include <sched.h>
|
||||
#include <sys/prctl.h>
|
||||
#include <sys/syscall.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cerrno>
|
||||
#include <cstddef>
|
||||
#include <fstream>
|
||||
#include <vector>
|
||||
|
||||
#include "spec.hpp"
|
||||
#include "util.hpp"
|
||||
|
||||
#ifndef SECCOMP_RET_KILL_PROCESS
|
||||
#define SECCOMP_RET_KILL_PROCESS 0x80000000U
|
||||
#endif
|
||||
#ifndef CAP_LAST_CAP_FALLBACK
|
||||
#define CAP_LAST_CAP_FALLBACK 40
|
||||
#endif
|
||||
|
||||
namespace doos {
|
||||
|
||||
namespace {
|
||||
|
||||
#if defined(__x86_64__)
|
||||
constexpr unsigned kArch = AUDIT_ARCH_X86_64;
|
||||
constexpr bool kHasX32 = true;
|
||||
#elif defined(__aarch64__)
|
||||
constexpr unsigned kArch = AUDIT_ARCH_AARCH64;
|
||||
constexpr bool kHasX32 = false;
|
||||
#else
|
||||
constexpr unsigned kArch = 0;
|
||||
constexpr bool kHasX32 = false;
|
||||
#endif
|
||||
|
||||
// Syscalls answered with EPERM. Entries the architecture does not have are skipped at compile time.
|
||||
std::vector<unsigned> deniedSyscalls() {
|
||||
std::vector<unsigned> v;
|
||||
#define DENY(name) v.push_back(static_cast<unsigned>(name))
|
||||
#ifdef SYS_acct
|
||||
DENY(SYS_acct);
|
||||
#endif
|
||||
#ifdef SYS_add_key
|
||||
DENY(SYS_add_key);
|
||||
#endif
|
||||
#ifdef SYS_bpf
|
||||
DENY(SYS_bpf);
|
||||
#endif
|
||||
#ifdef SYS_clock_adjtime
|
||||
DENY(SYS_clock_adjtime);
|
||||
#endif
|
||||
#ifdef SYS_clock_settime
|
||||
DENY(SYS_clock_settime);
|
||||
#endif
|
||||
#ifdef SYS_create_module
|
||||
DENY(SYS_create_module);
|
||||
#endif
|
||||
#ifdef SYS_delete_module
|
||||
DENY(SYS_delete_module);
|
||||
#endif
|
||||
#ifdef SYS_finit_module
|
||||
DENY(SYS_finit_module);
|
||||
#endif
|
||||
#ifdef SYS_init_module
|
||||
DENY(SYS_init_module);
|
||||
#endif
|
||||
#ifdef SYS_get_kernel_syms
|
||||
DENY(SYS_get_kernel_syms);
|
||||
#endif
|
||||
#ifdef SYS_keyctl
|
||||
DENY(SYS_keyctl);
|
||||
#endif
|
||||
#ifdef SYS_kexec_file_load
|
||||
DENY(SYS_kexec_file_load);
|
||||
#endif
|
||||
#ifdef SYS_kexec_load
|
||||
DENY(SYS_kexec_load);
|
||||
#endif
|
||||
#ifdef SYS_lookup_dcookie
|
||||
DENY(SYS_lookup_dcookie);
|
||||
#endif
|
||||
#ifdef SYS_nfsservctl
|
||||
DENY(SYS_nfsservctl);
|
||||
#endif
|
||||
#ifdef SYS_open_by_handle_at
|
||||
DENY(SYS_open_by_handle_at);
|
||||
#endif
|
||||
#ifdef SYS_perf_event_open
|
||||
DENY(SYS_perf_event_open);
|
||||
#endif
|
||||
#ifdef SYS_pivot_root
|
||||
DENY(SYS_pivot_root);
|
||||
#endif
|
||||
#ifdef SYS_process_vm_readv
|
||||
DENY(SYS_process_vm_readv);
|
||||
#endif
|
||||
#ifdef SYS_process_vm_writev
|
||||
DENY(SYS_process_vm_writev);
|
||||
#endif
|
||||
#ifdef SYS_query_module
|
||||
DENY(SYS_query_module);
|
||||
#endif
|
||||
#ifdef SYS_quotactl
|
||||
DENY(SYS_quotactl);
|
||||
#endif
|
||||
#ifdef SYS_reboot
|
||||
DENY(SYS_reboot);
|
||||
#endif
|
||||
#ifdef SYS_request_key
|
||||
DENY(SYS_request_key);
|
||||
#endif
|
||||
#ifdef SYS_settimeofday
|
||||
DENY(SYS_settimeofday);
|
||||
#endif
|
||||
#ifdef SYS_stime
|
||||
DENY(SYS_stime);
|
||||
#endif
|
||||
#ifdef SYS_swapon
|
||||
DENY(SYS_swapon);
|
||||
#endif
|
||||
#ifdef SYS_swapoff
|
||||
DENY(SYS_swapoff);
|
||||
#endif
|
||||
#ifdef SYS_sysfs
|
||||
DENY(SYS_sysfs);
|
||||
#endif
|
||||
#ifdef SYS__sysctl
|
||||
DENY(SYS__sysctl);
|
||||
#endif
|
||||
#ifdef SYS_uselib
|
||||
DENY(SYS_uselib);
|
||||
#endif
|
||||
#ifdef SYS_userfaultfd
|
||||
DENY(SYS_userfaultfd);
|
||||
#endif
|
||||
#ifdef SYS_ustat
|
||||
DENY(SYS_ustat);
|
||||
#endif
|
||||
#ifdef SYS_mount
|
||||
DENY(SYS_mount);
|
||||
#endif
|
||||
#ifdef SYS_umount2
|
||||
DENY(SYS_umount2);
|
||||
#endif
|
||||
#ifdef SYS_umount
|
||||
DENY(SYS_umount);
|
||||
#endif
|
||||
#ifdef SYS_mount_setattr
|
||||
DENY(SYS_mount_setattr);
|
||||
#endif
|
||||
#ifdef SYS_move_mount
|
||||
DENY(SYS_move_mount);
|
||||
#endif
|
||||
#ifdef SYS_open_tree
|
||||
DENY(SYS_open_tree);
|
||||
#endif
|
||||
#ifdef SYS_fsopen
|
||||
DENY(SYS_fsopen);
|
||||
#endif
|
||||
#ifdef SYS_fsconfig
|
||||
DENY(SYS_fsconfig);
|
||||
#endif
|
||||
#ifdef SYS_fsmount
|
||||
DENY(SYS_fsmount);
|
||||
#endif
|
||||
#ifdef SYS_fspick
|
||||
DENY(SYS_fspick);
|
||||
#endif
|
||||
#ifdef SYS_io_uring_setup
|
||||
DENY(SYS_io_uring_setup);
|
||||
#endif
|
||||
#ifdef SYS_io_uring_enter
|
||||
DENY(SYS_io_uring_enter);
|
||||
#endif
|
||||
#ifdef SYS_io_uring_register
|
||||
DENY(SYS_io_uring_register);
|
||||
#endif
|
||||
#ifdef SYS_setns
|
||||
DENY(SYS_setns);
|
||||
#endif
|
||||
#ifdef SYS_kcmp
|
||||
DENY(SYS_kcmp);
|
||||
#endif
|
||||
#undef DENY
|
||||
return v;
|
||||
}
|
||||
|
||||
sock_filter stmt(unsigned short code, unsigned k) { return BPF_STMT(code, k); }
|
||||
sock_filter jump(unsigned short code, unsigned k, unsigned char jt, unsigned char jf) { return BPF_JUMP(code, k, jt, jf); }
|
||||
|
||||
constexpr unsigned kOffNr = offsetof(seccomp_data, nr);
|
||||
constexpr unsigned kOffArch = offsetof(seccomp_data, arch);
|
||||
constexpr unsigned kOffArg0 = offsetof(seccomp_data, args); // low 32 bits on little-endian
|
||||
|
||||
std::vector<sock_filter> buildFilter() {
|
||||
const unsigned eperm = SECCOMP_RET_ERRNO | EPERM;
|
||||
const unsigned enosys = SECCOMP_RET_ERRNO | ENOSYS;
|
||||
std::vector<sock_filter> f;
|
||||
// wrong architecture: kill, never guess
|
||||
f.push_back(stmt(BPF_LD | BPF_W | BPF_ABS, kOffArch));
|
||||
f.push_back(jump(BPF_JMP | BPF_JEQ | BPF_K, kArch, 1, 0));
|
||||
f.push_back(stmt(BPF_RET | BPF_K, SECCOMP_RET_KILL_PROCESS));
|
||||
f.push_back(stmt(BPF_LD | BPF_W | BPF_ABS, kOffNr));
|
||||
if (kHasX32) { // x32 syscall numbers must not dodge the numbers below
|
||||
f.push_back(jump(BPF_JMP | BPF_JSET | BPF_K, 0x40000000U, 0, 1));
|
||||
f.push_back(stmt(BPF_RET | BPF_K, enosys));
|
||||
}
|
||||
// clone3 cannot be inspected (flags live in a struct): ENOSYS makes libc fall back to clone()
|
||||
#ifdef SYS_clone3
|
||||
f.push_back(jump(BPF_JMP | BPF_JEQ | BPF_K, static_cast<unsigned>(SYS_clone3), 0, 1));
|
||||
f.push_back(stmt(BPF_RET | BPF_K, enosys));
|
||||
#endif
|
||||
// clone/unshare with CLONE_NEWUSER: EPERM (nested user namespaces are the usual first step of kernel exploits)
|
||||
for (unsigned nr : {static_cast<unsigned>(SYS_clone), static_cast<unsigned>(SYS_unshare)}) {
|
||||
f.push_back(jump(BPF_JMP | BPF_JEQ | BPF_K, nr, 0, 4));
|
||||
f.push_back(stmt(BPF_LD | BPF_W | BPF_ABS, kOffArg0));
|
||||
f.push_back(jump(BPF_JMP | BPF_JSET | BPF_K, CLONE_NEWUSER, 0, 1));
|
||||
f.push_back(stmt(BPF_RET | BPF_K, eperm));
|
||||
f.push_back(stmt(BPF_LD | BPF_W | BPF_ABS, kOffNr)); // restore the accumulator for the next compare
|
||||
}
|
||||
for (unsigned nr : deniedSyscalls()) {
|
||||
f.push_back(jump(BPF_JMP | BPF_JEQ | BPF_K, nr, 0, 1));
|
||||
f.push_back(stmt(BPF_RET | BPF_K, eperm));
|
||||
}
|
||||
f.push_back(stmt(BPF_RET | BPF_K, SECCOMP_RET_ALLOW));
|
||||
if (f.size() > 4096) throw std::runtime_error("seccomp program too large");
|
||||
return f;
|
||||
}
|
||||
|
||||
void dropCapabilities() {
|
||||
int last = CAP_LAST_CAP_FALLBACK;
|
||||
std::ifstream in("/proc/sys/kernel/cap_last_cap");
|
||||
if (in) in >> last;
|
||||
for (int cap = 0; cap <= last; ++cap) {
|
||||
if (std::find(spec::kKeptCaps.begin(), spec::kKeptCaps.end(), cap) != spec::kKeptCaps.end()) continue;
|
||||
if (::prctl(PR_CAPBSET_DROP, cap, 0, 0, 0) != 0 && errno != EINVAL) {
|
||||
throw SysError("drop capability " + std::to_string(cap));
|
||||
}
|
||||
}
|
||||
::prctl(PR_CAP_AMBIENT, PR_CAP_AMBIENT_CLEAR_ALL, 0, 0, 0);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
void Hardening::apply() {
|
||||
dropCapabilities();
|
||||
if (kArch == 0) {
|
||||
std::fprintf(stderr, "doos: no seccomp filter for this CPU architecture, capabilities were still reduced\n");
|
||||
return;
|
||||
}
|
||||
std::vector<sock_filter> prog = buildFilter();
|
||||
sock_fprog fp{static_cast<unsigned short>(prog.size()), prog.data()};
|
||||
if (::prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &fp) != 0) throw SysError("install seccomp filter");
|
||||
}
|
||||
|
||||
} // namespace doos
|
||||
@@ -0,0 +1,15 @@
|
||||
#pragma once
|
||||
|
||||
namespace doos {
|
||||
|
||||
// Confinement applied to every process that runs a container command, right before exec:
|
||||
// 1. capability bounding set reduced to the Docker default set (spec::kKeptCaps),
|
||||
// 2. a seccomp-bpf denylist (keyring, bpf, perf, userfaultfd, io_uring, mount family, kexec, module loading,
|
||||
// nested user namespaces and the like), with an architecture check and x32 guard.
|
||||
// Both need no_new_privs, which the caller sets. Throws when the filter cannot be installed.
|
||||
class Hardening {
|
||||
public:
|
||||
static void apply();
|
||||
};
|
||||
|
||||
} // namespace doos
|
||||
+148
@@ -0,0 +1,148 @@
|
||||
#include "idmap.hpp"
|
||||
|
||||
#include <fcntl.h>
|
||||
#include <pwd.h>
|
||||
#include <sched.h>
|
||||
#include <sys/prctl.h>
|
||||
#include <signal.h>
|
||||
#include <sys/wait.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <fstream>
|
||||
#include <sstream>
|
||||
#include <vector>
|
||||
|
||||
#include "spec.hpp"
|
||||
#include "util.hpp"
|
||||
|
||||
namespace doos {
|
||||
|
||||
namespace {
|
||||
|
||||
void writeProcFile(pid_t pid, const char* file, const std::string& content) {
|
||||
std::string path = "/proc/" + std::to_string(pid) + "/" + file;
|
||||
int fd = ::open(path.c_str(), O_WRONLY | O_CLOEXEC);
|
||||
if (fd < 0) throw SysError("open " + path);
|
||||
Fd guard(fd);
|
||||
if (::write(fd, content.data(), content.size()) != static_cast<ssize_t>(content.size())) {
|
||||
throw SysError("write " + path + " (user namespaces blocked? see apparmor/doos and docs/RESEARCH.md)");
|
||||
}
|
||||
}
|
||||
|
||||
bool onPath(std::string_view bin) {
|
||||
const char* path = std::getenv("PATH");
|
||||
std::stringstream ss(path ? path : "/usr/bin:/bin");
|
||||
for (std::string dir; std::getline(ss, dir, ':');) {
|
||||
std::string full = dir + "/" + std::string(bin);
|
||||
if (::access(full.c_str(), X_OK) == 0) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
IdMode IdMap::parse(const std::string& s) {
|
||||
if (s == "single") return IdMode::Single;
|
||||
if (s == "subuid") return IdMode::Subuid;
|
||||
throw std::runtime_error("unknown id mode '" + s + "' (use single or subuid)");
|
||||
}
|
||||
|
||||
std::string IdMap::name(IdMode m) { return m == IdMode::Subuid ? "subuid" : "single"; }
|
||||
|
||||
std::optional<SubRange> IdMap::range(const char* file) {
|
||||
std::ifstream in(file);
|
||||
if (!in) return std::nullopt;
|
||||
std::string user;
|
||||
if (const passwd* pw = ::getpwuid(::geteuid())) user = pw->pw_name;
|
||||
std::string uid = std::to_string(::geteuid());
|
||||
for (std::string line; std::getline(in, line);) {
|
||||
std::vector<std::string> f;
|
||||
std::stringstream ss(line);
|
||||
for (std::string t; std::getline(ss, t, ':');) f.push_back(t);
|
||||
if (f.size() != 3 || (f[0] != user && f[0] != uid)) continue;
|
||||
try {
|
||||
SubRange r{std::stoul(f[1]), std::stoul(f[2])};
|
||||
if (r.count > 0) return r;
|
||||
} catch (const std::exception&) {
|
||||
}
|
||||
}
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
bool IdMap::subuidAvailable() {
|
||||
return range("/etc/subuid") && range("/etc/subgid") && onPath(spec::kNewUidMap) && onPath(spec::kNewGidMap);
|
||||
}
|
||||
|
||||
IdMode IdMap::best() { return subuidAvailable() ? IdMode::Subuid : IdMode::Single; }
|
||||
|
||||
void IdMap::apply(pid_t pid, IdMode mode) {
|
||||
std::string uid = std::to_string(::geteuid()), gid = std::to_string(::getegid());
|
||||
if (mode == IdMode::Single) {
|
||||
writeProcFile(pid, "setgroups", "deny");
|
||||
writeProcFile(pid, "uid_map", "0 " + uid + " 1\n");
|
||||
writeProcFile(pid, "gid_map", "0 " + gid + " 1\n");
|
||||
return;
|
||||
}
|
||||
auto u = range("/etc/subuid");
|
||||
auto g = range("/etc/subgid");
|
||||
if (!u || !g || !onPath(spec::kNewUidMap) || !onPath(spec::kNewGidMap)) {
|
||||
throw std::runtime_error("subuid mapping unavailable: need /etc/subuid, /etc/subgid, newuidmap and newgidmap "
|
||||
"(install uidmap, or use --ids single)");
|
||||
}
|
||||
auto n = [](const SubRange& r) { return std::to_string(std::min<unsigned long>(r.count, spec::kMaxIdRange - 1)); };
|
||||
std::string p = std::to_string(pid);
|
||||
ProcResult a = runProcess({std::string(spec::kNewUidMap), p, "0", uid, "1", "1", std::to_string(u->start), n(*u)});
|
||||
if (a.code != 0) throw std::runtime_error("newuidmap failed (exit " + std::to_string(a.code) + ")");
|
||||
ProcResult b = runProcess({std::string(spec::kNewGidMap), p, "0", gid, "1", "1", std::to_string(g->start), n(*g)});
|
||||
if (b.code != 0) throw std::runtime_error("newgidmap failed (exit " + std::to_string(b.code) + ")");
|
||||
}
|
||||
|
||||
namespace {
|
||||
struct JobCtx {
|
||||
const std::function<int()>* fn;
|
||||
int syncFd;
|
||||
};
|
||||
|
||||
int jobMain(void* arg) {
|
||||
auto* c = static_cast<JobCtx*>(arg);
|
||||
::prctl(PR_SET_PDEATHSIG, SIGKILL);
|
||||
char go = 0;
|
||||
if (::read(c->syncFd, &go, 1) != 1 || go != 'G') ::_exit(125);
|
||||
::close(c->syncFd);
|
||||
int code = 125;
|
||||
try {
|
||||
code = (*c->fn)();
|
||||
} catch (const std::exception& e) {
|
||||
::dprintf(2, "doos: %s\n", e.what());
|
||||
}
|
||||
::_exit(code);
|
||||
}
|
||||
} // namespace
|
||||
|
||||
int UserNsJob::run(IdMode mode, const std::function<int()>& fn) {
|
||||
int sync[2];
|
||||
if (::pipe2(sync, O_CLOEXEC) != 0) throw SysError("pipe");
|
||||
Fd rd(sync[0]), wr(sync[1]);
|
||||
std::vector<char> stack(1 << 20);
|
||||
JobCtx ctx{&fn, rd.get()};
|
||||
pid_t child = ::clone(jobMain, stack.data() + stack.size(), CLONE_NEWUSER | SIGCHLD, &ctx);
|
||||
if (child < 0) throw SysError("clone (user namespaces available?)");
|
||||
try {
|
||||
IdMap::apply(child, mode);
|
||||
} catch (...) {
|
||||
::kill(child, SIGKILL);
|
||||
::waitpid(child, nullptr, 0);
|
||||
throw;
|
||||
}
|
||||
char go = 'G';
|
||||
if (::write(wr.get(), &go, 1) != 1) throw SysError("sync job");
|
||||
wr.reset();
|
||||
int st = 0;
|
||||
while (::waitpid(child, &st, 0) < 0) {
|
||||
if (errno != EINTR) throw SysError("waitpid");
|
||||
}
|
||||
return WIFEXITED(st) ? WEXITSTATUS(st) : 128 + WTERMSIG(st);
|
||||
}
|
||||
|
||||
} // namespace doos
|
||||
@@ -0,0 +1,41 @@
|
||||
#pragma once
|
||||
#include <sys/types.h>
|
||||
|
||||
#include <functional>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
|
||||
namespace doos {
|
||||
|
||||
// How container ids map to host ids.
|
||||
// single: container 0 -> invoking user, nothing else mappable.
|
||||
// subuid: container 0 -> invoking user, 1..N -> the user's /etc/subuid and /etc/subgid range (newuidmap).
|
||||
enum class IdMode { Single, Subuid };
|
||||
|
||||
struct SubRange {
|
||||
unsigned long start = 0;
|
||||
unsigned long count = 0;
|
||||
};
|
||||
|
||||
class IdMap {
|
||||
public:
|
||||
static IdMode parse(const std::string& s); // "single" or "subuid"
|
||||
static std::string name(IdMode m);
|
||||
// Subuid when both ranges and both helper binaries exist, else Single.
|
||||
static IdMode best();
|
||||
static bool subuidAvailable();
|
||||
// Applies the map to the user namespace of `pid`. Throws with the reason when it cannot.
|
||||
static void apply(pid_t pid, IdMode mode);
|
||||
|
||||
private:
|
||||
static std::optional<SubRange> range(const char* file);
|
||||
};
|
||||
|
||||
// Runs `fn` as root of a fresh user namespace carrying the same id map as containers of `mode`.
|
||||
// Needed to touch files owned by subuids (extract images, delete trees). Returns fn's exit code.
|
||||
class UserNsJob {
|
||||
public:
|
||||
static int run(IdMode mode, const std::function<int()>& fn);
|
||||
};
|
||||
|
||||
} // namespace doos
|
||||
+384
@@ -0,0 +1,384 @@
|
||||
#include "image.hpp"
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <sys/stat.h>
|
||||
#include <cstdio>
|
||||
#include <sys/utsname.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cctype>
|
||||
#include <ctime>
|
||||
#include <map>
|
||||
#include <sstream>
|
||||
|
||||
#include "spec.hpp"
|
||||
#include "util.hpp"
|
||||
|
||||
namespace fs = std::filesystem;
|
||||
|
||||
namespace doos {
|
||||
|
||||
namespace {
|
||||
|
||||
std::string cpuArch(bool debianStyle) {
|
||||
utsname u;
|
||||
if (::uname(&u) != 0) throw SysError("uname");
|
||||
std::string m = u.machine;
|
||||
if (m == "x86_64") return debianStyle ? "amd64" : "x86_64";
|
||||
if (m == "aarch64") return debianStyle ? "arm64" : "aarch64";
|
||||
throw std::runtime_error("no catalog images for architecture " + m + "; use a file or URL with --sha256");
|
||||
}
|
||||
|
||||
std::string fetchText(const std::string& url) {
|
||||
ProcResult r = runProcess({"curl", "--fail", "--silent", "--show-error", "--location", "--proto", "=https",
|
||||
"--max-time", "60", url},
|
||||
true);
|
||||
if (r.code != 0) throw std::runtime_error("download failed: " + url + " (curl exit " + std::to_string(r.code) + ")");
|
||||
return r.out;
|
||||
}
|
||||
|
||||
void fetchFile(const std::string& url, const fs::path& out) {
|
||||
ProcResult r = runProcess({"curl", "--fail", "--silent", "--show-error", "--location", "--proto", "=https",
|
||||
"--max-time", "60", "--max-filesize", std::to_string(spec::kMaxCaptureBytes),
|
||||
"--output", out.string(), url});
|
||||
if (r.code != 0) throw std::runtime_error("download failed: " + url + " (curl exit " + std::to_string(r.code) + ")");
|
||||
}
|
||||
|
||||
// Private scratch directory, removed on scope exit.
|
||||
class TempDir {
|
||||
public:
|
||||
TempDir() {
|
||||
std::string tmpl = (fs::temp_directory_path() / "doos-XXXXXX").string();
|
||||
if (!::mkdtemp(tmpl.data())) throw SysError("mkdtemp");
|
||||
path_ = tmpl;
|
||||
}
|
||||
~TempDir() {
|
||||
std::error_code ec;
|
||||
fs::remove_all(path_, ec);
|
||||
}
|
||||
TempDir(const TempDir&) = delete;
|
||||
TempDir& operator=(const TempDir&) = delete;
|
||||
const fs::path& path() const { return path_; }
|
||||
|
||||
private:
|
||||
fs::path path_;
|
||||
};
|
||||
|
||||
std::string upper(std::string s) {
|
||||
for (auto& c : s) c = static_cast<char>(std::toupper(static_cast<unsigned char>(c)));
|
||||
return s;
|
||||
}
|
||||
|
||||
// Verifies a detached signature against a publisher key that is fetched over https and accepted only when its
|
||||
// fingerprint equals the pinned one. Returns false (with a loud note) when gpg is not installed; throws when the
|
||||
// signature is bad, missing, or made by another key.
|
||||
bool verifySignature(const fs::path& data, const fs::path& sig, const std::string& keyUrl, const std::string& fpr) {
|
||||
if (runProcess({"sh", "-c", "command -v gpg"}, true, true).code != 0) {
|
||||
std::fprintf(stderr, "note: signature NOT checked (gpg is not installed); relying on https and the publisher's checksum\n");
|
||||
return false;
|
||||
}
|
||||
TempDir home;
|
||||
::chmod(home.path().c_str(), 0700);
|
||||
fs::path keyFile = home.path() / "key.asc";
|
||||
fetchFile(keyUrl, keyFile);
|
||||
std::string h = home.path().string();
|
||||
if (runProcess({"gpg", "--batch", "--quiet", "--homedir", h, "--import", keyFile.string()}, false, true).code != 0) {
|
||||
throw std::runtime_error("cannot import the publisher key from " + keyUrl);
|
||||
}
|
||||
ProcResult fp = runProcess({"gpg", "--batch", "--homedir", h, "--with-colons", "--fingerprint"}, true, true);
|
||||
if (fp.out.find(":::::::::" + upper(fpr) + ":") == std::string::npos) {
|
||||
throw std::runtime_error("the key at " + keyUrl + " does not have the pinned fingerprint " + fpr);
|
||||
}
|
||||
ProcResult v = runProcess({"gpg", "--batch", "--homedir", h, "--status-fd", "1", "--verify", sig.string(), data.string()}, true, true);
|
||||
bool good = false, valid = false;
|
||||
std::istringstream in(v.out);
|
||||
for (std::string line; std::getline(in, line);) {
|
||||
if (line.find("BADSIG") != std::string::npos || line.find("ERRSIG") != std::string::npos ||
|
||||
line.find("EXPKEYSIG") != std::string::npos || line.find("REVKEYSIG") != std::string::npos) {
|
||||
throw std::runtime_error("bad signature on " + data.filename().string());
|
||||
}
|
||||
if (line.find("[GNUPG:] GOODSIG") == 0) good = true;
|
||||
if (line.find("[GNUPG:] VALIDSIG") == 0) {
|
||||
std::istringstream ls(line);
|
||||
std::vector<std::string> tok;
|
||||
for (std::string t; ls >> t;) tok.push_back(upper(t));
|
||||
if (tok.size() > 2 && (tok[2] == upper(fpr) || tok.back() == upper(fpr))) valid = true;
|
||||
}
|
||||
}
|
||||
if (v.code != 0 || !good || !valid) {
|
||||
throw std::runtime_error("signature check failed for " + data.filename().string() + " (not signed by " + fpr + ")");
|
||||
}
|
||||
std::fprintf(stderr, "signature verified (key %s)\n", fpr.c_str());
|
||||
return true;
|
||||
}
|
||||
|
||||
bool isHex64(const std::string& s) {
|
||||
return s.size() == 64 && std::all_of(s.begin(), s.end(), [](unsigned char c) { return std::isxdigit(c); });
|
||||
}
|
||||
|
||||
std::vector<unsigned long> versionParts(const std::string& v) {
|
||||
std::vector<unsigned long> out;
|
||||
std::stringstream ss(v);
|
||||
for (std::string t; std::getline(ss, t, '.');) out.push_back(t.empty() ? 0 : std::stoul(t));
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string lowerCase(std::string s) {
|
||||
for (auto& c : s) c = static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
|
||||
return s;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::vector<std::string> Catalog::names() { return {"alpine", "ubuntu", "ubuntu-24.04", "debian"}; }
|
||||
|
||||
bool Catalog::knows(const std::string& n) {
|
||||
return n == "alpine" || n == "ubuntu" || (n.rfind("ubuntu-", 0) == 0 && n.size() > 7) || n == "debian" ||
|
||||
(n.rfind("debian-", 0) == 0 && n.size() > 7);
|
||||
}
|
||||
|
||||
ImageSource Catalog::resolve(const std::string& n) {
|
||||
ImageSource src;
|
||||
src.name = n;
|
||||
if (n == "alpine") {
|
||||
std::string arch = cpuArch(false);
|
||||
std::string base = std::string(spec::kAlpineBase) + arch + "/";
|
||||
std::string yaml = fetchText(base + "latest-releases.yaml");
|
||||
std::istringstream in(yaml);
|
||||
std::string file, sha;
|
||||
bool inBlock = false;
|
||||
for (std::string line; std::getline(in, line);) {
|
||||
if (line.find("flavor: alpine-minirootfs") != std::string::npos) { inBlock = true; continue; }
|
||||
if (inBlock && line.rfind("-", 0) == 0) break;
|
||||
auto val = [&](const char* key) {
|
||||
auto p = line.find(key);
|
||||
return p == std::string::npos ? std::string() : line.substr(p + std::string(key).size());
|
||||
};
|
||||
if (inBlock && file.empty()) file = val("file: ");
|
||||
if (inBlock && sha.empty()) sha = val("sha256: ");
|
||||
}
|
||||
if (file.empty() || !isHex64(sha) || file.find('/') != std::string::npos) {
|
||||
throw std::runtime_error("cannot find alpine-minirootfs in latest-releases.yaml");
|
||||
}
|
||||
src.location = base + file;
|
||||
src.sha256 = sha;
|
||||
src.sigUrl = src.location + ".asc";
|
||||
src.keyUrl = std::string(spec::kAlpineKeyUrl);
|
||||
src.keyFpr = std::string(spec::kAlpineKeyFpr);
|
||||
return src;
|
||||
}
|
||||
if (n == "debian" || n.rfind("debian-", 0) == 0) {
|
||||
std::string rel = n == "debian" ? std::string(spec::kDebianDefaultRelease) : n.substr(7);
|
||||
if (rel.empty() || !std::all_of(rel.begin(), rel.end(), [](unsigned char c) { return std::islower(c); })) {
|
||||
throw std::runtime_error("invalid debian release '" + rel + "' (example: debian-bookworm)");
|
||||
}
|
||||
std::string arch = cpuArch(true);
|
||||
std::string dist = arch == "amd64" ? "dist-amd64" : "dist-arm64v8";
|
||||
std::string base = std::string(spec::kDebianBase) + dist + "/" + rel + "/slim/oci/";
|
||||
std::string index = fetchText(base + "index.json");
|
||||
// The manifest is embedded (base64) in the index; its single layer digest is the sha256 of the rootfs tarball.
|
||||
auto d = index.find("\"data\":\"");
|
||||
if (d == std::string::npos) throw std::runtime_error("unexpected debian index.json (no embedded manifest)");
|
||||
d += 8;
|
||||
std::string manifest = base64Decode(index.substr(d, index.find('"', d) - d));
|
||||
auto l = manifest.find("\"layers\"");
|
||||
auto dg = l == std::string::npos ? l : manifest.find("sha256:", l);
|
||||
if (dg == std::string::npos) throw std::runtime_error("unexpected debian manifest (no layer digest)");
|
||||
std::string sha = manifest.substr(dg + 7, 64);
|
||||
if (!isHex64(sha)) throw std::runtime_error("unexpected debian layer digest");
|
||||
src.location = base + "blobs/rootfs.tar.gz";
|
||||
src.sha256 = sha; // content-addressed: the digest is the integrity check; there is no publisher signature
|
||||
return src;
|
||||
}
|
||||
if (n == "ubuntu" || n.rfind("ubuntu-", 0) == 0) {
|
||||
std::string ver = n == "ubuntu" ? std::string(spec::kUbuntuDefaultVersion) : n.substr(7);
|
||||
if (ver.empty() || !std::all_of(ver.begin(), ver.end(), [](unsigned char c) { return std::isdigit(c) || c == '.'; })) {
|
||||
throw std::runtime_error("invalid ubuntu version '" + ver + "' (example: ubuntu-24.04)");
|
||||
}
|
||||
std::string arch = cpuArch(true);
|
||||
std::string base = std::string(spec::kUbuntuBase) + ver + "/release/";
|
||||
TempDir tmp;
|
||||
fetchFile(base + "SHA256SUMS", tmp.path() / "SHA256SUMS");
|
||||
fetchFile(base + "SHA256SUMS.gpg", tmp.path() / "SHA256SUMS.gpg");
|
||||
verifySignature(tmp.path() / "SHA256SUMS", tmp.path() / "SHA256SUMS.gpg", std::string(spec::kUbuntuKeyUrl),
|
||||
std::string(spec::kUbuntuKeyFpr));
|
||||
std::string sums = fsutil::readFile(tmp.path() / "SHA256SUMS");
|
||||
std::string suffix = "-base-" + arch + ".tar.gz";
|
||||
std::string bestFile, bestSha;
|
||||
std::vector<unsigned long> bestVer;
|
||||
std::istringstream in(sums);
|
||||
for (std::string line; std::getline(in, line);) {
|
||||
auto sp = line.find(" *");
|
||||
if (sp == std::string::npos) continue;
|
||||
std::string sha = line.substr(0, sp), file = line.substr(sp + 2);
|
||||
if (file.rfind("ubuntu-base-", 0) != 0 || file.size() <= suffix.size() ||
|
||||
file.compare(file.size() - suffix.size(), suffix.size(), suffix) != 0 || !isHex64(sha)) {
|
||||
continue;
|
||||
}
|
||||
auto v = versionParts(file.substr(12, file.size() - 12 - suffix.size()));
|
||||
if (bestFile.empty() || v > bestVer) { bestFile = file; bestSha = sha; bestVer = v; }
|
||||
}
|
||||
if (bestFile.empty()) throw std::runtime_error("no ubuntu-base " + ver + " for " + arch + " found at " + base);
|
||||
src.location = base + bestFile;
|
||||
src.sha256 = bestSha;
|
||||
return src;
|
||||
}
|
||||
throw std::runtime_error("unknown catalog image '" + n + "' (known: alpine, ubuntu, ubuntu-VERSION, debian, debian-RELEASE)");
|
||||
}
|
||||
|
||||
std::string ImageStore::normalize(const std::string& name) {
|
||||
std::string s = lowerCase(name);
|
||||
std::replace(s.begin(), s.end(), ':', '-');
|
||||
return s;
|
||||
}
|
||||
|
||||
void ImageStore::validateName(const std::string& name) {
|
||||
bool ok = !name.empty() && name.size() <= spec::kMaxNameLen && std::isalnum(static_cast<unsigned char>(name[0]));
|
||||
for (char c : name) ok = ok && (std::islower(static_cast<unsigned char>(c)) || std::isdigit(static_cast<unsigned char>(c)) ||
|
||||
c == '-' || c == '_' || c == '.');
|
||||
if (!ok) throw std::runtime_error("invalid image name '" + name + "': use [a-z0-9][a-z0-9._-]{0,31}");
|
||||
}
|
||||
|
||||
ImageStore::ImageStore(fs::path root) : root_(std::move(root)) { fsutil::mkdirs(root_, 0700); }
|
||||
|
||||
bool ImageStore::has(const std::string& name) const { return fs::exists(dir(name) / "meta"); }
|
||||
fs::path ImageStore::rootfs(const std::string& name) const { return dir(name) / "rootfs"; }
|
||||
|
||||
namespace {
|
||||
std::map<std::string, std::string> readMeta(const fs::path& p) {
|
||||
std::map<std::string, std::string> kv;
|
||||
std::istringstream in(fsutil::readFile(p));
|
||||
for (std::string line; std::getline(in, line);) {
|
||||
auto eq = line.find('=');
|
||||
if (eq != std::string::npos) kv[line.substr(0, eq)] = line.substr(eq + 1);
|
||||
}
|
||||
return kv;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
IdMode ImageStore::ids(const std::string& name) const {
|
||||
auto kv = readMeta(dir(name) / "meta");
|
||||
return IdMap::parse(kv.count("ids") ? kv["ids"] : "single");
|
||||
}
|
||||
|
||||
std::vector<ImageInfo> ImageStore::list() const {
|
||||
std::vector<ImageInfo> out;
|
||||
for (const auto& e : fs::directory_iterator(root_)) {
|
||||
std::string name = e.path().filename().string();
|
||||
if (!e.is_directory() || name.rfind(".", 0) == 0 || !has(name)) continue;
|
||||
auto kv = readMeta(e.path() / "meta");
|
||||
ImageInfo i;
|
||||
i.name = name;
|
||||
i.source = kv["source"];
|
||||
i.ids = IdMap::parse(kv.count("ids") ? kv["ids"] : "single");
|
||||
i.created = kv.count("created") ? std::stol(kv["created"]) : 0;
|
||||
i.bytes = kv.count("bytes") ? std::stoull(kv["bytes"]) : 0;
|
||||
out.push_back(i);
|
||||
if (out.size() >= spec::kMaxImages) break;
|
||||
}
|
||||
std::sort(out.begin(), out.end(), [](const ImageInfo& a, const ImageInfo& b) { return a.name < b.name; });
|
||||
return out;
|
||||
}
|
||||
|
||||
ImageInfo ImageStore::add(const ImageSource& src, IdMode ids) {
|
||||
validateName(src.name);
|
||||
if (has(src.name)) throw std::runtime_error("image '" + src.name + "' already exists");
|
||||
std::size_t count = 0;
|
||||
for (const auto& e : fs::directory_iterator(root_)) count += (e.is_directory() && has(e.path().filename().string())) ? 1 : 0;
|
||||
if (count >= spec::kMaxImages) {
|
||||
throw std::runtime_error("image limit reached (" + std::to_string(spec::kMaxImages) + "); delete one first");
|
||||
}
|
||||
for (const char* tool : {"curl", "tar", "sha256sum"}) {
|
||||
if (runProcess({"sh", "-c", std::string("command -v ") + tool}, true, true).code != 0) {
|
||||
throw std::runtime_error(std::string("required tool not found: ") + tool);
|
||||
}
|
||||
}
|
||||
fs::path tmp = root_ / (".tmp-" + src.name);
|
||||
{
|
||||
std::error_code ec;
|
||||
if (fs::exists(tmp, ec)) {
|
||||
int rc = UserNsJob::run(ids, [&] { fsutil::removeTree(tmp); return 0; });
|
||||
if (rc != 0) throw std::runtime_error("cannot clean stale " + tmp.string());
|
||||
}
|
||||
}
|
||||
fsutil::mkdirs(tmp, 0700);
|
||||
auto cleanup = [&] { UserNsJob::run(ids, [&] { try { fsutil::removeTree(tmp); } catch (...) { return 1; } return 0; }); };
|
||||
try {
|
||||
fs::path archive;
|
||||
if (src.isFile) {
|
||||
archive = src.location;
|
||||
if (!fs::is_regular_file(archive)) throw std::runtime_error("not a file: " + archive.string());
|
||||
} else {
|
||||
archive = tmp / "download";
|
||||
std::fprintf(stderr, "downloading %s\n", src.location.c_str());
|
||||
ProcResult d = runProcess({"curl", "--fail", "--location", "--proto", "=https", "--progress-bar",
|
||||
"--max-filesize", std::to_string(spec::kMaxImageDownloadBytes), "--max-time",
|
||||
std::to_string(spec::kDownloadTimeoutSec), "--output", archive.string(), src.location});
|
||||
if (d.code != 0) throw std::runtime_error("download failed (curl exit " + std::to_string(d.code) + ")");
|
||||
}
|
||||
if (!src.sha256.empty()) {
|
||||
ProcResult h = runProcess({"sha256sum", archive.string()}, true);
|
||||
std::string got = h.out.substr(0, 64);
|
||||
if (h.code != 0 || got != lowerCase(src.sha256)) {
|
||||
throw std::runtime_error("sha256 mismatch for " + src.location + ": expected " + src.sha256 + ", got " + got);
|
||||
}
|
||||
std::fprintf(stderr, "sha256 verified\n");
|
||||
if (!src.sigUrl.empty()) {
|
||||
TempDir sigDir;
|
||||
fetchFile(src.sigUrl, sigDir.path() / "file.asc");
|
||||
verifySignature(archive, sigDir.path() / "file.asc", src.keyUrl, src.keyFpr);
|
||||
}
|
||||
} else {
|
||||
std::fprintf(stderr, "note: local file used without --sha256, not verified\n");
|
||||
}
|
||||
fs::path root = tmp / "rootfs";
|
||||
fsutil::mkdirs(root, 0755);
|
||||
std::vector<std::string> tar = {"tar", "-x", "-p", "--numeric-owner", "--exclude=./dev/*", "--exclude=dev/*",
|
||||
"-f", archive.string(), "-C", root.string()};
|
||||
if (ids == IdMode::Single) tar.insert(tar.begin() + 1, "--no-same-owner");
|
||||
std::fprintf(stderr, "extracting (ids: %s)\n", IdMap::name(ids).c_str());
|
||||
int rc = UserNsJob::run(ids, [&] {
|
||||
std::vector<char*> a;
|
||||
for (auto& s : tar) a.push_back(s.data());
|
||||
a.push_back(nullptr);
|
||||
::execvp(a[0], a.data());
|
||||
return 127;
|
||||
});
|
||||
if (rc != 0) throw std::runtime_error("tar failed (exit " + std::to_string(rc) + ")");
|
||||
std::size_t skipped = 0;
|
||||
std::uint64_t bytes = fsutil::diskUsage(root, skipped);
|
||||
ImageInfo info;
|
||||
info.name = src.name;
|
||||
info.source = src.location;
|
||||
info.ids = ids;
|
||||
info.created = std::time(nullptr);
|
||||
info.bytes = bytes;
|
||||
fsutil::writeFile(tmp / "meta", "source=" + src.location + "\nsha256=" + src.sha256 + "\nids=" + IdMap::name(ids) +
|
||||
"\ncreated=" + std::to_string(info.created) + "\nbytes=" + std::to_string(bytes) + "\n");
|
||||
if (!src.isFile) std::remove(archive.c_str());
|
||||
if (::rename(tmp.c_str(), dir(src.name).c_str()) != 0) throw SysError("install image " + src.name);
|
||||
return info;
|
||||
} catch (...) {
|
||||
cleanup();
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
std::string ImageStore::remove(const std::string& name, bool confirmed, const std::vector<std::string>& usedBy) {
|
||||
validateName(name);
|
||||
if (!has(name)) throw std::runtime_error("no such image '" + name + "'");
|
||||
if (!usedBy.empty()) {
|
||||
std::string list;
|
||||
for (const auto& u : usedBy) list += (list.empty() ? "" : ", ") + u;
|
||||
throw std::runtime_error("image '" + name + "' is used by: " + list);
|
||||
}
|
||||
if (!confirmed) return "dry run: would delete image " + name + ". Add --yes to confirm.";
|
||||
IdMode ids = this->ids(name);
|
||||
fs::path d = dir(name);
|
||||
int rc = UserNsJob::run(ids, [&] { fsutil::removeTree(d); return 0; });
|
||||
if (rc != 0) throw std::runtime_error("could not remove image " + name);
|
||||
return "deleted image " + name;
|
||||
}
|
||||
|
||||
} // namespace doos
|
||||
@@ -0,0 +1,63 @@
|
||||
#pragma once
|
||||
#include <cstdint>
|
||||
#include <filesystem>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "idmap.hpp"
|
||||
|
||||
namespace doos {
|
||||
|
||||
// Where an image comes from, after resolving a catalog name, URL or file.
|
||||
struct ImageSource {
|
||||
std::string name; // store name
|
||||
std::string location; // https URL or local file path
|
||||
std::string sha256; // expected, empty only for local files
|
||||
bool isFile = false;
|
||||
// Optional detached signature over the downloaded file; the key is fetched and must match the pinned fingerprint.
|
||||
std::string sigUrl;
|
||||
std::string keyUrl;
|
||||
std::string keyFpr;
|
||||
};
|
||||
|
||||
struct ImageInfo {
|
||||
std::string name;
|
||||
std::string source;
|
||||
IdMode ids = IdMode::Single;
|
||||
long created = 0;
|
||||
std::uint64_t bytes = 0;
|
||||
};
|
||||
|
||||
// Built-in sources: alpine, ubuntu (latest LTS), ubuntu-VERSION, debian, debian-RELEASE. Resolved from the
|
||||
// publishers over https; Alpine and Ubuntu are also checked against their pinned release signing keys.
|
||||
class Catalog {
|
||||
public:
|
||||
static std::vector<std::string> names();
|
||||
static bool knows(const std::string& imageName);
|
||||
static ImageSource resolve(const std::string& imageName);
|
||||
};
|
||||
|
||||
class ImageStore {
|
||||
public:
|
||||
explicit ImageStore(std::filesystem::path root);
|
||||
|
||||
static std::string normalize(const std::string& name); // "ubuntu:24.04" -> "ubuntu-24.04"
|
||||
static void validateName(const std::string& name);
|
||||
|
||||
bool has(const std::string& name) const;
|
||||
std::filesystem::path rootfs(const std::string& name) const;
|
||||
IdMode ids(const std::string& name) const;
|
||||
std::vector<ImageInfo> list() const;
|
||||
|
||||
// Downloads (catalog/URL) or reads (file), verifies the sha256, extracts under the id map.
|
||||
ImageInfo add(const ImageSource& src, IdMode ids);
|
||||
// Dry run unless confirmed. `usedBy` lists containers referencing the image; non-empty refuses.
|
||||
std::string remove(const std::string& name, bool confirmed, const std::vector<std::string>& usedBy);
|
||||
|
||||
private:
|
||||
std::filesystem::path dir(const std::string& name) const { return root_ / name; }
|
||||
std::filesystem::path root_;
|
||||
};
|
||||
|
||||
} // namespace doos
|
||||
+50
-2
@@ -18,14 +18,16 @@ namespace doos {
|
||||
|
||||
NetMode Network::parse(const std::string& s) {
|
||||
if (s == "slirp") return NetMode::Slirp;
|
||||
if (s == "pasta") return NetMode::Pasta;
|
||||
if (s == "host") return NetMode::Host;
|
||||
if (s == "none") return NetMode::None;
|
||||
throw std::runtime_error("unknown network mode '" + s + "' (use slirp, host or none)");
|
||||
throw std::runtime_error("unknown network mode '" + s + "' (use slirp, pasta, host or none)");
|
||||
}
|
||||
|
||||
std::string Network::name(NetMode m) {
|
||||
switch (m) {
|
||||
case NetMode::Slirp: return "slirp";
|
||||
case NetMode::Pasta: return "pasta";
|
||||
case NetMode::Host: return "host";
|
||||
case NetMode::None: return "none";
|
||||
}
|
||||
@@ -36,8 +38,10 @@ std::string Network::resolvConf(NetMode m) {
|
||||
switch (m) {
|
||||
case NetMode::Slirp: return "nameserver " + std::string(spec::kSlirpDns) + "\n";
|
||||
case NetMode::None: return "";
|
||||
case NetMode::Pasta:
|
||||
case NetMode::Host: {
|
||||
// Host netns is shared, so the host resolver config is valid. Stub on 127.0.0.53 works too.
|
||||
// pasta forwards the namespace's loopback to the host and host mode shares it, so the host resolver
|
||||
// config is valid as is, including the 127.0.0.53 stub.
|
||||
std::ifstream in("/etc/resolv.conf");
|
||||
return std::string(std::istreambuf_iterator<char>(in), {});
|
||||
}
|
||||
@@ -84,3 +88,47 @@ void SlirpSession::stop() {
|
||||
}
|
||||
|
||||
} // namespace doos
|
||||
|
||||
namespace doos {
|
||||
|
||||
void PastaSession::start(pid_t targetPid) {
|
||||
pid_ = ::fork();
|
||||
if (pid_ < 0) throw SysError("fork pasta");
|
||||
if (pid_ == 0) {
|
||||
::prctl(PR_SET_PDEATHSIG, SIGKILL);
|
||||
int nul = ::open("/dev/null", O_WRONLY);
|
||||
if (nul >= 0) { ::dup2(nul, 1); ::dup2(nul, 2); }
|
||||
std::string target = std::to_string(targetPid);
|
||||
::execlp("pasta", "pasta", "--foreground", "--quiet", "--config-net", target.c_str(), static_cast<char*>(nullptr));
|
||||
::_exit(127);
|
||||
}
|
||||
// pasta has no ready fd: wait until the target namespace has an interface besides lo.
|
||||
std::string dev = "/proc/" + std::to_string(targetPid) + "/net/dev";
|
||||
for (int waited = 0; waited < spec::kNetReadyTimeoutMs; waited += 20) {
|
||||
int st = 0;
|
||||
if (::waitpid(pid_, &st, WNOHANG) == pid_) {
|
||||
pid_ = -1;
|
||||
throw std::runtime_error("pasta exited early (is it installed and permitted?)");
|
||||
}
|
||||
std::ifstream in(dev);
|
||||
int lines = 0;
|
||||
for (std::string line; std::getline(in, line);) ++lines;
|
||||
if (lines > 3) return; // two header lines + lo + at least one more interface
|
||||
::poll(nullptr, 0, 20);
|
||||
}
|
||||
stop();
|
||||
throw std::runtime_error("pasta not ready in time");
|
||||
}
|
||||
|
||||
void PastaSession::stop() {
|
||||
if (pid_ <= 0) return;
|
||||
// pasta drops to another uid after start, so we cannot signal it, and it leaves by itself shortly after its
|
||||
// namespace is gone. Reap it if it is already done; otherwise let init adopt it instead of blocking the exit.
|
||||
for (int waited = 0; waited < 20; waited += 5) {
|
||||
if (::waitpid(pid_, nullptr, WNOHANG) == pid_) break;
|
||||
::poll(nullptr, 0, 5);
|
||||
}
|
||||
pid_ = -1;
|
||||
}
|
||||
|
||||
} // namespace doos
|
||||
|
||||
+20
-1
@@ -5,7 +5,12 @@
|
||||
|
||||
namespace doos {
|
||||
|
||||
enum class NetMode { Slirp, Host, None };
|
||||
// slirp: private netns, outbound only, host loopback blocked (default).
|
||||
// pasta: private netns that behaves like the host network (host loopback, bound ports forwarded), but with
|
||||
// its own abstract unix sockets.
|
||||
// host: the host's network namespace itself (shares abstract sockets), explicit opt-in.
|
||||
// none: loopback only.
|
||||
enum class NetMode { Slirp, Pasta, Host, None };
|
||||
|
||||
class Network {
|
||||
public:
|
||||
@@ -17,6 +22,20 @@ public:
|
||||
static std::string resolvConf(NetMode m);
|
||||
};
|
||||
|
||||
// Owns a pasta process attached to a container's network namespace.
|
||||
class PastaSession {
|
||||
public:
|
||||
PastaSession() = default;
|
||||
PastaSession(const PastaSession&) = delete;
|
||||
PastaSession& operator=(const PastaSession&) = delete;
|
||||
~PastaSession() { stop(); }
|
||||
void start(pid_t targetPid);
|
||||
void stop();
|
||||
|
||||
private:
|
||||
pid_t pid_ = -1;
|
||||
};
|
||||
|
||||
// Owns a slirp4netns process attached to a container's network namespace.
|
||||
class SlirpSession {
|
||||
public:
|
||||
|
||||
+203
-22
@@ -15,7 +15,15 @@
|
||||
#include <unistd.h>
|
||||
|
||||
#include <cstring>
|
||||
#include <fstream>
|
||||
|
||||
#include <poll.h>
|
||||
#include <algorithm>
|
||||
|
||||
#include <chrono>
|
||||
|
||||
#include "cgroup.hpp"
|
||||
#include "hardening.hpp"
|
||||
#include "spec.hpp"
|
||||
#include "util.hpp"
|
||||
|
||||
@@ -26,8 +34,10 @@ namespace doos {
|
||||
namespace {
|
||||
|
||||
volatile sig_atomic_t g_forwardTo = 0;
|
||||
volatile sig_atomic_t g_terminating = 0;
|
||||
|
||||
void forwardSignal(int sig) {
|
||||
g_terminating = 1;
|
||||
if (g_forwardTo > 0) ::kill(g_forwardTo, sig);
|
||||
}
|
||||
|
||||
@@ -95,17 +105,24 @@ void bindHostDevice(const fs::path& root, const std::string& dev) {
|
||||
doMount(("/dev/" + dev).c_str(), target, nullptr, MS_BIND);
|
||||
}
|
||||
|
||||
void setupRoot(const Container& c) {
|
||||
fs::path root = c.rootfs();
|
||||
doMount(nullptr, "/", nullptr, MS_REC | MS_PRIVATE);
|
||||
doMount(root.c_str(), root.string(), nullptr, MS_BIND | MS_REC);
|
||||
// apt drops to its _apt user for downloads; with one mappable id that user does not exist.
|
||||
void writeAptSandboxOverride(const Container& c, const fs::path& lowerEtc) {
|
||||
std::error_code ec;
|
||||
if (c.ids() != IdMode::Single || !fs::is_directory(lowerEtc / "apt", ec)) return;
|
||||
fs::path dir = c.etcDir() / "apt/apt.conf.d";
|
||||
fsutil::mkdirs(dir);
|
||||
fsutil::writeFile(dir / "99doos-sandbox", "APT::Sandbox::User \"root\";\n");
|
||||
}
|
||||
|
||||
// resolv.conf lives in the /etc upperdir so it matches the network mode on every start.
|
||||
fs::path resolv = c.layer("/etc") / "upper" / "resolv.conf";
|
||||
void writeResolvConf(const Container& c) {
|
||||
fs::path resolv = c.etcDir() / "resolv.conf";
|
||||
fsutil::mkdirs(resolv.parent_path());
|
||||
std::error_code ec;
|
||||
fs::remove(resolv, ec);
|
||||
fsutil::writeFile(resolv, Network::resolvConf(c.net()));
|
||||
}
|
||||
|
||||
void mountHostLayers(const Container& c, const fs::path& root) {
|
||||
for (auto d : spec::kOverlayDirs) {
|
||||
std::error_code e;
|
||||
if (fs::is_directory(std::string(d), e)) mountOverlay(c, root, d);
|
||||
@@ -125,6 +142,32 @@ void setupRoot(const Container& c) {
|
||||
}
|
||||
}
|
||||
for (auto d : spec::kPrivateDirs) doMount(c.privateDir(std::string(d)).c_str(), sub(root, d), nullptr, MS_BIND);
|
||||
// overlays that live inside a private dir must be mounted after it
|
||||
for (auto d : spec::kVarOverlayDirs) {
|
||||
std::error_code e;
|
||||
if (fs::is_directory(std::string(d), e)) mountOverlay(c, root, d);
|
||||
}
|
||||
}
|
||||
|
||||
void mountImageRoot(const Container& c, const fs::path& root, const fs::path& imageRoot) {
|
||||
std::string opts = "lowerdir=" + imageRoot.string() + ",upperdir=" + (c.layer("/root") / "upper").string() +
|
||||
",workdir=" + (c.layer("/root") / "work").string() + ",userxattr";
|
||||
doMount("overlay", root.string(), "overlay", MS_NODEV | MS_NOSUID, opts);
|
||||
for (auto d : spec::kEmptyDirs) fsutil::mkdirs(sub(root, d));
|
||||
}
|
||||
|
||||
void setupRoot(const Container& c, const fs::path& imageRoot) {
|
||||
fs::path root = c.rootfs();
|
||||
doMount(nullptr, "/", nullptr, MS_REC | MS_PRIVATE);
|
||||
writeResolvConf(c);
|
||||
if (c.imageMode()) {
|
||||
writeAptSandboxOverride(c, imageRoot / "etc");
|
||||
mountImageRoot(c, root, imageRoot);
|
||||
} else {
|
||||
writeAptSandboxOverride(c, "/etc");
|
||||
doMount(root.c_str(), root.string(), nullptr, MS_BIND | MS_REC);
|
||||
mountHostLayers(c, root);
|
||||
}
|
||||
|
||||
// tmpfs mounts
|
||||
doMount("tmpfs", sub(root, "/tmp"), "tmpfs", MS_NOSUID | MS_NODEV, "mode=1777,size=" + std::string(spec::kTmpSize));
|
||||
@@ -160,12 +203,16 @@ void applyLimitsAndEnv(const Container& c) {
|
||||
if (::prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) != 0) throw SysError("no_new_privs");
|
||||
rlimit core{0, 0};
|
||||
::setrlimit(RLIMIT_CORE, &core);
|
||||
// No single file can outgrow the size cap, which bounds the overshoot between two usage scans.
|
||||
rlimit fsize{c.sizeCap(), c.sizeCap()};
|
||||
::setrlimit(RLIMIT_FSIZE, &fsize);
|
||||
rlimit nofile;
|
||||
if (::getrlimit(RLIMIT_NOFILE, &nofile) == 0) {
|
||||
if (nofile.rlim_max > spec::kMaxOpenFiles) nofile.rlim_max = spec::kMaxOpenFiles;
|
||||
if (nofile.rlim_cur > nofile.rlim_max) nofile.rlim_cur = nofile.rlim_max;
|
||||
::setrlimit(RLIMIT_NOFILE, &nofile);
|
||||
}
|
||||
if (!c.relaxed()) Hardening::apply();
|
||||
const char* term = std::getenv("TERM");
|
||||
std::string termv = term && *term ? term : "xterm";
|
||||
const char* lang = std::getenv("LANG");
|
||||
@@ -179,7 +226,9 @@ void applyLimitsAndEnv(const Container& c) {
|
||||
::setenv("HOSTNAME", c.name().c_str(), 1);
|
||||
::setenv("container", "doos", 1);
|
||||
::setenv("PS1", ("(doos:" + c.name() + ") \\u@\\h:\\w\\$ ").c_str(), 1);
|
||||
if (!langv.empty()) ::setenv("LANG", langv.c_str(), 1);
|
||||
// The host's locale usually does not exist inside an image; C.UTF-8 is built into glibc and musl.
|
||||
if (c.imageMode()) ::setenv("LANG", "C.UTF-8", 1);
|
||||
else if (!langv.empty()) ::setenv("LANG", langv.c_str(), 1);
|
||||
}
|
||||
|
||||
[[noreturn]] void execCommand(std::vector<std::string> cmd) {
|
||||
@@ -202,6 +251,7 @@ struct ChildCtx {
|
||||
const Container* container;
|
||||
const std::vector<std::string>* command;
|
||||
int syncFd;
|
||||
fs::path imageRoot;
|
||||
int lockFd; // inherited copy must be closed or the flock outlives unlock()
|
||||
};
|
||||
|
||||
@@ -216,7 +266,7 @@ int containerInit(void* arg) {
|
||||
::close(ctx->syncFd);
|
||||
|
||||
const Container& c = *ctx->container;
|
||||
setupRoot(c);
|
||||
setupRoot(c, ctx->imageRoot);
|
||||
if (::sethostname(c.name().c_str(), c.name().size()) != 0) throw SysError("sethostname");
|
||||
if (Network::isolated(c.net())) bringLoopbackUp();
|
||||
applyLimitsAndEnv(c);
|
||||
@@ -244,23 +294,60 @@ int containerInit(void* arg) {
|
||||
::_exit(code);
|
||||
} catch (const std::exception& e) {
|
||||
::dprintf(2, "doos: %s\n", e.what());
|
||||
auto* se = dynamic_cast<const SysError*>(&e);
|
||||
std::ifstream restrictUserns("/proc/sys/kernel/apparmor_restrict_unprivileged_userns");
|
||||
int restricted = 0;
|
||||
if (restrictUserns) restrictUserns >> restricted;
|
||||
if (se && (se->code() == EPERM || se->code() == EACCES) && restricted == 1) {
|
||||
::dprintf(2, "doos: AppArmor restricts user namespaces for this binary. Install a profile once with:\n %s\n",
|
||||
Sandbox::apparmorInstallHint().c_str());
|
||||
}
|
||||
::_exit(125);
|
||||
}
|
||||
}
|
||||
|
||||
void writeMap(pid_t pid, const char* file, const std::string& content) {
|
||||
std::string path = "/proc/" + std::to_string(pid) + "/" + file;
|
||||
int fd = ::open(path.c_str(), O_WRONLY | O_CLOEXEC);
|
||||
if (fd < 0) throw SysError("open " + path);
|
||||
Fd guard(fd);
|
||||
if (::write(fd, content.data(), content.size()) != static_cast<ssize_t>(content.size())) {
|
||||
throw SysError("write " + path +
|
||||
" (user namespaces blocked? see apparmor/doos and docs/RESEARCH.md)");
|
||||
} // namespace
|
||||
|
||||
namespace {
|
||||
fs::path selfPath() {
|
||||
std::error_code ec;
|
||||
fs::path self = fs::read_symlink("/proc/self/exe", ec);
|
||||
if (ec) throw std::runtime_error("cannot resolve the path of this binary: " + ec.message());
|
||||
std::string path = self.string();
|
||||
if (path.find_first_of("{}[]*?\"\n,") != std::string::npos) {
|
||||
throw std::runtime_error("binary path contains characters AppArmor treats as globs: " + path);
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
// One profile per binary path, so installing doos in a second place does not replace the first profile.
|
||||
std::string profileName(const fs::path& self) {
|
||||
std::string n = "doos" + self.string();
|
||||
for (auto& c : n) {
|
||||
if (c == '/') c = '-';
|
||||
}
|
||||
return n;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
std::string Sandbox::apparmorProfile() {
|
||||
fs::path self = selfPath();
|
||||
return "# Lets doos use unprivileged user namespaces where kernel.apparmor_restrict_unprivileged_userns=1.\n"
|
||||
"# Generated by: doos apparmor (profile is for exactly this path)\n"
|
||||
"abi <abi/5.0>,\n"
|
||||
"include <tunables/global>\n"
|
||||
"\n"
|
||||
"profile " + profileName(self) + " " + self.string() + " flags=(unconfined) {\n"
|
||||
" userns,\n"
|
||||
" @{exec_path} mr,\n"
|
||||
"}\n";
|
||||
}
|
||||
|
||||
std::string Sandbox::apparmorInstallHint() {
|
||||
std::string file = "/etc/apparmor.d/" + profileName(selfPath());
|
||||
return "doos apparmor | sudo tee " + file + " >/dev/null && sudo apparmor_parser -r " + file;
|
||||
}
|
||||
|
||||
int Sandbox::run() {
|
||||
if (command_.size() > spec::kMaxCommandArgs) throw std::runtime_error("too many command arguments");
|
||||
// Decide start-or-join under the lock so concurrent attaches agree.
|
||||
@@ -273,6 +360,14 @@ int Sandbox::run() {
|
||||
|
||||
int Sandbox::start(FileLock& lock) {
|
||||
NetMode net = container_.net();
|
||||
fs::path imageRoot;
|
||||
if (container_.imageMode()) {
|
||||
Store store = Store::fromEnvironment();
|
||||
imageRoot = store.images().rootfs(container_.image());
|
||||
if (!fs::is_directory(imageRoot)) {
|
||||
throw std::runtime_error("image '" + container_.image() + "' is missing; add it again with: doos image add");
|
||||
}
|
||||
}
|
||||
int flags = CLONE_NEWUSER | CLONE_NEWNS | CLONE_NEWPID | CLONE_NEWUTS | CLONE_NEWIPC | SIGCHLD;
|
||||
if (Network::isolated(net)) flags |= CLONE_NEWNET;
|
||||
|
||||
@@ -281,7 +376,7 @@ int Sandbox::start(FileLock& lock) {
|
||||
Fd syncRead(sync[0]), syncWrite(sync[1]);
|
||||
|
||||
std::vector<char> stack(1 << 20);
|
||||
ChildCtx ctx{&container_, &command_, syncRead.get(), lock.fd()};
|
||||
ChildCtx ctx{&container_, &command_, syncRead.get(), imageRoot, lock.fd()};
|
||||
pid_t child = ::clone(containerInit, stack.data() + stack.size(), flags, &ctx);
|
||||
if (child < 0) throw SysError("clone (user namespaces available?)");
|
||||
|
||||
@@ -297,12 +392,23 @@ int Sandbox::start(FileLock& lock) {
|
||||
}
|
||||
} reaper{child};
|
||||
|
||||
writeMap(child, "setgroups", "deny");
|
||||
writeMap(child, "uid_map", "0 " + std::to_string(::geteuid()) + " 1\n");
|
||||
writeMap(child, "gid_map", "0 " + std::to_string(::getegid()) + " 1\n");
|
||||
IdMap::apply(child, container_.ids());
|
||||
|
||||
SlirpSession slirp;
|
||||
PastaSession pasta;
|
||||
if (net == NetMode::Slirp) slirp.start(child);
|
||||
if (net == NetMode::Pasta) pasta.start(child);
|
||||
|
||||
Limits limits{container_.memoryCap(), container_.pidsCap(), container_.milliCpus()};
|
||||
std::string cgroupWhy, notEnforced;
|
||||
std::unique_ptr<Cgroup> cgroup = Cgroup::create(container_.name(), limits, cgroupWhy, notEnforced);
|
||||
if (cgroup) {
|
||||
cgroup->addMain(child);
|
||||
if (!notEnforced.empty()) std::fprintf(stderr, "doos: limits not enforced (controller missing): %s\n", notEnforced.c_str());
|
||||
} else {
|
||||
std::fprintf(stderr, "doos: memory, pids and cpu limits are not enforced and the size cap only warns: %s\n",
|
||||
cgroupWhy.c_str());
|
||||
}
|
||||
|
||||
container_.recordPid(child);
|
||||
struct PidGuard {
|
||||
@@ -324,12 +430,82 @@ int Sandbox::start(FileLock& lock) {
|
||||
::sigaction(SIGQUIT, &ign, nullptr);
|
||||
installForwarding(child, {SIGTERM, SIGHUP});
|
||||
|
||||
int status = watch(child, cgroup.get());
|
||||
reaper.armed = false;
|
||||
if (cgroup && cgroup->oomKills() > 0) {
|
||||
std::fprintf(stderr, "doos: '%s' was killed by its memory limit (%s); raise it with --memory\n",
|
||||
container_.name().c_str(), formatSize(container_.memoryCap()).c_str());
|
||||
}
|
||||
if (cgroup && cgroup->pidFailures() > 0) {
|
||||
std::fprintf(stderr, "doos: '%s' hit its process limit (%u) %llu times; raise it with --pids\n",
|
||||
container_.name().c_str(), container_.pidsCap(),
|
||||
static_cast<unsigned long long>(cgroup->pidFailures()));
|
||||
}
|
||||
return exitCodeOf(status);
|
||||
}
|
||||
|
||||
// Waits for the container's init while enforcing the size cap: warn at kSizeWarnPercent, freeze at the cap,
|
||||
// thaw again once usage drops below kSizeThawPercent. Nothing is ever deleted by this loop.
|
||||
int Sandbox::watch(pid_t child, Cgroup* cgroup) {
|
||||
using Clock = std::chrono::steady_clock;
|
||||
long fd = ::syscall(SYS_pidfd_open, child, 0);
|
||||
Fd pidfd(static_cast<int>(fd));
|
||||
const std::uint64_t cap = container_.sizeCap();
|
||||
bool warned = false;
|
||||
auto nextScan = Clock::now() + std::chrono::milliseconds(spec::kSizeCheckMinIntervalMs);
|
||||
for (;;) {
|
||||
auto left = std::chrono::duration_cast<std::chrono::milliseconds>(nextScan - Clock::now()).count();
|
||||
int timeout = static_cast<int>(std::max<long>(left, 50));
|
||||
if (pidfd.valid()) {
|
||||
pollfd p{pidfd.get(), POLLIN, 0};
|
||||
int r = ::poll(&p, 1, timeout);
|
||||
if (r > 0) break;
|
||||
if (r < 0 && errno != EINTR) break;
|
||||
} else {
|
||||
int st = 0;
|
||||
if (::waitpid(child, &st, WNOHANG) == child) return st;
|
||||
::poll(nullptr, 0, std::min(timeout, 1000));
|
||||
}
|
||||
if (g_terminating && cgroup && cgroup->frozen()) cgroup->freeze(false); // a frozen init cannot see SIGTERM
|
||||
if (Clock::now() < nextScan) continue;
|
||||
|
||||
auto began = Clock::now();
|
||||
std::size_t skipped = 0;
|
||||
std::uint64_t used = container_.usedBytes(skipped);
|
||||
// The scan may use at most 1/kSizeScanDutyFactor of the time: big trees are scanned less often.
|
||||
auto took = Clock::now() - began;
|
||||
nextScan = Clock::now() + std::max(std::chrono::milliseconds(spec::kSizeCheckMinIntervalMs),
|
||||
std::chrono::duration_cast<std::chrono::milliseconds>(took) * spec::kSizeScanDutyFactor);
|
||||
const char* name = container_.name().c_str();
|
||||
if (used >= cap) {
|
||||
if (cgroup && !cgroup->frozen()) {
|
||||
cgroup->freeze(true);
|
||||
std::fprintf(stderr, "doos: '%s' reached its size cap (%s of %s): FROZEN. Free space with 'doos attach %s -- ...'; it thaws below %d%%.\n",
|
||||
name, formatSize(used).c_str(), formatSize(cap).c_str(), name, spec::kSizeThawPercent);
|
||||
} else if (!cgroup && !warned) {
|
||||
std::fprintf(stderr, "doos: '%s' is over its size cap (%s of %s), cannot freeze\n", name,
|
||||
formatSize(used).c_str(), formatSize(cap).c_str());
|
||||
warned = true;
|
||||
}
|
||||
} else if (used * 100 >= cap * spec::kSizeWarnPercent) {
|
||||
if (!warned) {
|
||||
std::fprintf(stderr, "doos: '%s' uses %s of its %s size cap\n", name, formatSize(used).c_str(),
|
||||
formatSize(cap).c_str());
|
||||
warned = true;
|
||||
}
|
||||
} else {
|
||||
warned = false;
|
||||
}
|
||||
if (cgroup && cgroup->frozen() && used * 100 < cap * spec::kSizeThawPercent) {
|
||||
cgroup->freeze(false);
|
||||
std::fprintf(stderr, "doos: '%s' thawed (%s used)\n", name, formatSize(used).c_str());
|
||||
}
|
||||
}
|
||||
int status = 0;
|
||||
while (::waitpid(child, &status, 0) < 0) {
|
||||
if (errno != EINTR) throw SysError("waitpid");
|
||||
}
|
||||
reaper.armed = false;
|
||||
return exitCodeOf(status);
|
||||
return status;
|
||||
}
|
||||
|
||||
int Sandbox::join(pid_t initPid, FileLock& lock) {
|
||||
@@ -347,10 +523,15 @@ int Sandbox::join(pid_t initPid, FileLock& lock) {
|
||||
if (!rootFd.valid()) throw SysError("open container root");
|
||||
|
||||
lock.unlock();
|
||||
// Later attaches live in the `shell` leaf: same limits as the container, but never frozen.
|
||||
std::unique_ptr<Cgroup> cgroup = Cgroup::open(container_.name());
|
||||
pid_t child = ::fork();
|
||||
if (child < 0) throw SysError("fork");
|
||||
if (child == 0) {
|
||||
try {
|
||||
if (cgroup) {
|
||||
try { cgroup->addShell(::getpid()); } catch (const std::exception&) { /* limits only; keep going */ }
|
||||
}
|
||||
for (int i = 0; i < 6; ++i) {
|
||||
if (fds[i].valid() && ::setns(fds[i].get(), kinds[i]) != 0) {
|
||||
throw SysError(std::string("setns ") + names[i]);
|
||||
|
||||
@@ -7,6 +7,8 @@
|
||||
|
||||
namespace doos {
|
||||
|
||||
class Cgroup;
|
||||
|
||||
// Runs a command inside a container: starts the namespaces if the container is
|
||||
// stopped, joins them if it is already running. Returns the command's exit code.
|
||||
class Sandbox {
|
||||
@@ -16,8 +18,14 @@ public:
|
||||
|
||||
int run();
|
||||
|
||||
// AppArmor profile (text) that lets this very binary use user namespaces on Ubuntu 24.04+.
|
||||
static std::string apparmorProfile();
|
||||
// Shell command that installs it; shown when a start fails because of AppArmor.
|
||||
static std::string apparmorInstallHint();
|
||||
|
||||
private:
|
||||
int start(FileLock& lock);
|
||||
int watch(pid_t child, Cgroup* cgroup);
|
||||
int join(pid_t initPid, FileLock& lock);
|
||||
|
||||
Container container_;
|
||||
|
||||
+104
-2
@@ -2,6 +2,7 @@
|
||||
#pragma once
|
||||
#include <array>
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <string_view>
|
||||
|
||||
namespace spec {
|
||||
@@ -18,6 +19,57 @@ constexpr std::string_view kTmpSize = "256m";
|
||||
constexpr std::string_view kRunSize = "64m";
|
||||
constexpr std::string_view kShmSize = "64m";
|
||||
|
||||
// Writable-layer size cap per container (--size). Enforced by a watchdog that freezes the container.
|
||||
constexpr std::uint64_t kDefaultSizeBytes = 4ULL << 30;
|
||||
constexpr std::uint64_t kMinSizeBytes = 64ULL << 20;
|
||||
constexpr std::uint64_t kMaxSizeBytes = 1ULL << 40;
|
||||
constexpr int kSizeWarnPercent = 80;
|
||||
constexpr int kSizeThawPercent = 95; // thaw again below this share of the cap
|
||||
constexpr int kSizeCheckMinIntervalMs = 2000; // floor between usage scans, never faster
|
||||
constexpr int kSizeScanDutyFactor = 10; // next scan waits at least 10x the last scan's duration
|
||||
constexpr std::size_t kMaxScanEntries = 5000000; // usage scan stops (and reports) beyond this
|
||||
constexpr std::string_view kCgroupDir = "doos"; // under the delegated user@UID.service cgroup
|
||||
|
||||
// Resource limits per container (--memory, --pids, --cpus). Enforced by cgroup v2; cpus 0 means "all cores".
|
||||
constexpr std::uint64_t kDefaultMemoryBytes = 8ULL << 30;
|
||||
constexpr std::uint64_t kMinMemoryBytes = 64ULL << 20;
|
||||
constexpr std::uint64_t kMaxMemoryBytes = 1ULL << 40;
|
||||
constexpr std::uint32_t kDefaultPids = 4096;
|
||||
constexpr std::uint32_t kMinPids = 16;
|
||||
constexpr std::uint32_t kMaxPids = 4194304;
|
||||
constexpr std::uint32_t kMaxMilliCpus = 1024000; // 1024 cpus
|
||||
constexpr std::uint32_t kMinMilliCpus = 100; // 0.1 cpu
|
||||
constexpr int kCpuPeriodUs = 100000;
|
||||
|
||||
// Seeding limits
|
||||
constexpr std::size_t kMaxSkeletonDirs = 200000;
|
||||
constexpr std::size_t kMaxEtcCopyFiles = 50000;
|
||||
constexpr std::uint64_t kMaxEtcCopyBytes = 256ULL << 20;
|
||||
|
||||
// Id mapping: container ids 0..kMaxIdRange. 0 maps to the invoking user, the rest to the subuid range.
|
||||
constexpr std::uint32_t kMaxIdRange = 65536;
|
||||
constexpr std::string_view kNewUidMap = "newuidmap";
|
||||
constexpr std::string_view kNewGidMap = "newgidmap";
|
||||
|
||||
// Images
|
||||
constexpr std::size_t kMaxImages = 16;
|
||||
constexpr std::uint64_t kMaxImageDownloadBytes = 2ULL << 30;
|
||||
constexpr int kDownloadTimeoutSec = 900;
|
||||
constexpr std::size_t kMaxCaptureBytes = 1 << 20; // listing / checksum fetches
|
||||
constexpr std::string_view kAlpineBase = "https://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/";
|
||||
constexpr std::string_view kUbuntuBase = "https://cdimage.ubuntu.com/ubuntu-base/releases/";
|
||||
constexpr std::string_view kUbuntuDefaultVersion = "26.04";
|
||||
constexpr std::string_view kDebianBase = "https://raw.githubusercontent.com/debuerreotype/docker-debian-artifacts/";
|
||||
constexpr std::string_view kDebianDefaultRelease = "trixie";
|
||||
|
||||
// Release signing keys, pinned by fingerprint. Keys are fetched from the publisher and rejected unless the
|
||||
// fingerprint matches. Sources of the fingerprints: alpinelinux.org/downloads, ubuntu.com verify tutorial.
|
||||
constexpr std::string_view kAlpineKeyUrl = "https://alpinelinux.org/keys/ncopa.asc";
|
||||
constexpr std::string_view kAlpineKeyFpr = "0482D84022F52DF1C4E7CD43293ACD0907D9495A";
|
||||
constexpr std::string_view kUbuntuKeyUrl =
|
||||
"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x843938DF228D22F7B3742BC0D94AA3F0EFE21092";
|
||||
constexpr std::string_view kUbuntuKeyFpr = "843938DF228D22F7B3742BC0D94AA3F0EFE21092";
|
||||
|
||||
// Networking
|
||||
constexpr std::string_view kSlirpBinary = "slirp4netns";
|
||||
constexpr std::string_view kSlirpDns = "10.0.2.3";
|
||||
@@ -29,14 +81,64 @@ constexpr std::string_view kContainerPath =
|
||||
constexpr std::string_view kHome = "/root";
|
||||
|
||||
// Host directories exposed as overlay lowerdirs. Writes land in the per-container upperdir.
|
||||
constexpr std::array<std::string_view, 3> kOverlayDirs = {"/usr", "/etc", "/opt"};
|
||||
constexpr std::array<std::string_view, 2> kOverlayDirs = {"/usr", "/opt"};
|
||||
|
||||
// Host directories copied once into a private per-container dir (owned by the container root, so
|
||||
// chown --reference and in-place edits work). Unreadable entries are skipped and counted.
|
||||
constexpr std::array<std::string_view, 1> kCopiedDirs = {"/etc"};
|
||||
|
||||
// Host mode /etc under --hardening default: only these entries are copied (relaxed copies everything readable).
|
||||
// Entries that may hold credentials are skipped even inside allowed directories.
|
||||
constexpr std::array<std::string_view, 53> kEtcAllow = {
|
||||
"passwd", "group", "nsswitch.conf", "hosts", "hostname", "host.conf", "localtime", "timezone", "machine-id",
|
||||
"ld.so.conf", "ld.so.conf.d", "ld.so.cache", "ssl", "ca-certificates", "ca-certificates.conf", "pki",
|
||||
"alternatives", "apt", "dpkg", "bash.bashrc", "bashrc", "profile", "profile.d", "environment", "inputrc",
|
||||
"os-release", "lsb-release", "issue", "issue.net", "debian_version", "shells", "protocols", "services",
|
||||
"mime.types", "magic", "gai.conf", "locale.gen", "locale.conf", "default", "gitconfig", "bash_completion",
|
||||
"bash_completion.d", "vim", "emacs", "xdg", "fonts", "mailcap", "wgetrc", "ssh/ssh_config",
|
||||
"ssh/ssh_config.d", "perl", "python3", "terminfo"};
|
||||
constexpr std::array<std::string_view, 8> kEtcDenyNames = {"private", "shadow", "gshadow", "sudoers", "sudoers.d",
|
||||
"auth.conf", "auth.conf.d", "ssh_host_*"};
|
||||
|
||||
// Capabilities kept in the bounding set under --hardening default (the Docker default set).
|
||||
constexpr std::array<int, 14> kKeptCaps = {0 /*CHOWN*/, 1 /*DAC_OVERRIDE*/, 3 /*FOWNER*/, 4 /*FSETID*/,
|
||||
5 /*KILL*/, 6 /*SETGID*/, 7 /*SETUID*/, 8 /*SETPCAP*/,
|
||||
10 /*NET_BIND_SERVICE*/, 13 /*NET_RAW*/, 18 /*SYS_CHROOT*/,
|
||||
27 /*MKNOD*/, 29 /*AUDIT_WRITE*/, 31 /*SETFCAP*/};
|
||||
|
||||
// Extra overlay mounted inside the private /var, only when the host has the directory (dpkg database).
|
||||
constexpr std::array<std::string_view, 1> kVarOverlayDirs = {"/var/lib/dpkg"};
|
||||
|
||||
// Lock files that exist on the host owned by an unmapped uid: an empty copy is created in the upper layer so
|
||||
// the container can open them for writing. Each must live under one of the overlay dirs above.
|
||||
constexpr std::array<std::string_view, 3> kUpperLockFiles = {"/var/lib/dpkg/lock", "/var/lib/dpkg/lock-frontend",
|
||||
"/var/lib/dpkg/triggers/Lock"};
|
||||
|
||||
// Directories whose regular files (not subdirs) are copied into the upper layer at creation, so the container
|
||||
// owns them: dpkg hard-links `status` to `status-old`, which the kernel refuses for files owned by nobody.
|
||||
constexpr std::array<std::string_view, 3> kUpperCopyDirs = {"/var/lib/dpkg", "/var/lib/dpkg/triggers",
|
||||
"/var/lib/dpkg/alternatives"};
|
||||
|
||||
struct DirSpec {
|
||||
std::string_view path;
|
||||
unsigned mode;
|
||||
};
|
||||
// Directories created in the private /var (package managers expect them).
|
||||
constexpr std::array<DirSpec, 8> kVarDirs = {{{"lib/apt/lists/partial", 0755},
|
||||
{"lib/dpkg", 0755},
|
||||
{"cache/apt/archives/partial", 0755},
|
||||
{"log/apt", 0755},
|
||||
{"tmp", 01777},
|
||||
{"lib/apt", 0755},
|
||||
{"cache", 0755},
|
||||
{"log", 0755}}};
|
||||
|
||||
// Host top-level entries: recreated as symlinks when the host has them as symlinks (usr-merge),
|
||||
// overlaid like kOverlayDirs when they are real directories, skipped when absent.
|
||||
constexpr std::array<std::string_view, 5> kCompatDirs = {"/bin", "/sbin", "/lib", "/lib32", "/lib64"};
|
||||
|
||||
// Persistent per-container private directories (no host lower layer).
|
||||
constexpr std::array<std::string_view, 2> kPrivateDirs = {"/root", "/var"};
|
||||
constexpr std::array<std::string_view, 3> kPrivateDirs = {"/root", "/var", "/etc"};
|
||||
|
||||
// Empty mountpoints / tmpfs
|
||||
constexpr std::array<std::string_view, 6> kEmptyDirs = {"/mnt", "/srv", "/tmp", "/run", "/dev", "/proc"};
|
||||
|
||||
+263
@@ -4,10 +4,15 @@
|
||||
#include <poll.h>
|
||||
#include <sys/file.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/wait.h>
|
||||
#include <dirent.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <csignal>
|
||||
#include <cstring>
|
||||
#include <cctype>
|
||||
#include <fstream>
|
||||
#include <unordered_set>
|
||||
#include <sstream>
|
||||
|
||||
#include "spec.hpp"
|
||||
@@ -46,8 +51,266 @@ FileLock::FileLock(const fs::path& file, int timeoutMs) {
|
||||
}
|
||||
}
|
||||
|
||||
std::uint64_t parseSize(const std::string& text) {
|
||||
std::size_t i = 0;
|
||||
while (i < text.size() && std::isdigit(static_cast<unsigned char>(text[i]))) ++i;
|
||||
if (i == 0 || i > 15) throw std::runtime_error("invalid size '" + text + "' (examples: 512m, 4g)");
|
||||
std::uint64_t v = std::stoull(text.substr(0, i));
|
||||
std::string unit = text.substr(i);
|
||||
for (auto& c : unit) c = static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
|
||||
if (unit == "b" || unit.empty()) {
|
||||
} else {
|
||||
if (unit.size() == 2 && unit[1] == 'b') unit.pop_back();
|
||||
if (unit == "k") v <<= 10;
|
||||
else if (unit == "m") v <<= 20;
|
||||
else if (unit == "g") v <<= 30;
|
||||
else if (unit == "t") v <<= 40;
|
||||
else throw std::runtime_error("invalid size unit in '" + text + "' (use k, m, g or t)");
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
std::string formatSize(std::uint64_t b) {
|
||||
const char* u[] = {"B", "K", "M", "G", "T"};
|
||||
double v = static_cast<double>(b);
|
||||
int i = 0;
|
||||
while (v >= 1024 && i < 4) { v /= 1024; ++i; }
|
||||
char buf[32];
|
||||
if (i == 0) std::snprintf(buf, sizeof buf, "%lluB", static_cast<unsigned long long>(b));
|
||||
else std::snprintf(buf, sizeof buf, v < 10 ? "%.1f%s" : "%.0f%s", v, u[i]);
|
||||
return buf;
|
||||
}
|
||||
|
||||
std::string base64Decode(const std::string& in) {
|
||||
std::string out;
|
||||
unsigned val = 0;
|
||||
int bits = -8;
|
||||
for (unsigned char c : in) {
|
||||
int d;
|
||||
if (c >= 'A' && c <= 'Z') d = c - 'A';
|
||||
else if (c >= 'a' && c <= 'z') d = c - 'a' + 26;
|
||||
else if (c >= '0' && c <= '9') d = c - '0' + 52;
|
||||
else if (c == '+' || c == '-') d = 62;
|
||||
else if (c == '/' || c == '_') d = 63;
|
||||
else if (c == '=' || std::isspace(c)) continue;
|
||||
else throw std::runtime_error("invalid base64 input");
|
||||
val = (val << 6) | static_cast<unsigned>(d);
|
||||
bits += 6;
|
||||
if (bits >= 0) {
|
||||
out.push_back(static_cast<char>((val >> bits) & 0xFF));
|
||||
bits -= 8;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
ProcResult runProcess(const std::vector<std::string>& argv, bool capture, bool quiet) {
|
||||
int pipefd[2] = {-1, -1};
|
||||
if (capture && ::pipe2(pipefd, O_CLOEXEC) != 0) throw SysError("pipe");
|
||||
Fd readEnd(pipefd[0]), writeEnd(pipefd[1]);
|
||||
pid_t pid = ::fork();
|
||||
if (pid < 0) throw SysError("fork " + argv.at(0));
|
||||
if (pid == 0) {
|
||||
if (capture) ::dup2(writeEnd.get(), 1);
|
||||
if (quiet) {
|
||||
int nul = ::open("/dev/null", O_WRONLY);
|
||||
if (nul >= 0) { ::dup2(nul, 2); if (!capture) ::dup2(nul, 1); }
|
||||
}
|
||||
std::vector<char*> a;
|
||||
for (const auto& x : argv) a.push_back(const_cast<char*>(x.c_str()));
|
||||
a.push_back(nullptr);
|
||||
::execvp(a[0], a.data());
|
||||
::dprintf(2, "doos: cannot run %s: %s\n", a[0], std::strerror(errno));
|
||||
::_exit(127);
|
||||
}
|
||||
writeEnd.reset();
|
||||
ProcResult r;
|
||||
if (capture) {
|
||||
char buf[4096];
|
||||
ssize_t n;
|
||||
while ((n = ::read(readEnd.get(), buf, sizeof buf)) != 0) {
|
||||
if (n < 0) { if (errno == EINTR) continue; break; }
|
||||
if (r.out.size() + static_cast<std::size_t>(n) > spec::kMaxCaptureBytes) {
|
||||
::kill(pid, SIGKILL);
|
||||
::waitpid(pid, nullptr, 0);
|
||||
throw std::runtime_error("output of " + argv.at(0) + " exceeds the capture cap");
|
||||
}
|
||||
r.out.append(buf, static_cast<std::size_t>(n));
|
||||
}
|
||||
}
|
||||
int st = 0;
|
||||
while (::waitpid(pid, &st, 0) < 0) {
|
||||
if (errno != EINTR) throw SysError("waitpid " + argv.at(0));
|
||||
}
|
||||
r.code = WIFEXITED(st) ? WEXITSTATUS(st) : 128 + WTERMSIG(st);
|
||||
return r;
|
||||
}
|
||||
|
||||
namespace fsutil {
|
||||
|
||||
namespace {
|
||||
void mirrorRec(const fs::path& lower, const fs::path& upper, SeedStats& st, std::size_t depth) {
|
||||
if (depth > spec::kMaxTreeDepth) throw std::runtime_error("tree too deep: " + lower.string());
|
||||
std::error_code ec;
|
||||
fs::directory_iterator it(lower, ec), end;
|
||||
if (ec) { ++st.skipped; return; }
|
||||
for (; it != end; it.increment(ec)) {
|
||||
if (ec) { ++st.skipped; return; }
|
||||
struct stat sb;
|
||||
if (::lstat(it->path().c_str(), &sb) != 0 || !S_ISDIR(sb.st_mode)) continue;
|
||||
if (st.made >= spec::kMaxSkeletonDirs) throw std::runtime_error("directory skeleton cap reached");
|
||||
fs::path target = upper / it->path().filename();
|
||||
if (::mkdir(target.c_str(), (sb.st_mode & 07777) | 0700) != 0 && errno != EEXIST) {
|
||||
throw SysError("mkdir " + target.string());
|
||||
}
|
||||
++st.made;
|
||||
mirrorRec(it->path(), target, st, depth + 1);
|
||||
}
|
||||
}
|
||||
|
||||
void copyRec(const fs::path& from, const fs::path& to, SeedStats& st, std::size_t depth, const SkipFn& skip) {
|
||||
if (depth > spec::kMaxTreeDepth) throw std::runtime_error("tree too deep: " + from.string());
|
||||
std::error_code ec;
|
||||
fs::directory_iterator it(from, ec), end;
|
||||
if (ec) { ++st.skipped; return; }
|
||||
for (; it != end; it.increment(ec)) {
|
||||
if (ec) { ++st.skipped; return; }
|
||||
struct stat sb;
|
||||
const fs::path& src = it->path();
|
||||
fs::path dst = to / src.filename();
|
||||
if (::lstat(src.c_str(), &sb) != 0) { ++st.skipped; continue; }
|
||||
if (skip && skip(src)) { ++st.skipped; continue; }
|
||||
if (st.made >= spec::kMaxEtcCopyFiles) throw std::runtime_error("copy file cap reached: " + from.string());
|
||||
if (S_ISDIR(sb.st_mode)) {
|
||||
if (::mkdir(dst.c_str(), (sb.st_mode & 07777) | 0700) != 0 && errno != EEXIST) {
|
||||
throw SysError("mkdir " + dst.string());
|
||||
}
|
||||
++st.made;
|
||||
copyRec(src, dst, st, depth + 1, skip);
|
||||
} else if (S_ISLNK(sb.st_mode)) {
|
||||
std::error_code e2;
|
||||
fs::remove(dst, e2);
|
||||
fs::create_symlink(fs::read_symlink(src), dst);
|
||||
++st.made;
|
||||
} else if (S_ISREG(sb.st_mode)) {
|
||||
if (::access(src.c_str(), R_OK) != 0) { ++st.skipped; continue; }
|
||||
st.bytes += static_cast<std::uint64_t>(sb.st_size);
|
||||
if (st.bytes > spec::kMaxEtcCopyBytes) throw std::runtime_error("copy byte cap reached: " + from.string());
|
||||
std::error_code e2;
|
||||
fs::copy_file(src, dst, fs::copy_options::overwrite_existing, e2);
|
||||
if (e2) { ++st.skipped; continue; }
|
||||
::chmod(dst.c_str(), (sb.st_mode & 07777) | 0600);
|
||||
++st.made;
|
||||
} else {
|
||||
++st.skipped;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct InodeHash {
|
||||
std::size_t operator()(const std::pair<dev_t, ino_t>& k) const { return std::hash<unsigned long>()(k.first * 1000003UL ^ k.second); }
|
||||
};
|
||||
using InodeSet = std::unordered_set<std::pair<dev_t, ino_t>, InodeHash>;
|
||||
|
||||
std::uint64_t usageRec(const fs::path& p, std::size_t& seen, std::size_t& skipped, std::size_t depth, InodeSet& links) {
|
||||
struct stat sb;
|
||||
if (::lstat(p.c_str(), &sb) != 0) { ++skipped; return 0; }
|
||||
std::uint64_t total = static_cast<std::uint64_t>(sb.st_blocks) * 512;
|
||||
if (!S_ISDIR(sb.st_mode)) {
|
||||
// hard links share their blocks: count each inode once
|
||||
if (sb.st_nlink > 1 && !links.insert({sb.st_dev, sb.st_ino}).second) return 0;
|
||||
return total;
|
||||
}
|
||||
if (depth > spec::kMaxTreeDepth || seen >= spec::kMaxScanEntries) { ++skipped; return total; }
|
||||
std::error_code ec;
|
||||
fs::directory_iterator it(p, ec), end;
|
||||
if (ec) { ++skipped; return total; }
|
||||
for (; it != end; it.increment(ec)) {
|
||||
if (ec) { ++skipped; break; }
|
||||
++seen;
|
||||
total += usageRec(it->path(), seen, skipped, depth + 1, links);
|
||||
}
|
||||
return total;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
SeedStats mirrorDirs(const fs::path& lower, const fs::path& upper) {
|
||||
SeedStats st;
|
||||
mkdirs(upper);
|
||||
mirrorRec(lower, upper, st, 0);
|
||||
return st;
|
||||
}
|
||||
|
||||
SeedStats copyTree(const fs::path& from, const fs::path& to, const SkipFn& skip) {
|
||||
SeedStats st;
|
||||
mkdirs(to);
|
||||
copyRec(from, to, st, 0, skip);
|
||||
return st;
|
||||
}
|
||||
|
||||
SeedStats copyEntries(const fs::path& from, const fs::path& to, const std::vector<std::string>& entries,
|
||||
const SkipFn& skip) {
|
||||
SeedStats total;
|
||||
mkdirs(to);
|
||||
for (const auto& e : entries) {
|
||||
fs::path src = from / e, dst = to / e;
|
||||
struct stat sb;
|
||||
if (::lstat(src.c_str(), &sb) != 0) continue; // not present on this host: nothing to copy, nothing skipped
|
||||
if (skip && skip(src)) { ++total.skipped; continue; }
|
||||
mkdirs(dst.parent_path());
|
||||
if (S_ISDIR(sb.st_mode)) {
|
||||
SeedStats s = copyTree(src, dst, skip);
|
||||
total.made += s.made; total.skipped += s.skipped; total.bytes += s.bytes;
|
||||
} else if (S_ISLNK(sb.st_mode)) {
|
||||
std::error_code ec;
|
||||
fs::remove(dst, ec);
|
||||
fs::create_symlink(fs::read_symlink(src), dst);
|
||||
++total.made;
|
||||
} else if (S_ISREG(sb.st_mode) && ::access(src.c_str(), R_OK) == 0) {
|
||||
std::error_code ec;
|
||||
fs::copy_file(src, dst, fs::copy_options::overwrite_existing, ec);
|
||||
if (ec) { ++total.skipped; continue; }
|
||||
::chmod(dst.c_str(), (sb.st_mode & 07777) | 0600);
|
||||
total.bytes += static_cast<std::uint64_t>(sb.st_size);
|
||||
++total.made;
|
||||
} else {
|
||||
++total.skipped;
|
||||
}
|
||||
if (total.bytes > spec::kMaxEtcCopyBytes) throw std::runtime_error("copy byte cap reached: " + from.string());
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
SeedStats copyFiles(const fs::path& from, const fs::path& to) {
|
||||
SeedStats st;
|
||||
std::error_code ec;
|
||||
fs::directory_iterator it(from, ec), end;
|
||||
if (ec) { ++st.skipped; return st; }
|
||||
for (; it != end; it.increment(ec)) {
|
||||
if (ec) { ++st.skipped; break; }
|
||||
struct stat sb;
|
||||
const fs::path& src = it->path();
|
||||
if (::lstat(src.c_str(), &sb) != 0 || !S_ISREG(sb.st_mode)) continue;
|
||||
fs::path dst = to / src.filename();
|
||||
std::error_code e2;
|
||||
if (fs::exists(dst, e2)) continue;
|
||||
if (::access(src.c_str(), R_OK) != 0) { ++st.skipped; continue; }
|
||||
st.bytes += static_cast<std::uint64_t>(sb.st_size);
|
||||
if (st.bytes > spec::kMaxEtcCopyBytes) throw std::runtime_error("copy byte cap reached: " + from.string());
|
||||
fs::copy_file(src, dst, fs::copy_options::none, e2);
|
||||
if (e2) { ++st.skipped; continue; }
|
||||
::chmod(dst.c_str(), (sb.st_mode & 07777) | 0600);
|
||||
++st.made;
|
||||
}
|
||||
return st;
|
||||
}
|
||||
|
||||
std::uint64_t diskUsage(const fs::path& p, std::size_t& skipped) {
|
||||
std::size_t seen = 0;
|
||||
InodeSet links;
|
||||
return usageRec(p, seen, skipped, 0, links);
|
||||
}
|
||||
|
||||
void mkdirs(const fs::path& p, mode_t mode) {
|
||||
std::error_code ec;
|
||||
fs::create_directories(p, ec);
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
#pragma once
|
||||
#include <cerrno>
|
||||
#include <cstdint>
|
||||
#include <filesystem>
|
||||
#include <functional>
|
||||
#include <stdexcept>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
namespace doos {
|
||||
|
||||
@@ -45,7 +48,38 @@ private:
|
||||
Fd fd_;
|
||||
};
|
||||
|
||||
// Parses "512m", "2g", "1t" (k/m/g/t, optional trailing b) into bytes; throws on garbage.
|
||||
std::uint64_t parseSize(const std::string& text);
|
||||
std::string formatSize(std::uint64_t bytes);
|
||||
|
||||
std::string base64Decode(const std::string& in); // lenient on whitespace, throws on bad characters
|
||||
|
||||
struct ProcResult {
|
||||
int code = -1;
|
||||
std::string out; // stdout, only when captured
|
||||
};
|
||||
// fork+exec without a shell. Captured output is capped; stderr is inherited unless quiet.
|
||||
ProcResult runProcess(const std::vector<std::string>& argv, bool capture = false, bool quiet = false);
|
||||
|
||||
namespace fsutil {
|
||||
struct SeedStats {
|
||||
std::size_t made = 0;
|
||||
std::size_t skipped = 0; // unreadable or unsupported entries, never silent
|
||||
std::uint64_t bytes = 0;
|
||||
};
|
||||
// Creates every directory of `lower` in `upper` (owner rwx added) so a unmapped-owner lower stays writable.
|
||||
SeedStats mirrorDirs(const std::filesystem::path& lower, const std::filesystem::path& upper);
|
||||
// Copies dirs, regular files and symlinks readable by the caller. Capped by spec.hpp limits.
|
||||
// `skip` (optional) is asked for every entry and returns true to leave it out (counted as skipped).
|
||||
using SkipFn = std::function<bool(const std::filesystem::path&)>;
|
||||
SeedStats copyTree(const std::filesystem::path& from, const std::filesystem::path& to, const SkipFn& skip = nullptr);
|
||||
// Copies only the named entries (relative paths, files, links or dirs) of `from` into `to`.
|
||||
SeedStats copyEntries(const std::filesystem::path& from, const std::filesystem::path& to,
|
||||
const std::vector<std::string>& entries, const SkipFn& skip);
|
||||
// Copies only the regular files directly inside `from` (readable ones) into `to`; existing targets stay.
|
||||
SeedStats copyFiles(const std::filesystem::path& from, const std::filesystem::path& to);
|
||||
// Sum of allocated bytes (st_blocks) below p, lstat only. `skipped` counts unreadable dirs.
|
||||
std::uint64_t diskUsage(const std::filesystem::path& p, std::size_t& skipped);
|
||||
void mkdirs(const std::filesystem::path& p, mode_t mode = 0755);
|
||||
void writeFile(const std::filesystem::path& p, const std::string& content);
|
||||
std::string readFile(const std::filesystem::path& p);
|
||||
|
||||
+148
@@ -57,5 +57,153 @@ wait; sleep 0.5
|
||||
"$DOOS" delete t | grep -q "dry run" && [ -d "$DOOS_HOME/containers/t" ] && ok "delete is dry-run" || bad "delete is dry-run"
|
||||
"$DOOS" delete --yes t | grep -q deleted && [ ! -e "$DOOS_HOME/containers/t" ] && ok "delete --yes removes" || bad "delete --yes removes"
|
||||
|
||||
# ---- 1.0: skeleton, seeded /etc, uid mapping (host mode, container t was deleted above)
|
||||
"$DOOS" get_or_create s >/dev/null 2>&1
|
||||
outs() { "$DOOS" attach s -- sh -c "$1" 2>&1; }
|
||||
[ "$(outs 'touch /usr/bin/doos-x && echo y')" = "y" ] && [ ! -e /usr/bin/doos-x ] && ok "skeleton: write into /usr/bin" || bad "skeleton: write into /usr/bin"
|
||||
[ "$(outs 'stat -c %u /etc/shells')" = "0" ] && ok "/etc owned by container root" || bad "/etc owned by container root"
|
||||
[ "$(outs 'chown 100:100 /root && stat -c %u:%g /root')" = "100:100" ] && ok "subuid: chown to other ids" || bad "subuid: chown to other ids"
|
||||
[ "$(outs 'ls /var/lib/dpkg/status >/dev/null && echo y')" = "y" ] && ok "/var/lib/dpkg present" || bad "/var/lib/dpkg present"
|
||||
"$DOOS" get_or_create --ids single s1 >/dev/null 2>&1
|
||||
[ "$("$DOOS" attach s1 -- cat /proc/self/uid_map | wc -l)" = "1" ] && ok "ids single: one mapping" || bad "ids single: one mapping"
|
||||
"$DOOS" attach s1 -- sh -c 'cat /etc/apt/apt.conf.d/99doos-sandbox' 2>&1 | grep -q 'Sandbox::User "root"' && ok "ids single: apt sandbox override" || bad "ids single: apt sandbox override"
|
||||
check "ids fixed at creation" bash -c "! '$DOOS' get_or_create --ids subuid s1"
|
||||
check "bad size rejected" bash -c "! '$DOOS' get_or_create --size 1x z1"
|
||||
check "size below minimum rejected" bash -c "! '$DOOS' get_or_create --size 1m z2"
|
||||
"$DOOS" attach s -- sh -c 'apt-get update >/dev/null 2>&1; apt-get install -y --no-install-recommends cowsay >/dev/null 2>&1; dpkg --audit && /usr/games/cowsay ok >/dev/null' && ok "host mode: apt install completes" || bad "host mode: apt install completes"
|
||||
[ ! -e /usr/games/cowsay ] && ok "host untouched by apt install" || bad "host untouched by apt install"
|
||||
"$DOOS" delete --yes s >/dev/null; "$DOOS" delete --yes s1 >/dev/null
|
||||
|
||||
# ---- images
|
||||
echo "--- images (network)"
|
||||
"$DOOS" image list | grep -q "alpine.*available" && ok "image list shows catalog" || bad "image list shows catalog"
|
||||
check "http url refused" bash -c "! '$DOOS' image add http://example.com/x.tar.gz"
|
||||
check "url without sha256 refused" bash -c "! '$DOOS' image add https://example.com/x.tar.gz"
|
||||
check "unknown image refused" bash -c "! '$DOOS' get_or_create --image nosuch i0"
|
||||
tmpd=$(mktemp -d); mkdir -p $tmpd/r/bin; echo hi > $tmpd/r/bin/f; tar -C $tmpd/r -czf $tmpd/mini.tar.gz .
|
||||
check "wrong sha256 rejected" bash -c "! '$DOOS' image add --sha256 $(printf '0%.0s' $(seq 64)) $tmpd/mini.tar.gz mini"
|
||||
[ ! -e "$DOOS_HOME/images/mini" ] && [ ! -e "$DOOS_HOME/images/.tmp-mini" ] && ok "failed add leaves nothing" || bad "failed add leaves nothing"
|
||||
check "local tar added" "$DOOS" image add $tmpd/mini.tar.gz mini
|
||||
rm -rf $tmpd; "$DOOS" image delete --yes mini >/dev/null
|
||||
|
||||
"$DOOS" attach --image alpine a1 -- true >/dev/null 2>&1
|
||||
"$DOOS" image list | grep -q "alpine.*installed" && ok "image auto-fetched on demand" || bad "image auto-fetched on demand"
|
||||
[ "$("$DOOS" attach a1 -- sh -c '. /etc/os-release; echo $ID')" = "alpine" ] && ok "alpine image runs" || bad "alpine image runs"
|
||||
"$DOOS" attach a1 -- sh -c 'apk add --no-cache jq >/dev/null 2>&1 && jq --version' | grep -q jq && ok "alpine: apk add" || bad "alpine: apk add"
|
||||
[ "$("$DOOS" attach a1 -- sh -c 'chown 100:100 /root && stat -c %u /root')" = "100" ] && ok "image: subuid chown" || bad "image: subuid chown"
|
||||
"$DOOS" attach a1 -- sh -c 'exec 3<>/dev/tcp/1.1.1.1/443' >/dev/null 2>&1; "$DOOS" attach a1 -- sh -c 'wget -qO- https://example.com | grep -q html' && ok "image: network + dns" || bad "image: network + dns"
|
||||
check "image delete refused while used" bash -c "! '$DOOS' image delete --yes alpine"
|
||||
check "image in use is a dry run too" bash -c "! '$DOOS' image delete alpine"
|
||||
"$DOOS" get_or_create --image alpine --ids single a2 >/dev/null 2>&1 && bad "image ids mismatch rejected" || ok "image ids mismatch rejected"
|
||||
"$DOOS" delete --yes a1 >/dev/null
|
||||
"$DOOS" image delete alpine | grep -q "dry run" && [ -d "$DOOS_HOME/images/alpine" ] && ok "image delete is dry-run" || bad "image delete is dry-run"
|
||||
"$DOOS" image delete --yes alpine | grep -q deleted && [ ! -e "$DOOS_HOME/images/alpine" ] && ok "image delete --yes removes" || bad "image delete --yes removes"
|
||||
|
||||
"$DOOS" attach --image ubuntu u1 -- sh -c 'apt-get update >/dev/null 2>&1; apt-get install -y --no-install-recommends cowsay >/dev/null 2>&1; dpkg --audit && /usr/games/cowsay ok >/dev/null' && ok "ubuntu image: apt install completes" || bad "ubuntu image: apt install completes"
|
||||
"$DOOS" delete --yes u1 >/dev/null; "$DOOS" image delete --yes ubuntu >/dev/null
|
||||
|
||||
# ---- size cap (needs cgroup v2 delegation; scans every 2 s at the earliest)
|
||||
echo "--- size cap, accounting of other ids"
|
||||
"$DOOS" get_or_create --size 64m cap >/dev/null 2>&1
|
||||
# 80 MB written as container uid 5 into a mode-700 directory the host user cannot read: only an id-map scan sees it
|
||||
"$DOOS" attach cap -- python3 -c '
|
||||
import os,time
|
||||
os.mkdir("/root/d5"); os.chown("/root/d5",5,5); os.chmod("/root/d5",0o700); os.setgid(5); os.setuid(5)
|
||||
for n in (1,2):
|
||||
with open("/root/d5/f%d"%n,"wb") as f: f.write(b"x"*(40*1024*1024)); f.flush(); os.fsync(f.fileno())
|
||||
time.sleep(40)' >/tmp/doos-cap.$$ 2>&1 &
|
||||
sleep 9
|
||||
if grep -q "FROZEN" /tmp/doos-cap.$$; then
|
||||
ok "size cap freezes (counts other uids)"
|
||||
"$DOOS" attach cap -- sh -c 'grep -q /shell /proc/self/cgroup 2>/dev/null; rm -rf /root/d5' >/dev/null 2>&1; sleep 5
|
||||
grep -q thawed /tmp/doos-cap.$$ && ok "size cap thaws after cleanup" || bad "size cap thaws after cleanup"
|
||||
elif grep -q "cannot freeze\|not enforced" /tmp/doos-cap.$$; then
|
||||
echo "skip size cap: no cgroup delegation here (warn-only mode confirmed)"
|
||||
else
|
||||
bad "size cap freezes (counts other uids)"
|
||||
fi
|
||||
kill %% 2>/dev/null; wait 2>/dev/null; rm -f /tmp/doos-cap.$$
|
||||
"$DOOS" delete --yes cap >/dev/null 2>&1
|
||||
|
||||
# ---- resource limits, rescue leaf, hardening
|
||||
echo "--- limits and hardening"
|
||||
"$DOOS" get_or_create --memory 256m --pids 32 --cpus 0.5 lim >/dev/null 2>&1
|
||||
"$DOOS" list | grep -q "^lim" && ok "limit options accepted" || bad "limit options accepted"
|
||||
r=$("$DOOS" attach lim -- python3 -c 'b=bytearray(b"x"*(400*1024*1024)); print("survived")' 2>&1)
|
||||
echo "$r" | grep -q "killed by its memory limit" && ! echo "$r" | grep -q survived && ok "memory limit kills the container" || bad "memory limit kills the container"
|
||||
r=$("$DOOS" attach lim -- sh -c 'for i in $(seq 1 80); do sleep 3 & done 2>&1; wait' 2>&1)
|
||||
echo "$r" | grep -q "process limit" && ok "pids limit enforced and reported" || bad "pids limit enforced and reported"
|
||||
check "limits can change later" "$DOOS" get_or_create --memory 512m lim
|
||||
check "memory below minimum refused" bash -c "! '$DOOS' get_or_create --memory 1m z3"
|
||||
[ ! -e "$DOOS_HOME/containers/z3" ] && ok "failed create leaves nothing" || bad "failed create leaves nothing"
|
||||
check "bad cpus refused" bash -c "! '$DOOS' get_or_create --cpus abc z4"
|
||||
"$DOOS" attach lim -- sleep 6 >/dev/null 2>&1 & sleep 1.5
|
||||
[ "$("$DOOS" attach lim -- sh -c 'grep -c /shell /proc/self/cgroup')" = "1" ] && ok "second attach runs in the shell cgroup" || bad "second attach runs in the shell cgroup"
|
||||
wait
|
||||
h() { "$DOOS" attach lim -- python3 -c "$1" 2>&1; }
|
||||
[ "$(h 'import ctypes,errno;l=ctypes.CDLL(None,use_errno=True);print(l.syscall(250,0,0,0,0,0),errno.errorcode[ctypes.get_errno()])')" = "-1 EPERM" ] && ok "seccomp: keyctl denied" || bad "seccomp: keyctl denied"
|
||||
[ "$(h 'import ctypes,errno;l=ctypes.CDLL(None,use_errno=True);print(l.syscall(272,0x10000000),errno.errorcode[ctypes.get_errno()])')" = "-1 EPERM" ] && ok "seccomp: nested user namespace denied" || bad "seccomp: nested user namespace denied"
|
||||
[ "$(h 'import ctypes,errno;l=ctypes.CDLL(None,use_errno=True);print(l.syscall(435,0,0),errno.errorcode[ctypes.get_errno()])')" = "-1 ENOSYS" ] && ok "seccomp: clone3 answers ENOSYS" || bad "seccomp: clone3 answers ENOSYS"
|
||||
"$DOOS" attach lim -- sh -c 'mount -t tmpfs t /mnt' >/dev/null 2>&1 && bad "mount denied" || ok "mount denied"
|
||||
[ "$("$DOOS" attach lim -- sh -c 'grep ^CapEff /proc/self/status | cut -f2')" = "00000000a80425fb" ] && ok "capabilities reduced to the default set" || bad "capabilities reduced to the default set"
|
||||
"$DOOS" attach lim -- sh -c 'gcc --version >/dev/null && curl -sS -o /dev/null https://example.com && git --version >/dev/null && python3 -c "import ssl" && ping -c1 -W3 1.1.1.1 >/dev/null' && ok "common tools work under hardening" || bad "common tools work under hardening"
|
||||
[ "$("$DOOS" attach lim -- sh -c 'ls /etc | wc -l')" -lt 80 ] && ok "etc is an allowlist" || bad "etc is an allowlist"
|
||||
"$DOOS" attach lim -- sh -c 'ls /etc/shadow /etc/sudoers /etc/ssh/ssh_host_rsa_key' >/dev/null 2>&1 && bad "etc: no credentials" || ok "etc: no credentials"
|
||||
"$DOOS" get_or_create --hardening relaxed rx >/dev/null 2>&1
|
||||
"$DOOS" attach rx -- sh -c 'unshare -U true && grep -q "^Seccomp:.0" /proc/self/status' >/dev/null 2>&1 && ok "relaxed: no seccomp, nested userns allowed" || bad "relaxed: no seccomp, nested userns allowed"
|
||||
check "hardening fixed at creation" bash -c "! '$DOOS' get_or_create --hardening relaxed lim"
|
||||
"$DOOS" delete --yes lim >/dev/null; "$DOOS" delete --yes rx >/dev/null
|
||||
|
||||
# ---- pasta network mode
|
||||
echo "--- pasta"
|
||||
python3 -c '
|
||||
import socket,time
|
||||
s=socket.socket(); s.bind(("127.0.0.1",48765)); s.listen(5)
|
||||
a=socket.socket(socket.AF_UNIX); a.bind("\0doos-abstract-test"); a.listen(5)
|
||||
time.sleep(25)' & lp=$!
|
||||
sleep 1
|
||||
"$DOOS" get_or_create --net pasta pa >/dev/null 2>&1
|
||||
pr() { "$DOOS" attach pa -- python3 -c "$1" 2>&1; }
|
||||
[ "$(pr 'import socket;s=socket.create_connection(("127.0.0.1",48765),2);print("ok")')" = "ok" ] && ok "pasta: host loopback reachable" || bad "pasta: host loopback reachable"
|
||||
[ "$(pr 'import socket;s=socket.socket(socket.AF_UNIX)
|
||||
try: s.connect("\0doos-abstract-test");print("reachable")
|
||||
except Exception: print("blocked")')" = "blocked" ] && ok "pasta: host abstract sockets not shared" || bad "pasta: host abstract sockets not shared"
|
||||
[ "$(pr 'import urllib.request;print(urllib.request.urlopen("https://example.com",timeout=6).status)')" = "200" ] && ok "pasta: dns and https" || bad "pasta: dns and https"
|
||||
"$DOOS" attach pa -- python3 -c '
|
||||
import socket
|
||||
s=socket.socket(); s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1); s.bind(("0.0.0.0",48911)); s.listen(1); s.settimeout(8)
|
||||
c,_=s.accept(); print(c.recv(5).decode())' >/tmp/doos-in.$$ 2>&1 &
|
||||
sleep 2.5; python3 -c 'import socket;s=socket.create_connection(("127.0.0.1",48911),2);s.sendall(b"hello")' 2>/dev/null; wait %2 2>/dev/null; sleep 0.5
|
||||
grep -q hello /tmp/doos-in.$$ && ok "pasta: inbound port forwarded" || bad "pasta: inbound port forwarded"
|
||||
rm -f /tmp/doos-in.$$; kill $lp 2>/dev/null; wait 2>/dev/null
|
||||
"$DOOS" delete --yes pa >/dev/null
|
||||
|
||||
# ---- documented network examples that were not covered yet, and the apparmor command
|
||||
echo "--- network examples"
|
||||
"$DOOS" apparmor | grep -q "profile doos.* $DOOS flags=(unconfined)" && ok "apparmor: profile names this binary" || bad "apparmor: profile names this binary"
|
||||
"$DOOS" attach --net none iso -- sh -c 'curl -sS -m 3 https://example.com >/dev/null 2>&1; [ $? -eq 6 ] && [ "$(tail -n +3 /proc/net/dev | cut -d: -f1 | tr -d " ")" = lo ]' >/dev/null 2>&1 && ok "none: curl exits 6, only lo" || bad "none: curl exits 6, only lo"
|
||||
"$DOOS" attach nets -- timeout 5 python3 -m http.server 8077 --bind 0.0.0.0 >/dev/null 2>&1 & sleep 2.5
|
||||
curl -sS -m 2 -o /dev/null http://127.0.0.1:8077/ >/dev/null 2>&1 && bad "slirp: no inbound path" || ok "slirp: no inbound path"
|
||||
wait %% 2>/dev/null
|
||||
"$DOOS" get_or_create --net pasta pa1 >/dev/null 2>&1; "$DOOS" get_or_create --net pasta pa2 >/dev/null 2>&1
|
||||
"$DOOS" attach pa1 -- timeout 12 python3 -m http.server 8066 --bind 0.0.0.0 >/dev/null 2>&1 & sleep 5
|
||||
[ "$("$DOOS" attach pa2 -- curl -sS -m 2 -o /dev/null -w '%{http_code}' http://127.0.0.1:8066/ 2>/dev/null)" = "200" ] && ok "pasta: container to container via loopback" || bad "pasta: container to container via loopback"
|
||||
wait 2>/dev/null
|
||||
"$DOOS" delete --yes pa1 >/dev/null; "$DOOS" delete --yes pa2 >/dev/null; "$DOOS" delete --yes nets >/dev/null; "$DOOS" delete --yes iso >/dev/null
|
||||
|
||||
# ---- signatures and more catalog images
|
||||
echo "--- signatures"
|
||||
"$DOOS" image add alpine 2>&1 | grep -q "signature verified" && ok "alpine signature verified against pinned key" || bad "alpine signature verified against pinned key"
|
||||
"$DOOS" image delete --yes alpine >/dev/null
|
||||
shim=$(mktemp -d)
|
||||
printf '#!/bin/sh\ncase "$*" in\n *--with-colons*) echo "fpr:::::::::0482D84022F52DF1C4E7CD43293ACD0907D9495A:";;\n *--verify*) echo "[GNUPG:] GOODSIG AAAAAAAAAAAAAAAA M"; echo "[GNUPG:] VALIDSIG 1111111111111111111111111111111111111111 2026-01-01 1 0 4 0 1 10 00 1111111111111111111111111111111111111111";;\nesac\nexit 0\n' > $shim/gpg; chmod +x $shim/gpg
|
||||
PATH=$shim:$PATH "$DOOS" image add alpine >/dev/null 2>&1 && bad "signature by the wrong key refused" || ok "signature by the wrong key refused"
|
||||
[ ! -e "$DOOS_HOME/images/alpine" ] && ok "refused signature leaves no image" || bad "refused signature leaves no image"
|
||||
sed -i 's/GOODSIG AAAAAAAAAAAAAAAA M/BADSIG 293ACD0907D9495A N/' $shim/gpg
|
||||
PATH=$shim:$PATH "$DOOS" image add alpine >/dev/null 2>&1 && bad "BADSIG refused" || ok "BADSIG refused"
|
||||
rm -rf $shim
|
||||
"$DOOS" attach --image debian d1 -- sh -c '. /etc/os-release; echo $ID; apt-get update >/dev/null 2>&1; apt-get install -y --no-install-recommends cowsay >/dev/null 2>&1; dpkg --audit && /usr/games/cowsay ok >/dev/null' 2>/dev/null | grep -q debian && ok "debian image: apt install completes" || bad "debian image: apt install completes"
|
||||
"$DOOS" delete --yes d1 >/dev/null; "$DOOS" image delete --yes debian >/dev/null
|
||||
|
||||
echo "passed=$pass failed=$fail"
|
||||
[ "$fail" -eq 0 ]
|
||||
|
||||
Reference in New Issue
Block a user