Remove encryption; user-data filters; SQLite safety; recovery, retention, scheduler
- No client-side encryption: plaintext content-addressed remote (SHA-256 names), no backup key, no cryptography dependency (server disk encryption is the trust model). - Filters back user data: images, archives, databases, PDFs accepted; 10 MiB cap; binary sniffing removed; temp names hardened (~$, #..#, .temp). - SQLite zero-error policy: backup-API snapshots + integrity_check, journal folding, locked/corrupt loud skips, verified restores. - Recovery: reindex from manifests, remote adopt, on-demand blob fetch, 5-day retention + thinning, GC, date-guarded remote purge, metrics. - Scheduler with WebDAV quota signal and 70% pressure backstop (floor kept). - 37 tests incl. live-monitor capture safety and DB safety.
This commit is contained in:
retoor
@@ -0,0 +1,104 @@
|
||||
# Compression, Deduplication, and Integrity in Restic, BorgBackup, and Kopia
|
||||
|
||||
## Fixed-file vs content-defined chunking: algorithm, average chunk sizes, how dedup works with encryption
|
||||
|
||||
### Takeaway
|
||||
All three use content-defined chunking (CDC) by default with a fixed-size fallback option; deduplication is on plaintext hashes before compression/encryption (so no convergent encryption), with Borg and Kopia keying chunk IDs while restic uses plain SHA-256.
|
||||
|
||||
### Cited Findings
|
||||
- Restic splits each file independently with Rabin-fingerprint CDC over a 64-byte sliding window; a random irreducible polynomial is generated at `init` and stored as `chunker_polynomial` in `config` to harden against watermarking — [Source](https://restic.readthedocs.io/en/stable/100_references.html); background and worked example — [Source](https://restic.net/blog/2015-09-12/restic-foundation1-cdc/)
|
||||
- Restic chunking parameters: files <512 KiB are not split; blobs are 512 KiB–8 MiB with ~1 MiB average target; modified files only re-store changed blobs, robust to insertions at arbitrary offsets — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic 0.18.0 mitigates chunk-size fingerprinting (Alexeev/Percival/Zhang 2025) by randomly assigning chunks to pack files so attackers observing the repo cannot map chunk sizes to files — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic deduplication happens before encryption: blob ID is SHA-256 of plaintext; index maps plaintext hash to pack location; pack/index filenames are SHA-256 of ciphertext for accident detection only — [Source](https://forum.restic.net/t/how-are-blobs-deduplicated-with-encryption/6478); all content referenced by SHA-256 of plaintext, one blob holds data from only one file, multiple blobs packed per pack file — [Source](https://github.com/restic/restic/issues/2401)
|
||||
- Restic uses random per-repository master keys and random 16-byte IV per encryption (not convergent/deterministic encryption); identical plaintexts deduplicate via index lookup, not via identical ciphertexts — [Source](https://restic.readthedocs.io/en/stable/design.html); external audit notes AES-256-CTR + Poly1305-AES with separate keys — [Source](https://words.filippo.io/restic-cryptography/)
|
||||
- Borg default chunker is `fastcdc` (window-less keyed Gear hash); alternatives: `fixed` (fixed blocksize, optional different header block), `buzhash`/`buzhash64` (rolling Buzhash), `rabin-aes`/`toeplitz-aes`/`goldilocks-aes` (UHF-then-PRF: rolling universal hash + AES-128, cut decision only on AES output) — [Source](https://borgbackup.readthedocs.io/en/latest/internals/data-structures.html); comparison table and guidance — [Source](https://borgbackup.readthedocs.io/en/master/internals/chunker.html)
|
||||
- Borg classic `buzhash` defaults: `CHUNK_MIN_EXP=19` (512 KiB), `CHUNK_MAX_EXP=23` (8 MiB), `HASH_MASK_BITS=21` (~2 MiB target), `HASH_WINDOW_SIZE=4095` bytes; tunable via `--chunker-params`; min/max clamp where rolling-hash cuts may occur — [Source](https://github.com/borgbackup/borg/blob/master/docs/misc/create_chunker-params.txt); `fastcdc` uses `CHUNK_MIN_EXP,CHUNK_MAX_EXP,HASH_MASK_BITS,NC_LEVEL` with normalized chunking — [Source](https://borgbackup.readthedocs.io/en/latest/internals/data-structures.html)
|
||||
- Borg chunker secrets are derived from repository key material (buzhash table XOR seed stored encrypted in keyfile; AES chunkers derive table/polynomial/AES key from `id_key` per-chunker domain) to resist fingerprinting; 2025 research (Truong et al. CCS 2025 eprint 2025/558; eprint 2025/532) showed keyed rolling-hash-only chunkers allow key recovery, motivating AES-based chunkers — [Source](https://borgbackup.readthedocs.io/en/master/internals/chunker.html)
|
||||
- Borg deduplication is global across all archives/hosts/files on chunk `id_hash`, which is a keyed MAC over plaintext (`HMAC-SHA256` for `--id-hash sha256`, keyed `BLAKE3` for `--id-hash blake3`) using secret `id_key`, not plain hash, so attackers cannot confirm small-file presence without the key — [Source](https://borgbackup.readthedocs.io/en/latest/internals/security.html); file metadata (msgpacked items) is chunked with finer params and deduplicated the same way — [Source](https://borgbackup.readthedocs.io/en/latest/internals/data-structures.html)
|
||||
- Borg 1.x-compatible dedup requires `buzhash`; `fastcdc`/`buzhash64` give same dedup but different cut points; `fixed` gives positional (not content-shift-resilient) dedup, suited to disk images — [Source](https://borgbackup.readthedocs.io/en/master/internals/chunker.html)
|
||||
- Kopia calls chunkers “splitters”: `BUZHASH`, `RABINKARP` (rolling-hash CDC) and `fixed`, selectable size 1M–8M — [Source](https://kopia.discourse.group/t/does-kopia-use-content-defined-chunking-cdc/1417); default object splitter is `DYNAMIC-4M-BUZHASH` (also the default for every `repository create` backend) — [Source](https://kopia.io/docs/reference/command-line/common/repository-create-filesystem/)
|
||||
- Kopia splitter history: original `DYNAMIC` splitter (silvasur/buzhash dependency) was deprecated for license reasons and replaced by a faster but incompatible buzhash implementation; old repos remain readable but large objects re-upload instead of deduping across the change — [Source](https://github.com/kopia/kopia/commit/03339c18afedb810f31320ef01c707e7acbdc374)
|
||||
- Kopia pipeline is split → hash → compare against index → discard if known; otherwise compress → encrypt → pack multiple small blocks into ~20–40 MB packs with random names; content hash default `BLAKE2B-256-128` — [Source](https://kopia.io/docs/advanced/compression/); pack/index architecture — [Source](https://kopia.io/docs/advanced/architecture/)
|
||||
- Kopia deduplication is unaffected by compression settings because hashing precedes compression since index v2 (see next section); same file under compressed and uncompressed policies still dedupes — [Source](https://kopia.discourse.group/t/deduplication-and-compression/714)
|
||||
|
||||
### Inferences
|
||||
- None of the three use convergent encryption (deterministic, plaintext-derived keys); all use random repository keys + random nonces/session keys, with dedup achieved by a client-side plaintext-hash index lookup before encryption.
|
||||
- Keyed chunk IDs (Borg MAC, Kopia HMAC-derived keys/format secret) hide confirmation-of-file attacks from repo-only attackers; restic's plain-SHA-256 blob IDs do not, which is part of why restic added pack-mixing and per-repo random polynomials.
|
||||
|
||||
### Gaps
|
||||
- Exact numeric defaults for Kopia `DYNAMIC-4M-BUZHASH` (min/max/expected size, window bytes) were not found in fetched docs; only the 4M-average naming and 1M–8M range were confirmed.
|
||||
- Whether Kopia content IDs themselves are HMAC-keyed (vs. plain hash + separately HMAC-derived encryption keys) could not be confirmed from fetched pages; per-content key derivation via HMAC-SHA256 is confirmed but ID-keying needs source-code confirmation.
|
||||
|
||||
## Which compression algorithms are supported, how is the algorithm recorded per object, and what happens mixing versions
|
||||
|
||||
### Takeaway
|
||||
Restic supports only zstd (repo v2, per-blob type byte + unpacked version byte); Borg supports none/lz4/zstd/zlib/lzma with per-object ctype+clevel metadata and free mixing; Kopia supports a large s2/pgzip/gzip/deflate/zstd matrix recorded per-content in index v2 and also freely mixable going forward.
|
||||
|
||||
### Cited Findings
|
||||
- Restic repo v2 adds compression; data and tree blobs may use zstandard only; v1 has no compressed types — [Source](https://restic.readthedocs.io/en/stable/100_references.html); changelog entry “Support compression for blobs (data/tree) and index/lock/snapshot files” — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic per-run negotiation: `--compression off|fastest|auto(default)|better|max` (also `RESTIC_COMPRESSION`); maps to klauspost/compress zstd levels `SpeedFastest`/`SpeedDefault`/`SpeedBetterCompression`/`SpeedBestCompression` with 512 KiB window and CRC disabled — [Source](https://github.com/restic/restic/blob/master/internal/repository/repository.go); `auto` is default and uses implementation default level — [Source](https://forum.restic.net/t/what-compression-is-used/6850); tuning doc confirms `auto` default for v2 repos — [Source](https://restic.readthedocs.io/en/stable/047_tuning_parameters.html)
|
||||
- Restic pack header per-blob recording: 1-byte type `0b00` data / `0b01` tree / `0b10` compressed-data / `0b11` compressed-tree, followed by `Length(encrypted_blob)||Hash(plaintext)` (uncompressed) or `Length(encrypted_blob)||Length(plaintext)||Hash(plaintext)` (compressed), little-endian uint32; index adds `uncompressed_length` only for compressed blobs — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic unpacked files (index/snapshot/lock): plaintext is `encoding_version||data` with 1-byte version; `[` (0x5b)/`{` (0x7b) mean “whole plaintext is JSON” (v1 back-compat); version `2` means zstd-compressed JSON; new v2 writes always version 2; implementation compresses unpacked data before encryption via `compressUnpacked`, and compresses tree blobs even when `--compression off` for data blobs (`if Compression != off || t != DataBlob`) — [Source](https://restic.readthedocs.io/en/stable/100_references.html); code — [Source](https://github.com/restic/restic/blob/master/internal/repository/repository.go); design rationale (null-byte/version-byte history) — [Source](https://github.com/restic/restic/pull/3666)
|
||||
- Restic mixing/compat: compressed and uncompressed blobs of same type may be mixed in one pack; in v2, data and tree blobs must be in separate packs; v1-repo data remains valid in v2 (no re-upload required); new data compressed per run-level, `prune` recompresses only repacked chunks; v2 repos unreadable by pre-compression restic versions; new repos default to v2 since 0.14.0 so compression is on by default — [Source](https://restic.readthedocs.io/en/stable/100_references.html); release/upgrade behavior — [Source](https://forum.restic.net/t/restic-0-14-0-released/5359)
|
||||
- Restic dedup uses plaintext hash, so future zstd-output changes do not break dedup (compressed bytes never hashed for identity) — [Source](https://github.com/restic/restic/pull/3666)
|
||||
- Borg compression set: `none` (0x00), `lz4` (0x01), `zstd` (0x03, levels -128..22), `zlib` (0x05, levels 0–9), `lzma` (0x02, levels 0–9); `ctype` byte + `clevel` byte (zstd `int8_t` so -1→255, -128→128; others unsigned with 255 = n/a); speed order none>lz4>zlib>lzma, lz4>zstd; compression lzma>zlib>lz4>none, zstd>lz4 — [Source](https://borgbackup.readthedocs.io/en/latest/internals/data-structures.html); legacy 1.x zlib has no ID bytes (detected by `0x.8` header) — [Source](https://borgbackup.readthedocs.io/en/stable/internals/data-structures.html)
|
||||
- Borg per-object recording (borg2): msgpacked metadata dict holds `ctype`, `clevel`, `csize` (compressed+obfuscated size), `psize` (payload w/o obfuscation trailer, when obfuscated), `olevel`, `size` (uncompressed), `type` (ro_type `A`/`C`/`S`/`F`); metadata and data slots separately encrypted with header bound as AAD — [Source](https://borgbackup.readthedocs.io/en/latest/internals/data-structures.html); code `RepoObj.format/parse` — [Source](https://github.com/borgbackup/borg/blob/86fd77fd/src/borg/repoobj.py); compressor base auto-detection via ID header — [Source](https://github.com/borgbackup/borg/blob/86fd77fd/src/borg/compress.pyx)
|
||||
- Borg negotiation/mixing: default compression is `lz4`; mixing methods in one repo is fine because dedup is on source chunks, not compressed bytes; first writer of a chunk determines its stored compression; `borg recreate`/`repo-compress` can recompress; wrappers `auto,C[,L]` (lz4 compressibility heuristic → `none` vs `C`) and `obfuscate,SPEC,C[,L]` (Padmé deterministic padding, ≤12% overhead, `MAX_DATA_SIZE` ~20 MiB cap) — [Source](https://manpages.debian.org/trixie/borgbackup2/borg2-compression.1.en.html)
|
||||
- Kopia compression is disabled by default and controlled per-policy (global/host/path: `--compression=...`, min/max file size, extensions); new setting applies going forward only, does not retroactively recompress — [Source](https://kopia.io/docs/faqs/)
|
||||
- Kopia algorithm menu: `none|deflate-best-compression|deflate-best-speed|deflate-default|gzip|gzip-best-compression|gzip-best-speed|pgzip|pgzip-best-compression|pgzip-best-speed|s2-better|s2-default|s2-parallel-4|s2-parallel-8|zstd|zstd-better-compression|zstd-fastest` (`zstd` recommended default choice); benchmark table shows s2 fastest (~GB/s, largest), zstd smallest, pgzip balanced — [Source](https://kopia.io/docs/faqs/); details and memory/I-O guidance — [Source](https://kopia.io/docs/advanced/compression/)
|
||||
- Kopia per-object recording: since content-level compression (v0.9, `--index-version=2`), compression done after hashing; per-content compression ID kept in content manager bookkeeping, visible via `kopia content list -c` / `content stats` (e.g. `(uncompressed)` vs `zstd` vs `zstd-fastest` with counts/sizes), no longer encoded as `Z`-prefixed content ID — [Source](https://github.com/kopia/kopia/pull/1076); if compressed chunk grows, original stored uncompressed — [Source](https://kopia.io/docs/advanced/compression/)
|
||||
- Kopia mixing/compat: dedup unaffected by algorithm changes or library-output drift because ID is pre-compression hash; enables future recompression during maintenance; changing policy does not rewrite old contents; newer Kopia cannot read legacy LZ4-compressed contents — must migrate with an older version (restore/repack) before upgrading — [Source](https://github.com/kopia/kopia/pull/1076); LZ4 removal notice — [Source](https://kopia.io/docs/advanced/compression/)
|
||||
|
||||
### Inferences
|
||||
- Recording compression outside the content ID (restic type bits/index field, Borg metadata dict, Kopia index v2) is what allows free mixing and future recompression without breaking content addressing.
|
||||
- Restic's single-codec choice simplifies negotiation to levels only; Borg/Kopia pay codec-matrix complexity for finer speed/ratio control.
|
||||
|
||||
### Gaps
|
||||
- Exact zstd numeric levels behind restic `auto/better/max` (klauspost `SpeedDefault` etc. numeric equivalents) were not pinned to zstd CLI levels in fetched sources.
|
||||
- Kopia on-disk bytes for per-content compression IDs (enum values/header layout) were not found; only CLI-visible behavior is documented.
|
||||
|
||||
## How is integrity verified (HMAC, AEAD tag, checksums, parity)? How are manifests/snapshots authenticated?
|
||||
|
||||
### Takeaway
|
||||
Restic uses Encrypt-then-MAC (AES-256-CTR + Poly1305-AES) on every blob/file plus SHA-256 content addressing; Borg 2 uses AEAD (AES-OCB or ChaCha20-Poly1305) with chunk-ID-as-AAD plus layered checksums; Kopia uses AEAD (AES-GCM or ChaCha20-Poly1305) with HMAC-derived per-content keys plus optional Reed-Solomon ECC — none provide parity by default.
|
||||
|
||||
### Cited Findings
|
||||
- Restic envelope: all files except `keys/` (and pack containers) are `IV(16)||CIPHERTEXT||MAC(16)` (32 B overhead, random IV per file); pack files hold multiple independently encrypted/authenticated blobs + encrypted header + LE `Header_Length`; primitives AES-256-CTR + Poly1305-AES, Encrypt-then-MAC (MAC over ciphertext), keys via scrypt KDF → 32 B enc key + 32 B MAC key (`k`+`r`) unlocking master keys in `keys/` JSON — [Source](https://restic.readthedocs.io/en/stable/100_references.html); audit summary — [Source](https://words.filippo.io/restic-cryptography/)
|
||||
- Restic content integrity: filenames are hex SHA-256 of file ciphertext (verifiable with `sha256sum`); trees/data addressed by SHA-256 of plaintext with deterministic JSON for trees; `restic check` verifies structure plus optional `--read-data` payload reads; tampered data fails MAC and is not decrypted — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic snapshots/manifests: snapshots are JSON (`time/tree/paths/hostname/...`) stored as unpacked encrypted files (v2: version-byte + zstd then `IV||C||MAC`), filename = storage ID; snapshot→tree→blob DAG; no separate manifest signature — authentication is the per-file MAC + content-hash reference; write-order rules (packs → index → snapshot; read reverse) keep repo consistent — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic has no parity/ECC; repair is `rebuild-index` + re-backup + `check`; threat model explicitly excludes deletion protection — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Borg 2 AEAD modes: `--encryption aes256-ocb|chacha20-poly1305` × `--id-hash sha256|blake3`, orthogonal to `--key-location repokey|keyfile`; per-session random `sessionid`, `session_key` via SHA-256 KDF, counter IVs; each object has two separately encrypted slots (metadata + data) behind unencrypted header (`OBJ_MAGIC||version||chunk_id||meta_size||data_size`); `AAD = header||slot_tag||id||...`, tag authenticates metadata, payload, header prefix and chunk ID — [Source](https://borgbackup.readthedocs.io/en/latest/internals/security.html)
|
||||
- Borg chunk-ID binding: `id = MAC(id_key, plaintext)`; AEAD tag binds ID to ciphertext, so repo cannot swap content under an ID; post-decrypt `id == MAC(decompressed)` check is optional by default (only detects malicious client writes) but enforced by `borg check --verify-data` / `BORG_ASSERT_ID` — [Source](https://borgbackup.readthedocs.io/en/latest/internals/security.html)
|
||||
- Borg manifest/archive authentication (Horton principle DAG): every object referenced by parent ID up to manifest; borg2 stores `ro_type` in metadata and verifies expected vs. actual type, binding meaning + ID via AAD; no TAM in borg2; borg 1.x used TAM (`HKDF-SHA-512(id_key||enc_key||enc_hmac_key, 64 B salt, "borg-metadata-authentication-manifest")` → `HMAC` over packed manifest) to anchor fixed-ID manifest (CVE-2016-10099) — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg non-encrypting modes: `authenticated-sha256|blake3` carry keyed MAC (not encryption) over payload+header+slot; `none-*` carry only unkeyed checksums (accidental-corruption detection, no tamper protection); legacy 1.x modes were AES-CTR Encrypt-then-MAC with IV reservation; key blobs wrapped by argon2-derived KEK + chacha20-poly1305 (IV 0) — [Source](https://borgbackup.readthedocs.io/en/latest/internals/security.html)
|
||||
- Borg layered checksums: `IntegrityCheckedFile` streaming checksums for cache/index/hints with `integrity.<TXN>` msgpack files and `[integrity]` cache-config section; corrupt index/hints are deleted and rebuilt; `config/config` (version/id/encryption/id_hash) is plaintext/unauthenticated, protected instead by client security-dir `EncryptionMethodMismatch` checks — [Source](https://borgbackup.readthedocs.io/en/latest/internals/data-structures.html)
|
||||
- Kopia content encryption: default `AES256-GCM-HMAC-SHA256` (alt `CHACHA20-POLY1305-HMAC-SHA256`), set at `repository create` and immutable after; per-content AEAD keys derived via HMAC-SHA256; blocks packed into 20 MB packs (20–40 MB on wire) with local index at pack end + top-level index mapping block ID → (blob, offset, length) — [Source](https://kopia.io/docs/advanced/architecture/); cipher history (deprecated unauthenticated AES-CTR/SALSA20) — [Source](https://github.com/kopia/kopia/pull/277)
|
||||
- Kopia format/manifest envelope: `kopia.repository` format blob holds `uniqueID` (salt), `keyAlgo scrypt-65536-8-1`, `encryption AES256_GCM`, `encryptedBlockFormat` = JSON (`ContentFormat{version,hash,encryption,HMACSecret 32 B,MasterKey 32 B,MaxPackSize 20 MiB}` + `ObjectFormat{splitter}`) encrypted with passphrase-derived `Km=PBKDF(pass,uniqueID)`, `Ke=HKDF(SHA256,Km,uniqueID,"AES",32)`, `AD=HKDF(...,"CHECKSUM",32)`; snapshots/manifests are ordinary encrypted contents in the same CABS/object layers — [Source](https://kopia.io/docs/advanced/encryption/); defaults (`--block-hash BLAKE2B-256-128`, `--object-splitter DYNAMIC-4M-BUZHASH`, `--encryption AES256-GCM-HMAC-SHA256`) — [Source](https://kopia.io/docs/reference/command-line/common/repository-create-filesystem/)
|
||||
- Kopia extra redundancy: experimental Reed-Solomon `REED-SOLOMON-CRC32` ECC with `--ecc-overhead-percent` (default 0 = disabled); must be enabled at creation, cannot be added later; cloud backends already ECC-protected so often redundant — [Source](https://kopia.io/docs/advanced/ecc/); consistency/verify docs cover validity checks and repair — [Source](https://kopia.io/docs/advanced/architecture/)
|
||||
|
||||
### Inferences
|
||||
- All three authenticate snapshots/manifests the same way as data (no detached signatures); trust anchors are the client-held keys (restic master keys, Borg key+TAM/AAD chain, Kopia format-block passphrase), giving a key-anchored DAG from manifest to chunks.
|
||||
- Only Kopia offers built-in parity (ECC); restic/Borg rely on backend durability + authenticated detection and rebuild/re-upload repair.
|
||||
|
||||
### Gaps
|
||||
- Kopia manifest-object specifics (manifest content type, index-epoch authentication, `kopia snapshot verify` coverage) were not fetched; snapshot authentication is inferred from “all contents encrypted/authenticated” architecture.
|
||||
- Exact Borg `MAX_DATA_SIZE`/obfuscation interaction with AEAD tag and `BORG_ASSERT_ID` default scope need code-level confirmation beyond docs snippets.
|
||||
|
||||
## Order of operations: compress-then-encrypt vs alternatives, and why
|
||||
|
||||
### Takeaway
|
||||
All three do hash → compress → encrypt (dedup first, compress second, encrypt last); encrypting last is mandatory because ciphertext is incompressible and randomized, while hashing/compressing first preserves dedup and ratio.
|
||||
|
||||
### Cited Findings
|
||||
- Restic code order in `saveAndEncrypt`: plaintext hash for ID/dedup (`saveBlob` → `Hash(buf)`) → zstd compress (if v2 and enabled) → random nonce → `Seal` (AES-CTR + Poly1305 MAC) → pack; unpacked files: `compressUnpacked` (prepend version byte + zstd) → `Seal` — [Source](https://github.com/restic/restic/blob/master/internal/repository/repository.go); PR states “Unpacked files like lock, index and snapshot files are also compressed before encryption” — [Source](https://github.com/restic/restic/pull/3666)
|
||||
- Restic why: backing up pre-compressed (e.g. `.gz`) data defeats CDC dedup because small input changes avalanche through the compressor and shift cut points; project advises backing up uncompressed data and letting restic chunk-then-compress per blob; chunk-then-compress discussion — [Source](https://github.com/restic/restic/issues/790); zstd-dictionary debate concludes no dictionary needed precisely because compression runs after chunking/dedup — [Source](https://github.com/restic/restic/issues/3775)
|
||||
- Borg order: `id = MAC(id_key, data)` → `compressed = compress(data)` → `AEAD_encrypt(session_key, iv, compressed, aad=id||header)` (borg2); legacy: `id=AUTH(data)` → `compress` → `AES-CTR` → `MAC(encrypted)` — [Source](https://borgbackup.readthedocs.io/en/latest/internals/security.html); docs state “Compression is applied after deduplication, thus using different compression methods in one repo does not influence deduplication” — [Source](https://manpages.debian.org/trixie/borgbackup2/borg2-compression.1.en.html)
|
||||
- Kopia order: “splits into chunks → hash → compare → if new, compress chunk → encrypt → pack” — [Source](https://kopia.io/docs/advanced/compression/); content-level compression PR explicitly rewired pipeline from `read → split → compress → hash → store` to `read → split → hash → compress → store` to stop compressor-version drift from changing IDs and breaking dedup and to enable server-side/recompression — [Source](https://github.com/kopia/kopia/pull/1076)
|
||||
- General rationale shared across docs: encryption must be last because (a) compressed ciphertext does not compress, (b) randomized encryption (IV/session key) destroys dedup if hashed after, (c) MAC/AEAD must cover the stored (compressed) bytes; compressing whole files before chunking is measurably worse than chunk-then-compress (Kopia split-vs-whole table: s2 28.6% vs 25.6%, gzip 18.8% vs 17.3% — small loss accepted for dedup wins) — [Source](https://kopia.io/docs/advanced/compression/)
|
||||
|
||||
### Inferences
|
||||
- The universal pattern is dedup-on-plaintext → compress → authenticated-encrypt; any deviation (pre-compressed inputs, compress-after-encrypt, hash-after-compress) is documented as an anti-pattern in all three projects’ issues/docs.
|
||||
- Per-chunk compression inherently sacrifices cross-chunk dictionary context; all three accept slightly worse ratios in exchange for shift-resilient dedup and independent integrity domains.
|
||||
|
||||
### Gaps
|
||||
- No source quantified restic’s incompressible-blob shortcut (whether zstd `EncodeAll` output larger than input is stored raw vs. kept); forum claims “stores raw data” but code path stores `uncompressedLength` flag — exact skip-threshold logic needs code confirmation.
|
||||
@@ -0,0 +1,147 @@
|
||||
# Backup encryption security choices
|
||||
|
||||
## Which cipher and mode does each use? How are nonces/IVs generated and stored?
|
||||
|
||||
### Takeaway
|
||||
Restic uses non-standard AES-256-CTR + Poly1305-AES (Encrypt-then-MAC) with random 16-byte IV per file/blob; Borg 1.x uses AES-256-CTR + HMAC-SHA256 or keyed BLAKE2b-256 with 64-bit counters tracked by reservation, while Borg 2 moves to AEAD AES256-OCB and ChaCha20-Poly1305; Kopia uses AES256-GCM-HMAC-SHA256 by default (or CHACHA20-POLY1305-HMAC-SHA256) with per-content keys and random IVs; Duplicity delegates entirely to external GnuPG/OpenPGP (symmetric passphrase or public-key session-key hybrid).
|
||||
|
||||
### Cited Findings
|
||||
- Restic: “All data stored by restic in the repository is encrypted with AES-256 in counter mode and authenticated using Poly1305-AES” — [Source](https://github.com/restic/restic/blob/master/doc/design.rst)
|
||||
- Restic: “For encrypting new data first 16 bytes are read from a cryptographically secure pseudo-random number generator as a random nonce. This is used both as the IV for counter mode and the nonce for Poly1305” — [Source](https://github.com/restic/restic/blob/master/doc/design.rst)
|
||||
- Restic: format is “IV || CIPHERTEXT || MAC”, “complete encryption overhead is 32 bytes. For each file, a new random IV is selected”, 16-byte IV stored first, 16-byte MAC last — [Source](https://restic.readthedocs.io/en/v0.4.0/Design)
|
||||
- Restic: needs three keys: “a 32-byte key for AES-256 encryption, a 16-byte AES key and a 16-byte key for Poly1305”, the last 32 bytes split into 16-byte AES key `k` + 16-byte `r` then masked for Poly1305 per Bernstein paper — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic: pack files contain multiple independently encrypted/authenticated blobs plus encrypted header + 4-byte little-endian header length; blob types 0b00 data, 0b01 tree, 0b10/0b11 compressed data/tree (repo format v2, zstandard) — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic: passwords wrapped via scrypt KDF (`N`, `r`, `p`, `salt`); example `N=65536, r=8, p=1`, also observed `N=32768, r=8, p=5` in 2026 docs; derived 64 bytes split into 32-byte AES key + 32-byte MAC key; multiple key files per repo allow password change without re-encrypting data — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic current as of 2026 is v0.19.1 (5 Jul 2026) / v0.19.0, repo format version 1 or 2, v2 adds compression — [Source](https://restic.net/)
|
||||
- Borg 1.x: “repokey and keyfile use AES-CTR-256 for encryption and HMAC-SHA256 for authentication in an encrypt-then-MAC (EtM) construction” — [Source](https://manpages.debian.org/bullseye/borgbackup/borg-init.1.en.html)
|
||||
- Borg 1.x blake2 variants: “repokey-blake2 and keyfile-blake2 are also authenticated encryption modes, but use BLAKE2b-256 instead of HMAC-SHA256”, chunk ID is keyed BLAKE2b-256 — [Source](https://manpages.debian.org/bullseye/borgbackup/borg-init.1.en.html)
|
||||
- Borg chunk header: “TYPE(1) + HMAC(32) + NONCE(8) + CIPHERTEXT. Encryption and HMAC use two different keys” — [Source](https://borgbackup.readthedocs.io/en/1.0-maint/internals.html)
|
||||
- Borg CTR IV: “A 64bit initialization vector is used”, “only 8 bytes of the 16 bytes nonce is saved in the payload, the first 8 bytes are always zeros”, limits capacity to 2**64 * 16 bytes (~295 exabytes) — [Source](https://borgbackup.readthedocs.io/en/1.0-maint/internals.html)
|
||||
- Borg IV uniqueness via reservation: “initializes the encryption counter to be higher than any previously used counter value”, commits reservation by “taking the current counter value and adding 4 GiB / 16 bytes to the counter”, persisted via SaveFile to security DB + repository before encrypting — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg pseudocode: `iv = reserve_iv()`, `encrypted = AES-256-CTR(enc_key, 8-null-bytes || iv, compressed)`, `authenticated = type-byte || AUTHENTICATOR(enc_hmac_key, encrypted) || iv || encrypted` — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg offline key wrapping: “256 bit key encryption key (KEK) is derived from the passphrase using PBKDF2-HMAC-SHA256 with a random 256 bit salt”, then Encrypt-and-MAC with “AES-256-CTR with a constant initialization vector of 0”, base64 keyblob in keyfile or repo config — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg stable in 2026 is 1.4.5; crypto section still states “actual encryption is currently always AES-256 in CTR mode” for 1.x line — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg 2 (beta 2.0.0b22/b23 as of 2026): modes `aes256-ocb`, `chacha20-poly1305`, `authenticated-sha256/blake3`, `none-sha256/blake3`; “AES256 in OCB mode (encryption + authentication)” and “ChaCha20 + Poly1305” — [Source](https://borgbackup.readthedocs.io/en/latest/usage/repo-create.html)
|
||||
- Borg 2: “All data can be protected client-side using 256-bit authenticated encryption (AES-OCB or chacha20-poly1305)” — [Source](https://github.com/borgbackup/borg)
|
||||
- Borg 2 goals: “get rid of AES-CTR mode and use ‘session keys’”, “use more modern / faster AEAD ciphers: AES-OCB and chacha20-poly1305”, “use a more modern KDF: argon2” — [Source](https://github.com/borgbackup/borg/wiki/Borg-2.0)
|
||||
- Kopia default: `const DefaultAlgorithm = "AES256-GCM-HMAC-SHA256"` — [Source](https://pkg.go.dev/github.com/kopia/kopia/repo/encryption)
|
||||
- Kopia options: “By default, Kopia uses the AES256-GCM-HMAC-SHA256 encryption algorithm … but you can choose CHACHA20-POLY1305-HMAC-SHA256”, immutable after repo creation, selected via `--encryption=` or Advanced Options — [Source](https://github.com/kopia/kopia/blob/master/site/content/docs/FAQs/_index.md)
|
||||
- Kopia per-content keys: registered as “AES-256-GCM using per-content key generated using HMAC-SHA256”, derives key via `deriveKey(p, purposeEncryptionKey)` + `hmac.New(sha256.New, keyDerivationSecret)` pool, overhead 28 bytes — [Source](https://github.com/kopia/kopia/blob/master/repo/encryption/aes256_gcm_hmac_sha256_encryptor.go)
|
||||
- Kopia format blob: `encryption` field (default `AES256_GCM`), `encryptedBlockFormat` = JSON `EncryptedRepositoryConfig` encrypted with random IV prepended, key `Ke = HKDF(SHA256, Km, UniqueID, "AES", 32)` where `Km = PBKDF(passphrase, UniqueID)` via scrypt `N=65536, r=8, p=1`, AD = `HKDF(SHA256, Km, UniqueID, "CHECKSUM", 32)` — [Source](https://kopia.io/docs/advanced/encryption/)
|
||||
- Kopia `ContentFormat` holds `Hash`, `Encryption`, `HMACSecret`, `MasterKey (SIV-mode only)`, `MaxPackSize`; repository config stored encrypted because it contains key material — [Source](https://kopia.io/docs/advanced/encryption/)
|
||||
- Kopia observed version v0.23.1 in Go docs (2026 architecture doc updated Feb 2026) — [Source](https://pkg.go.dev/github.com/kopia/kopia/repo/encryption)
|
||||
- Duplicity: “incrementally backs up files and folders into tar-format volumes encrypted with GnuPG” — [Source](https://man.archlinux.org/man/duplicity.1.en)
|
||||
- Duplicity key modes: `--encrypt-key key` (public-key, repeatable), `--hidden-encrypt-key` uses `gpg --hidden-recipient` to hide recipient key ID, `--sign-key`, symmetric via `PASSPHRASE` env, `--gpg-options`, `--gpg-binary` — [Source](https://man.archlinux.org/man/duplicity.1.en)
|
||||
- Duplicity relies on OpenPGP hybrid: random session key `s`, `Enc_ri(s)` per recipient + `Enc_s(data)` — [Source](https://gnupg.org/ftp/blurbs/an-advanced-introduction-to-gnupg.pdf)
|
||||
- Duplicity current lineage includes 2.x/3.x (Debian bookworm-backports, Arch manuals); backend still GnuPG + librsync incremental — [Source](https://manpages.debian.org/bookworm-backports/duplicity/duplicity.1.en.html)
|
||||
|
||||
### Inferences
|
||||
- Restic’s construction is bespoke AES-CTR + Poly1305-AES (not AES-GCM or ChaCha20-Poly1305); random IV per encryption avoids counter-reuse tracking at cost of 16-byte IV storage and reliance on CSPRNG.
|
||||
- Borg 1.x’s 8-byte stored counter is the scaling bottleneck that motivated Borg 2 session keys + AEAD nonces; Borg 2 OCB vs ChaCha choice is performance-driven (hardware AES vs portable).
|
||||
- Kopia’s “-HMAC-SHA256” suffix denotes per-content key derivation + outer HMAC layer, not just plain GCM; format-blob AD binding to passphrase-derived Km prevents config swapping without password.
|
||||
- Duplicity inherits whatever GnuPG/OpenPGP negotiates (modern GnuPG 2.x defaults to AES-256 + MDC/AEAD), so its cipher agility is a GnuPG property, not a Duplicity design choice.
|
||||
|
||||
### Gaps
|
||||
- Exact GnuPG symmetric cipher/MDC vs AEAD (OCB/EAX) negotiated by Duplicity 3.x in 2026 not confirmed from Duplicity docs; depends on installed GnuPG version and `--gpg-options`.
|
||||
- Kopia nonce length/placement for content blobs (vs format blob) not fully detailed in fetched docs; source states overhead 28 bytes but byte layout not quoted.
|
||||
- Borg 2 session-key derivation and nonce construction details not yet in stable internals doc (beta docs only list mode names).
|
||||
|
||||
## What is encrypted vs plaintext (content, filenames, metadata, manifests/snapshots)?
|
||||
|
||||
### Takeaway
|
||||
Restic, Borg (in encrypted modes) and Kopia encrypt file contents, filenames/paths, metadata and snapshots/manifests client-side; only key-file wrappers, config envelopes, outer pack/segment sizes, timestamps and directory layout remain visible. Duplicity encrypts tar volumes including embedded filenames + manifest/sigtar, but volume filenames, sizes and increment metadata stay plaintext. None hide sizes, counts or access patterns.
|
||||
|
||||
### Cited Findings
|
||||
- Restic guarantee: “Unencrypted content of stored files and metadata cannot be accessed without a password … Everything except the metadata included for informational purposes in the key files is encrypted and authenticated. The cache is also encrypted to prevent metadata leaks” — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic snapshots are “JSON document … stored in a file below snapshots”, encrypted as `IV || Ciphertext || MAC` (v1 JSON, v2 `encoding_version || zstd(JSON)`); example snapshot contains `paths, hostname, username, uid/gid, tags, tree` only after decryption — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic trees contain `name, type, mode, mtime/atime/ctime, uid/gid/user, inode, size, content[plaintext hashes], subtree, linktarget` encrypted inside tree blobs — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic index JSON (`packs[].blobs[].id/type/offset/length`) and pack headers (plaintext hashes, offsets) are encrypted; only after decryption are blob IDs visible — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic config (`version, id, chunker_polynomial`) is encrypted; `keys/` files expose only informational `hostname, username, created, kdf params, salt` + encrypted `data` wrapping master keys — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Borg attack model: client trusted, repository/server untrusted with full read/write/MITM; guarantees attacker cannot modify data, rename/remove/add archive, recover plaintext, or recover definite structural info (object graph) undetected — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg: “authenticated encryption technique makes it suitable for backups to targets not fully trusted” — [Source](https://manpages.debian.org/bookworm/borgbackup2/borg2.1.en.html)
|
||||
- Borg authenticated-only modes (`authenticated-sha256/blake3`) provide tamper detection without confidentiality; `none` modes provide neither — [Source](https://borgbackup.readthedocs.io/en/latest/usage/repo-create.html)
|
||||
- Borg compression happens before encryption: `compressed = compress(data)` then encrypt-then-MAC; optional `obfuscate` pseudo-compressor pads with 0x00 before encryption to hide compressed sizes — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Kopia: “Encryption is at the repository level, and Kopia encrypts all snapshots in all repositories by default” via repository password — [Source](https://github.com/kopia/kopia/blob/master/site/content/docs/FAQs/_index.md)
|
||||
- Kopia layers: BLOB store holds packs; CABS encrypts blocks after hashing (`AES256-GCM-HMAC-SHA256` or `CHACHA20-POLY1305-HMAC-SHA256`); CAOS directory listings (`k` prefix), manifests (`m`), indirect JSON (`x`) are CABS blocks; LAMS manifests (snapshots, policies) stored as CABS blocks — [Source](https://kopia.io/docs/advanced/architecture/)
|
||||
- Kopia: “Pack files in blob storage have random names and don’t reveal anything about their contents or structure. Their sizes are also generally unrelated to their content due to splitting and merging” — [Source](https://kopia.io/docs/advanced/architecture/)
|
||||
- Kopia blob prefixes `p` (data packs), `q` (metadata packs), `x` (indices), object prefixes `k/m/x`, `I` virtual indirection — visible types but not contents — [Source](https://kopia.io/docs/advanced/architecture/)
|
||||
- Duplicity volumes are gzipped tar + GnuPG; OpenPGP literal-data packet (filename, mode, mtime) is inside encryption, so embedded names hidden, but outer `duplicity-full|inc` volume filenames, `manifest.gpg`/`sigtar.gpg` names, sizes and S3 storage-class split (manifest/sigtar on Standard for quick retrieval, data on Glacier/Deep Archive) are server-visible — [Source](https://man.archlinux.org/man/duplicity.1.en)
|
||||
- Duplicity over S3 with `--s3-use-glacier` notes: “Duplicity will store the manifest.gpg and sigtar.gpg files … on AWS S3 standard storage … all other data is stored in S3 Glacier” — [Source](https://man.archlinux.org/man/duplicity.1.en)
|
||||
- Duplicity `--no-encryption` writes only gzipped volumes (no confidentiality) — [Source](https://linux.die.net/man/1/duplicity)
|
||||
|
||||
### Inferences
|
||||
- For Restic/Borg/Kopia, filenames, symlink targets, ownership, timestamps and snapshot manifests are as confidential as file bytes; server compromise reveals at most sizes/counts/timing, not names.
|
||||
- Kopia’s p/q/x prefix separation intentionally leaks data-vs-metadata partitioning to aid storage management while hiding actual object graphs.
|
||||
- Duplicity’s S3 Glacier optimization trades metadata confidentiality (manifests pinned to Standard, easily listable) for cheaper restores.
|
||||
|
||||
### Gaps
|
||||
- Whether Borg archive names themselves (manifest entries) are fully encrypted vs length-leaking was not explicitly quoted; inferred encrypted from manifest DAG but needs source confirmation.
|
||||
- Kopia policy/manifest label (key=value) visibility to server (for server-managed repos) not clarified in fetched docs.
|
||||
- Duplicity filename encryption inside OpenPGP (literal packet) assumed from GnuPG behavior, not Duplicity doc; no Duplicity doc explicitly states embedded names are hidden.
|
||||
|
||||
## How are object names derived (plaintext hash, HMAC, random IDs) and what does that leak to an untrusted server?
|
||||
|
||||
### Takeaway
|
||||
Restic outer filenames are SHA-256 of ciphertext (verifiable via sha256sum) while inner blob references are SHA-256 of plaintext hidden in encrypted index/headers; Borg chunk IDs are HMAC/keyed hashes of plaintext (dedup without revealing content); Kopia content IDs are hashes/HMACs of plaintext with secret, packed into randomly named blobs; Duplicity names are sequential timestamps leaking backup chain. All leak dedup equality, counts, sizes and access patterns to varying degrees.
|
||||
|
||||
### Cited Findings
|
||||
- Restic: “storage ID is the SHA-256 hash of the content of a file”, filename is “lower case hexadecimal representation of the storage ID”, verifiable by running `sha256sum` on file and comparing to filename — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic: “All content … is referenced according to its SHA-256 hash”, file split into CDC blobs, “SHA-256 hashes of all Blobs are saved in ordered list”, tree `content` holds plaintext hashes, `subtree` holds plaintext tree ID — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic: index/pack header “yields all plaintext hashes, types, offsets and lengths”, pack `id` in index is outer pack hash; `restic cat pack` verifies hash and warns on mismatch — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic data/keys layout: `data/XX/<full-hash>`, `snapshots/<id>`, `index/<id>`, `keys/<id>`, `locks/`, `tmp/`; snapshot filename is storage ID — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic attacker with read access can “Infer which packs probably contain trees via file access patterns”, “Infer the size of backups by using creation timestamps”, and pre-0.18.0 could derive chunker polynomial from observed chunk sizes per 2025 IACR paper; 0.18.0 mitigates by “randomly assigning chunks to pack files” — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Borg: “key = id = id_hash(unencrypted_data)”, `id_hash` is `sha256` (no keys) or `hmac-sha256` (with keys); must be cryptographically strong for dedup — [Source](https://borgbackup.readthedocs.io/en/1.0-maint/internals.html)
|
||||
- Borg 1.x: chunk ID `id = AUTHENTICATOR(id_key, data)` with independent `id_key` vs `enc_hmac_key`; decryption asserts `CONSTANT-TIME-COMPARISON(chunk-id, AUTHENTICATOR(id_key, decompressed))` — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg 2: “A chunk is considered duplicate if its id_hash value is identical”, `id_hash` e.g. “(hmac-)sha256 or (keyed) blake3”, selectable via `--id-hash=blake3` — [Source](https://github.com/borgbackup/borg)
|
||||
- Borg manifest has fixed ID `000…000`, anchored via TAM: `tam_key = HKDF-SHA-512(ikm=id_key||enc_key||enc_hmac_key, salt, "borg-metadata-authentication-manifest")`, `HMAC(tam_key, packed)` — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg fingerprinting: “repository does not hide size of chunks”, buzhash chunking uses “secret, random per-repo chunker seed”, small files <512 KiB yield single chunk; attacker with candidate files could brute-force fingerprint by sizes; mitigations: chunker choice/params, secret seed, compression choice, `obfuscate` padding; proximity/order (inode-order scan, segment adjacency) may leak additional info — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Kopia: “Block ID is generated by applying cryptographic hash function such as SHA2 or BLAKE2S”, identical blocks yield identical IDs for natural dedup; after hashing, block encrypted; multiple blocks merged into 20-40MB packs; index maps block ID -> (blob name, offset, length) — [Source](https://kopia.io/docs/advanced/architecture/)
|
||||
- Kopia: pack blob names random (e.g. `pb4cf8…`, `q7a99…`, `xn0_20db…`); content list/show via `kopia content list/show` requires decryption — [Source](https://kopia.io/docs/advanced/architecture/)
|
||||
- Kopia format `UniqueID` is 32 random bytes, also PBKDF salt and HKDF info, preventing cross-repo ID correlation — [Source](https://kopia.io/docs/advanced/encryption/)
|
||||
- Duplicity sets: “files in full backup sets will start with duplicity-full while the incremental sets start with duplicity-inc”, ordered patches; deleting full invalidates dependent incrementals — [Source](http://duplicity.nongnu.org/vers7/duplicity.1.html)
|
||||
- Duplicity S3 observer can tell “that you are using Duplicity, the name of the bucket, your AWS Access Key ID, the increment dates and the amount of data in each increment” (affects connection, not GPG payload) — [Source](https://linux.die.net/man/1/duplicity)
|
||||
- Duplicity default recipient key IDs visible in OpenPGP packets unless `--hidden-encrypt-key` (`--hidden-recipient`) used; restore then tries all secret keys — [Source](https://man.archlinux.org/man/duplicity.1.en)
|
||||
|
||||
### Inferences
|
||||
- Restic outer names (ciphertext hashes) look random to server; inner plaintext hashes never appear plaintext on server, so equality of files is hidden unless attacker correlates sizes/access patterns — unlike Borg/Kopia where HMAC IDs are server-visible keys enabling dedup-equality oracle.
|
||||
- Borg/Kopia HMAC-ID design intentionally reveals equality (required for server-side dedup) but not content; without `id_key`/`HMACSecret` server cannot confirm guesses except via size/proximity fingerprinting.
|
||||
- Duplicity leaks the most metadata: full vs incremental, chain order, timestamps and volume counts directly encode backup history even when payloads are opaque.
|
||||
|
||||
### Gaps
|
||||
- No reliable source confirming whether Restic pack filename is hash of ciphertext vs plaintext; doc says hash of content but pack verification suggests ciphertext — ambiguity noted.
|
||||
- Kopia whether content IDs are plain hash vs HMAC-SHA256 with `HMACSecret` not fully resolved: architecture says hash, `ContentFormat.HMACSecret` implies keyed; exact construction needs source read.
|
||||
- Borg 2 object store layout (keys/ namespace, segment naming) not fetched; assumed similar to 1.x but with AEAD.
|
||||
|
||||
## Any documented rationale for their choices (audit reports, design docs)?
|
||||
|
||||
### Takeaway
|
||||
Restic cites simplicity + Encrypt-then-MAC robustness and documents threat model + 2025 chunking-attack mitigation; Borg cites Encrypt-then-MAC robustness, Horton principle + TAM (CVE-2016-10099), and Borg 2 session-key/AEAD/Argon2 motivations; Kopia cites envelope encryption + per-content keys + random packing; Duplicity cites delegation to GnuPG/librsync. No Cure53-style audit report was found for these four in the searches; strongest external reviews found are Filippo Valsorda’s Restic note and Borg issue discussions.
|
||||
|
||||
### Cited Findings
|
||||
- Restic design: “Encryption is a first-class feature, the implementation looks sane and I guess the deduplication trade-off is worth it” — Filippo Valsorda, quoted in Restic encryption docs — [Source](https://github.com/restic/restic/blob/master/doc/070_encryption.rst)
|
||||
- Restic threat model: trusted client + authentic restic + secret password; guarantees vs assumptions listed; “Advances … against … (AES-256-CTR-Poly1305-AES and SHA-256) have not occurred”, brute-force infeasible, leaked key requires full re-encryption via `copy`/new repo — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic 2025 chunking attack: paper “Chunking Attacks on File Backup Services using Content-Defined Chunking” by Alexeev/Percival/Zhang, mitigated in 0.18.0 by random pack assignment; random irreducible CDC polynomial stored in encrypted `config` to harden watermark attacks — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Restic CDC: Rabin fingerprints, 64-byte window, files <512 KiB unsplit, 512 KiB–8 MiB blobs targeting 1 MiB average — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- Borg: “Encryption is currently based on the Encrypt-then-MAC construction, which is generally seen as the most robust way” — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg: follows “Horton principle … not only the message must be authenticated, but also its meaning”, object ID MACs plaintext, parent reference assigns meaning, DAG anchored by TAM — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg TAM added since 1.0.9 for “Pre-1.0.9 manifest spoofing vulnerability (CVE-2016-10099)” — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg: “Borg does not support unauthenticated encryption — only authenticated encryption … No unauthenticated schemes will be added” — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg multi-client caveat: with multiple independent clients on same repo, “Borg fails to provide confidentiality” due to counter-reservation replay; trusted sync channel needed — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg compression+encryption discussion in issue #1040 concluded “no problem at all” to “hard and extremely slow to exploit”; user can disable compression — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg primitives rationale: HMAC-SHA256 vs BLAKE2b chosen on SHA hardware support (Ryzen, Intel 10th+ mobile/11th+ desktop, M1+, ARM64 SHA ext favour HMAC-SHA256; 64-bit CPUs without SHA ext favour BLAKE2b); uses OpenSSL libcrypto only, not libssl/TLS/X.509 — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg 2 rationale: global AES+MAC key requires perfect counter tracking across clients/threads (XOR-plaintext leak risk + sync complexity); session keys ease multithreading — [Source](https://github.com/borgbackup/borg/wiki/Borg-2.0)
|
||||
- Borg keyed chunkers (`toeplitz-aes/rabin-aes/goldilocks-aes`) rationale: “make chunk-boundary fingerprinting attacks much harder” — [Source](https://github.com/borgbackup/borg)
|
||||
- Kopia rationale: “standard envelope encryption technique to de-couple the repository passphrase from keys used for encrypting/authenticating contents”; format blob holds params, `UniqueID` random per repo, scrypt PBKDF + HKDF-SHA256 for Ke/AD — [Source](https://kopia.io/docs/advanced/encryption/)
|
||||
- Kopia FAQ: encryption mandatory, algorithm fixed at creation, password unrecoverable — [Source](https://github.com/kopia/kopia/blob/master/site/content/docs/FAQs/_index.md)
|
||||
- Duplicity rationale: uses librsync for space-efficient incrementals + GnuPG so “they will be safe from spying and/or modification by the server” — [Source](https://packages.debian.org/bullseye/arm64/utils/duplicity)
|
||||
- Duplicity signing+symmetric-encrypt via CLI gpg noted as “specifically challenging”, only certain passphrase/agent combos tested working — [Source](http://duplicity.nongnu.org/vers7/duplicity.1.html)
|
||||
|
||||
### Inferences
|
||||
- Restic/Borg explicitly prefer Encrypt-then-MAC over AEAD for 1.x-era compatibility and library constraints (OpenSSL libcrypto, Go stdlib); Borg 2 and Kopia converge on modern AEAD (OCB/GCM/ChaCha20-Poly1305) once widely available.
|
||||
- Borg’s extensive fingerprinting/TAM documentation is the most explicit untrusted-server rationale among the four; Restic’s 2025 mitigation shows similar concerns emerging later.
|
||||
- Duplicity offloads rationale to GnuPG/OpenPGP; its docs contain no cipher-selection rationale beyond backend/storage-class notes.
|
||||
|
||||
### Gaps
|
||||
- No Cure53 or similar third-party audit report for restic/Borg/Kopia/Duplicity found in searches (results returned unrelated Ente/Project11/Monocypher audits); if audits exist, they were not retrievable with queries used.
|
||||
- Kopia independent security review or design-rationale doc beyond envelope-encryption page not found.
|
||||
- Duplicity successor scope (e.g. duplicacy, duplicity 3.x forks) not clarified; findings cover classic Duplicity only.
|
||||
@@ -0,0 +1,114 @@
|
||||
# Backup encryption security choices: key management
|
||||
|
||||
## How is the master key generated and derived from a password (scrypt, Argon2, PBKDF2 parameters)?
|
||||
|
||||
### Takeaway
|
||||
All five systems use password-derived envelope encryption with a random master/data key, but KDFs and parameters differ: restic pins scrypt N=65536/r=8/p=1 (auto-calibrated on newer adds); Borg 1.x uses PBKDF2-HMAC-SHA256 while Borg 2.x defaults to Argon2id + ChaCha20-Poly1305; Kopia uses scrypt-65536-8-1 over UniqueID salt plus HKDF-SHA256 subkeys (PBKDF2/scrypt configurability added ~2025); rclone crypt uses scrypt N=16384/r=8/p=1; Tarsnap generates keys locally and only uses scrypt to passphrase-wrap the key file.
|
||||
|
||||
### Cited Findings
|
||||
- restic: all repo data encrypted with AES-256-CTR + Poly1305-AES MAC as `IV || CIPHERTEXT || MAC` (32 bytes overhead, random 16-byte nonce per file); three keys needed (32-byte AES-256 key + 16-byte AES key + 16-byte Poly1305 key) — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- restic: on open, password + per-key `N`, `r`, `p`, `salt` feed scrypt to derive 64 bytes; first 32 = AES-256 encryption key, last 32 split into 16-byte AES key `k` + 16-byte Poly1305 key `r` (masked); these decrypt the `data` field to reveal the master key JSON — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- restic: canonical key-file example pins `"kdf": "scrypt", "N": 65536, "r": 8, "p": 1`; code rejects any KDF other than scrypt (`only supported KDF is scrypt()`) — [Source](https://github.com/restic/restic/blob/9e2d60e2/internal/repository/key.go)
|
||||
- restic: `AddKey` calibrates KDF parameters (`crypto.Calibrate(KDFTimeout, KDFMemory)`) when adding keys, generates a random salt (`crypto.NewSalt()`) and either a fresh random master key (`crypto.NewRandomKey()`) or copies the existing master key template — [Source](https://github.com/restic/restic/blob/9e2d60e2/internal/repository/key.go)
|
||||
- Borg 1.x/legacy: key file encrypted with PBKDF2-HMAC-SHA256 (32-byte salt, `PBKDF2_ITERATIONS`), AES-CTR with zero IV plus HMAC-SHA256 encrypt-and-MAC construction — [Source](https://github.com/borgbackup/borg/blob/86fd77fd/src/borg/legacy/crypto/key.py)
|
||||
- Borg 2.x: 256-bit key-encryption key (KEK) derived from passphrase with Argon2 + random 256-bit salt, then Encrypt-then-MAC of packed key material with ChaCha20-Poly1305 AEAD and constant IV==0 (safe because salt makes KEK unique per encryption) — [Source](https://borgbackup.readthedocs.io/en/2.0.0b9/internals/security.html)
|
||||
- Borg: new key files support `argon2 chacha20-poly1305` vs legacy `sha256` (PBKDF2) algorithms; Argon2 path derives 32-byte KEK via `argon2.low_level.hash_secret_raw` with configurable time/memory/parallelism/type — [Source](https://github.com/borgbackup/borg/blob/da3105f1/src/borg/crypto/key.py)
|
||||
- Borg: `--key-algorithm argon2` is default (Argon2id); `pbkdf2` kept for old-client compatibility; docs note Argon2 path also fixes two issues at once (separate encrypt vs MAC keys, encrypt-then-MAC instead of encrypt-and-MAC) — [Source](https://git.uninsane.org/shelvacu-mirrors/borg/commit/08f82ee40867f605ca6994db6dc32218d7b85cbc)
|
||||
- Borg: random Borg key itself consists of three random secrets (crypt key, id key, chunker seed); passphrase only locks/encrypts this key, chunking and IDs also derive from it — [Source](https://borgbackup.readthedocs.io/en/stable/usage/init.html)
|
||||
- Kopia: envelope encryption; format blob carries `uniqueID` (random 32 bytes), `keyAlgo` (e.g. `scrypt-65536-8-1`), `encryption: AES256_GCM`, and `encryptedBlockFormat` holding the real content-encryption secrets (`HMACSecret`, `MasterKey`) — [Source](https://kopia.io/docs/advanced/encryption/)
|
||||
- Kopia: `Km = PBKDF(passphrase, UniqueID, cost params)` (32 bytes) → `Ke = HKDF(SHA256, Km, UniqueID, "AES", 32)` and `AD = HKDF(SHA256, Km, UniqueID, "CHECKSUM", 32)`; format block encrypted with AES256-GCM using Ke, random IV prepended, AD authenticated — [Source](https://kopia.io/docs/advanced/encryption/)
|
||||
- Kopia: only scrypt N=65536/r=8/p=1 documented as supported ("at the moment"), field reserved for future algorithm/cost changes — [Source](https://kopia.io/docs/advanced/encryption/)
|
||||
- Kopia: 2025 PR adds configurable KDF: `--pbkdf (pbkdf2|scrypt)`, `--pbkdf-iter` (default 600000 for PBKDF2), `--pbkdf-memory` (default 64 MB for scrypt), usable at `repo create` and `change-password` — [Source](https://github.com/kopia/kopia/pull/5145)
|
||||
- rclone crypt: file content uses NaCl SecretBox (XSalsa20 + Poly1305), 64 KiB chunks each with 16-byte authenticator, header = 8-byte magic `RCLONE\x00\x00` + 24-byte random nonce incremented per chunk — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt: filename segments PKCS#7-padded to 16 bytes, encrypted deterministically with EME (ECB-Mix-ECB) AES-256, emitted as lowercase unpadded base32 — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt: 80 bytes of key material derived via scrypt `N=16384, r=8, p=1` from password (+ optional `password2` salt); without user salt a built-in internal salt is used — [Source](https://rclone.org/crypt/)
|
||||
- Tarsnap: key files hold authentication keys (server access proofs) and encryption keys (archive encrypt/sign/verify/decrypt) separately, so server compromise does not disclose data — [Source](https://www.tarsnap.com/security.html)
|
||||
- Tarsnap: no password-derived master key by default; keys generated by `tarsnap-keygen`; passphrase protection is optional (`--passphrased`) and wraps the key file with keys from the scrypt KDF, with tunable `--passphrase-mem` / `--passphrase-time` — [Source](https://www.tarsnap.com/man-tarsnap-keygen.1.html)
|
||||
|
||||
### Inferences
|
||||
- restic/Kopia/rclone all standardize on scrypt but with 4x cost difference (16384 vs 65536), so rclone password guessing is cheaper; Borg's Argon2id move is the most modern KDF posture.
|
||||
- Kopia's double-HKDF (separate Ke and AD from one Km) is the cleanest domain separation; restic's split-and-mask of 64 scrypt bytes is older Poly1305-AES style.
|
||||
|
||||
### Gaps
|
||||
- restic's current auto-calibration targets (`KDFTimeout`, `KDFMemory`) were not retrieved; documented example still shows 65536/8/1 but newly added keys may use higher N.
|
||||
- Borg's exact default Argon2 time/memory/parallelism constants (`ARGON2_ARGS`, `ARGON2_SALT_BYTES`) were not captured.
|
||||
- Tarsnap's default scrypt mem/time when `--passphrase-mem/--passphrase-time` are omitted was not found.
|
||||
|
||||
## Where is key material stored (repo config, local key file, OS keyring, TPM)? What is the exact UX for export/backup of keys?
|
||||
|
||||
### Takeaway
|
||||
restic and Kopia keep the (password-locked) key material inside the repository; Borg offers repokey (in repo) vs keyfile (client `~/.config/borg/keys`) as a first-class choice; Tarsnap and rclone keep keys strictly client-side (key file / `rclone.conf`), making client-side backup mandatory. None document TPM; OS-credential integration exists only for Kopia (server/connect password caching) and via external helpers for the rest.
|
||||
|
||||
### Cited Findings
|
||||
- restic: `keys/` directory in repo holds JSON key files (hostname, username, created, kdf params, salt, encrypted `data`); `restic cat masterkey` decrypts and pretty-prints master encryption/MAC keys — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- restic: passwords supplied via prompt, `--password-file`, or `--password-command` (`$RESTIC_PASSWORD_FILE` / `$RESTIC_PASSWORD_COMMAND`); empty passwords refused by default, `--insecure-no-password` required since 0.17.0 — [Source](https://man.archlinux.org/man/restic-key-add.1.en)
|
||||
- Borg 2.x: repokey modes store encrypted key in repo (`repo_dir/config`); keyfile modes store in home dir (`~/.config/borg/keys`); `--key-location` repokey|keyfile chosen at `rcreate`, movable later via `key change-location` — [Source](https://borgbackup.readthedocs.io/en/2.0.0b6/usage/rcreate.html)
|
||||
- Borg 1.x: identical split — repokey inside repo directory, keyfile in `~/.config/borg/keys` (macOS: `~/Library/Application Support/borg/keys`); remote repos via ssh never see passphrase, plaintext key, or plaintext files — [Source](https://borgbackup.readthedocs.io/en/master/usage/repo-create.html)
|
||||
- Borg export UX: `borg key export [PATH]`, `--paper` (printable, per-line checksums for type-in), `--qr-html` (QR + paper copy); export stays passphrase-encrypted (no passphrase included); `borg key import [--paper]` restores; paper-key web tool at `paperkey.html` — [Source](https://borgbackup.readthedocs.io/en/master/usage/key.html); [Source](https://borgbackup.readthedocs.io/en/stable/paperkey.html)
|
||||
- Borg: for keyfile repos the key must be backed up independently ("NOT sufficient" to keep copy on the backed-up system); for repokey a backup is "not strictly needed" but guards against corruption/loss of the in-repo key — [Source](https://man.archlinux.org/man/borg-key-export.1.en)
|
||||
- Kopia: format blob (`kopia.repository` / `kopia.blobcfg` objects) lives in storage; local `repository-*.config` holds connection parameters (not the password); quick-reconnect token via `kopia repository status -t` (opaque) and `-s` embeds password (trivially decodable, must be guarded) — [Source](https://kopia.io/docs/reference/command-line/)
|
||||
- Kopia: repository password cached in OS-specific credential storage (Keychain on macOS, Credential Manager on Windows, Keyring on Linux) — [Source](https://kopia.io/docs/reference/command-line/)
|
||||
- rclone crypt: password + optional salt (`password2`) stored in `rclone.conf` in lightly obscured form (AES-CTR with static shared key, random IV prepended) — explicitly "not secure" without overall config-file encryption (`rclone config` password protection); env vars `RCLONE_CRYPT_PASSWORD` / `RCLONE_CRYPT_PASSWORD2` (must be `rclone obscure`d) — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt: recovery = remember password or keep `rclone.conf`; same passwords re-entered on another machine reproduce access (obscured strings differ due to salt) — [Source](https://rclone.org/crypt/)
|
||||
- Tarsnap: single key file from `tarsnap-keygen --keyfile <file> --user <email> --machine <name>`; every `tarsnap` invocation needs `--keyfile`; passphrase via `--passphrase method:arg` (`dev:tty-stdin` default, `env:VAR`, `file:FILENAME` flagged as risky) — [Source](https://www.tarsnap.com/man-tarsnap-keygen.1.html); [Source](https://man.archlinux.org/man/tarsnap.1.en)
|
||||
- Tarsnap: `tarsnap-keymgmt --outkeyfile <new> [-r] [-w] [-d] [--nuke]` mints restricted sub-keys (read-only, write-only, delete, nuke-only); `-d` implies `-r` — [Source](https://www.tarsnap.com/man-tarsnap-keymgmt.1.html)
|
||||
|
||||
### Inferences
|
||||
- Only Borg gives a deliberate two-factor-style choice (possession of keyfile + knowledge of passphrase); restic/Kopia/rclone-default are passphrase-only if the repo/config is exfiltrated.
|
||||
- No system in scope documents TPM / hardware-backed key storage as of 2026; OS keyring appears only as a convenience cache (Kopia), not as a KDF or sealing mechanism.
|
||||
|
||||
### Gaps
|
||||
- No reliable source found on native TPM2 / Secure Enclave / Windows Hello integration for any of the four; community wrappers (pass, keyring scripts, systemd-creds) exist but were not verified as documented behavior.
|
||||
- restic's recommended master-key backup workflow beyond `cat masterkey` (e.g. official paper-key guidance) was not found.
|
||||
|
||||
## Do any support key rotation, multiple keys/passphrases, or Shamir/shared recovery? How?
|
||||
|
||||
### Takeaway
|
||||
restic and Borg (2.x/master) support multiple concurrent keys/passphrases sharing one master secret; Kopia and rclone crypt support only password change (re-wrapping the same secrets), with rclone requiring full re-upload; Tarsnap supports restricted sub-keys but not multi-passphrase unlock. No system documents Shamir secret sharing natively.
|
||||
|
||||
### Cited Findings
|
||||
- restic: `key` command with `list`, `add`, `remove`, `passwd` subcommands manages multiple access keys per repo; `key add` prompts for current password then new password and saves a new key wrapping the same master key; list marks current key with `*` — [Source](https://restic.readthedocs.io/en/stable/070_encryption.html)
|
||||
- restic: `key passwd` creates a new key ID and removes the old one; `key remove <ID>` refuses to remove the currently-used key — [Source](https://manpages.opensuse.org/Tumbleweed/restic/restic-key-passwd.1.en.html); [Source](https://man.archlinux.org/man/restic-key-remove.1.en.raw)
|
||||
- restic: `AddKey` with non-nil `template` copies master keys from the old key instead of generating new ones, confirming rotation re-wraps rather than re-encrypts data — [Source](https://github.com/restic/restic/blob/9e2d60e2/internal/repository/key.go)
|
||||
- Borg: `key change-passphrase` only re-locks the same secrets ("does not protect future nor past backups" if key+passphrase were compromised) — [Source](https://borgbackup.readthedocs.io/en/stable/usage/key.html)
|
||||
- Borg: `key change-algorithm argon2|pbkdf2` upgrades/downgrades the KDF wrapping without changing secrets — [Source](https://git.uninsane.org/shelvacu-mirrors/borg/commit/08f82ee40867f605ca6994db6dc32218d7b85cbc)
|
||||
- Borg (master/2.x): multiple borg keys per repo — each key holds the same secret material under an independent passphrase and label (first key labeled `admin`, protected from removal); `key list/add/remove/export --label|--key` selectors; passphrase tried against every key — [Source](https://github.com/borgbackup/borg/pull/9762)
|
||||
- Borg: `rcreate --other-repo SRC --copy-crypt-key` can reuse crypt key across related repos (default: fresh random crypt key, shared chunker/ID keys for dedup) — [Source](https://borgbackup.readthedocs.io/en/2.0.0b6/usage/rcreate.html)
|
||||
- Kopia: `kopia repository change-password` (CLI only, not GUI) re-encrypts the format block with a new password; must already be connected (so a still-connected client can reset a forgotten password, a disconnected one cannot) — [Source](https://kopia.io/docs/faqs/); [Source](https://kopia.io/docs/reference/command-line/common/repository-change-password/)
|
||||
- rclone crypt: no in-place password change — changing the configured password orphans existing content; must re-upload everything (in place via second crypt remote + `rclone copy` decrypting/re-encrypting on the fly, at 2x bandwidth/quota cost) — [Source](https://rclone.org/crypt/)
|
||||
- Tarsnap: no multi-passphrase unlock or rotation of archive encryption keys documented; capability separation via `tarsnap-keymgmt` restricted keys is the only delegation mechanism — [Source](https://www.tarsnap.com/man-tarsnap-keymgmt.1.html)
|
||||
- Shamir/shared recovery: no hits in official docs for restic, Borg, Kopia, rclone crypt, or Tarsnap; only community workarounds (splitting password/paper key with external SSS tools) exist — no primary-source citation available.
|
||||
|
||||
### Inferences
|
||||
- True cryptographic rotation (new data key, re-encryption) is offered by none of the five for existing snapshots; all "rotation" is re-wrapping or passphrase change.
|
||||
- Borg's multi-key design is closest to shared/admin recovery (per-user passphrases + protected admin key), but all keys still share one secret — compromise of the secret compromises all slots.
|
||||
|
||||
### Gaps
|
||||
- Whether `borg key add` (multi-key) is released in stable 2.0 vs master-only could not be pinned down; primary evidence is the PR plus master `usage/key.html`.
|
||||
- No documented Shamir/threshold scheme in any official docs; confirm as absent-by-design vs merely undocumented.
|
||||
|
||||
## What happens on key loss — is data unrecoverable by design? Any documented footguns?
|
||||
|
||||
### Takeaway
|
||||
All five declare password/key loss unrecoverable by design. The sharpest footguns: restic `key remove` of the sole key and deleted/corrupt `keys/` files; Borg keyfile loss without export and repokey-with-empty-passphrase on exposed storage; Kopia disconnected-password loss; rclone salt (`password2`) loss and obscured-config confusion; Tarsnap key-file loss (including billing trap).
|
||||
|
||||
### Cited Findings
|
||||
- restic docs repeat on every backend page: "knowledge of your password is required… Losing your password means that your data is irrecoverably lost" — [Source](https://restic.readthedocs.io/en/stable/030%5Fpreparing%5Fa%5Fnew%5Frepo.html)
|
||||
- restic forum (maintainer fd0): if key-file `data` or `salt` is missing/corrupt there is "no way to decrypt the data again. Even if you know the password"; intact `data`+`salt` (+N/r/p, brute-forceable) is required — [Source](https://forum.restic.net/t/recover-a-damaged-missing-corrupted-key-file/1798)
|
||||
- restic footgun case: user generated a random key file, ran `key remove` on the old key, then lost the random file (only copy was inside the backup) — recovery required disk forensics for the deleted key file; `key remove` is just a file deletion in `keys/` — [Source](https://forum.restic.net/t/recover-from-a-previous-password/2592)
|
||||
- restic: brute force only viable if much of the password structure is known; otherwise "consider it lost" — [Source](https://forum.restic.net/t/forgotten-password/2990)
|
||||
- Borg: "always need both the Borg key and passphrase"; keyfile loss = repo loss ("if you lose the key, you lose access"); mandated offsite export ("leaving your keys inside your car" warning); empty passphrase with repokey = anyone reading the repo can unlock (as good as no encryption); keyfile+empty passphrase acceptable only if client disk is encrypted — [Source](https://borgbackup.readthedocs.io/en/stable/usage/init.html)
|
||||
- Borg: exported key stays encrypted — recovery needs both export file AND original passphrase, stored in separate safe places — [Source](https://borgbackup.readthedocs.io/en/stable/usage/key.html)
|
||||
- Kopia: "you cannot restore your files if you forget your password, there is no way to recover a forgotten password because only you know it"; FAQ: "There is no way to recover it or the files and folders within that repository. Store your repository password in a safe place, such as a password manager" — [Source](https://github.com/kopia/kopia/blob/master/site/content/docs/Features/_index.md); [Source](https://kopia.io/docs/faqs/)
|
||||
- Kopia footgun nuance: password CAN be reset while still connected (`change-password`), which rescues forgotten-but-connected GUI users via CLI, but a disconnected client with a forgotten password is unrecoverable — [Source](https://kopia.io/docs/faqs/)
|
||||
- Kopia reconnect token (`status -t -s`) trivially decodes to the password — storing it insecurely leaks the repo — [Source](https://kopia.io/docs/reference/command-line/)
|
||||
- rclone crypt: custom salt is "effectively a second password that must be memorized" and is NOT stored with the data; losing `password2` loses data even with correct `password` — [Source](https://rclone.org/crypt/)
|
||||
- rclone footguns: obscured passwords look encrypted but use a static shared AES-CTR key (cursory-inspection only); 1.49.0–1.53.2 random-password generator bug produced insecure passwords (fixed 1.53.3, must rotate by re-upload) — [Source](https://rclone.org/crypt/)
|
||||
- Tarsnap FAQ: "You can't. Your key file contains the only copy of the cryptographic keys… if you lose them there is no way to get your data back" (same for forgotten key-file passphrase); separate FAQ covers being unable to stop billing after losing all keys — [Source](https://www.tarsnap.com/faq.html)
|
||||
|
||||
### Inferences
|
||||
- The envelope designs make server-side recovery cryptographically impossible, so every vendor pushes the same operational answer: paper/physical export + password manager + offsite separation of key-export and passphrase.
|
||||
- rclone's model is the most fragile operationally (two secrets + weak-by-default config obscuring + no re-wrap), Tarsnap second (single local file, no passphrase fallback).
|
||||
|
||||
### Gaps
|
||||
- Corrupt-repo recovery (bitrot, backend errors) vs key loss is covered by Kopia consistency docs and restic `rebuild-index`/`recover`, but systematic per-tool data on partial recovery without keys was out of scope and not collected.
|
||||
@@ -0,0 +1,126 @@
|
||||
# Untrusted-Server Threat Model: restic, BorgBackup, Kopia, rclone crypt on dumb backends
|
||||
|
||||
## What plaintext metadata does each system leave on the backend (repo layout, config files, sizes, timestamps)?
|
||||
|
||||
### Takeaway
|
||||
All four encrypt file content client-side but leak different amounts of structural metadata: restic leaks backend object sizes/counts/timestamps and config filenames; Borg leaks chunk sizes/proximity unless obfuscation is enabled; Kopia leaks only prefixed blob names (p/q/x) and approximate pack sizes; rclone crypt leaks file sizes (±16B) and mtimes and optionally directory structure.
|
||||
|
||||
### Cited Findings
|
||||
- restic design: “Apart from the files stored within the `keys` directory, all files are encrypted with AES-256 in counter mode (CTR)” with Poly1305-AES MAC, IV in first 16 bytes — keys directory itself is the exception and contains KDF parameters in plaintext JSON (`hostname`, `username`, `kdf=scrypt`, `N=65536,r=8,p=1`, `salt`, `created`) — [Source](https://github.com/restic/restic/blob/master/doc/design.rst); same guarantee restated as “Everything except the metadata included for informational purposes in the key files is encrypted and authenticated” — [Source](https://rustic.cli.rs/dev-docs/design/threat_model.html)
|
||||
- restic repo layout on dumb backend is content-defined: `config`, `keys/`, `locks/`, `snapshots/`, `index/`, `data/` pack files named by hex SHA-256 of plaintext content; filenames of packs/snapshots/indexes are plaintext hashes, sizes and mtimes visible to server — [Source](https://restic.readthedocs.io/en/stable/100_references.html)
|
||||
- restic explicitly warns attacker with write access “can determine which files belong to what snapshot (e.g. based on the timestamps of the stored files)” — [Source](https://rustic.cli.rs/dev-docs/design/threat_model.html)
|
||||
- restic local cache “is also encrypted to prevent metadata leaks” — [Source](https://rustic.cli.rs/dev-docs/design/threat_model.html)
|
||||
- Borg attack model: “environment of the client process (e.g. borg create) is trusted and the repository (server) is not. The attacker has any and all access to the repository, including interactive manipulation (man-in-the-middle)” — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg guarantees vs untrusted server: attacker cannot (1) modify archive data undetected, (2) rename/remove/add archive undetected, (3) recover plaintext, (4) recover definite structural information such as object graph — but “heuristics based on access patterns are possible” — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg multi-client caveat: “When the above attack model is extended to include multiple clients independently updating the same repository, then Borg fails to provide confidentiality (i.e. guarantees 3) and 4) do not apply any more)” — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg chunk-ID is MAC of plaintext (`id = AUTHENTICATOR(id_key, data)`), encryption is Encrypt-then-MAC AES-256-CTR + HMAC-SHA256 or BLAKE2b-256; IV counter “added in plaintext” and tracked via client security DB + repo reservation — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg 2.0 changes to AEAD (`AES-256-OCB` or `chacha20-poly1305`) with session keys, chunk ID via MAC over plaintext, object ID bound as AAD so attacker cannot “change the type of the object or move content to a different object ID” — [Source](https://borgbackup.readthedocs.io/en/2.0.0b12/internals/security.html)
|
||||
- Borg fingerprinting: “A borg repository does not hide the size of the chunks it stores”; small files <512KiB yield single chunk; buzhash chunker uses secret per-repo chunker seed; optional `obfuscate` pseudo-compressor pads with 0x00 bytes (only adds size) to hinder size fingerprinting — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg proximity leak: “Borg does not try to obfuscate order / proximity of files”; sorts by inode order not name, but “when new files are close to each other [in recursion order], the resulting chunks will be also stored close to each other in segment file(s)” — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Kopia envelope encryption: repo has plaintext JSON `formatBlob` with `tool`, `buildVersion`, `uniqueID`, `keyAlgo=scrypt-65536-8-1`, `version`, `encryption` (e.g. `AES256_GCM`), plus `encryptedBlockFormat` ciphertext; `UniqueID` doubles as PBKDF salt and HKDF info — [Source](https://kopia.io/docs/advanced/encryption)
|
||||
- Kopia passphrase → 32B master key `Km = PBKDF(passphrase, UniqueID)` (scrypt N=65536,r=8,p=1), then `Ke = HKDF(SHA256,Km,UniqueID,"AES",32)` and `AD = HKDF(SHA256,Km,UniqueID,"CHECKSUM",32)` for format-blob AES-GCM — [Source](https://kopia.io/docs/advanced/encryption)
|
||||
- Kopia content encryption options are `AES256-GCM-HMAC-SHA256` (default) or `CHACHA20-POLY1305-HMAC-SHA256`, chosen at repo creation, cannot be changed after — [Source](https://github.com/kopia/kopia/blob/master/site/content/docs/FAQs/_index.md)
|
||||
- Kopia blob layer: pack blobs have random names, “don’t reveal anything about their contents or structure”; prefixes `p` (data packs), `q` (metadata packs), `x` (indices); object-ID single-letter prefixes `k` (directory), `m` (manifest block), `x` (indirect JSON) route to `q` vs `p` packs — [Source](https://kopia.io/docs/advanced/architecture)
|
||||
- Kopia packs are 20–40MB merged blobs; “sizes are also generally unrelated to their content due to splitting and merging”; per-pack trailing local index enables recovery if global index lost — [Source](https://kopia.io/docs/advanced/architecture)
|
||||
- Kopia manifests (snapshots/policies) are small JSON stored as encrypted CABS blocks, addressed by `key=value` labels (e.g. `type:policy`, `hostname:… path:… username:…` visible via `kopia manifest list` client-side, not plaintext on backend) — [Source](https://kopia.io/docs/getting-started/)
|
||||
- Kopia: “encrypts these snapshots before they leave your computer”, “password never leaves your machine”, single password per repo, “currently no access control mechanism when sharing a repository” — [Source](https://kopia.io/docs/features/)
|
||||
- rclone crypt: wraps any backend; “automatically encrypt (before uploading) and decrypt (after downloading) on your local system … leaving the data encrypted at rest”; direct access to wrapped remote bypasses crypto — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt file format: 8B magic `RCLONE\x00\x00` + 24B nonce header, then 64KiB chunks each with 16B Poly1305 tag (XSalsa20+Poly1305 SecretBox); 1B file → 49B total, 1MiB → 1048864B (+0.03%) — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt name encryption: path split on `/`, PKCS#7 padded to 16B, EME-AES-256 deterministic, base32-lowercase-no-pad; identical names → identical ciphertexts; common prefixes hidden — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt explicitly “does not encrypt file length - this can be calculated within 16 bytes” nor “modification time - used for syncing”; versions suffix `-vYYYY-MM-DD…` left plaintext; directory names optionally unencrypted — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt filename modes: `standard` (encrypted, ~143 char limit, dir structure visible), `obfuscate` (“Very simple filename obfuscation … cannot be relied upon for strong protection”), `off` (adds `.bin` only) — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt: “Any metadata supported by the underlying remote is read and written” — i.e. WebDAV/SFTP xattrs/mtimes pass through unencrypted — [Source](https://rclone.org/crypt/)
|
||||
- rclone WebDAV backend: “Plain WebDAV does not support modified times” nor hashes except via Fastmail/ownCloud/Nextcloud vendor extensions — [Source](https://rclone.org/webdav/)
|
||||
|
||||
### Inferences
|
||||
- For size-hiding, Kopia pack merging (>20MB) is the strongest default; restic pack (~4-8MB default, content-defined) is intermediate; Borg chunk-level sizes leak most without `obfuscate` compressor.
|
||||
- Deterministic name encryption (rclone crypt standard, restic hash-named packs are content-derived not name-derived) still leaks equality + directory shape; only Kopia random pack names hide shape by default.
|
||||
- Plaintext `keys/` (restic) and `formatBlob` (Kopia) both expose KDF parameters and repo unique IDs — useful for offline dictionary attack cost estimation but not plaintext.
|
||||
|
||||
### Gaps
|
||||
- No reliable primary-doc figure found for exact restic snapshot/index plaintext filename entropy or padding policy; restic docs do not claim filename obfuscation or padding.
|
||||
- Borg `obfuscate` compressor size-overhead curve (“few percent [cheap] to ridiculously larger [expensive]”) has no numeric table in fetched docs.
|
||||
|
||||
## How do they handle an actively malicious or compromised server (authentication of manifests, rollback protection, replay of old snapshots)?
|
||||
|
||||
### Takeaway
|
||||
Borg has the strongest formal malicious-server story (Horton-anchored DAG + TAM-signed manifest + nonce tracking); restic authenticates all objects but explicitly does not detect deletion/rollback or timestamp-grouping attacks; Kopia relies on content-addressed HMAC + maintenance/verify with server-side immutability as add-on; rclone crypt has per-chunk authentication but no manifest/rollback concept.
|
||||
|
||||
### Cited Findings
|
||||
- restic guarantees: “Modifications to data … can be detected” and “Data that has been tampered will not be decrypted” (MAC checked before decrypt) — [Source](https://rustic.cli.rs/dev-docs/design/threat_model.html)
|
||||
- restic non-goals: “not designed to protect against attackers deleting files”; attacker deleting timestamp-correlated packs makes “particular snapshot vanish … Restic is not designed to detect this attack” — [Source](https://rustic.cli.rs/dev-docs/design/threat_model.html)
|
||||
- restic compromised-host cases: attacker with repo write can “Create snapshots (containing garbage data) which cover all modified files and wait until a trusted host has used forget often enough to remove all correct snapshots” and “Create a garbage snapshot for every existing snapshot with slightly different timestamp” to trick rotation — [Source](https://rustic.cli.rs/dev-docs/design/threat_model.html)
|
||||
- restic append-only caveat: attacker with append-only access can still “Capture the password and decrypt past and future backups” (no forward secrecy); safe `forget` requires separate doc procedure — [Source](https://rustic.cli.rs/dev-docs/design/threat_model.html)
|
||||
- restic key-leak: “impossible to securely revoke a leaked key without re-encrypting the whole repository” — [Source](https://rustic.cli.rs/dev-docs/design/threat_model.html)
|
||||
- restic open issue #5041 proposes extending model with append-only enforcement + management/data key separation so “adversary who compromises the host cannot retroactively publish a snapshot which will trick forget into deletion” — still open/discussion as of 2026 — [Source](https://github.com/restic/restic/issues/5041)
|
||||
- Borg structural auth follows “Horton principle”: every object referenced by parent via plaintext-MAC object ID up to manifest, forming authenticated DAG — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg manifest has fixed ID `000…000` so cannot be DAG-authenticated; protected since 1.0.9 by TAM: `tam_key = HKDF-SHA-512(id_key||enc_key||enc_hmac_key, RANDOM(64), "borg-metadata-authentication-manifest")`, `HMAC(tam_key, packed_manifest)` stored in manifest — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg nonce/counter anti-reuse: client commits +4GiB reservation to security DB + repo before encrypting; crash-safe via SaveFile; but “in a multiple-client scenario a repository can trick a client into reusing counter values by ignoring counter reservations and replaying the manifest (which will fail if the client has seen a more recent manifest or has a more recent nonce reservation)” — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg RPC: over system SSH (no own network code), server can only send responses not requests; msgpack limited Unpacker; worst-case server can impose repo DoS; log-channel confusion limited to in-progress requests — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg CVE-2023-36811: “flaw in the cryptographic authentication scheme … allowed an attacker to fake archives and potentially indirectly cause backup data loss”; requires inserting files without headers + repo write; “does not disclose plaintext … nor affect authenticity of existing archives”; fixed in 1.2.5 + upgrade procedure; mitigate by reviewing archives after `check --repair` before `prune` — [Source](https://nvd.nist.gov/vuln/detail/CVE-2023-36811)
|
||||
- Borg append-only: `borg config repo append_only 1` or `borg serve --append-only`; “never overwrite or delete committed data” at segment level, but `delete/prune` still allowed (appear to succeed, only tag as deleted in new transaction); `compact` becomes no-op with no warning — [Source](https://raw.githubusercontent.com/borgbackup/borg/1.4.5/docs/usage/notes.rst)
|
||||
- Borg append-only rollback: transaction log (`transactions` file with UTC timestamps) allows manual rollback by deleting segment files from attack point onward (e.g. `rm data/**/{6..13}`), provided `compact` has not run; must clear client cache after (`borg delete --cache-only`) — [Source](https://raw.githubusercontent.com/borgbackup/borg/1.4.5/docs/usage/notes.rst)
|
||||
- Borg append-only limits: “Append-only mode is not respected by tools other than Borg. rm still works”; clients must only access via `borg serve`; any non-append-only write (admin prune/create) permanently compacts away “deleted” data; SSH `authorized_keys` split (`--append-only` key for untrusted clients, full key for admin) is recommended pattern — [Source](https://borgbackup.readthedocs.io/en/1.1-maint/usage/notes.html)
|
||||
- Kopia consistency: `kopia snapshot verify` walks snapshot roots, checks index structures + blob existence; runs automatically during daily full maintenance; `--verify-files-percent=N` samples downloads/decrypts (100% ≈ test restore, discarded after check); e.g. 1% daily → ~98% coverage over a year — [Source](https://kopia.io/docs/advanced/consistency/)
|
||||
- Kopia corruption causes listed as non-POSIX/networked filesystems, silent bit-rot, large clock skew (few minutes tolerated, larger can cause self-deletion); recommends mature POSIX FS or cloud storage + NTP — [Source](https://kopia.io/docs/advanced/consistency/)
|
||||
- Kopia ransomware model: malware may exfiltrate cloud keys and delete snapshots; mitigation is provider-side restricted keys (no delete) + object-lock/retention (COMPLIANCE), not client crypto alone — [Source](https://kopia.io/docs/advanced/ransomware-protection/)
|
||||
- Kopia object-lock: `--retention-mode COMPLIANCE --retention-period <e.g.30d>` at `repo create s3`, plus `maintenance set --extend-object-locks true` with `full-interval` ≥1 day shorter than retention; supports S3 (+B2-via-S3), Azure version-level immutability, GCS versioning+retention; `--point-in-time` reconnect for S3 restores — [Source](https://kopia.io/docs/advanced/ransomware-protection/)
|
||||
- rclone crypt integrity: per-chunk Poly1305 via SecretBox; “data integrity is protected by an extremely strong crypto authenticator”; `cryptcheck` (not `check`) required since hashes not stored — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt has no manifest/snapshot ledger: replay of old encrypted files, rollback, or server-side deletion is undetectable by crypt layer; `--pass-bad-blocks` (zero-fill corrupt chunks) is opt-in recovery only — [Source](https://rclone.org/crypt/)
|
||||
|
||||
### Inferences
|
||||
- Only Borg attempts to bind manifest meaning (TAM) against a fully MITM server with persistent client state; restic/Kopia assume server may be read/write-malicious for confidentiality/integrity of objects but push rollback/availability to out-of-band controls (append-only buckets, object-lock, separate admin host).
|
||||
- For WebDAV-class backends with no compute, Borg-style `serve --append-only` enforcement is unavailable; closest analogues are restic append-only-capable REST backends (not WebDAV) or Kopia S3 object-lock — WebDAV must rely on server ACLs/versioning.
|
||||
|
||||
### Gaps
|
||||
- No primary-source evidence found that Kopia authenticates manifest labels/meaning against malicious-server manifest substitution beyond content HMAC; treat as gap.
|
||||
- No restic primary doc found describing rollback detection (e.g. monotonic snapshot counter); restic docs explicitly disclaim it.
|
||||
|
||||
## How do they rate-limit, retry, and verify uploads (idempotent PUTs, integrity re-checks)?
|
||||
|
||||
### Takeaway
|
||||
All target dumb blob stores with idempotent content-addressed PUTs and client-side re-verification (`check`/`verify`/`cryptcheck`); retry/backoff is client-configured and backend-specific; none documents server-side rate-limiting — throttling is via client concurrency limits and provider quotas.
|
||||
|
||||
### Cited Findings
|
||||
- restic repo design “allows parallel access of multiple instances … even parallel writes”; locks with `--retry-lock` retry until timeout; `restic check` verifies pack hash vs pack ID, `cat pack <id>` warns on mismatch — [Source](https://restic.readthedocs.io/en/latest/100_references.html?highlight=threat)
|
||||
- Kopia architecture: content-addressed blocks (`Block ID = hash(data)`); identical blocks dedup naturally; uploads merged into packs; index maps block→(blob,offset,len) — enabling idempotent re-PUT (same ID = same bytes) — [Source](https://kopia.io/docs/advanced/architecture)
|
||||
- Kopia verification tiers: metadata-only `snapshot verify` every full maintenance + opt-in content download sample `--verify-files-percent` + `--file-parallelism` for throughput control — [Source](https://kopia.io/docs/advanced/consistency/)
|
||||
- Kopia storage requirements imply retry need: “strong read-after-write consistency and eventual list-after-write consistency”; “can compensate for such inconsistent behaviors for up to several minutes, but larger inconsistencies can lead to data loss” — [Source](https://kopia.io/docs/advanced/consistency/)
|
||||
- Borg RPC worst-case throttling noted only as server-imposed “denial of repository service”; client uses limited msgpack Unpacker to avoid memory-DoS from large messages — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- rclone crypt backup guidance: use `sync` on encrypted paths with same passwords so “will check the checksums while copying”; `check` between two encrypted remotes works, `cryptcheck` needed vs plaintext — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt chunking (64KiB + 16B tag) bounds retry unit; header nonce incremented per chunk, reuse probability ~2e-32 per exabyte — [Source](https://rclone.org/crypt/)
|
||||
|
||||
### Inferences
|
||||
- Content-addressing (restic pack ID, Borg chunk ID, Kopia block ID) makes PUTs naturally idempotent and safe to retry; verification is always client-driven (`check`/`verify`/`cryptcheck`) since dumb backends cannot attest.
|
||||
- Parallelism knobs (`--file-parallelism`, rclone `--transfers/--checkers`, Borg client concurrency) are the de facto rate-limiters; no evidence of server-push backpressure handling in fetched docs.
|
||||
|
||||
### Gaps
|
||||
- No primary-source values found in fetched docs for restic `--retry-lock` defaults, backend backoff schedule, or WebDAV-specific retry/idempotency; mark as gap — need restic backend docs + rclone WebDAV backend options.
|
||||
- No Kopia primary-doc retry/backoff parameters or blob `Put` atomicity guarantees surfaced in fetched pages; need `repo/blob` Go API docs.
|
||||
- No Borg primary-doc upload retry/rate-limit parameters surfaced; Borg docs focus on correctness not throttling.
|
||||
|
||||
## Any guidance specific to WebDAV-class dumb backends (no server compute) that applies to a local daemon syncing to WebDAV?
|
||||
|
||||
### Takeaway
|
||||
Treat WebDAV as untrusted byte store: do all crypto/index/manifest work client-side, never depend on server mtime/hash, use atomic PUT + verify-after-write, and add out-of-band append-only/versioning since WebDAV has no compute to enforce it.
|
||||
|
||||
### Cited Findings
|
||||
- Kopia repository model: “All Repository features are implemented client-side, without any need for a custom server, thus encryption keys never leave the client”; layers are Object/Manifest/Block/Raw-BLOB over simple blob API — directly applicable to WebDAV — [Source](https://github.com/kopia/repo/blob/master/README.md)
|
||||
- Kopia supports “Any remote server or cloud storage that supports WebDAV” and SFTP as first-class repo storage, plus Rclone-wrapped Dropbox/OneDrive/Google-Drive (experimental) — [Source](https://github.com/kopia/kopia?pubDate=20260701)
|
||||
- Kopia caveat for dumb/networked FS: requires “POSIX semantics, atomic writes (either native or emulated), strong read-after-write … eventual list-after-write”; “avoid emulated, layered, or networked filesystems which may not be fully compliant. Alternatively, use cloud storage”; WebDAV falls in risky class — [Source](https://kopia.io/docs/advanced/consistency/)
|
||||
- Kopia clock rule: “clock skews of few minutes are tolerated, but bigger clock skews can lead to major inconsistencies, including Kopia deleting its own data”; run NTP on clients and servers — [Source](https://kopia.io/docs/advanced/consistency/)
|
||||
- rclone WebDAV limits: “Plain WebDAV does not support modified times … does not support hashes” (except Fastmail/ownCloud/Nextcloud extensions) — so sync daemons must not trust mtime/hash for change detection — [Source](https://rclone.org/webdav/)
|
||||
- rclone crypt on WebDAV pattern: point crypt at `remote:path` subdirectory, access exclusively via crypt remote; `.bin` suffix added when names unencrypted to prevent provider interpreting content; `strict_names` errors on mixed encrypted/unencrypted dirs — [Source](https://rclone.org/crypt/)
|
||||
- rclone crypt password rotation requires full re-upload (decrypt-with-old → encrypt-with-new), either from local source or crypt-to-crypt `move`; “half the bandwidth and charged twice” on metered backends — [Source](https://rclone.org/crypt/)
|
||||
- restic threat-model implication for WebDAV: server sees object sizes/counts/timestamps; timestamp-grouping enables targeted snapshot deletion — mitigation is to avoid leaking timing (jitter uploads, shared pack sizes) though not prescribed in docs — [Source](https://rustic.cli.rs/dev-docs/design/threat_model.html)
|
||||
- Borg inode-order (not alpha) directory traversal + secret chunker seed + optional size obfuscation are concrete dumb-backend-hardening techniques portable to any daemon: randomize scan order, per-repo secret for chunking, pad to size classes — [Source](https://borgbackup.readthedocs.io/en/stable/internals/security.html)
|
||||
- Borg SSH `authorized_keys` split (`borg serve --append-only` for untrusted clients vs full for admin) has no WebDAV equivalent; WebDAV equivalent is provider ACL/versioning or Kopia-style object-lock where available (S3/Azure/GCS only — not WebDAV) — [Source](https://borgbackup.readthedocs.io/en/1.1-maint/usage/notes.html); Kopia object-lock “currently only supports object locks when using an S3 repo” (native B2 excluded, use S3 mode) — [Source](https://kopia.io/docs/advanced/ransomware-protection/)
|
||||
- Filippo Valsorda review quoted in restic docs: “The design might not be perfect, but it’s good. Encryption is a first-class feature, the implementation looks sane and … deduplication trade-off is worth it” — informal audit signal, not formal audit — [Source](https://github.com/restic/restic/blob/master/doc/070_encryption.rst)
|
||||
|
||||
### Inferences
|
||||
- For a local daemon syncing to WebDAV: (1) encrypt+MAC everything client-side with random pack names (Kopia-style) or hash-named packs (restic-style); (2) keep manifest/index signed with local key (Borg TAM-style) and verify on every sync; (3) never use server mtime/ETag as source of truth; (4) verify-after-write via GET+MAC before deleting local staging; (5) mitigate rollback by keeping local monotonic snapshot counter + out-of-band copy; (6) pad/obfuscate sizes and jitter upload times to reduce grouping leaks.
|
||||
- Since WebDAV cannot enforce append-only, ransomware safety must come from server-side versioning/quotas + separate prune identity, mirroring Borg append-only and Kopia restricted-keys guidance.
|
||||
|
||||
### Gaps
|
||||
- No primary WebDAV-server hardening guide (append-only ACLs, versioning) surfaced for restic/Borg/Kopia on plain WebDAV; restic WebDAV backend doc and rclone WebDAV option reference not fetched (tool-call budget) — need follow-up.
|
||||
- No formal third-party audit reports (e.g. Cure53/Quarkslab) confirmed for restic/Borg/Kopia in fetched sources; only Valsorda informal review and CVE record found — need dedicated audit search.
|
||||
Reference in New Issue
Block a user