versioning/research_notes/Retention scheduling adaptation/scheduling-models.md

# Retention / pruning / maintenance scheduling models

## On-demand vs scheduled vs automatic-after-each-backup: which model does each tool use, and what are the documented tradeoffs?

### Takeaway
restic and BorgBackup are fully manual/external-scheduler tools (no built-in scheduler; prune-after-backup is a script/wrapper convention with lock-contention and cost tradeoffs), Kopia is automatic/built-in (maintenance fires opportunistically on client use with a single elected owner), Time Machine is fully automatic and continuous (thinning driven by schedule + disk pressure), and Veeam is built-in-scheduler enterprise (retention applied inline after job sessions plus a nightly background process, with health-check/compact on separate schedules).

### Cited Findings
- restic has no built-in scheduler: `forget` only deletes snapshot objects and a separate `prune` must remove unreferenced data; `--prune` on `forget` automates the two-step sequence only when snapshots were actually removed — [restic forget docs](https://restic.readthedocs.io/en/stable/060_forget.html)
- restic docs warn pruning is time-consuming, takes an exclusive repository lock so "backups cannot be completed" during prune, and advise planning prune windows plus running `restic check` afterwards — [restic forget docs](https://restic.readthedocs.io/en/stable/060_forget.html)
- Community restic practice is external cron (e.g. `0 0 * * *` daily backup script running backup → check → `forget --keep-daily N --prune`) or systemd timers; third-party wrappers (restic-scheduler, autorestic/resticprofile) add per-task systemd services where backup runs often (e.g. daily) and retention+check runs monthly — [restic cron example](https://gist.github.com/perfecto25/f528f8d14e1c4b6e2a912513539a5af7); [restic-scheduler](https://github.com/AenonDynamics/restic-scheduler)
- BorgBackup `prune` is documented as "normally used by automated backup scripts"; the official quickstart pattern is one script doing `create` → `prune` → `compact` in sequence, and `borgmatic`'s default actions are create+prune+compact+check (i.e. after-each-backup by convention, not by daemon) — [borg prune docs](https://borgbackup.readthedocs.io/en/stable/usage/prune.html); [borg quickstart](https://github.com/borgbackup/borg/blob/master/docs/quickstart.rst); [borgmatic manpage](https://manpages.ubuntu.com/manpages/focal/man1/borgmatic.1.html)
- borgmatic docs warn the default every-run prune/compact/check is fine for small repos but too slow for very large ones, so large repos should decouple them (skip actions / separate schedules) — [borgmatic large backups guide](https://github.com/borgmatic-collective/borgmatic/blob/main/docs/how-to/deal-with-very-large-backups.md)
- Vorta (Borg desktop GUI) exposes retention as a "Prune after each backup" checkbox, i.e. after-each-backup as an opt-in — [Vorta prune docs](https://vorta.borgbase.com/usage/prune)
- Kopia maintenance is automatic since v0.6.0: it "will happen occasionally when the `kopia` command-line client is used", with quick tasks ~hourly and full tasks every 24h by default — [Kopia maintenance docs](https://kopia.io/docs/advanced/maintenance/)
- Kopia defaults are quick interval 1h and full interval 24h, both enabled (from source defaults `QuickCycle 1h`, `FullCycle 24h`) — [maintenance_params.go v0.17.0](https://raw.githubusercontent.com/kopia/kopia/v0.17.0/repo/maintenance/maintenance_params.go); corroborated by user-observed "every hour for quick and every day for full" — [kopia issue #1439](https://github.com/kopia/kopia/issues/1439)
- Kopia retention policy (which snapshots to keep: keep-latest/hourly/daily/weekly/monthly/annual) is separate from maintenance (GC of unreferenced blobs); retention is applied by deleting expired snapshots, full-maintenance Snapshot-GC then reclaims the data — [kopia policy set reference](https://kopia.io/docs/reference/command-line/common/policy-set/); [Kopia maintenance docs](https://kopia.io/docs/advanced/maintenance/)
- Time Machine "automatically makes hourly backups for the past 24 hours, daily backups for the past month, and weekly backups for all previous months", deleting oldest backups when the disk is full — [Apple Support](https://support.apple.com/en-la/104984)
- Time Machine local snapshots are created hourly, stored on the source disk, kept up to 24h or until space is needed, and removed automatically under pressure — [Apple local snapshots guide](https://support.apple.com/en-euro/guide/mac-help/mh35933/mac)
- Veeam short-term retention is applied inline at the end of each job session (chain transform/merge), while GFS retention is enforced by a background process / nightly retention job (v11+ Cloud Connect docs describe a nightly "retention job"; run `History > System`, filter "retention") — [VCSP GFS retention docs](https://veeamvcsp.github.io/docs/vcc/gfs)
- Veeam health check and defrag/compact are opt-in scheduled operations attached to backup jobs, not run after every backup: health check off by default schedule wording, default monthly (last Sat/Sun 05:00 depending on product/generation), compact disabled by default — [Veeam maintenance settings](https://helpcenter.veeam.com/docs/vbr/userguide/backup_copy_settings_backup.html); [Veeam health check](https://helpcenter.veeam.com/docs/vbr/userguide/backup_health_check.html)

### Inferences
- The spectrum is: manual+external (restic, borg) → automatic-opportunistic (kopia) → fully automatic OS-driven (Time Machine) → policy-engine with built-in scheduler (Veeam).
- Documented tradeoff axis is consistent across tools: running retention/GC after every backup minimizes staleness but costs time, I/O, lock contention and (cloud) egress; decoupling expensive steps to daily/weekly/monthly windows is the universally recommended mitigation.

### Gaps
- No authoritative 2026 statement found on restic ever adding a built-in scheduler; docs and ecosystem still assume cron/systemd as of current stable docs.
- Exact Veeam short-term-retention merge timing (end-of-session inline vs deferred background) varies by chain type (forever-forward vs forward-incremental vs reverse); per-type timing not fully pinned down from fetched pages.

## What are the concrete scheduling mechanisms (systemd timers, cron, built-in scheduler, maintenance windows) with example intervals?

### Takeaway
restic/Borg rely on cron or systemd timers you write (daily backup typical; prune often piggybacked, check/compact weekly–monthly); Kopia uses built-in intervals (1h quick / 24h full, tunable, plus snapshot scheduling via interval/time-of-day/cron policy); Time Machine uses undocumented-internal launchd scheduling (~hourly, customizable via tools); Veeam uses a built-in per-job scheduler plus nightly background retention and monthly health-check defaults.

### Cited Findings
- restic: no internal scheduler; typical community cron is daily `0 0 * * *` running a backup script; systemd-wrapper example ships `restic-scheduler@.timer` (backup, commonly daily) plus `restic-retention@.timer` (forget+prune and check, commonly monthly) with e.g. `RETENTION_POLICY_DAYS=14 WEEKS=12 MONTHS=18 YEARS=2` — [restic cron example](https://gist.github.com/perfecto25/f528f8d14e1c4b6e2a912513539a5af7); [restic-scheduler](https://github.com/AenonDynamics/restic-scheduler)
- restic `check --read-data-subset=n/t` (or `x%`, or size like `50M`) exists precisely to spread full-data verification across scheduled runs (e.g. 1/7..7/7 across a week, or weekly `5%`); community practice converges on daily small-subset or weekly rotating-part checks rather than full `--read-data` each run — [restic 0.13 working-with-repos](https://restic.readthedocs.io/en/v0.13.0/045_working_with_repos.html); [restic forum practice](https://forum.restic.net/t/do-you-use-check-read-data/8930)
- Borg: no daemon; scheduling via cron/systemd calling a script or `borgmatic` (sample `borgmatic.timer` ships in repo); official quickstart script runs backup→prune→compact every invocation with example policy `--keep-daily 7 --keep-weekly 4 --keep-monthly 6` — [borg quickstart](https://github.com/borgbackup/borg/blob/master/docs/quickstart.rst); [borgmatic systemd sample](https://github.com/witten/borgmatic/blob/master/sample/systemd/borgmatic.timer)
- Borg `check --max-duration SECONDS` supports splitting a long repo check into partial checks; documented example: full check would take 7h, daily `--max-duration=3600` yields one full check per week — [borg-check manpage](https://manpages.debian.org/testing/borgbackup/borg-check.1.en.html)
- Borg 2.x adds `--max-age` so repeated `--max-duration`-bounded runs re-check each pack at most once per age window (example `--max-duration=3600 --max-age=1w` daily ≈ full verification weekly); partial checks require `--repository-only` — [borg 2 check docs](https://borgbackup.readthedocs.io/en/latest/usage/check.html)
- Kopia snapshot scheduling is policy-driven: `--snapshot-interval`, `--snapshot-time HH:mm,...`, `--snapshot-time-crontab`, `--run-missed`, `--manual` — [kopia policy set reference](https://kopia.io/docs/reference/command-line/common/policy-set/)
- Kopia maintenance intervals are built-in and tunable: `maintenance set --quick-interval=2h --full-interval=8h`, enable/disable flags, and `--pause-quick/--pause-full=DURATION` to suspend — [Kopia maintenance docs](https://kopia.io/docs/advanced/maintenance/)
- KopiaUI/server runs a "Periodic maintenance" check roughly every 10 minutes and executes quick/full work only when due (observed behavior with default 1h/24h) — [kopia issue #1439](https://github.com/kopia/kopia/issues/1439)
- Time Machine: hourly automatic backups driven by `com.apple.backupd-auto` LaunchDaemon (`StartInterval 3600` default, adjustable via `sudo defaults write ... StartInterval -int 7200` or TimeMachineEditor interval/calendar modes) — [Apple Gazette schedule customization](https://www.applegazette.com/applegazette-mac/customize-time-machine-backups-schedule)
- Time Machine thinning triggers: hourly→24h, daily→~30d, weekly→until-full on the backup volume; local APFS snapshots thin on age (>24h) or space pressure, manually via `tmutil thinlocalsnapshots <mount> [bytes] [urgency 1-4]` and verifiable via `tmutil verifychecksums` / Option-click "Verify Backups" (network targets) — [Apple Support](https://support.apple.com/en-la/104984); [Apple local snapshots](https://support.apple.com/en-euro/guide/mac-help/mh35933/mac); [tmutil reference](https://ss64.com/mac/tmutil.html); [Apple verify backups](https://support.apple.com/en-mn/guide/mac-help/mh26840/mac)
- Veeam: per-job backup schedule + GFS calendar (weekly day-of-week, monthly first/second/third/fourth/last week, yearly month) with GFS fulls created on scheduled days (synthetic); since v11 GFS creation happens right on scheduled days and a nightly background retention job enforces GFS deletions — [Veeam GFS cycles](https://helpcenter.veeam.com/docs/vbr/userguide/backup_copy_gfs_periods.html); [VCSP GFS retention docs](https://veeamvcsp.github.io/docs/vcc/gfs)
- Veeam health check default: monthly, 05:00 last Saturday (VBR backup jobs) / last Sunday (backup-copy jobs) / last Friday (Windows agent); runs piggybacked on the first incremental session of the scheduled day, or the next session if the job didn't run that day — [Veeam health check](https://helpcenter.veeam.com/docs/vbr/userguide/backup_health_check.html); [Veeam copy health check](https://helpcenter.veeam.com/docs/vbr/userguide/backup_copy_health_check.html); [Veeam agent health check](https://helpcenter.veeam.com/docs/agentforwindows/userguide/backup_health_check.html)
- Veeam defrag/compact-full is disabled by default, scheduled via job Maintenance settings when enabled; requires free space for an auxiliary VBK and is incompatible with GFS retention enabled — [Veeam maintenance settings](https://helpcenter.veeam.com/docs/vbr/userguide/backup_copy_settings_backup.html)

### Inferences
- Example "sane defaults" synthesis for adaptation: backup cadence hourly/daily (cheap op); retention-mark pass daily or after each backup; repack/GC weekly–monthly; integrity verification monthly or continuous-subset.
- Time Machine is the only system with zero user-visible retention scheduling: policy is fixed, enforcement is event-driven (post-backup + on-full).

### Gaps
- Precise modern macOS scheduler internals (whether current macOS still uses `StartInterval`-based `backupd-auto` vs newer XPC/Activity scheduler) not verified from a primary Apple source; third-party descriptions may lag macOS 14–15 changes.
- borgmatic's current recommended check frequency knobs (e.g. `checks:` list with `frequency:` per check) were not fetched in detail; schema confirms `checks` and `compact_threshold` exist but exact frequency syntax needs a docs fetch.

## How do they separate cheap policy runs (mark/delete metadata) from expensive ones (repack/GC), and how often is each recommended?

### Takeaway
All five systems split cheap metadata marking from expensive space reclamation; cheap passes run often (per-backup or daily), expensive passes rarely (weekly to monthly) with explicit thresholds/tuning.

### Cited Findings
- restic: `forget` (cheap: deletes snapshot metadata objects only) vs `prune` (expensive: scans all snapshots, classifies packs used/partly/unused, downloads+re-uploads repacked data — "very time-consuming for remote repositories"); `--max-unused` (default `5%`) bounds repacking, `--max-repack-size` caps work per run, `--repack-cacheable-only` restricts to metadata for a fast pass — [restic forget docs](https://restic.readthedocs.io/en/stable/060_forget.html)
- restic: `forget --prune` couples them (prune runs only if snapshots were removed); `--max-unused unlimited` minimizes time/bandwidth (keeps partly-used packs), `0` minimizes space; `--max-repack-size 0` is the documented low-scratch-space recovery mode — [restic forget docs](https://restic.readthedocs.io/en/stable/060_forget.html)
- Borg ≤1.x: `prune` deletes archives and `compact` (segments) was fused into prune; since 1.2 they are split: "Repository disk space is not freed until you run `borg compact`", docs recommend running compact regularly but "not after each borg command", e.g. once a month possibly with check, or when space is needed — [borg prune docs](https://borgbackup.readthedocs.io/en/stable/usage/prune.html); [borg compact docs](https://borgbackup.readthedocs.io/en/stable/usage/compact.html)
- Borg 1.4 `compact --threshold 10%` (default) compacts a segment only above that saving; `--threshold 0` forces maximum compaction, much slower — [borg compact docs](https://borgbackup.readthedocs.io/en/stable/usage/compact.html)
- Borg 2.x `compact` is further gated (acts only when reclaimable space ≥ threshold/5, i.e. 2% at default) plus tiny-pack merging only when small packs combine to a full-size pack; `undelete` possible after prune/delete until compact runs — [borg 2 compact docs](https://borgbackup.readthedocs.io/en/latest/usage/compact.html)
- Kopia: quick maintenance (keeps frequently-accessed `q`/`n` blobs low; never deletes metadata without another copy existing; ~hourly) vs full maintenance (Snapshot GC marking + `p`-pack compaction + dropping deleted contents; every 24h); docs warn full-maintenance effects take several hours/cycles to materialize — [Kopia maintenance docs](https://kopia.io/docs/advanced/maintenance/)
- Kopia task list makes the split explicit: `snapshot-gc`, `quick-delete-blobs` vs `full-delete-blobs`, `quick-rewrite-contents` vs `full-rewrite-contents`, `full-drop-deleted-content`, `index-compaction`, epoch tasks — [maintenance package reference](https://pkg.go.dev/github.com/kopia/kopia/repo/maintenance)
- Time Machine: thinning is metadata-cheap (hardlink/APFS-clone based; dropping a snapshot only frees blocks unique to it); local-snapshot thinning is pressure-driven, backup-volume thinning continuous; heavyweight equivalent (`hdiutil compact` of network sparsebundle, `tmutil verifychecksums`) is manual/occasional — [Apple Support](https://support.apple.com/en-la/104984); [Time Machine cheatsheet](https://gist.github.com/chrisbranson/4bf59fc6f3b1be6dd5058600959564b8)
- Veeam: cheap = per-session retention application + transform/merge of chains; expensive = active/synthetic fulls (scheduled weekly/monthly), defrag+compact full (disabled by default, scheduled separately), health check with CRC+hash of latest restore point only (monthly default, not whole chain) — [Veeam maintenance settings](https://helpcenter.veeam.com/docs/vbr/userguide/backup_copy_settings_backup.html); [Veeam health check](https://helpcenter.veeam.com/docs/vbr/userguide/backup_health_check.html)
- Veeam health check verifies only the latest restore point per chain (not history), bounding cost; if it doesn't finish before the next scheduled run the old session stops and a new one starts — [Veeam health check](https://helpcenter.veeam.com/docs/vbr/userguide/backup_health_check.html)

### Inferences
- Recommended-frequency pattern: mark/expire cheaply and often (per-backup/daily); reclaim rarely (weekly/monthly/on-pressure); verify on a third, slowest cadence (monthly/rolling-subset).
- Kopia's quick/full and Borg's prune/compact and restic's forget/prune are structurally the same two-tier design with different default couplings.

### Gaps
- No hard vendor numbers for "how often to prune" in restic docs (only "plan so it completes without interfering"); frequencies above are community practice, not vendor prescription.
- Veeam synthetic-full vs compact interaction details (when compact is worthwhile vs transform) rely on forum lore more than docs; not fully sourced here.

## What safety rails exist (dry-run defaults, locking against concurrent runs, what happens if a scheduled run is interrupted)?

### Takeaway
Every tool offers dry-run previews (none dry-run by default); all serialize maintenance with locks/ownership (restic exclusive locks, Borg repo+cache locks, Kopia single owner + exclusive lock, Veeam job-serialization with health-check yielding); interrupted cheap passes are safe to rerun, interrupted expensive passes resume or need explicit recovery steps.

### Cited Findings
- restic `forget --dry-run` prints what would be removed without removing; docs present it as the always-available preview ("You can always use `--dry-run`") — [restic forget docs](https://restic.readthedocs.io/en/stable/060_forget.html)
- restic `prune --dry-run` likewise only shows what would be done — [restic-prune manpage](https://manpages.ubuntu.com/manpages/questing/man1/restic-prune.1.html)
- restic refuses "empty" policies (e.g. `--keep-last 0` removes nothing) and requires `--unsafe-allow-remove-all` plus a host/tag/path filter to delete all of a group (since 0.17.0); `--group-by host,paths` default scopes policy per backup set as a safety feature — [restic forget docs](https://restic.readthedocs.io/en/stable/060_forget.html)
- restic locks: two types (exclusive vs shared); `prune` (and `check`) require exclusive locks, backups take shared locks and can run concurrently; locks are files under `locks/` with 30-minute staleness timeout plus same-host liveness check; `--retry-lock DURATION` waits instead of failing; exit code 11 = already locked; `unlock` removes stale locks — [restic design locks](https://github.com/restic/restic/blob/master/doc/design.rst); [restic forum locking](https://forum.restic.net/t/potential-issues-with-concurrent-execution-of-restic-commands/8099); [restic-prune manpage](https://manpages.ubuntu.com/manpages/questing/man1/restic-prune.1.html)
- restic prune is documented as interrupt-safe ("repository remains usable no matter at which point the command is interrupted") but needs scratch space; last-resort `--unsafe-recover-no-free-space` can leave repo temporarily unusable if it fails (then remove `index/` + `repair index`) — [restic forget docs](https://restic.readthedocs.io/en/stable/060_forget.html)
- Borg docs "strongly recommend" always running `prune -v --list --dry-run` first; `--stats`/`--quick-stats` and `--dry-run` are mutually exclusive; since 1.2.0 Borg retains the oldest archive if no rule would otherwise keep anything — [borg prune docs](https://borgbackup.readthedocs.io/en/stable/usage/prune.html)
- Borg 2.x splits deletion into soft-delete (`prune`/`delete` mark) + `compact` (irreversible); `undelete` recovers until compact runs — [borg 2 prune docs](https://borgbackup.readthedocs.io/en/master/usage/prune.html); [borg 2 compact docs](https://borgbackup.readthedocs.io/en/latest/usage/compact.html)
- Borg `prune` auto-removes stale checkpoint archives from interrupted backups (except the latest checkpoint, still needed) — [borg prune docs](https://borgbackup.readthedocs.io/en/stable/usage/prune.html)
- Borg locking: repository and cache locks serialize access; `borg break-lock` exists for locks left by dead processes but must only be used when no borg process is accessing the repo/cache — [borg break-lock manpage](https://manpages.debian.org/testing/borgbackup/borg-break-lock.1.en.html)
- Borg `check` receiving SIGINT stops at the next safe boundary leaving repo and chunk index consistent; recorded partial results are kept so later checks resume where they stopped; `--repair` archive runs stop between whole archives — [borg 2 check docs](https://borgbackup.readthedocs.io/en/latest/usage/check.html)
- Borg `check` is read-only by default; `--repair` is flagged "POTENTIALLY DANGEROUS ... might lead to data loss"; `--find-lost-archives` can restore lost archives only before `compact` removes their data — [borg check manpage](https://manpages.ubuntu.com/manpages/questing/man1/borg2-check.1.html)
- Kopia: single maintenance owner (`user@host`, view/change via `maintenance info`/`set --owner=me`), others never auto-run maintenance; maintenance runs under an exclusive lock (`RunExclusive`); default safety windows (`PackDeleteMinAge 24h`, snapshot-GC margin for in-flight snapshots and eventual-consistency delay) mean GC takes multiple cycles; `--safety=none` disables all of it with an explicit corruption warning — [Kopia maintenance docs](https://kopia.io/docs/advanced/maintenance/); [maintenance package reference](https://pkg.go.dev/github.com/kopia/kopia/repo/maintenance)
- Kopia `maintenance run` (quick) / `run --full` require being the owner; `--force` overrides ownership ("unsafe") — [kopia maintenance run reference](https://kopia.io/docs/reference/command-line/advanced/maintenance-run)
- Time Machine: no dry-run for thinning; safety comes from the fixed policy + `tmutil delete` operating one snapshot at a time and verification tooling (`verifychecksums`, Verify Backups); local snapshots are APFS copy-on-write so thinning never endangers live data — [tmutil reference](https://ss64.com/mac/tmutil.html); [Apple verify backups](https://support.apple.com/en-mn/guide/mac-help/mh26840/mac)
- Veeam: health check yields to any job/operation touching the backup (stops if one starts); unfinished health-check sessions are superseded by the next scheduled run; corrupted blocks trigger Error status + automatic retry session re-transferring bad blocks from source; GFS-flagged restore points cannot be deleted/modified during their retention window; immutable (hardened Linux/object-lock) repos block repair/deletion paths — [Veeam health check](https://helpcenter.veeam.com/docs/vbr/userguide/backup_health_check.html); [Veeam GFS cycles](https://helpcenter.veeam.com/docs/vbr/userguide/backup_copy_gfs_periods.html)
- Veeam append-only/immutable analog to restic: with immutability, retention cannot delete until the lock expires; Linux immutable repos "do not support repair" per health-check limitations — [Veeam health check](https://helpcenter.veeam.com/docs/vbr/userguide/backup_health_check.html)

### Inferences
- Common rail inventory for adaptation: (1) preview/dry-run, (2) empty-policy refusal / oldest-retention floor, (3) scope limiters (group-by, prefix/glob, per-machine chains), (4) exclusive locks + stale-lock recovery, (5) interrupt-safe/resumable expensive passes, (6) undo window (Borg undelete-before-compact; Kopia multi-cycle GC delay; Veeam GFS immutability windows).
- restic's append-only + `--keep-within` guidance is the sharpest documented footgun: count-based `--keep-*` policies let injected attacker snapshots displace legitimate ones at `forget` time; time-window policies bound the damage — [restic forget docs](https://restic.readthedocs.io/en/stable/060_forget.html)

### Gaps
- Borg 1.x concurrent-access failure mode specifics (which commands take exclusive vs shared locks) not re-verified from primary docs in this pass; `break-lock` semantics fetched, lock-type matrix not.
- Time Machine behavior when a scheduled backup/thinning is interrupted (power loss mid-thin) not found in Apple docs; generally treated as crash-safe via APFS but unsourced — left as gap rather than claimed.