packfs/CHANGELOG.md

# Changelog

All notable changes to this project are documented in this file, in the
style of [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This
project follows [Semantic Versioning](https://semver.org/); see
`PACKFS_VERSION_STRING` in `include/packfs.h` for the authoritative current
version (`packfs.pc`, generated by `make install`, is derived from it, not
maintained separately).

This project is pre-release (`0.x`, an initial, partial implementation of
`concept.md`, not a completed one; see `README.md`'s "Status"). `v0.1.0` is
tagged locally in this repository's own git history but has not been
pushed to any remote — there is no public release yet, only a local one.

## [Unreleased]

### Fixed
- **A failed journal write (disk full, quota, an I/O error) used to be
  silently reported as success.** `journal_append_record` and its callers
  were `void` and never checked `fwrite`/`fflush`/`fsync`'s return
  values, so `vfs_write`/`vfs_mkdir`/`vfs_unlink`/`vfs_rename` on an
  overlay-backed file all reported success even when the durable journal
  append actually failed. Fixed by propagating success/failure through
  the whole call chain; the affected call now returns `VFS_ERR_IO`.
- **A second bug, found only by reproducing the first fix's edge case
  directly: a partial write left a torn record in the journal that made
  every later, individually-*successful* write unrecoverable on reopen
  too**, not just the failed one — `journal_replay` stops at the first
  record it can't fully read, regardless of what valid records follow it.
  Fixed by rolling the journal file back to its exact pre-record length
  whenever a record fails partway. Both bugs covered permanently by the
  new `tests/test_journal_failure.c`, which forces a real write failure
  via `RLIMIT_FSIZE` rather than a mock.
- Neither bug was caught by an existing test — both were found by code
  review while investigating a question about data-integrity
  trustworthiness, then confirmed by direct reproduction before and after
  each fix, not merely inspected and assumed correct.
- **`tests/test_journal_failure.c` itself leaked memory on its normal,
  expected code path** (`vfs_free(v2)` was only called on the unreached
  `else` branch, in two separate reopen blocks) — invisible locally
  because local sanitizer runs had been using
  `ASAN_OPTIONS=detect_leaks=0`, caught by Gitea CI, which does not set
  that option. Fixed, and the local/CI sanitizer-invocation mismatch that
  let it go unnoticed is now called out explicitly in `CONTRIBUTING.md`.
  Also fixed a `-Wunused-result` warning on an intentionally-ignored
  `write()` return value in `tests/test_crash_consistency.c`'s progress
  side-channel.

### Added
- `tests/test_crash_consistency.c`: real `fork()`+`SIGKILL` fault
  injection (not a hand-truncated file) against journaled writes and
  compaction, with an independent progress side-channel so the test's own
  oracle can distinguish "never attempted before the kill" from
  "attempted and lost" — a distinction its own first draft got wrong,
  reporting 128 false failures before that side-channel was added. Zero
  real corruption found across dozens of genuine interruptions per run,
  both scenarios, on repeated runs including under ASan/UBSan.

### Documented
- The pack integrity checksum's (FNV-1a64) non-cryptographic nature —
  already documented for the compaction-dedup bug — was never explicitly
  connected to its *other* job, load-time pack integrity validation,
  where it matters more (`SECURITY.md`).
- TSan's inability to run is now confirmed in a *second*, independent
  sandboxed environment (a separate remote cloud sandbox, tried
  specifically to check whether the restriction was one environment's
  fluke), not just this project's own — same root cause
  (`personality(ADDR_NO_RANDOMIZE)` blocked by seccomp) confirmed both
  generically and against this project's actual test suite.
- **`POSTMORTEM.md`**: a detailed, permanent record of every real bug and
  environment issue found across this project's development, how each was
  actually diagnosed (including false starts), the final fix, and the
  regression-prevention artifact for each — written so a future session
  doesn't have to re-diagnose something already answered there.

### Fixed (CI)
- **`.gitea/workflows/ci.yml`'s ThreadSanitizer step was failing CI for an
  environment reason, but the actual bug was in the CI script, not the
  code under test.** Gitea Actions' `run:` steps execute under `set -e`;
  an unwrapped `OUT=$("./binary" 2>&1)` aborts the whole script on the
  first nonzero exit *before* the step's own classification logic (which
  exists specifically to not fail the build over the already-documented
  TSan-can't-start-here limitation) ever runs. Fixed by wrapping every
  such invocation in `if cmd; then RC=0; else RC=$?; fi`. Fixing this
  surfaced two more real issues, both also fixed: capturing unbounded
  flake output into a bash variable (rather than a file with a bounded
  read-back) caused a genuine misclassification at scale, and a second,
  previously undocumented segfault variant of the same TSan-can't-start
  flake (confirmed via hitting multiple unrelated binaries at random, not
  a per-binary bug). The ASan/UBSan step's retry budget was also raised
  from 3 to 5 after a real 3-in-a-row flake exhaustion was observed during
  this same verification work. See `POSTMORTEM.md` §6.2 for the full
  diagnosis, including the exact reproduction technique used to confirm
  each fix (not just reasoned about).
- A `-Wunused-result` warning on `tests/test_crash_consistency.c`'s
  intentionally-ignored `write()` return value built clean locally (a
  `(void)` cast) but still warned on the Gitea runner's own gcc — a real
  toolchain configuration difference, confirmed by reproducing clean
  locally with identical flags first. Fixed with an actual conditional
  branch on the return value instead, which is portably honored.

## [0.1.0] - 2026-09-14

### Added
- Initial implementation of `concept.md`: `vfs_*` core, `mem`/`dir`/`pack`
  backends, the copy-on-write overlay (copy-up, whiteouts, compaction, an
  append journal), path containment (`openat2`/Landlock on Linux 5.6+/5.13+,
  a documented weaker fallback elsewhere), and pack integrity validation.
- `tests/test_*.c` covering each backend, concurrency, and randomized
  structural-index stress testing against an independent reference model.
- `bench/bench.c` (`make bench`) and `BENCH.md`, comparing PackFS against
  the host filesystem across metadata operations, large sequential I/O,
  random-access pack reads, mount-table scaling, and concurrent workloads.
- `PACKFS_VERSION_MAJOR`/`MINOR`/`PATCH`/`STRING` and `pfs_version()` for
  compile-time and runtime version/ABI checks.
- `packfs.pc` (pkg-config), generated by `make install` from
  `packfs.pc.in`, version always derived from `include/packfs.h`.
- `SPDX-License-Identifier: MIT` on every file under `src/` and
  `include/`.

### Fixed
- **Bulk sequential `create`/`unlink`/`mkdir` on `mem`/`dir` was O(n²) in
  file count.** The index (`UpperSnapshot`) was a flat sorted array copied
  in full on every structural write; rewritten as a persistent treap
  (`src/upper.c`), reducing a structural write to the O(log n) nodes on
  the path to the change. See `BENCH.md`'s "Resolution" for the full
  before/after measurement and complexity-class confirmation.
- **`pack_write`'s compaction-time exact-duplicate elimination (Section
  9.2) was O(n²) in entry count, and trusted a hash match without
  comparing actual bytes** (a latent correctness bug: FNV-1a64 is
  explicitly not collision-resistant). Rewritten as an open-addressing
  hash table with a `memcmp` verification before ever reusing a
  `data_off`, fixing both at once. See `BENCH.md`'s "Resolution #2".
- **`backend_pack_new` was declared in `include/packfs.h` but never
  implemented** — any program calling it failed at link time. Implemented
  as a standalone, read-only `pack` `Backend`.
- The CI config's (`.gitea/workflows/ci.yml`) sanitizer-build steps never
  passed `-D_GNU_SOURCE` when compiling test files (only the library
  object files got it), which was harmless until a test included
  `internal.h` (needed for `pthread_rwlock_t`) and became a link failure.

### Changed
- CI moved from a GitHub-Actions-convention config (`.github/workflows/`)
  to Gitea Actions (`.gitea/workflows/`) — this project is hosted on
  Gitea, not GitHub, and was never actually hosted on GitHub; only the
  scaffolding's starting convention changed. `SECURITY.md`, `CLAUDE.md`,
  `CONTRIBUTING.md`, and `README.md` updated to match.

### Documented
- **`vfs.c`'s mount table (`MountSnapshot`) is O(n²) in mount count, the
  same pattern the file index used to have — confirmed, and deliberately
  left unfixed.** Mount counts are bounded by a program's own source code,
  not workload-driven, so they do not reach the scale that made the file
  index's O(n²) a real problem. See `BENCH.md`'s "Finding: mount table
  scaling" for the numbers and reasoning.
- A ThreadSanitizer environment limitation (some sandboxes block the
  `personality(ADDR_NO_RANDOMIZE)` syscall TSan needs) and a related
  ASan/UBSan sandbox startup flake (`AddressSanitizer:DEADLYSIGNAL`),
  both in `CONTRIBUTING.md` and `CLAUDE.md`, with the confirming tests and
  the required mitigation (`timeout`-wrapped sanitizer runs).

<!-- This repository has no remote configured yet (see `git remote -v`),
     so there is no comparison-link URL to cite here truthfully; add
     "[0.1.0]: <repo-url>/releases/tag/v0.1.0" once one exists, per Keep
     a Changelog's own convention. -->