packfs/CHANGELOG.md

Changelog

All notable changes to this project are documented in this file, in the style of Keep a Changelog. This project follows Semantic Versioning; see PACKFS_VERSION_STRING in include/packfs.h for the authoritative current version (packfs.pc, generated by make install, is derived from it, not maintained separately).

This project is pre-release (0.x, an initial, partial implementation of concept.md, not a completed one; see README.md's "Status"). v0.1.0 is tagged locally in this repository's own git history but has not been pushed to any remote — there is no public release yet, only a local one.

[Unreleased]

Fixed

• A failed journal write (disk full, quota, an I/O error) used to be silently reported as success. journal_append_record and its callers were void and never checked fwrite/fflush/fsync's return values, so vfs_write/vfs_mkdir/vfs_unlink/vfs_rename on an overlay-backed file all reported success even when the durable journal append actually failed. Fixed by propagating success/failure through the whole call chain; the affected call now returns VFS_ERR_IO.
• A second bug, found only by reproducing the first fix's edge case directly: a partial write left a torn record in the journal that made every later, individually-successful write unrecoverable on reopen too, not just the failed one — journal_replay stops at the first record it can't fully read, regardless of what valid records follow it. Fixed by rolling the journal file back to its exact pre-record length whenever a record fails partway. Both bugs covered permanently by the new tests/test_journal_failure.c, which forces a real write failure via RLIMIT_FSIZE rather than a mock.
• Neither bug was caught by an existing test — both were found by code review while investigating a question about data-integrity trustworthiness, then confirmed by direct reproduction before and after each fix, not merely inspected and assumed correct.
• tests/test_journal_failure.c itself leaked memory on its normal, expected code path (vfs_free(v2) was only called on the unreached else branch, in two separate reopen blocks) — invisible locally because local sanitizer runs had been using ASAN_OPTIONS=detect_leaks=0, caught by Gitea CI, which does not set that option. Fixed, and the local/CI sanitizer-invocation mismatch that let it go unnoticed is now called out explicitly in CONTRIBUTING.md. Also fixed a -Wunused-result warning on an intentionally-ignored write() return value in tests/test_crash_consistency.c's progress side-channel.

Added

• tests/test_crash_consistency.c: real fork()+SIGKILL fault injection (not a hand-truncated file) against journaled writes and compaction, with an independent progress side-channel so the test's own oracle can distinguish "never attempted before the kill" from "attempted and lost" — a distinction its own first draft got wrong, reporting 128 false failures before that side-channel was added. Zero real corruption found across dozens of genuine interruptions per run, both scenarios, on repeated runs including under ASan/UBSan.

Documented

• The pack integrity checksum's (FNV-1a64) non-cryptographic nature — already documented for the compaction-dedup bug — was never explicitly connected to its other job, load-time pack integrity validation, where it matters more (SECURITY.md).
• TSan's inability to run is now confirmed in a second, independent sandboxed environment (a separate remote cloud sandbox, tried specifically to check whether the restriction was one environment's fluke), not just this project's own — same root cause (personality(ADDR_NO_RANDOMIZE) blocked by seccomp) confirmed both generically and against this project's actual test suite.
• POSTMORTEM.md: a detailed, permanent record of every real bug and environment issue found across this project's development, how each was actually diagnosed (including false starts), the final fix, and the regression-prevention artifact for each — written so a future session doesn't have to re-diagnose something already answered there.

Fixed (CI)

• .gitea/workflows/ci.yml's ThreadSanitizer step was failing CI for an environment reason, but the actual bug was in the CI script, not the code under test. Gitea Actions' run: steps execute under set -e; an unwrapped OUT=$("./binary" 2>&1) aborts the whole script on the first nonzero exit before the step's own classification logic (which exists specifically to not fail the build over the already-documented TSan-can't-start-here limitation) ever runs. Fixed by wrapping every such invocation in if cmd; then RC=0; else RC=$?; fi. Fixing this surfaced two more real issues, both also fixed: capturing unbounded flake output into a bash variable (rather than a file with a bounded read-back) caused a genuine misclassification at scale, and a second, previously undocumented segfault variant of the same TSan-can't-start flake (confirmed via hitting multiple unrelated binaries at random, not a per-binary bug). The ASan/UBSan step's retry budget was also raised from 3 to 5 after a real 3-in-a-row flake exhaustion was observed during this same verification work. See POSTMORTEM.md §6.2 for the full diagnosis, including the exact reproduction technique used to confirm each fix (not just reasoned about).
• A -Wunused-result warning on tests/test_crash_consistency.c's intentionally-ignored write() return value built clean locally (a (void) cast) but still warned on the Gitea runner's own gcc — a real toolchain configuration difference, confirmed by reproducing clean locally with identical flags first. Fixed with an actual conditional branch on the return value instead, which is portably honored.

[0.1.0] - 2026-09-14

Added

• Initial implementation of concept.md: vfs_* core, mem/dir/pack backends, the copy-on-write overlay (copy-up, whiteouts, compaction, an append journal), path containment (openat2/Landlock on Linux 5.6+/5.13+, a documented weaker fallback elsewhere), and pack integrity validation.
• tests/test_*.c covering each backend, concurrency, and randomized structural-index stress testing against an independent reference model.
• bench/bench.c (make bench) and BENCH.md, comparing PackFS against the host filesystem across metadata operations, large sequential I/O, random-access pack reads, mount-table scaling, and concurrent workloads.
• PACKFS_VERSION_MAJOR/MINOR/PATCH/STRING and pfs_version() for compile-time and runtime version/ABI checks.
• packfs.pc (pkg-config), generated by make install from packfs.pc.in, version always derived from include/packfs.h.
• SPDX-License-Identifier: MIT on every file under src/ and include/.

Fixed

• Bulk sequential create/unlink/mkdir on mem/dir was O(n²) in file count. The index (UpperSnapshot) was a flat sorted array copied in full on every structural write; rewritten as a persistent treap (src/upper.c), reducing a structural write to the O(log n) nodes on the path to the change. See BENCH.md's "Resolution" for the full before/after measurement and complexity-class confirmation.
• pack_write's compaction-time exact-duplicate elimination (Section 9.2) was O(n²) in entry count, and trusted a hash match without comparing actual bytes (a latent correctness bug: FNV-1a64 is explicitly not collision-resistant). Rewritten as an open-addressing hash table with a memcmp verification before ever reusing a data_off, fixing both at once. See BENCH.md's "Resolution #2".
• backend_pack_new was declared in include/packfs.h but never implemented — any program calling it failed at link time. Implemented as a standalone, read-only pack Backend.
• The CI config's (.gitea/workflows/ci.yml) sanitizer-build steps never passed -D_GNU_SOURCE when compiling test files (only the library object files got it), which was harmless until a test included internal.h (needed for pthread_rwlock_t) and became a link failure.

Changed

• CI moved from a GitHub-Actions-convention config (.github/workflows/) to Gitea Actions (.gitea/workflows/) — this project is hosted on Gitea, not GitHub, and was never actually hosted on GitHub; only the scaffolding's starting convention changed. SECURITY.md, CLAUDE.md, CONTRIBUTING.md, and README.md updated to match.

Documented

• vfs.c's mount table (MountSnapshot) is O(n²) in mount count, the same pattern the file index used to have — confirmed, and deliberately left unfixed. Mount counts are bounded by a program's own source code, not workload-driven, so they do not reach the scale that made the file index's O(n²) a real problem. See BENCH.md's "Finding: mount table scaling" for the numbers and reasoning.
• A ThreadSanitizer environment limitation (some sandboxes block the personality(ADDR_NO_RANDOMIZE) syscall TSan needs) and a related ASan/UBSan sandbox startup flake (AddressSanitizer:DEADLYSIGNAL), both in CONTRIBUTING.md and CLAUDE.md, with the confirming tests and the required mitigation (timeout-wrapped sanitizer runs).