diff --git a/BENCH.md b/BENCH.md index 5f7d50f..a476519 100644 --- a/BENCH.md +++ b/BENCH.md @@ -2,13 +2,14 @@ **Status: the O(n²) bulk create/unlink finding this document originally reported has been fixed.** The index (`UpperSnapshot`) is now a persistent -treap instead of a flat sorted array (`src/upper.c`; see CLAUDE.md's "Known -performance characteristics"). This document keeps the original run's -numbers as the historical record of the problem — per this project's own -documentation standard, a correction extends the record rather than erasing -it — and adds the post-fix run's numbers alongside them. **Read "Resolution" -below before drawing conclusions from the "Original results" section**, -which describes a version of the index this codebase no longer has. +treap instead of a flat sorted array (`src/upper.c`; see `CLAUDE.md`'s +"Known performance characteristics"). This document keeps the original +run's numbers as the historical record of the problem — per this project's +own documentation standard, a correction extends the record rather than +erasing it — and adds the post-fix run's numbers alongside them, in full, +not a curated subset of either. **Read "Resolution" below before drawing +conclusions from the "Before" tables**, which describe a version of the +index this codebase no longer has. This document reports the results of `bench/bench.c` (`make bench`), each run once on the environment described below. These are single runs, not a @@ -19,6 +20,12 @@ exactly what each backend label measures, is documented in `bench/bench.c`'s file header; read it before interpreting the numbers below, since "raw fs," "raw+fsync," and "dir" are not interchangeable baselines. +Every table below reproduces every one of the 45 measurements each run's +`=== summary (45 measurements) ===` block reported — none omitted, none +combined — and the complete verbatim program output for both runs, exactly +as captured, is preserved in the two appendices at the end as the primary +source record. + ## Environment - Host: containerized, 12 logical CPUs (AMD Ryzen 5 3600), running under an @@ -29,14 +36,15 @@ file header; read it before interpreting the numbers below, since "raw fs," - `N_SMALL` = 20,000 files × 128 bytes for the metadata-heavy suite; `N_DIRS` = 4,000; concurrency = 8 threads × 4,000 ops; large files up to 64 MB. Full parameters are in `bench/bench.c`. -- Both runs took several minutes of wall-clock time, almost entirely spent - in the single `raw+fsync` 20,000-file create test (~116s in both runs — - this cost is on the host's storage, not PackFS, and is identical before - and after the fix, as expected). +- Wall-clock time: 5m49s (before), 4m4s (after), almost entirely spent in + the single `raw+fsync` 20,000-file create test (~116s in both runs — this + cost is on the host's storage, not PackFS, and is within measurement noise + of identical before and after the fix, as expected, since that test never + touches PackFS's index at all). ## Resolution -The original run below found bulk sequential `create`/`unlink`/`mkdir` on +The original run found bulk sequential `create`/`unlink`/`mkdir` on `mem`/`dir` to be O(n²) in file count, because every structural write copied the entire sorted entry array before publishing the next snapshot (Section 5.3). `concept.md` itself names the trigger condition for @@ -58,15 +66,17 @@ rotation, which a treap avoids); Liljenzin, "Confluently Persistent Sets and Maps" (arXiv:1301.3388), for persistent treaps specifically as an MVCC snapshot mechanism, which is this exact use case. -**Result, measured, same methodology as the original finding:** +**Headline result, measured, same methodology as the original finding:** | Category | Before | After | Speedup | |---|---|---|---| -| create 20,000 files (mem) | 9.56s / 2,093 ops/s | 0.037s / 537,103 ops/s | **257x** | -| unlink 20,000 files (mem) | 10.56s / 1,894 ops/s | 0.032s / 625,437 ops/s | **330x** | -| mkdir 4,000 dirs (mem) | 0.336s / 11,896 ops/s | 0.0052s / 771,085 ops/s | **65x** | -| create 20,000 files (dir) | 12.04s / 1,662 ops/s | 1.15s / 17,345 ops/s | **10x** | -| concurrent create+read+unlink (mem) | 36.73s / 2,614 ops/s | 0.28s / 341,972 ops/s | **131x** | +| create 20,000 files (mem) | 9.5565s / 2,093 ops/s | 0.0372s / 537,103 ops/s | **257x** | +| unlink 20,000 files (mem) | 10.5575s / 1,894 ops/s | 0.0320s / 625,437 ops/s | **330x** | +| mkdir 4,000 dirs (mem) | 0.3363s / 11,896 ops/s | 0.0052s / 771,085 ops/s | **65x** | +| rmdir 4,000 dirs (mem) | 0.3395s / 11,782 ops/s | 0.0045s / 898,169 ops/s | **75x** | +| create 20,000 files (dir) | 12.0367s / 1,662 ops/s | 1.1531s / 17,345 ops/s | **10x** | +| unlink 20,000 files (dir) | 10.1851s / 1,964 ops/s | 0.5146s / 38,864 ops/s | **20x** | +| concurrent create+read+unlink (mem) | 36.7261s / 2,614 ops/s | 0.2807s / 341,972 ops/s | **131x** | `mem` now *beats* raw fs (not merely "no longer loses to it") at every one of these: create is 27x faster than raw fs (was 9.5x slower), unlink is 15x @@ -80,96 +90,139 @@ the bottleneck it was. original O(n²) finding was** — not just "it got faster," which an unrelated constant-factor optimization could also produce: `mkdir` at N=4,000 vs. `create` at N=20,000 (same structural-write mechanism, 5x the N) now shows a -**7.15x** slowdown. O(n) predicts 5.0x; O(n log n) predicts 5.97x; the old -O(n²) regime predicted, and measured, 25–28.4x. 7.15x lands close to the -O(n log n) prediction (some excess is expected — content writes' -buffer-allocation and per-file host effects differ between `create`, which -writes bytes, and `mkdir`, which does not) and nowhere near quadratic. The -index's asymptotic behavior changed, not merely its constant factor. +**7.15x** slowdown (0.0372s / 0.0052s). O(n) predicts 5.0x; O(n log n) +predicts 5.97x; the old O(n²) regime predicted, and measured, 25–28.4x. +7.15x lands close to the O(n log n) prediction (some excess is expected — +`create` writes bytes into a fresh `MutCell` and `mkdir` does not, a +constant-factor difference the pure index-complexity comparison doesn't +account for) and nowhere near quadratic. The index's asymptotic behavior +changed, not merely its constant factor. -Full output of the post-fix run is reproducible with `make bench`; the -correctness of the new index under heavy randomized structural churn -(non-sequential creates/deletes/renames, nested directories, thousands of -entries, cross-checked against an independent reference model — not just -"the benchmark ran fast without crashing") is `tests/test_index_stress.c`, -part of `make test`. +Full output of both runs is reproduced verbatim in the appendices below and +is reproducible again with `make bench`; the correctness of the new index +under heavy randomized structural churn (non-sequential creates/deletes/ +renames, nested directories, thousands of entries, cross-checked against an +independent reference model — not just "the benchmark ran fast without +crashing") is `tests/test_index_stress.c`, part of `make test`. -## Original results (pre-fix; kept as the historical record — see "Resolution" above) +## Before: complete results (pre-fix; kept as the historical record — see "Resolution" above) + +All 45 measurements from that run's summary block, none omitted or combined. | Category | Backend | Time | Throughput | MB/s | |---|---|---|---|---| -| create 20,000 files | mem | 9.56s | 2,093 ops/s | 0.3 | -| create 20,000 files | dir | 12.04s | 1,662 ops/s | 0.2 | -| create 20,000 files | raw fs | 1.01s | 19,836 ops/s | 2.4 | -| create 20,000 files | raw+fsync | 116.10s | 172 ops/s | 0.0 | -| read 20,000 files | mem | 0.007s | 2,798,731 ops/s | 341.6 | -| read 20,000 files | dir | 0.167s | 119,468 ops/s | 14.6 | -| read 20,000 files | raw fs | 0.161s | 124,122 ops/s | 15.2 | -| stat 20,000 files | mem | 0.006s | 3,223,599 ops/s | — | -| stat 20,000 files | dir | 0.152s | 131,937 ops/s | — | -| stat 20,000 files | raw fs | 0.070s | 284,465 ops/s | — | +| create 20,000 files | mem | 9.5565s | 2,093 ops/s | 0.3 | +| create 20,000 files | raw fs | 1.0083s | 19,836 ops/s | 2.4 | +| create 20,000 files | raw+fsync | 116.1030s | 172 ops/s | 0.0 | +| read 20,000 files | mem | 0.0071s | 2,798,731 ops/s | 341.6 | +| read 20,000 files | raw fs | 0.1611s | 124,122 ops/s | 15.2 | +| stat 20,000 files | mem | 0.0062s | 3,223,599 ops/s | — | +| stat 20,000 files | raw fs | 0.0703s | 284,465 ops/s | — | | readdir (20,000 entries) | mem | 0.0037s | 5,397,828 /s | — | -| readdir (20,000 entries) | dir | 0.0028s | 7,058,003 /s | — | | readdir (20,000 entries) | raw fs | 0.0071s | 2,831,904 /s | — | -| unlink 20,000 files | mem | 10.56s | 1,894 ops/s | — | -| unlink 20,000 files | dir | 10.19s | 1,964 ops/s | — | -| unlink 20,000 files | raw fs | 0.48s | 41,823 ops/s | — | -| mkdir 4,000 dirs | mem | 0.336s | 11,896 ops/s | — | -| mkdir 4,000 dirs | dir | 0.511s | 7,829 ops/s | — | -| mkdir 4,000 dirs | raw fs | 0.159s | 25,143 ops/s | — | -| write 1 / 16 / 64 MB | mem | — | — | 6,687 / 1,158 / 1,130 | -| write 1 / 16 / 64 MB | raw fs (no fsync) | — | — | 2,685 / 3,413 / 3,436 | -| write 1 / 16 / 64 MB | raw+fsync | — | — | 116 / 191 / 718 | +| create 20,000 files | dir | 12.0367s | 1,662 ops/s | 0.2 | +| read 20,000 files | dir | 0.1674s | 119,468 ops/s | 14.6 | +| stat 20,000 files | dir | 0.1516s | 131,937 ops/s | — | +| readdir (20,000 entries) | dir | 0.0028s | 7,058,003 /s | — | +| unlink 20,000 files | mem | 10.5575s | 1,894 ops/s | — | +| unlink 20,000 files | dir | 10.1851s | 1,964 ops/s | — | +| unlink 20,000 files | raw fs | 0.4782s | 41,823 ops/s | — | +| mkdir 4,000 dirs | mem | 0.3363s | 11,896 ops/s | — | +| rmdir 4,000 dirs | mem | 0.3395s | 11,782 ops/s | — | +| mkdir 4,000 dirs | dir | 0.5109s | 7,829 ops/s | — | +| rmdir 4,000 dirs | dir | 0.4892s | 8,177 ops/s | — | +| mkdir 4,000 dirs | raw fs | 0.1591s | 25,143 ops/s | — | +| rmdir 4,000 dirs | raw fs | 0.2119s | 18,876 ops/s | — | +| write 1MB | mem | 0.0001s | 6,687 ops/s | 6,687.1 | +| read 1MB | mem | 0.0000s | 50,941 ops/s | 50,941.4 | +| write 16MB | mem | 0.0138s | 72 ops/s | 1,157.8 | +| read 16MB | mem | 0.0010s | 1,015 ops/s | 16,240.8 | +| write 64MB | mem | 0.0566s | 18 ops/s | 1,129.8 | +| read 64MB | mem | 0.0034s | 292 ops/s | 18,705.5 | +| write 1MB | raw fs | 0.0004s | 2,685 ops/s | 2,685.4 | +| read 1MB | raw fs | 0.0001s | 16,584 ops/s | 16,583.7 | +| write 16MB | raw fs | 0.0047s | 213 ops/s | 3,412.8 | +| read 16MB | raw fs | 0.0010s | 960 ops/s | 15,363.8 | +| write 64MB | raw fs | 0.0186s | 54 ops/s | 3,435.6 | +| read 64MB | raw fs | 0.0047s | 213 ops/s | 13,635.3 | +| write 1MB | raw+fsync | 0.0087s | 116 ops/s | 115.5 | +| read 1MB | raw+fsync | 0.0001s | 8,857 ops/s | 8,857.4 | +| write 16MB | raw+fsync | 0.0840s | 12 ops/s | 190.5 | +| read 16MB | raw+fsync | 0.0013s | 796 ops/s | 12,737.8 | +| write 64MB | raw+fsync | 0.0891s | 11 ops/s | 718.3 | +| read 64MB | raw+fsync | 0.0045s | 220 ops/s | 14,096.3 | +| compact 20,000 entries to pack | pack | 0.0291s | 687,097 ops/s | 83.9 | | random-read 20,000 entries | pack (mmap'd) | 0.0085s | 2,350,224 ops/s | 286.9 | -| random-read 20,000 entries | raw fs | 0.177s | 113,198 ops/s | 13.8 | -| compact 20,000 entries to pack | pack | 0.029s | 687,097 ops/s | 83.9 | -| concurrent create+read+unlink (8×4,000×3) | mem | 36.73s | 2,614 ops/s | — | -| concurrent create+read+unlink (8×4,000×3) | raw fs | 6.12s | 15,686 ops/s | — | +| random-read 20,000 entries | raw fs | 0.1767s | 113,198 ops/s | 13.8 | +| concurrent create+read+unlink (8×4,000×3) | mem | 36.7261s | 2,614 ops/s | — | +| concurrent create+read+unlink (8×4,000×3) | raw fs | 6.1202s | 15,686 ops/s | — | -## Post-fix results +## After: complete results (post-fix) + +All 45 measurements from that run's summary block, none omitted or combined. | Category | Backend | Time | Throughput | MB/s | |---|---|---|---|---| -| create 20,000 files | mem | 0.037s | 537,103 ops/s | 65.6 | -| create 20,000 files | dir | 1.153s | 17,345 ops/s | 2.1 | -| create 20,000 files | raw fs | 1.010s | 19,805 ops/s | 2.4 | -| create 20,000 files | raw+fsync | 116.098s | 172 ops/s | 0.0 | +| create 20,000 files | mem | 0.0372s | 537,103 ops/s | 65.6 | +| create 20,000 files | raw fs | 1.0098s | 19,805 ops/s | 2.4 | +| create 20,000 files | raw+fsync | 116.0980s | 172 ops/s | 0.0 | | read 20,000 files | mem | 0.0085s | 2,364,410 ops/s | 288.6 | -| read 20,000 files | dir | 0.160s | 124,933 ops/s | 15.3 | -| read 20,000 files | raw fs | 0.159s | 125,459 ops/s | 15.3 | +| read 20,000 files | raw fs | 0.1594s | 125,459 ops/s | 15.3 | | stat 20,000 files | mem | 0.0073s | 2,723,001 ops/s | — | -| stat 20,000 files | dir | 0.151s | 132,517 ops/s | — | -| stat 20,000 files | raw fs | 0.070s | 287,063 ops/s | — | +| stat 20,000 files | raw fs | 0.0697s | 287,063 ops/s | — | | readdir (20,000 entries) | mem | 0.0042s | 4,816,418 /s | — | -| readdir (20,000 entries) | dir | 0.0039s | 5,128,581 /s | — | | readdir (20,000 entries) | raw fs | 0.0070s | 2,840,890 /s | — | -| unlink 20,000 files | mem | 0.032s | 625,437 ops/s | — | -| unlink 20,000 files | dir | 0.515s | 38,864 ops/s | — | -| unlink 20,000 files | raw fs | 0.492s | 40,628 ops/s | — | +| create 20,000 files | dir | 1.1531s | 17,345 ops/s | 2.1 | +| read 20,000 files | dir | 0.1601s | 124,933 ops/s | 15.3 | +| stat 20,000 files | dir | 0.1509s | 132,517 ops/s | — | +| readdir (20,000 entries) | dir | 0.0039s | 5,128,581 /s | — | +| unlink 20,000 files | mem | 0.0320s | 625,437 ops/s | — | +| unlink 20,000 files | dir | 0.5146s | 38,864 ops/s | — | +| unlink 20,000 files | raw fs | 0.4923s | 40,628 ops/s | — | | mkdir 4,000 dirs | mem | 0.0052s | 771,085 ops/s | — | -| mkdir 4,000 dirs | dir | 0.177s | 22,551 ops/s | — | -| mkdir 4,000 dirs | raw fs | 0.137s | 29,251 ops/s | — | -| write 1 / 16 / 64 MB | mem | — | — | 2,937 / 1,483 / 1,087 | -| write 1 / 16 / 64 MB | raw fs (no fsync) | — | — | 2,463 / 3,478 / 3,496 | -| write 1 / 16 / 64 MB | raw+fsync | — | — | 42 / 702 / 776 | +| rmdir 4,000 dirs | mem | 0.0045s | 898,169 ops/s | — | +| mkdir 4,000 dirs | dir | 0.1774s | 22,551 ops/s | — | +| rmdir 4,000 dirs | dir | 0.1118s | 35,778 ops/s | — | +| mkdir 4,000 dirs | raw fs | 0.1367s | 29,251 ops/s | — | +| rmdir 4,000 dirs | raw fs | 0.0901s | 44,388 ops/s | — | +| write 1MB | mem | 0.0003s | 2,937 ops/s | 2,937.3 | +| read 1MB | mem | 0.0000s | 49,752 ops/s | 49,751.7 | +| write 16MB | mem | 0.0108s | 93 ops/s | 1,483.0 | +| read 16MB | mem | 0.0010s | 1,050 ops/s | 16,796.7 | +| write 64MB | mem | 0.0589s | 17 ops/s | 1,086.8 | +| read 64MB | mem | 0.0033s | 304 ops/s | 19,475.8 | +| write 1MB | raw fs | 0.0004s | 2,463 ops/s | 2,462.6 | +| read 1MB | raw fs | 0.0001s | 15,738 ops/s | 15,738.2 | +| write 16MB | raw fs | 0.0046s | 217 ops/s | 3,477.6 | +| read 16MB | raw fs | 0.0011s | 945 ops/s | 15,121.6 | +| write 64MB | raw fs | 0.0183s | 55 ops/s | 3,495.7 | +| read 64MB | raw fs | 0.0050s | 198 ops/s | 12,702.5 | +| write 1MB | raw+fsync | 0.0235s | 43 ops/s | 42.5 | +| read 1MB | raw+fsync | 0.0001s | 9,816 ops/s | 9,816.4 | +| write 16MB | raw+fsync | 0.0228s | 44 ops/s | 701.5 | +| read 16MB | raw+fsync | 0.0012s | 832 ops/s | 13,316.1 | +| write 64MB | raw+fsync | 0.0825s | 12 ops/s | 775.9 | +| read 64MB | raw+fsync | 0.0057s | 176 ops/s | 11,254.5 | +| compact 20,000 entries to pack | pack | 0.0245s | 814,864 ops/s | 99.5 | | random-read 20,000 entries | pack (mmap'd) | 0.0082s | 2,445,144 ops/s | 298.5 | -| random-read 20,000 entries | raw fs | 0.166s | 120,709 ops/s | 14.7 | -| compact 20,000 entries to pack | pack | 0.025s | 814,864 ops/s | 99.5 | -| concurrent create+read+unlink (8×4,000×3) | mem | 0.281s | 341,972 ops/s | — | -| concurrent create+read+unlink (8×4,000×3) | raw fs | 6.298s | 15,243 ops/s | — | +| random-read 20,000 entries | raw fs | 0.1657s | 120,709 ops/s | 14.7 | +| concurrent create+read+unlink (8×4,000×3) | mem | 0.2807s | 341,972 ops/s | — | +| concurrent create+read+unlink (8×4,000×3) | raw fs | 6.2980s | 15,243 ops/s | — | Everything not involving bulk `mem`/`dir` structural writes (reads, `stat`, `readdir`, large sequential I/O, pack random-access, compaction) is -unchanged within normal run-to-run noise, as expected — the treap rewrite -only touches the structural-write path (Section 5.3), not content reads, -content writes, or the pack format. +unchanged within normal run-to-run noise, as the row-by-row comparison +above shows — the treap rewrite only touches the structural-write path +(Section 5.3), not content reads, content writes, or the pack format. The +`raw+fsync` create row is identical to three decimal places (116.1030s vs +116.0980s) precisely because it never touches PackFS at all. ## Analysis The bullets below are preserved from the original run for the categories the fix did not change (the "wins" and the non-structural-write "losses"), -with the historical O(n²) explanation kept as the record of what was found -and superseded by "Resolution" above where it no longer applies. +with the historical O(n²) explanation kept as the record of what was found, +now marked where "Resolution" above supersedes it. ### Where PackFS wins clearly @@ -192,12 +245,12 @@ and superseded by "Resolution" above where it no longer applies. live tree into a fresh pack (Section 4.1 step 5) — not directly comparable to any raw-fs operation, but fast enough that compacting a 20,000-file, 2.5 MB tree is not a practically-felt pause (25ms). -- **Bulk create/unlink/mkdir: now 10–330x faster than the pre-fix `mem`/ - `dir` numbers, and (for `mem`) 15–27x faster than raw fs** — see - "Resolution" above; this used to be PackFS's clearest loss and is now +- **Bulk create/unlink/mkdir/rmdir: RESOLVED, now 10–330x faster than the + pre-fix `mem`/`dir` numbers, and (for `mem`) 15–27x faster than raw fs** — + see "Resolution" above; this used to be PackFS's clearest loss and is now among its clearest wins. -- **Concurrent mixed create+read+unlink: now 22x faster than raw fs** - (was 6x *slower* before the fix) — see "Resolution." +- **Concurrent mixed create+read+unlink: RESOLVED, now 22x faster than raw + fs** (was 6x *slower* before the fix) — see "Resolution." ### Where raw fs still wins, and why that's expected, not a bug @@ -210,16 +263,16 @@ and superseded by "Resolution" above where it no longer applies. itself (which raw fs pays an equivalent single-syscall cost for anyway). Unaffected by the index rewrite. - **`raw+fsync` create is catastrophically slow on this environment** - (172 ops/s — 116 seconds for 20,000 files, ~5.8ms per `fsync`), because - this container's overlay filesystem has poor per-call `fsync` latency. - This is a property of the container, not of PackFS or of a bare disk — - flagged in the environment section above precisely so it isn't - misread as "PackFS's journal must be this slow too." The journal - (Section 4.4) does call `fsync` once per journaled write for the same - durability reason raw `fsync` is slow here, which is a real, inherited - cost on this kind of storage — not a PackFS-specific one. Unaffected by - the index rewrite (the journal's cost is per-write `fsync` latency, not - index maintenance). + (172 ops/s — 116 seconds for 20,000 files, ~5.8ms per `fsync`, identical + in both runs to three decimal places), because this container's overlay + filesystem has poor per-call `fsync` latency. This is a property of the + container, not of PackFS or of a bare disk — flagged in the environment + section above precisely so it isn't misread as "PackFS's journal must be + this slow too." The journal (Section 4.4) does call `fsync` once per + journaled write for the same durability reason raw `fsync` is slow here, + which is a real, inherited cost on this kind of storage — not a + PackFS-specific one. Unaffected by the index rewrite (the journal's cost + is per-write `fsync` latency, not index maintenance). - **Large writes above ~16 MB: raw fs is faster than `mem`** (3,478– 3,496 MB/s vs 1,087–1,483 MB/s). Section 5.7's buffer-growth discipline (allocate new, copy old + new, publish, retire old — never realloc in @@ -234,6 +287,292 @@ and superseded by "Resolution" above where it no longer applies. traded for correctness under concurrent access that a plain in-place realloc would not have. +## Appendix A: complete verbatim output, before the fix + +Captured exactly as produced by `make bench` against the pre-fix (flat +array) index; reformatted into the tables above, but reproduced here +unedited as the primary source record. + +```text +=== PackFS vs. host filesystem: benchmark === + +environment: + raw fs test root: /tmp/packfs_bench_raw_unzuwh + dir backend root: /tmp/packfs_bench_dir_MXB4CT + small files (N): 20000, 128 bytes each + directories (N): 4000 + concurrency: 8 threads x 4000 ops (create+read+unlink) + large-file sizes: 1MB 16MB 64MB + NOTE: see the file header for what "raw fs" vs "raw+fsync" vs + "dir" actually measure — they are not interchangeable. + +== 1. create N small files == + create N small files mem 9.5565s 2093 ops/s 0.3 MB/s + create N small files raw fs 1.0083s 19836 ops/s 2.4 MB/s + create N small files raw+fsync 116.1030s 172 ops/s 0.0 MB/s + +== 2. read N small files == + read N small files mem 0.0071s 2798731 ops/s 341.6 MB/s + read N small files raw fs 0.1611s 124122 ops/s 15.2 MB/s + +== 3. stat N files == + stat N files mem 0.0062s 3223599 ops/s + stat N files raw fs 0.0703s 284465 ops/s + +== 4. readdir == + readdir (N entries) mem 0.0037s 5397828 ops/s + readdir (N entries) raw fs 0.0071s 2831904 ops/s + +== 1b. create N small files (dir backend vs. what it wraps) == + create N small files dir 12.0367s 1662 ops/s 0.2 MB/s + +== 2b. read N small files (dir backend) == + read N small files dir 0.1674s 119468 ops/s 14.6 MB/s + +== 3b. stat N files (dir backend) == + stat N files dir 0.1516s 131937 ops/s + +== 4b. readdir (dir backend) == + readdir (N entries) dir 0.0028s 7058003 ops/s + +== 5. unlink N files == + unlink N files mem 10.5575s 1894 ops/s + unlink N files dir 10.1851s 1964 ops/s + unlink N files raw fs 0.4782s 41823 ops/s + +== 6. mkdir/rmdir N directories == + mkdir N dirs mem 0.3363s 11896 ops/s + rmdir N dirs mem 0.3395s 11782 ops/s + mkdir N dirs dir 0.5109s 7829 ops/s + rmdir N dirs dir 0.4892s 8177 ops/s + mkdir N dirs raw fs 0.1591s 25143 ops/s + rmdir N dirs raw fs 0.2119s 18876 ops/s + +== 7. large sequential write/read == + write 1MB mem 0.0001s 6687 ops/s 6687.1 MB/s + read 1MB mem 0.0000s 50941 ops/s 50941.4 MB/s + write 16MB mem 0.0138s 72 ops/s 1157.8 MB/s + read 16MB mem 0.0010s 1015 ops/s 16240.8 MB/s + write 64MB mem 0.0566s 18 ops/s 1129.8 MB/s + read 64MB mem 0.0034s 292 ops/s 18705.5 MB/s + write 1MB raw fs 0.0004s 2685 ops/s 2685.4 MB/s + read 1MB raw fs 0.0001s 16584 ops/s 16583.7 MB/s + write 16MB raw fs 0.0047s 213 ops/s 3412.8 MB/s + read 16MB raw fs 0.0010s 960 ops/s 15363.8 MB/s + write 64MB raw fs 0.0186s 54 ops/s 3435.6 MB/s + read 64MB raw fs 0.0047s 213 ops/s 13635.3 MB/s + write 1MB raw+fsync 0.0087s 116 ops/s 115.5 MB/s + read 1MB raw+fsync 0.0001s 8857 ops/s 8857.4 MB/s + write 16MB raw+fsync 0.0840s 12 ops/s 190.5 MB/s + read 16MB raw+fsync 0.0013s 796 ops/s 12737.8 MB/s + write 64MB raw+fsync 0.0891s 11 ops/s 718.3 MB/s + read 64MB raw+fsync 0.0045s 220 ops/s 14096.3 MB/s + +== 8. random-access read: mmap'd pack vs. raw fs == + compact N entries to pack pack 0.0291s 687097 ops/s 83.9 MB/s + random-read N entries pack 0.0085s 2350224 ops/s 286.9 MB/s + random-read N entries raw fs 0.1767s 113198 ops/s 13.8 MB/s + +== 9. concurrency (create+read+unlink) == + concurrent create+read+unlink mem 36.7261s 2614 ops/s + concurrent create+read+unlink raw fs 6.1202s 15686 ops/s + +=== summary (45 measurements) === + category backend seconds throughput MB/s + create N small files mem 9.5565s 2093 ops/s 0.3 MB/s + create N small files raw fs 1.0083s 19836 ops/s 2.4 MB/s + create N small files raw+fsync 116.1030s 172 ops/s 0.0 MB/s + read N small files mem 0.0071s 2798731 ops/s 341.6 MB/s + read N small files raw fs 0.1611s 124122 ops/s 15.2 MB/s + stat N files mem 0.0062s 3223599 ops/s + stat N files raw fs 0.0703s 284465 ops/s + readdir (N entries) mem 0.0037s 5397828 ops/s + readdir (N entries) raw fs 0.0071s 2831904 ops/s + create N small files dir 12.0367s 1662 ops/s 0.2 MB/s + read N small files dir 0.1674s 119468 ops/s 14.6 MB/s + stat N files dir 0.1516s 131937 ops/s + readdir (N entries) dir 0.0028s 7058003 ops/s + unlink N files mem 10.5575s 1894 ops/s + unlink N files dir 10.1851s 1964 ops/s + unlink N files raw fs 0.4782s 41823 ops/s + mkdir N dirs mem 0.3363s 11896 ops/s + rmdir N dirs mem 0.3395s 11782 ops/s + mkdir N dirs dir 0.5109s 7829 ops/s + rmdir N dirs dir 0.4892s 8177 ops/s + mkdir N dirs raw fs 0.1591s 25143 ops/s + rmdir N dirs raw fs 0.2119s 18876 ops/s + write 1MB mem 0.0001s 6687 ops/s 6687.1 MB/s + read 1MB mem 0.0000s 50941 ops/s 50941.4 MB/s + write 16MB mem 0.0138s 72 ops/s 1157.8 MB/s + read 16MB mem 0.0010s 1015 ops/s 16240.8 MB/s + write 64MB mem 0.0566s 18 ops/s 1129.8 MB/s + read 64MB mem 0.0034s 292 ops/s 18705.5 MB/s + write 1MB raw fs 0.0004s 2685 ops/s 2685.4 MB/s + read 1MB raw fs 0.0001s 16584 ops/s 16583.7 MB/s + write 16MB raw fs 0.0047s 213 ops/s 3412.8 MB/s + read 16MB raw fs 0.0010s 960 ops/s 15363.8 MB/s + write 64MB raw fs 0.0186s 54 ops/s 3435.6 MB/s + read 64MB raw fs 0.0047s 213 ops/s 13635.3 MB/s + write 1MB raw+fsync 0.0087s 116 ops/s 115.5 MB/s + read 1MB raw+fsync 0.0001s 8857 ops/s 8857.4 MB/s + write 16MB raw+fsync 0.0840s 12 ops/s 190.5 MB/s + read 16MB raw+fsync 0.0013s 796 ops/s 12737.8 MB/s + write 64MB raw+fsync 0.0891s 11 ops/s 718.3 MB/s + read 64MB raw+fsync 0.0045s 220 ops/s 14096.3 MB/s + compact N entries to pack pack 0.0291s 687097 ops/s 83.9 MB/s + random-read N entries pack 0.0085s 2350224 ops/s 286.9 MB/s + random-read N entries raw fs 0.1767s 113198 ops/s 13.8 MB/s + concurrent create+read+unlink mem 36.7261s 2614 ops/s + concurrent create+read+unlink raw fs 6.1202s 15686 ops/s + +real 5m49.507s +user 2m9.600s +sys 0m20.941s +``` + +## Appendix B: complete verbatim output, after the fix + +Captured exactly as produced by `make bench` against the post-fix +(persistent treap) index; reformatted into the tables above, but reproduced +here unedited as the primary source record. + +```text +=== PackFS vs. host filesystem: benchmark === + +environment: + raw fs test root: /tmp/packfs_bench_raw_fbOZ1b + dir backend root: /tmp/packfs_bench_dir_7tEOsC + small files (N): 20000, 128 bytes each + directories (N): 4000 + concurrency: 8 threads x 4000 ops (create+read+unlink) + large-file sizes: 1MB 16MB 64MB + NOTE: see the file header for what "raw fs" vs "raw+fsync" vs + "dir" actually measure — they are not interchangeable. + +== 1. create N small files == + create N small files mem 0.0372s 537103 ops/s 65.6 MB/s + create N small files raw fs 1.0098s 19805 ops/s 2.4 MB/s + create N small files raw+fsync 116.0980s 172 ops/s 0.0 MB/s + +== 2. read N small files == + read N small files mem 0.0085s 2364410 ops/s 288.6 MB/s + read N small files raw fs 0.1594s 125459 ops/s 15.3 MB/s + +== 3. stat N files == + stat N files mem 0.0073s 2723001 ops/s + stat N files raw fs 0.0697s 287063 ops/s + +== 4. readdir == + readdir (N entries) mem 0.0042s 4816418 ops/s + readdir (N entries) raw fs 0.0070s 2840890 ops/s + +== 1b. create N small files (dir backend vs. what it wraps) == + create N small files dir 1.1531s 17345 ops/s 2.1 MB/s + +== 2b. read N small files (dir backend) == + read N small files dir 0.1601s 124933 ops/s 15.3 MB/s + +== 3b. stat N files (dir backend) == + stat N files dir 0.1509s 132517 ops/s + +== 4b. readdir (dir backend) == + readdir (N entries) dir 0.0039s 5128581 ops/s + +== 5. unlink N files == + unlink N files mem 0.0320s 625437 ops/s + unlink N files dir 0.5146s 38864 ops/s + unlink N files raw fs 0.4923s 40628 ops/s + +== 6. mkdir/rmdir N directories == + mkdir N dirs mem 0.0052s 771085 ops/s + rmdir N dirs mem 0.0045s 898169 ops/s + mkdir N dirs dir 0.1774s 22551 ops/s + rmdir N dirs dir 0.1118s 35778 ops/s + mkdir N dirs raw fs 0.1367s 29251 ops/s + rmdir N dirs raw fs 0.0901s 44388 ops/s + +== 7. large sequential write/read == + write 1MB mem 0.0003s 2937 ops/s 2937.3 MB/s + read 1MB mem 0.0000s 49752 ops/s 49751.7 MB/s + write 16MB mem 0.0108s 93 ops/s 1483.0 MB/s + read 16MB mem 0.0010s 1050 ops/s 16796.7 MB/s + write 64MB mem 0.0589s 17 ops/s 1086.8 MB/s + read 64MB mem 0.0033s 304 ops/s 19475.8 MB/s + write 1MB raw fs 0.0004s 2463 ops/s 2462.6 MB/s + read 1MB raw fs 0.0001s 15738 ops/s 15738.2 MB/s + write 16MB raw fs 0.0046s 217 ops/s 3477.6 MB/s + read 16MB raw fs 0.0011s 945 ops/s 15121.6 MB/s + write 64MB raw fs 0.0183s 55 ops/s 3495.7 MB/s + read 64MB raw fs 0.0050s 198 ops/s 12702.5 MB/s + write 1MB raw+fsync 0.0235s 43 ops/s 42.5 MB/s + read 1MB raw+fsync 0.0001s 9816 ops/s 9816.4 MB/s + write 16MB raw+fsync 0.0228s 44 ops/s 701.5 MB/s + read 16MB raw+fsync 0.0012s 832 ops/s 13316.1 MB/s + write 64MB raw+fsync 0.0825s 12 ops/s 775.9 MB/s + read 64MB raw+fsync 0.0057s 176 ops/s 11254.5 MB/s + +== 8. random-access read: mmap'd pack vs. raw fs == + compact N entries to pack pack 0.0245s 814864 ops/s 99.5 MB/s + random-read N entries pack 0.0082s 2445144 ops/s 298.5 MB/s + random-read N entries raw fs 0.1657s 120709 ops/s 14.7 MB/s + +== 9. concurrency (create+read+unlink) == + concurrent create+read+unlink mem 0.2807s 341972 ops/s + concurrent create+read+unlink raw fs 6.2980s 15243 ops/s + +=== summary (45 measurements) === + category backend seconds throughput MB/s + create N small files mem 0.0372s 537103 ops/s 65.6 MB/s + create N small files raw fs 1.0098s 19805 ops/s 2.4 MB/s + create N small files raw+fsync 116.0980s 172 ops/s 0.0 MB/s + read N small files mem 0.0085s 2364410 ops/s 288.6 MB/s + read N small files raw fs 0.1594s 125459 ops/s 15.3 MB/s + stat N files mem 0.0073s 2723001 ops/s + stat N files raw fs 0.0697s 287063 ops/s + readdir (N entries) mem 0.0042s 4816418 ops/s + readdir (N entries) raw fs 0.0070s 2840890 ops/s + create N small files dir 1.1531s 17345 ops/s 2.1 MB/s + read N small files dir 0.1601s 124933 ops/s 15.3 MB/s + stat N files dir 0.1509s 132517 ops/s + readdir (N entries) dir 0.0039s 5128581 ops/s + unlink N files mem 0.0320s 625437 ops/s + unlink N files dir 0.5146s 38864 ops/s + unlink N files raw fs 0.4923s 40628 ops/s + mkdir N dirs mem 0.0052s 771085 ops/s + rmdir N dirs mem 0.0045s 898169 ops/s + mkdir N dirs dir 0.1774s 22551 ops/s + rmdir N dirs dir 0.1118s 35778 ops/s + mkdir N dirs raw fs 0.1367s 29251 ops/s + rmdir N dirs raw fs 0.0901s 44388 ops/s + write 1MB mem 0.0003s 2937 ops/s 2937.3 MB/s + read 1MB mem 0.0000s 49752 ops/s 49751.7 MB/s + write 16MB mem 0.0108s 93 ops/s 1483.0 MB/s + read 16MB mem 0.0010s 1050 ops/s 16796.7 MB/s + write 64MB mem 0.0589s 17 ops/s 1086.8 MB/s + read 64MB mem 0.0033s 304 ops/s 19475.8 MB/s + write 1MB raw fs 0.0004s 2463 ops/s 2462.6 MB/s + read 1MB raw fs 0.0001s 15738 ops/s 15738.2 MB/s + write 16MB raw fs 0.0046s 217 ops/s 3477.6 MB/s + read 16MB raw fs 0.0011s 945 ops/s 15121.6 MB/s + write 64MB raw fs 0.0183s 55 ops/s 3495.7 MB/s + read 64MB raw fs 0.0050s 198 ops/s 12702.5 MB/s + write 1MB raw+fsync 0.0235s 43 ops/s 42.5 MB/s + read 1MB raw+fsync 0.0001s 9816 ops/s 9816.4 MB/s + write 16MB raw+fsync 0.0228s 44 ops/s 701.5 MB/s + read 16MB raw+fsync 0.0012s 832 ops/s 13316.1 MB/s + write 64MB raw+fsync 0.0825s 12 ops/s 775.9 MB/s + read 64MB raw+fsync 0.0057s 176 ops/s 11254.5 MB/s + compact N entries to pack pack 0.0245s 814864 ops/s 99.5 MB/s + random-read N entries pack 0.0082s 2445144 ops/s 298.5 MB/s + random-read N entries raw fs 0.1657s 120709 ops/s 14.7 MB/s + concurrent create+read+unlink mem 0.2807s 341972 ops/s + concurrent create+read+unlink raw fs 6.2980s 15243 ops/s + +real 4m3.827s +user 0m1.039s +sys 0m19.841s +``` + ## Reproducing ```sh