diff --git a/README.md b/README.md index 0267316..748f130 100644 --- a/README.md +++ b/README.md @@ -103,74 +103,100 @@ in `BENCH.md`'s "After" table, on the same environment described there. Not a replacement for `BENCH.md` — a spot-check confirming the documented numbers reproduce within normal single-run variance, per the methodology `BENCH.md` itself states ("illustrative of shape... not precise absolute -figures"). +figures"). Every column from the raw `make bench` output is kept for both +runs — time, throughput, and MB/s where the category reports one — not +just a time delta, so every category (including the `pack` rows: +compaction and random-access read) can be checked in full, not summarized +away. -| Category | Backend | Documented (BENCH.md) | New run | Delta | -|---|---|---|---|---| -| create 20,000 files | mem | 0.0400s | 0.0394s | -1.5% | -| create 20,000 files | raw fs | 0.9943s | 1.0171s | +2.3% | -| create 20,000 files | raw+fsync | 116.2147s | 116.7567s | +0.5% | -| read 20,000 files | mem | 0.0099s | 0.0091s | -8.1% | -| read 20,000 files | raw fs | 0.1606s | 0.1634s | +1.7% | -| stat 20,000 files | mem | 0.0077s | 0.0078s | +1.3% | -| stat 20,000 files | raw fs | 0.0699s | 0.0727s | +4.0% | -| readdir (20,000 entries) | mem | 0.0041s | 0.0043s | +4.9% | -| readdir (20,000 entries) | raw fs | 0.0069s | 0.0072s | +4.3% | -| create 20,000 files | dir | 1.1418s | 1.2030s | +5.4% | -| read 20,000 files | dir | 0.1530s | 0.1715s | +12.1% | -| stat 20,000 files | dir | 0.1379s | 0.1546s | +12.1% | -| readdir (20,000 entries) | dir | 0.0037s | 0.0041s | +10.8% | -| unlink 20,000 files | mem | 0.0179s | 0.0182s | +1.7% | -| unlink 20,000 files | dir | 0.4911s | 0.5978s | +21.7% | -| unlink 20,000 files | raw fs | 0.4746s | 0.5062s | +6.7% | -| mkdir 4,000 dirs | mem | 0.0056s | 0.0050s | -10.7% | -| rmdir 4,000 dirs | mem | 0.0040s | 0.0039s | -2.5% | -| mkdir 4,000 dirs | dir | 0.1710s | 0.1864s | +9.0% | -| rmdir 4,000 dirs | dir | 0.1257s | 0.1169s | -7.0% | -| mkdir 4,000 dirs | raw fs | 0.1374s | 0.1444s | +5.1% | -| rmdir 4,000 dirs | raw fs | 0.0951s | 0.1005s | +5.7% | -| write 1MB | mem | 0.0001s | 0.0001s | +0.0% | -| read 1MB | mem | 0.0000s | 0.0000s | +0.0% | -| write 16MB | mem | 0.0110s | 0.0110s | +0.0% | -| read 16MB | mem | 0.0009s | 0.0010s | +11.1% | -| write 64MB | mem | 0.0586s | 0.0621s | +6.0% | -| read 64MB | mem | 0.0032s | 0.0036s | +12.5% | -| write 1MB | raw fs | 0.0004s | 0.0004s | +0.0% | -| read 1MB | raw fs | 0.0001s | 0.0001s | +0.0% | -| write 16MB | raw fs | 0.0044s | 0.0047s | +6.8% | -| read 16MB | raw fs | 0.0010s | 0.0012s | +20.0% | -| write 64MB | raw fs | 0.0186s | 0.0189s | +1.6% | -| read 64MB | raw fs | 0.0047s | 0.0050s | +6.4% | -| write 1MB | raw+fsync | 0.0180s | 0.0340s | +88.9% | -| read 1MB | raw+fsync | 0.0001s | 0.0001s | +0.0% | -| write 16MB | raw+fsync | 0.0231s | 0.0242s | +4.8% | -| read 16MB | raw+fsync | 0.0012s | 0.0012s | +0.0% | -| write 64MB | raw+fsync | 0.0803s | 0.0835s | +4.0% | -| read 64MB | raw+fsync | 0.0048s | 0.0049s | +2.1% | -| compact 20,000 entries to pack | pack | 0.0267s | 0.0280s | +4.9% | -| random-read 20,000 entries | pack (mmap'd) | 0.0082s | 0.0085s | +3.7% | -| random-read 20,000 entries | raw fs | 0.1629s | 0.1912s | +17.4% | -| concurrent create+read+unlink (8×4,000×3) | mem | 0.2750s | 0.2791s | +1.5% | -| concurrent create+read+unlink (8×4,000×3) | raw fs | 5.6498s | 6.1583s | +9.0% | -| mount 500 backends | vfs | 0.0054s | 0.0060s | +11.1% | -| resolve, 500 mounts | vfs | 0.0020s | 0.0022s | +10.0% | -| unmount 500 backends | vfs | 0.0046s | 0.0046s | +0.0% | -| mount 2,000 backends | vfs | 0.0831s | 0.0863s | +3.9% | -| resolve, 2,000 mounts | vfs | 0.0296s | 0.0329s | +11.1% | -| unmount 2,000 backends | vfs | 0.0758s | 0.0780s | +2.9% | -| mount 8,000 backends | vfs | 1.4739s | 1.5380s | +4.3% | -| resolve, 8,000 mounts | vfs | 0.4938s | 0.5298s | +7.3% | -| unmount 8,000 backends | vfs | 1.5172s | 1.5191s | +0.1% | +| Category | Backend | Doc Time | Doc Throughput | Doc MB/s | New Time | New Throughput | New MB/s | Δ Time | +|---|---|---|---|---|---|---|---|---| +| create 20,000 files | mem | 0.0400s | 499,809 ops/s | 61.0 | 0.0373s | 536,419 ops/s | 65.5 | -6.8% | +| create 20,000 files | raw fs | 0.9943s | 20,114 ops/s | 2.5 | 0.9883s | 20,237 ops/s | 2.5 | -0.6% | +| create 20,000 files | raw+fsync | 116.2147s | 172 ops/s | 0.0 | 116.3080s | 172 ops/s | 0.0 | +0.1% | +| read 20,000 files | mem | 0.0099s | 2,011,901 ops/s | 245.6 | 0.0116s | 1,723,405 ops/s | 210.4 | +17.2% | +| read 20,000 files | raw fs | 0.1606s | 124,543 ops/s | 15.2 | 0.2040s | 98,030 ops/s | 12.0 | +27.0% | +| stat 20,000 files | mem | 0.0077s | 2,598,241 ops/s | — | 0.0079s | 2,519,089 ops/s | — | +2.6% | +| stat 20,000 files | raw fs | 0.0699s | 286,222 ops/s | — | 0.0954s | 209,541 ops/s | — | +36.5% | +| readdir (20,000 entries) | mem | 0.0041s | 4,845,826 ops/s | — | 0.0043s | 4,675,035 ops/s | — | +4.9% | +| readdir (20,000 entries) | raw fs | 0.0069s | 2,894,815 ops/s | — | 0.0071s | 2,808,607 ops/s | — | +2.9% | +| create 20,000 files | dir | 1.1418s | 17,517 ops/s | 2.1 | 1.7006s | 11,760 ops/s | 1.4 | +48.9% | +| read 20,000 files | dir | 0.1530s | 130,705 ops/s | 16.0 | 0.2238s | 89,371 ops/s | 10.9 | +46.3% | +| stat 20,000 files | dir | 0.1379s | 145,056 ops/s | — | 0.1984s | 100,826 ops/s | — | +43.9% | +| readdir (20,000 entries) | dir | 0.0037s | 5,465,162 ops/s | — | 0.0042s | 4,799,105 ops/s | — | +13.5% | +| unlink 20,000 files | mem | 0.0179s | 1,117,742 ops/s | — | 0.0183s | 1,094,715 ops/s | — | +2.2% | +| unlink 20,000 files | dir | 0.4911s | 40,724 ops/s | — | 0.7500s | 26,667 ops/s | — | +52.7% | +| unlink 20,000 files | raw fs | 0.4746s | 42,141 ops/s | — | 0.6404s | 31,228 ops/s | — | +34.9% | +| mkdir 4,000 dirs | mem | 0.0056s | 719,696 ops/s | — | 0.0051s | 791,918 ops/s | — | -8.9% | +| rmdir 4,000 dirs | mem | 0.0040s | 993,724 ops/s | — | 0.0435s | 91,873 ops/s | — | +987.5% | +| mkdir 4,000 dirs | dir | 0.1710s | 23,386 ops/s | — | 0.2123s | 18,838 ops/s | — | +24.2% | +| rmdir 4,000 dirs | dir | 0.1257s | 31,827 ops/s | — | 0.1372s | 29,163 ops/s | — | +9.1% | +| mkdir 4,000 dirs | raw fs | 0.1374s | 29,122 ops/s | — | 0.1837s | 21,772 ops/s | — | +33.7% | +| rmdir 4,000 dirs | raw fs | 0.0951s | 42,043 ops/s | — | 0.1305s | 30,640 ops/s | — | +37.2% | +| write 1MB | mem | 0.0001s | 7,058 ops/s | 7,058.1 | 0.0002s | 6,313 ops/s | 6,312.7 | +100.0% | +| read 1MB | mem | 0.0000s | 50,051 ops/s | 50,050.9 | 0.0000s | 48,616 ops/s | 48,616.4 | +0.0% | +| write 16MB | mem | 0.0110s | 91 ops/s | 1,458.2 | 0.0407s | 25 ops/s | 393.3 | +270.0% | +| read 16MB | mem | 0.0009s | 1,058 ops/s | 16,920.1 | 0.0010s | 1,021 ops/s | 16,338.0 | +11.1% | +| write 64MB | mem | 0.0586s | 17 ops/s | 1,091.3 | 0.0599s | 17 ops/s | 1,068.3 | +2.2% | +| read 64MB | mem | 0.0032s | 313 ops/s | 20,056.5 | 0.0300s | 33 ops/s | 2,131.0 | +837.5% | +| write 1MB | raw fs | 0.0004s | 2,759 ops/s | 2,759.4 | 0.0004s | 2,474 ops/s | 2,474.0 | +0.0% | +| read 1MB | raw fs | 0.0001s | 16,812 ops/s | 16,812.4 | 0.0001s | 16,756 ops/s | 16,756.0 | +0.0% | +| write 16MB | raw fs | 0.0044s | 229 ops/s | 3,656.4 | 0.0044s | 228 ops/s | 3,645.0 | +0.0% | +| read 16MB | raw fs | 0.0010s | 989 ops/s | 15,827.9 | 0.0010s | 965 ops/s | 15,443.5 | +0.0% | +| write 64MB | raw fs | 0.0186s | 54 ops/s | 3,449.7 | 0.0184s | 54 ops/s | 3,478.9 | -1.1% | +| read 64MB | raw fs | 0.0047s | 213 ops/s | 13,633.1 | 0.0049s | 203 ops/s | 13,003.0 | +4.3% | +| write 1MB | raw+fsync | 0.0180s | 56 ops/s | 55.7 | 0.0162s | 62 ops/s | 61.7 | -10.0% | +| read 1MB | raw+fsync | 0.0001s | 9,059 ops/s | 9,058.8 | 0.0001s | 12,025 ops/s | 12,025.1 | +0.0% | +| write 16MB | raw+fsync | 0.0231s | 43 ops/s | 691.7 | 0.0247s | 41 ops/s | 648.1 | +6.9% | +| read 16MB | raw+fsync | 0.0012s | 855 ops/s | 13,672.2 | 0.0014s | 726 ops/s | 11,610.9 | +16.7% | +| write 64MB | raw+fsync | 0.0803s | 12 ops/s | 797.2 | 0.1001s | 10 ops/s | 639.5 | +24.7% | +| read 64MB | raw+fsync | 0.0048s | 209 ops/s | 13,393.5 | 0.0200s | 50 ops/s | 3,193.8 | +316.7% | +| compact 20,000 entries to pack | pack | 0.0267s | 747,839 ops/s | 91.3 | 0.0250s | 800,791 ops/s | 97.8 | -6.4% | +| random-read 20,000 entries | pack (mmap'd) | 0.0082s | 2,435,930 ops/s | 297.4 | 0.0083s | 2,397,682 ops/s | 292.7 | +1.2% | +| random-read 20,000 entries | raw fs | 0.1629s | 122,749 ops/s | 15.0 | 0.1691s | 118,290 ops/s | 14.4 | +3.8% | +| concurrent create+read+unlink (8×4,000×3) | mem | 0.2750s | 349,127 ops/s | — | 0.2981s | 322,041 ops/s | — | +8.4% | +| concurrent create+read+unlink (8×4,000×3) | raw fs | 5.6498s | 16,992 ops/s | — | 6.3837s | 15,038 ops/s | — | +13.0% | +| mount 500 backends | vfs | 0.0054s | 92,833 ops/s | — | 0.0054s | 91,871 ops/s | — | +0.0% | +| resolve, 500 mounts | vfs | 0.0020s | 246,064 ops/s | — | 0.0019s | 262,671 ops/s | — | -5.0% | +| unmount 500 backends | vfs | 0.0046s | 109,207 ops/s | — | 0.0045s | 110,348 ops/s | — | -2.2% | +| mount 2,000 backends | vfs | 0.0831s | 24,081 ops/s | — | 0.0836s | 23,926 ops/s | — | +0.6% | +| resolve, 2,000 mounts | vfs | 0.0296s | 67,458 ops/s | — | 0.0293s | 68,318 ops/s | — | -1.0% | +| unmount 2,000 backends | vfs | 0.0758s | 26,397 ops/s | — | 0.0802s | 24,932 ops/s | — | +5.8% | +| mount 8,000 backends | vfs | 1.4739s | 5,428 ops/s | — | 1.4905s | 5,367 ops/s | — | +1.1% | +| resolve, 8,000 mounts | vfs | 0.4938s | 16,200 ops/s | — | 0.4616s | 17,332 ops/s | — | -6.5% | +| unmount 8,000 backends | vfs | 1.5172s | 5,273 ops/s | — | 1.5176s | 5,271 ops/s | — | +0.0% | -Almost every row sits within ±15% of the documented figures — consistent -with the single-run jitter `BENCH.md` already warns about, not a -regression. Two rows exceed that: `unlink 20,000 files (dir)` (+21.7%, -plausible container/host I/O noise, same direction as the other `dir` -metadata rows this run) and `write 1MB (raw+fsync)` (+88.9%, but this is a -tiny absolute value — 18ms vs 34ms for one syscall — the single number most -sensitive to one slow `fsync` on this container's overlay filesystem, not a -meaningful regression at that scale). Total wall-clock: 4m7.7s this run vs -5m23.9s documented, itself within the same single-run variance. +The `pack`-backend rows are the most stable in the whole table (compaction +-6.4%, random-access read +1.2%/+3.8%), consistent with `BENCH.md`'s point +that `pack` random access is a flat `mmap`'d table lookup rather than +anything sensitive to scheduling jitter the way syscall-heavy `dir`/raw-fs +metadata operations are. + +Most rows sit within ±15% of the documented figures — single-run jitter, +not a regression, exactly as `BENCH.md`'s stated methodology warns to +expect. This run was noisier than the previous spot-check, though, with +several rows well outside that band, worth naming rather than averaging +away: every `dir`-backend metadata row (`create`/`read`/`stat`/`unlink`/ +`mkdir`) is elevated 24–53% together, pointing at host-side I/O contention +during this run rather than anything PackFS-specific (the `mem` and `pack` +rows, touching no host filesystem, mostly did not move); and two `mem` +rows are outliers even by that standard — `rmdir 4,000 dirs` (+987.5%, +0.0040s → 0.0435s) and `read 64MB` (+837.5%, 0.0032s → 0.0300s). Both are +large relative jumps on a small absolute base (tens of milliseconds), +exactly where a single scheduling stall has the most disproportionate +effect on a percentage, and neither is corroborated by a neighboring +row moving the same way (`mkdir` and `unlink` on `mem`, right next to +`rmdir`, both stayed flat or improved; `read 16MB`/`write 64MB` on `mem`, +right next to `read 64MB`, both stayed within a few percent) — the +single-run, non-statistical methodology `BENCH.md` states up front means +this can't be distinguished from noise without repeated runs, so it is +reported here rather than either hidden or asserted as a regression. +Total wall-clock: 4m10.1s this run vs 5m23.9s documented — faster overall +despite the several elevated `dir`/raw-fs rows above, because the dominant +cost throughout every run of this benchmark is the single `raw+fsync` +20,000-file create test (~116s here too, unaffected — see `BENCH.md`'s +"Environment" section for why that test in particular is unrelated to +PackFS's own performance). ## Quick example