Add a literal prefilter to the Pike VM, profile its memory with Massif
Closed two of the three gaps the previous commit's honest self-review left open (the third, full streaming/bounded-memory input, remains out of scope for this pass and is still documented as such). Literal prefilter (regexx.c, pike_find): when the NFA-only Prog's first instruction is a mandatory OP_CHAR or OP_CLASS, a pattern beginning with a required literal or class rather than a nullable loop or a leading assertion, injecting a fresh unanchored start thread at a position that instruction would reject is certain to die on the very next pike_step call regardless; checking that identical condition before injecting rather than after changes nothing about which threads ever exist, only how much wasted work is done finding out. This targeted exactly examples/bench_vs_posix.c's worst regression: the literal-search scenario went from roughly 70x slower than POSIX <regex.h> (up from roughly 22x before the Pike VM existed) down to roughly 11x-13x, better than the original pre-Pike-VM number; number extraction (starts with a class) improved more modestly; a*b (starts with a nullable loop, structurally unhelped) is unchanged, as expected. Verified with the full 3,252-case suite, three clean AddressSanitizer/UndefinedBehaviorSanitizer passes, and a rerun of the whitebox dual-engine cross-check (24,000 match/fullmatch/search plus ~2,700 finditer comparisons between the two engines on the same compiled patterns, zero mismatches). Memory profiling (concept.md 7.9): Valgrind/Massif on the same adversarial, prefilter-proof pattern shape (a*b, nullable leading loop) used for the backtracking engine's own worst case, for a fair comparison. Peak heap was almost entirely the 10MB input buffer itself; the Pike VM's own contribution was roughly 12KB, confirming the design's O(instruction count x group count), input-length- independent memory bound actually holds for the v1 implementation, not only on paper. concept.md Section 10's table, which had only a "not yet profiled" caveat for this row before, is updated with the measured result. README.md, docs/API.md, USAGE.md, and bench_vs_posix.c's own printed summary are updated throughout with the corrected numbers, rather than left describing the pre-prefilter regression as current. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EjuMk8kY9SDus1wWe2K9xY
This commit is contained in:
@@ -692,10 +692,11 @@ A pattern with no backreference, lookaround, or atomic group runs on the
|
||||
Pike VM (`README.md` "The Pike VM"), which is genuinely linear time,
|
||||
including on adversarial nested-quantifier shapes like `(a+)+b` that would
|
||||
invite catastrophic backtracking elsewhere, with no atomic group needed;
|
||||
it does not yet have that engine's own constant-factor optimizations
|
||||
(no literal prefilter, no lazy DFA state caching), so an ordinary pattern
|
||||
can currently measure slower in absolute terms than before this engine
|
||||
existed, a real, documented trade-off, not an oversight. Everything else
|
||||
it has a literal prefilter (`README.md` "The Pike VM") but still no lazy
|
||||
DFA state caching or allocation pooling, so an ordinary pattern can
|
||||
currently measure slower in absolute terms than before this engine
|
||||
existed, narrower than before the prefilter but a real, documented
|
||||
trade-off still, not an oversight. Everything else
|
||||
(a backreference, lookaround, or atomic group anywhere) runs on the
|
||||
recursive backtracking engine instead: `match`/`fullmatch` (a single
|
||||
anchored attempt) cost time proportional to that attempt; `search`/
|
||||
|
||||
Reference in New Issue
Block a user