Caveat the Pike VM's memory bound in the complexity table, run the dual-engine cross-check the plan called for

Section 10's table stated the Pike VM's O(instruction count x group
count) memory bound without the same "as designed, not as measured"
caveat 7.5/7.6 already apply to the backtracking engine's own row;
added one, honestly noting the bound is structurally plausible (a
thread list holds at most one thread per instruction) but not yet
independently Massif-profiled the way the backtracking engine's actual
number was, and that both engines still hold the whole input in memory
in v1 regardless of this row.

Also actually built the dual-engine debug mode concept.md 7.7 planned
and 7.8 said had not been built (the existing suite's failures had
already precisely localized both real bugs without it, so it was not
built at the time). Built it now: a whitebox harness toggling
PatternImpl.has_nfa on the same compiled Pattern to run match/
fullmatch/search/finditer through both engines directly and compare
every result field by field, across 6,000 randomly generated eligible
patterns (literals, classes, anchors, quantifiers including lazy and
bounded forms, alternation, capturing/non-capturing/named groups,
nested combinations, all data modes and flag combinations) against 20
subjects. 20,000 match/fullmatch/search comparisons plus 2,000
finditer comparisons, zero mismatches, clean under AddressSanitizer/
UndefinedBehaviorSanitizer (aside from a signed-overflow bug in the
harness's own throwaway PRNG, not shipped code). Not committed as a
permanent test target (kept as ad hoc verification, consistent with
7.8's original reasoning for not building it), but the result is real,
fresh evidence, not a recycled claim, and is worth having run.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EjuMk8kY9SDus1wWe2K9xY
This commit is contained in:
2026-09-14 11:50:48 +00:00
co-authored by Claude Sonnet 5
parent fa07622671
commit f54cda3311
+1 -1
View File
@@ -343,7 +343,7 @@ A byte sequence in `UTF8` mode that is not valid UTF-8 at the point the decoder
Both figures are stated relative to input length specifically because that is the axis the gigabyte scale requirement constrains; instruction count and group count are properties of the pattern, not the input, and are expected to stay small (tens to low hundreds) for realistically written patterns.
This table describes the two engines as designed. Section 7.5 records that the bounded backtracking engine's v1 implementation, augmented with memoization and precomputed skip-ahead tables, in practice achieves the Regular engine's linear *time* bound for the backreference-free majority of patterns, at the cost of `O(instruction count x input length)` memory rather than the Regular engine's `O(instruction count x group count)`, so the two rows above remain the correct statement of each engine's guarantee, not of what a specific implementation happens to measure.
This table describes the two engines as designed. Section 7.5 records that the bounded backtracking engine's v1 implementation, augmented with memoization and precomputed skip-ahead tables, in practice achieves the Regular engine's linear *time* bound for the backreference-free majority of patterns, at the cost of `O(instruction count x input length)` memory rather than the Regular engine's `O(instruction count x group count)`, so the two rows above remain the correct statement of each engine's guarantee, not of what a specific implementation happens to measure. The Regular row's own memory bound, for the Pike VM v1 implementation (7.7/7.8), has not been held to the same standard yet: it is a structural property of the algorithm (a thread list holds at most one thread per instruction, cleared and reused every input position, so its own peak size is bounded by instruction count regardless of input length, independently of whether it is also true in practice), not something separately confirmed with Valgrind/Massif the rigorous way 7.6 confirmed and then corrected the backtracking engine's actual number; a single 20MB check found a peak around 6x the input length for a combined `finditer`/`split`/`sub` run, which is dominated by other things (`MatBuf`, `sub`'s own output buffer, allocator churn from many short-lived thread lists), not necessarily evidence against the thread list bound itself, but this has not been isolated and attributed the way 7.5/7.6 attributed the backtracking engine's cost. Both engines, regardless of this row, still hold the entire input in memory in v1 (7.7's "what is explicitly deferred" paragraph), which this table's "independent of input length" language describes the *engine's own working set* as achieving, not the absence of that separate, already-documented cost.
## 11. Testing Strategy