# Test Plan: Combinatorial Expansion This records what `tests/cases.py` generates, and why, before generating it, so the plan is checkable against the result rather than only inferable from it. Every case listed here follows `concept.md` Section 11's strategy: ground truth comes from running the same pattern/subject/flags through a real CPython `re`, not from a hand-derived expectation. `tests/gen.py` prints the exact final case count on every run; this document fixes the *categories* and *dimensions*, not an exact count, since the combinatorics are generated programmatically from the lists below. ## Categories ### A. Quantifier x grouping x flags combinatorics (search, finditer) Cartesian product of: - Atoms: `a`, `[a-c]`, `\d`, `\w`, `.` (5) - Quantifier suffixes: `*`, `+`, `?`, `{2,3}`, `{1,}`, `*?`, `+?`, `??` (8, greedy and lazy) - Grouping wrapper: none, `(...)`, `(?:...)`, `(?P...)` (4) - Flag combination: none, `IGNORECASE`, `MULTILINE`+`DOTALL` (3) - Operation: `search`, `finditer` (2) Each atom paired with one subject string chosen to exercise it meaningfully (mixed case, a run of matching and non-matching characters, at least one empty-match-prone case for `*`/`?`). 5 x 8 x 4 x 3 x 2 = 960 cases. ### B. Alternation x backreference x groups (search, fullmatch) ~20 hand-written base patterns combining `|`, capturing/named groups, and numeric/named backreferences (for example `(cat|dog|bird)\1`, `(?Pfoo|bar)-(?P=x)`, `(a|b){2,3}\1?`), each run against 3 subjects, 2 flag combinations, 2 operations: ~20 x 3 x 2 x 2 = 240 cases. ### C. Lookaround combinatorics (search, fullmatch) ~20 hand-written patterns combining `(?=...)`, `(?!...)`, `(?<=...)`, `(?`, `\g`, `\N` backreferences in replacement templates, `count` limits, and `maxsplit` limits, including patterns with 0, 1, and multiple capturing groups (Python interleaves every group's text into `split`'s result list): ~150-200 cases. ### E. Real world "combination" patterns (search, finditer, fullmatch, sub) ~40-50 realistic patterns that combine many features at once rather than isolating one (the way patterns are actually written): email-shaped, URL-shaped, IPv4, ISO date, HH:MM:SS time, US-shaped phone number, hex color, semantic version, key=value pairs, CSV-shaped splitting, quoted strings with backslash escapes, log-line-shaped text with multiple named groups, Markdown-style emphasis markers, single-level balanced parentheses, and Unicode word matching across scripts. Each pattern run through 2-3 of the four operations above, whichever are meaningful for it: ~150 cases. ### F. Mode cross-checks (search, finditer) ~15 patterns run in `ascii`, `utf8`, and `binary` mode against matched subjects, to check mode-dependent `\w`/`\s`/`\d`/case-folding behavior specifically, not just syntax: ~90 cases. ### G. Flag combination stress (search, finditer) ~10 representative patterns run under a wider sweep of flag combinations (`IGNORECASE`, `MULTILINE`, `DOTALL`, `VERBOSE`, and pairwise combinations) than categories A-C use, to catch flag-interaction bugs specifically: ~160 cases. ### H. `BINARY` mode with genuinely arbitrary raw bytes Unlike Category F (which exercises `BINARY`/`ASCII`/`UTF8` mode with ordinary UTF-8-encoded text), these cases use real Python `bytes` objects containing embedded `NUL`, high bytes (`0x80`-`0xFF`), and byte sequences that are not, and are not meant to be, valid UTF-8 at all: ~20 hand-written cases across `search`, `finditer`, `sub`, and `split`. ### I. `UTF8` mode edge cases Real multi-byte content beyond simple accented Latin: 4-byte (astral plane) code points such as emoji, combining marks, right-to-left scripts (Arabic, Hebrew), CJK, mixed-width strings, and offset correctness for backreferences/groups/lookaround spanning multi-byte characters: ~20 hand-written cases across `search`, `fullmatch`, `finditer`, `sub`, and `split`. ### J. Seeded random combinatorics A fixed-seed (`0xC0FFEE`) random generator building 900 pattern/subject pairs from the same atom/quantifier/grouping vocabulary as Category A, but combined randomly rather than exhaustively, across all three modes and a sweep of flag combinations. The fixed seed makes this a deterministic, reproducible regression test, not a flaky fuzzer: the same 900 cases generate every run, so a failure here is exactly as reproducible and reportable as a hand-written one, while still exploring combinations no one sat down and thought to write by hand. ## Target Roughly 1,900 generated cases for Categories A-G, run against the original 81 hand-written ones (kept, not replaced); Categories H-J added later brought the total to just over 3,200. `make test` prints the exact current count and the pass/fail result on every run. ## Result 3,252 cases generated (0 skipped), all passing, including the original 81. This expansion found and fixed four real defects before they were ever released, load-bearing enough to be worth naming here rather than only in the commit history: - A C trigraph bug in `tests/gen.py`'s own string-literal encoder: any generated pattern containing `??)` (the lazy-`?` quantifier next to a closing paren, produced by Category A) was silently rewritten by the C compiler to `?]` before the test suite ever ran, because a strict-mode C compiler trigraph-converts `??)` inside a string literal, not just in code (confirmed by compiling and printing the corrupted string directly). Fixed by escaping every `?` as `\?` in the generated C. - A real, previously undocumented `Pattern_finditer`/`Pattern_split` defect: CPython's empty-match handling additionally searches for, and reports, a *second*, non-empty match at the same start position whenever the first match found there was empty, which this build did not do. Reverse engineered against a real CPython interpreter (not written down in CPython's own documentation), fixed in `regexx.c`, and now recorded precisely in `concept.md` Section 3 and `docs/API.md` Sections 3.5-3.6 and 3.7. - `tests/gen.py`'s own `ASCII`-mode ground truth was wrong (Python `str` + `re.ASCII`, which stays in code-point space, instead of a `bytes` pattern against a `bytes` subject, which is what regexx's byte-oriented `ASCII` mode actually is), caught by Category F's mode cross-checks against a subject containing a multi-byte character. - A real, verified-wrong implementation of the `LOCALE` flag: an earlier revision treated bytes `0x80`-`0xFF` as word characters under `LOCALE` in `BINARY`/`ASCII` mode, based on an unverified assumption about "C" locale behavior. Caught by Category H, checked directly against both the C standard's guarantee for `isalnum()` under the "C" locale and a real CPython interpreter with `re.LOCALE` and the "C" locale explicitly set, both of which classify only ASCII letters and digits. `LOCALE` is now documented as an accepted no-op in `BINARY`/`ASCII` mode, matching verified reality instead of a prior assumption (`concept.md` 13.4, `docs/API.md` Section 2, `README.md` "Known deviations"). Two further, smaller findings were concrete instances of an already-documented category (glibc's `wctype.h` Unicode tables not matching CPython's own bundled ones exactly, `README.md` "Known deviations"), not new defects: U+00A0 (NO-BREAK SPACE, Category F) and fullwidth digits `U+FF10`-`U+FF19` (Category I) are both recognized by CPython's `\s`/`\d` but not by glibc's `iswspace()`/`iswdigit()` under the `C.utf8` locale this build relies on. Recorded, not patched, for the same reason already given for the first instance: hand-patching individual code points would start down the path of maintaining an ad hoc table this design deliberately avoided. After these fixes, the random Category J generator was additionally run against five more seeds (`1`, `42`, `777`, `99999`, `123456`) at 3,000 iterations each, 26,760 further checks beyond the 900 committed here, all passing; only the committed seed's 900 cases are part of the regular test suite, the rest was exploratory validation, not committed as a permanent fixture, to avoid the build slowing down for marginal additional coverage over what is already committed. All of the above is described in full in `README.md`, `docs/API.md`, and `concept.md`, not only here; this section exists so the connection between "ran a lot of generated tests" and "found these specific, real bugs" is traceable from the plan that produced them. ## Explicitly out of scope for this expansion Constructs `regexx` intentionally rejects (conditional groups, scoped inline flags, `\N{NAME}`, POSIX bracket classes, `concept.md` Section 5) are not exercised here as *positive* cases, since they are not valid input to generate ground truth for; `tests/harness.c`'s existing hand-written checks already cover that they fail cleanly (a `PatternError`, not a crash), which this expansion does not need to duplicate at scale.