tai/taicpp/docs/fuzzy.md

FuzzyMatcher: "which candidate did the user mean"

Code: src/util/fuzzy.h, src/util/fuzzy.cpp. Tests: tests/fuzzy_test.cpp (build/fuzzy_test, step fuzzy.unit of tests/run_all.py). Research: research/fuzzy-matching-selectors/ (12 sources read, 53 listed, conclusion generic-fuzzy-scorer-design). This is a taicpp component; the Python app has no equivalent. The code is the only definition of the scoring; this page states what it does today.

1. What it is for

One scorer, used wherever a short, typed, possibly wrong word has to pick one item out of a list of identifiers. Today that is @agent, @agent:model and @model mentions (cap-agents.md). It is deliberately generic and has no dependency on anything else in the application: pass a query and a list of strings, get scores or the best index.

Design rules it follows (project rules, see AGENTS.md):

• Static ceilings, no auto tuning (section 6).
• Deterministic: the same inputs always give the same order, ties included.
• Loud limits: nothing is silently truncated beyond the ceilings listed in section 6.
• No dependency: it is a pure function over std::string.

2. API

#include "fuzzy.h"
using detz::FuzzyMatcher;

double s = FuzzyMatcher::Score("opus", "claude-opus-4-5");               // 0..1
auto ranked = FuzzyMatcher::Rank("opus", {"claude-opus-4-5", "opencode"}); // vector<Ranked{index, score}>
long best  = FuzzyMatcher::Best("opus", candidates, /*floor=*/0.3, &score); // index or -1

Function	Returns
Score(query, text)	0 when nothing relates; 1.0 only for a case-insensitive exact match; otherwise the maximum of the four signals below, never above 0.999
Subsequence, Typo, TokenTypo, Trigram	the individual signals, public for tests and tuning discussions
Rank(query, texts)	every candidate with score above 0, best first
Best(query, texts, floor, &score)	index of the top candidate, or -1 when there is none or its score is not strictly above floor

Empty query or empty text scores 0. Comparison is case-insensitive for ASCII only (section 7).

3. The four signals

Score = max( Subsequence, Typo, 0.95 x TokenTypo, 0.9 x Trigram ), then capped at 0.999 unless the folded strings are equal (then exactly 1.0).

3.1 Subsequence (letters in order, boundary aware)

The query bytes must appear in the text in order, case-insensitively, not necessarily next to each other. The best alignment is found by dynamic programming and scored with the constants of fzf's FuzzyMatchV2 (research/fuzzy-matching-selectors/github-fzf-algo-go.md), using the two-matrix recurrence of fzy (github-fzy-match-c.md):

• D[i][j] = best score of query[0..i] where query[i] is matched exactly at text[j].
• M[i][j] = best score of query[0..i] using text[0..j], matched or skipped.
• Matching text[j] for query[i]:
  • i == 0: kScoreMatch + bonus[j] * kFirstMultiplier + kGap * j (the skipped leading bytes cost kGap each).
  • i > 0: max( M[i-1][j-1] + kScoreMatch + bonus[j], D[i-1][j-1] + kScoreMatch + max(bonus[j], kBonusConsecutive) ). The second branch is "continuing a run"; it always earns at least kBonusConsecutive.
• A skipped byte costs kGap (-1), at the start, inside and at the end.
• The raw result is M[n-1][m-1]; it is normalized by the perfect score n * 16 + 10 * 2 + (n-1) * 4 (every byte matched, the first at a whitespace boundary, all consecutive) and clamped to [0, 1].

Constant	Value	Applies when
kScoreMatch	16	each matched byte
kGap	-1	each skipped byte
kBonusWhite	10	match at the start of the text or after whitespace
kBonusBoundary	8	match after a byte that is not a letter or digit (-, _, /, :, .)
kBonusCamel	7	lower to Upper, or a non-digit to a digit
kBonusConsecutive	4	minimum bonus for continuing a run
kFirstMultiplier	2	first query byte's bonus is doubled

Consequences you can rely on: a query matches best at the start of a word; fb finds FooBar (camel humps) better than fxxxxbxx; shorter texts lose fewer points to skipped bytes. A query longer than the text cannot be a subsequence and scores 0 here.

3.2 Typo (restricted Damerau-Levenshtein)

1 - d / max(len(query), len(text)) where d is the optimal-string-alignment distance: insert, delete, substitute, or swap two adjacent bytes, each costing 1. It catches typos and transpositions (snonet for sonnet) that are not subsequences. It is low for a short query against a long text, which is what the next signal is for.

3.3 TokenTypo (typo against each word)

The text is split into runs of letters and digits (claude-opus-4-5 gives claude, opus, 4, 5); the result is the best Typo of the query against any one token, scaled by 0.95. It finds a misspelled word inside a long identifier (opuss in claude-opus-4-5) and gives an exact word match (opus, free, pro) a score of 0.95.

3.4 Trigram (pg_trgm)

Each alphanumeric word gets two leading spaces and one trailing space; the set of three-byte windows is compared by Jaccard (shared / total distinct), the way PostgreSQL's pg_trgm does (postgresql-pg-trgm.md: word against words is 4/7 = 0.571). Scaled by 0.9. It helps with scrambled or partial words and is the weakest signal.

4. Ranking and ties

Rank keeps candidates with score above 0 and sorts by:

1. score, descending (differences under 1e-9 count as equal);
2. shorter text first (byte length), the default tie rule of fzf (github-fzf-man-page.md);
3. earlier position in the input list.

So two equal matches never produce an error or a coin flip: free against zen-free and devplace-free (both 0.95) always picks zen-free.

5. Worked examples

Measured with the code (Subsequence, Typo, TokenTypo before the 0.95 scale, Trigram before the 0.9 scale, then the final Score):

query	text	subseq	typo	token	trigram	score
opus	claude-opus-4-5	0.84	0.27	1.00	0.31	0.95
opus	opencode	0.00	0.25	0.25	0.17	0.25
son	claude-sonnet-4-5	0.76	0.18	0.50	0.16	0.76
sonet (dropped letter)	claude-sonnet-4-5	0.83	0.29	0.83	0.26	0.83
snonet (swap)	claude-sonnet-4-5	0.00	0.29	0.83	0.14	0.79
opuss (extra letter)	claude-opus-4-5	0.00	0.27	0.80	0.22	0.76
g4m (word starts)	gpt-4o-mini	1.00	0.27	0.33	0.07	1.00
gpt4	gpt-4o	1.00	0.67	0.75	0.33	1.00
gpt4	gpt-4o-mini	0.97	0.36	0.75	0.21	0.97
free	zen-free	0.92	0.50	1.00	0.56	0.95
pro	gemini-2.5-pro	0.80	0.21	1.00	0.27	0.95
fb	FooBar	0.98	0.33	0.33	0.11	0.98
fb	fxxxxbxx	0.82	0.25	0.25	0.10	0.82
clde	claude	0.94	0.67	0.67	0.33	0.94
word	words	0.99	0.80	0.80	0.57	0.99
zzqq	zen-free	0.00	0.12	0.25	0.08	0.24

(g4m and gpt4 show 1.00 because the subsequence is perfect after rounding; the function itself returns at most 0.999 unless the strings are equal.) Reading the table: opus finds claude-opus-4-5 mainly through the exact word (token signal) and not at all through opencode; a swap is rescued by the token signal; a nonsense query tops out around 0.25.

6. Ceilings

Constant	Value	Effect
kFuzzyQueryMax	64 bytes	longer queries are cut
kFuzzyTextMax	256 bytes	longer candidates are cut
kFuzzyCandidateMax	4096	candidates beyond this are not ranked
kTrigramScale	0.9
kTokenScale	0.95

Cost per score is at most 64 x 256 cells for the subsequence DP and for the edit distance, so ranking a full catalog stays in the low milliseconds.

7. Known limits

• ASCII case folding only; bytes of 0x80 and above are treated as lowercase letters and compared as they are. Accented text is not normalized (fzf can).
• The constants are fzf's, tuned for file paths, not for model identifiers. They were not tuned on a real corpus of model ids; the examples above are the evidence. No claim of optimality.
• A subsequence can match scattered letters inside a long identifier; boundary bonuses make this rare, not impossible.
• The caller decides whether a score is good enough: Best(..., floor) takes the floor. The CAP catalog uses 0.3 (CAP_FUZZY_FLOOR); zzqq against everything scores at most 0.24 and is rejected.

8. Reusing it

Anything that picks from a list of names can call it: pass the user's text and the list, take Best with a floor, show the chosen candidate to the user when it differs from what they typed (the catalog prints 'tia:zne' resolved to tai:zen-free (fuzzy 0.67)). Do not use it for security decisions or for free text search; it is built for short identifiers.