Commit graph

7 commits

Author SHA1 Message Date
trvon
466927f502
fix(search): strip matched XML-style tag pairs in linear time
stripXmlTags removed '<name>...</name>' pairs with a lazy backreference
std::regex; every unmatched opener rescanned the rest of the query, so a
32 KB query of '<a>x' took 4.4 s (libstdc++, -O2) and time grew 4x per
doubling. Search text comes from MCP clients and the CLI.

Index closing tags once and find each opener's first closer by binary
search, preserving the regex's left-to-right shortest-match semantics.
The suspected stack overflow did not reproduce (libc++ or libstdc++, down
to 256 KB thread stacks under ASan); the new test guards the time bound.
2026-09-23 19:26:25 -06:00
trvon
ccc310432b
fix(search): make grouped FTS5 operator queries explicit (found by fuzzing)
fuzz_fts5_query found two inputs the operator-query normalizer still
rejected at MATCH time:
- FTS5 applies implicit AND only between phrases, so '(foo OR bar) baz',
  'bar (foo)' and 'NEAR(a b) c' are syntax errors; insert an explicit AND
  next to a parenthesized or NEAR group.
- 'NEAR/<x>(...)' was treated as a NEAR group; FTS5 accepts only 'NEAR('.
2026-09-23 18:25:30 -06:00
trvon
56fd0535ed
fix(search): keep operator queries valid FTS5 instead of passing them through
Queries containing FTS5 operator syntax (a quote, uppercase AND/OR/NOT,
parentheses, column filters) were passed to MATCH almost verbatim, so
'foo "bar', '(foo OR bar', 'foo AND', 'NOT foo', 'IL-6 OR TNF' or
'ratio:2' failed with an fts5 syntax or no-such-column error instead of
returning results; MCP clients and the CLI reach this path.

Rebuild operator queries token by token: keep phrases, parentheses,
AND/OR/NOT, well-formed NEAR(...) groups and filters on real FTS columns,
quote any other non-bareword term, and fall back to the literal word
query when the structure is invalid. The existing assertion that
'hello NEAR/2 world' passes through encoded a query FTS5 rejects.
2026-09-23 17:43:12 -06:00
trvon
80e0724d0f
feat: expanding instrumentation for optimizations and improvements for search / retrieval functions 2026-03-04 23:18:35 -05:00
trvon
2e87c3c947
feat: search/grep retrieval based improvements, test expansions and bug fixes 2026-02-24 19:17:55 -05:00
trvon
6df2332515
feat: beginning sandbox support work and retrieval optimizations 2026-02-24 17:11:19 -05:00
trvon
b7aa881de2
test fixes, reranking model loading improvements 2026-01-18 22:27:30 -05:00