Commit graph

4 commits

Author SHA1 Message Date
Trevon
864f677269
[09/14] Bound rerank text and reuse search parsing (#149)
* perf(search): preview-based rerank text and compiled path globs

Cross-rerank built each window entry's text with a full getContent()
read, then kept only the first rerankSnippetMaxChars characters. The
window now hydrates with one batchGetContentPreview() call bounded to
that many characters; buildCrossRerankText takes the preview instead of
the repository. batchGetContentPreview is virtual so a test can count
it: three window results cost one preview batch and zero full reads.

Path filtering compiled a std::regex from the glob on every call, and
the call sits inside per-document loops in pathSearch, the hash-prefix
search, and metadata filtering. GlobMatcher (src/app/services) holds
the compiled form; wildcardMatch() reuses it through a small
per-thread cache. The glob semantics are unchanged and now pinned by a
test: '*' stays within one path segment, '**' crosses, '?' is one
character, regex metacharacters are literal. yams::common::wildcard_match
was considered and rejected because its '*' crosses separators.

portability_allowlist.txt: two search_service.cpp anchors reflowed.

Signed-off-by: trvon <me@trevon.dev>

* perf(search): parse edge anchor properties once, lower the path query once

Graph expansion parsed every visited edge's JSON properties with
nlohmann::json::parse to read at most two strings, region and scope,
in three places (relation weighting, seed anchoring, and neighbour
anchoring). Properties strings repeat heavily across edges and most
carry no anchor at all. edgeAnchorHints() now returns immediately when
the text cannot contain either key and otherwise parses each distinct
string once per thread through a small bounded cache. A process-wide
parse counter lets the test pin it: one hundred lookups of the same
anchored string cost one parse, and un-anchored or malformed strings
cost none.

computePathMatchScore lowercased and trimmed the query for every
candidate document; pathSearch now does that once and passes the
lowered query down. The existing path-search tests cover the scoring.

portability_allowlist.txt: two search_service.cpp anchors reflowed.

edgeAnchorHints returns its two short strings by value: the per-thread memo
clears itself at 1024 entries, so a reference into it could dangle across
calls even though today's three callers consume the result immediately.

Signed-off-by: trvon <me@trevon.dev>

* fix(search): preserve hydration failure semantics

Stop reranking before scorer calls or score changes when previews fail.
Commit anchor hints atomically and recognize escaped JSON keys.

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>
2026-09-12 10:09:20 -04:00
trvon
883eb1127e
feat: installation tests and build improvements. Test fixes related to optimizations for reranker 2026-04-08 14:01:18 -04:00
trvon
e8a6296797
fix: re-ranker embedding generation fix, test failures drift fix, search improvements for graph query expansion and gliner improvements for building graph 2026-04-06 18:38:54 -04:00
trvon
364a11476d
fix: improvements for embedding repair routing to hot model. Continued search / graph build retrieval improvements 2026-04-06 09:55:59 -04:00