yams/tests/unit/search/symspell_search_catch2_test.cpp
Trevon 3f37e8f564
Sync experimental into main: Search retrieval improvements (#60)
* fix(graph): handle qualified C++ symbols

Signed-off-by: trvon <me@trevon.dev>

* refactor(vector)!: remove dead transition APIs

Split Simeon vector and retrieval linkage so consumers inherit only the runtime surface they use.

BREAKING CHANGE: remove unused async/model/config helpers and the undefined vector-backend factory from public headers.

Signed-off-by: trvon <me@trevon.dev>

* fix(vector): harden PQ snapshot lifecycle

Validate and atomically install persisted generations, revalidate after races, and fall back to exact search when a snapshot is stale. Add the exact-scan reference lane and distinguish observed work counters from unavailable backend metrics.

Signed-off-by: trvon <me@trevon.dev>

* feat(topology): expose route coordinates

Persist chart distortion and local-dimension evidence, retain pre-scalarization route axes, and emit detailed trace rows without treating unavailable vector work as zero.

Signed-off-by: trvon <me@trevon.dev>

* feat(xplan): add vector decision gates

Track binary and corpus identity, preserve missing-work semantics, and add repeated PQ routing, crossover, and rerank plans with quality and cost summaries.

Signed-off-by: trvon <me@trevon.dev>

* feat(topology): formalize coordinate routing

Separate chart geometry from route-risk admission and prove candidate-stage recall and strict-focus boundaries. Record the measured atlas limitations and literature basis for the next construction experiment.

Signed-off-by: trvon <me@trevon.dev>

* fix(search): serialize SymSpell readers

SQLite-backed lookups reuse mutable prepared statements, so shared-reader locking is unsafe. Advance the owned SymSpell revision for batched delete insertion and cover concurrent root lookups under TSAN.

Signed-off-by: trvon <me@trevon.dev>

* fix(vector): make mutations atomic

Preserve stored vectors on replacement and commit failures, reject cross-thread writes during explicit transactions, and restore exact reranking for quantized-primary PQ rows.

* fix(search)!: surface SymSpell write errors

Propagate persistent dictionary failures through metadata ingestion, roll back failed batches, and make clear remove stored terms.

BREAKING CHANGE: SymSpellSearch mutations now return Result values.

* feat(topology): formalize candidate expansion

* feat(vector): add coordinate-aware embeddings

* feat(topology): preserve protected relation cover

* feat(search): add graph candidate rescue

* feat(daemon): expose retrieval hydration telemetry

* feat(xplan): add rescue and hydration gates

* refactor(content): dedupe media handlers

Reuse shared graph presenter helpers and centralize repeated audio/video/archive buffer processing, validation, lifecycle, and stats helper logic.

* feat(lifecycle): prune stale corpus data

Keep cleanup preview-first while retention stays opt-in and long prune work no longer holds IPC responses open.

* feat(search): add topology candidate rescue

Preserve high-value routed candidates across vector backends while keeping exact rerank diagnostics explicit.

* feat(search): expose evidence paths

* docs: define mobile host contract

* formal: model bounded candidate rescue

* fix(prune): inspect extensionless signatures

* fix(prune): tolerate stale extensions

---------

Signed-off-by: trvon <me@trevon.dev>
2026-07-24 19:00:22 -04:00

109 lines
3.8 KiB
C++

#include <catch2/catch_test_macros.hpp>
#include <sqlite3.h>
#include <atomic>
#include <future>
#include <string_view>
#include <thread>
#include <vector>
#include <yams/search/symspell_search.h>
TEST_CASE("SymSpellSearch serializes SQLite-backed readers",
"[search][symspell][concurrency][catch2]") {
sqlite3* db = nullptr;
REQUIRE(sqlite3_open(":memory:", &db) == SQLITE_OK);
{
const auto schema = yams::search::SymSpellSearch::initializeSchema(db);
REQUIRE(schema.has_value());
yams::search::SymSpellSearch index(db);
REQUIRE(index.addTermsBatch({{"alpha", 100}, {"beta", 80}, {"gamma", 60}}).has_value());
constexpr std::size_t kThreadCount = 8;
constexpr std::size_t kIterations = 256;
std::promise<void> startPromise;
const auto start = startPromise.get_future().share();
std::atomic<std::size_t> ready{0};
std::atomic<bool> failed{false};
std::vector<std::thread> readers;
readers.reserve(kThreadCount);
for (std::size_t threadIndex = 0; threadIndex < kThreadCount; ++threadIndex) {
readers.emplace_back([&, threadIndex] {
ready.fetch_add(1, std::memory_order_release);
start.wait();
for (std::size_t iteration = 0; iteration < kIterations; ++iteration) {
const std::string_view expected =
(threadIndex + iteration) % 2 == 0 ? "alpha" : "beta";
const auto results = index.search(std::string(expected));
if (results.empty() || results.front().term != expected ||
!index.hasExactMatch(expected)) {
failed.store(true, std::memory_order_relaxed);
break;
}
}
});
}
while (ready.load(std::memory_order_acquire) != kThreadCount) {
std::this_thread::yield();
}
startPromise.set_value();
for (auto& reader : readers) {
reader.join();
}
CHECK_FALSE(failed.load(std::memory_order_relaxed));
}
CHECK(sqlite3_close(db) == SQLITE_OK);
}
TEST_CASE("SymSpellSearch reports rejected SQLite writes",
"[search][symspell][persistence][errors][catch2]") {
sqlite3* db = nullptr;
REQUIRE(sqlite3_open(":memory:", &db) == SQLITE_OK);
{
REQUIRE(yams::search::SymSpellSearch::initializeSchema(db).has_value());
char* error = nullptr;
REQUIRE(sqlite3_exec(db,
"CREATE TRIGGER reject_symspell_term BEFORE INSERT ON symspell_terms "
"BEGIN SELECT RAISE(ABORT, 'forced term failure'); END",
nullptr, nullptr, &error) == SQLITE_OK);
sqlite3_free(error);
yams::search::SymSpellSearch index(db);
SECTION("single term") {
CHECK_FALSE(index.addTerm("alpha", 100).has_value());
}
SECTION("transactional batch") {
CHECK_FALSE(index.addTermsBatch({{"alpha", 100}, {"beta", 80}}).has_value());
CHECK_FALSE(index.hasExactMatch("beta"));
}
CHECK_FALSE(index.hasExactMatch("alpha"));
}
CHECK(sqlite3_close(db) == SQLITE_OK);
}
TEST_CASE("SymSpellSearch clear removes terms", "[search][symspell][persistence][catch2]") {
sqlite3* db = nullptr;
REQUIRE(sqlite3_open(":memory:", &db) == SQLITE_OK);
{
REQUIRE(yams::search::SymSpellSearch::initializeSchema(db).has_value());
yams::search::SymSpellSearch index(db);
REQUIRE(index.addTerm("alpha", 100).has_value());
REQUIRE(index.hasExactMatch("alpha"));
REQUIRE(index.clear().has_value());
CHECK_FALSE(index.hasExactMatch("alpha"));
}
CHECK(sqlite3_close(db) == SQLITE_OK);
}