trevon/yams
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2
yams/tests/unit/search/search_engine_config_catch2_test.cpp
Trevon d398556ee5
 fix(tests): replace frozen expectations and harden release gates (#323) 
* test(search,daemon,storage): assert behaviour instead of frozen literals

Several suites encoded exact internal values that legitimate tuning and
ranking changes are allowed to move: SCIENTIFIC-era profile weights and
rrfK values, corpus-statistic figures, and hardcoded timeout/cancellation
numbers. They failed on every platform after those changes landed, so the
real behaviour regressions they should catch were hidden behind noise.

Replace the frozen literals with contract assertions: weights are finite,
non-negative and normalize, state selection still holds, capability-gated
sources stay zeroed, and hybrid results keep matching canonical hash/path
sets. Deadlines are not widened and the daemon paths they cover are
unchanged.

Also add independent corpus and adapter oracles, deterministic collector
cancellation and late-result gates, and synchronize ServiceManager
shutdown observation, so the tests fail for the behaviour they name.

The portability allowlist keeps its two reviewed raw test-env reads; only
their line numbers moved.

Signed-off-by: trvon <git@trevon.dev>

* ci(tests): summarise Meson failures without flooding the log

A failing test run printed the whole Meson suite output, so the actual
failing test names and their assertion messages were buried and the
uploaded logs were dominated by passing-test chatter.

Add a bounded reporter that extracts the failing test names plus their
surrounding diagnostics, keeps an overall cap on emitted lines, and is
unit-tested against representative Meson output. Wire it into the Tests
workflow so failures stay readable.

Signed-off-by: trvon <git@trevon.dev>

* build(deb): validate candidate identity, mapping and upgrade ordering

The Debian packaging scripts took filenames and versions on trust, so a
misnamed candidate or a wrong package ordering could pass validation while
the real install path was never exercised.

Fail closed on candidate identity, version and architecture, verify the
package-to-source mapping, seed the legacy-only layout, wait on the actual
services, and assert install/upgrade/remove ordering for both Debian and
RPM in an offline container. Cases that cannot be proven on a host are
reported as skipped rather than silently passing.

Signed-off-by: trvon <git@trevon.dev>

---------

Signed-off-by: trvon <git@trevon.dev>
2026-10-19 19:16:57 -06:00

436 lines
21 KiB
C++
Raw Permalink Blame History

/**
* @file search_engine_config_catch2_test.cpp
* @brief Tests for SearchEngineConfig, CorpusProfile, ComponentResult helpers,
* and score attribution helpers (search_models.h)
*
* These are header-only or inline functions at 0% coverage. Tests here
* exercise forProfile(), detectProfile(), componentSourceToString(),
* isVectorComponent(), isTextAnchoringComponent(), and accumulateComponentScore().
*/
#include <catch2/catch_approx.hpp>
#include <catch2/catch_test_macros.hpp>
#include <yams/search/search_models.h>
#include <array>
#include <cmath>
#include <string>
#include <unordered_map>
using namespace yams::search;
using Catch::Approx;
using yams::metadata::SearchResult;
TEST_CASE("navigationZoomLevelToString covers all zoom levels", "[search][config][catch2]") {
using Z = SearchEngineConfig::NavigationZoomLevel;
CHECK(std::string(SearchEngineConfig::navigationZoomLevelToString(Z::Auto)) == "AUTO");
CHECK(std::string(SearchEngineConfig::navigationZoomLevelToString(Z::Map)) == "MAP");
CHECK(std::string(SearchEngineConfig::navigationZoomLevelToString(Z::Neighborhood)) ==
"NEIGHBORHOOD");
CHECK(std::string(SearchEngineConfig::navigationZoomLevelToString(Z::Street)) == "STREET");
}
// ────────────────────────────────────────────────────────────────────────────────
// forProfile
// ────────────────────────────────────────────────────────────────────────────────
TEST_CASE("forProfile preserves profile identity and valid fusion domains",
"[search][config][catch2]") {
using CP = SearchEngineConfig::CorpusProfile;
constexpr std::array profiles = {CP::CODE, CP::PROSE, CP::DOCS, CP::MIXED, CP::CUSTOM};
for (const auto profile : profiles) {
const auto cfg = SearchEngineConfig::forProfile(profile);
const std::array weights = {
cfg.textWeight, cfg.simeonTextWeight, cfg.graphTextWeight, cfg.pathTreeWeight,
cfg.kgWeight, cfg.vectorWeight, cfg.graphVectorWeight, cfg.entityVectorWeight,
cfg.tagWeight, cfg.metadataWeight,
};
CAPTURE(SearchEngineConfig::corpusProfileToString(profile));
CHECK(cfg.corpusProfile == profile);
for (const float weight : weights) {
CHECK(std::isfinite(weight));
CHECK(weight >= 0.0F);
}
}
}
// ────────────────────────────────────────────────────────────────────────────────
// detectProfile
// ────────────────────────────────────────────────────────────────────────────────
TEST_CASE("detectProfile empty map returns MIXED", "[search][config][catch2]") {
std::unordered_map<std::string, int64_t> empty;
auto profile = SearchEngineConfig::detectProfile(empty);
CHECK(profile == SearchEngineConfig::CorpusProfile::MIXED);
}
TEST_CASE("detectProfile dominant code extensions returns CODE", "[search][config][catch2]") {
std::unordered_map<std::string, int64_t> counts = {
{".py", 40}, {".cpp", 20}, {".h", 10}, {".md", 5}, {".txt", 5}};
// 70/80 = 87.5% code
auto profile = SearchEngineConfig::detectProfile(counts);
CHECK(profile == SearchEngineConfig::CorpusProfile::CODE);
}
TEST_CASE("detectProfile dominant prose extensions returns PROSE", "[search][config][catch2]") {
std::unordered_map<std::string, int64_t> counts = {
{".md", 50}, {".txt", 20}, {".pdf", 10}, {".py", 5}};
// 80/85 = 94% prose
auto profile = SearchEngineConfig::detectProfile(counts);
CHECK(profile == SearchEngineConfig::CorpusProfile::PROSE);
}
TEST_CASE("detectProfile mixed code + prose returns DOCS", "[search][config][catch2]") {
std::unordered_map<std::string, int64_t> counts = {
{".py", 40}, {".md", 35}, {".txt", 10}, {".dat", 15}};
// code: 40/100 = 40%, prose: 45/100 = 45%
auto profile = SearchEngineConfig::detectProfile(counts);
CHECK(profile == SearchEngineConfig::CorpusProfile::DOCS);
}
TEST_CASE("detectProfile all unknown extensions returns MIXED", "[search][config][catch2]") {
std::unordered_map<std::string, int64_t> counts = {{".xyz", 50}, {".zyx", 30}, {".qwerty", 20}};
auto profile = SearchEngineConfig::detectProfile(counts);
CHECK(profile == SearchEngineConfig::CorpusProfile::MIXED);
}
TEST_CASE("detectProfile case-insensitive extension matching", "[search][config][catch2]") {
// detectProfile lowercases extensions so ".PY" counts as code
std::unordered_map<std::string, int64_t> counts = {
{".PY", 30}, {".CPP", 30}, {".H", 20}, {".md", 5}};
// 80/85 = 94% code (lowercase matches)
auto profile = SearchEngineConfig::detectProfile(counts);
CHECK(profile == SearchEngineConfig::CorpusProfile::CODE);
}
// ────────────────────────────────────────────────────────────────────────────────
// componentSourceToString
// ────────────────────────────────────────────────────────────────────────────────
TEST_CASE("componentSourceToString covers all sources", "[search][config][catch2]") {
using S = ComponentResult::Source;
CHECK(std::string(componentSourceToString(S::Text)) == "text");
CHECK(std::string(componentSourceToString(S::SimeonText)) == "simeon_text");
CHECK(std::string(componentSourceToString(S::GraphText)) == "graph_text");
CHECK(std::string(componentSourceToString(S::PathTree)) == "path_tree");
CHECK(std::string(componentSourceToString(S::KnowledgeGraph)) == "kg");
CHECK(std::string(componentSourceToString(S::Vector)) == "vector");
CHECK(std::string(componentSourceToString(S::EntityVector)) == "entity_vector");
CHECK(std::string(componentSourceToString(S::Tag)) == "tag");
CHECK(std::string(componentSourceToString(S::Metadata)) == "metadata");
CHECK(std::string(componentSourceToString(S::Symbol)) == "symbol");
CHECK(std::string(componentSourceToString(S::Unknown)) == "unknown");
}
TEST_CASE("componentSourceWeight matches config fields", "[search][config][catch2]") {
SearchEngineConfig cfg;
cfg.textWeight = 0.41f;
cfg.simeonTextWeight = 0.11f;
cfg.graphTextWeight = 0.12f;
cfg.pathTreeWeight = 0.09f;
cfg.kgWeight = 0.07f;
cfg.vectorWeight = 0.31f;
cfg.graphVectorWeight = 0.13f;
cfg.entityVectorWeight = 0.05f;
cfg.tagWeight = 0.02f;
cfg.metadataWeight = 0.03f;
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::Text) == Approx(0.41f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::SimeonText) ==
Approx(0.11f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::GraphText) ==
Approx(0.12f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::PathTree) ==
Approx(0.09f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::KnowledgeGraph) ==
Approx(0.07f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::Vector) ==
Approx(0.31f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::GraphVector) ==
Approx(0.13f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::EntityVector) ==
Approx(0.05f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::Tag) == Approx(0.02f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::Metadata) ==
Approx(0.03f));
CHECK(yams::search::componentSourceWeight(cfg, ComponentResult::Source::Symbol) ==
Approx(0.0f));
}
// ────────────────────────────────────────────────────────────────────────────────
// isVectorComponent / isTextAnchoringComponent
// ────────────────────────────────────────────────────────────────────────────────
TEST_CASE("isVectorComponent is true for Vector and EntityVector", "[search][config][catch2]") {
using S = ComponentResult::Source;
CHECK(isVectorComponent(S::Vector) == true);
CHECK(isVectorComponent(S::EntityVector) == true);
CHECK(isVectorComponent(S::Text) == false);
CHECK(isVectorComponent(S::PathTree) == false);
CHECK(isVectorComponent(S::KnowledgeGraph) == false);
CHECK(isVectorComponent(S::Tag) == false);
CHECK(isVectorComponent(S::Metadata) == false);
CHECK(isVectorComponent(S::Symbol) == false);
CHECK(isVectorComponent(S::Unknown) == false);
}
TEST_CASE("isTextAnchoringComponent identifies non-vector components", "[search][config][catch2]") {
using S = ComponentResult::Source;
CHECK(isTextAnchoringComponent(S::Text) == true);
CHECK(isTextAnchoringComponent(S::SimeonText) == true);
CHECK(isTextAnchoringComponent(S::GraphText) == true);
CHECK(isTextAnchoringComponent(S::PathTree) == true);
CHECK(isTextAnchoringComponent(S::KnowledgeGraph) == true);
CHECK(isTextAnchoringComponent(S::Tag) == true);
CHECK(isTextAnchoringComponent(S::Metadata) == true);
CHECK(isTextAnchoringComponent(S::Symbol) == true);
CHECK(isTextAnchoringComponent(S::Vector) == false);
CHECK(isTextAnchoringComponent(S::EntityVector) == false);
CHECK(isTextAnchoringComponent(S::Unknown) == false);
}
// ────────────────────────────────────────────────────────────────────────────────
// accumulateComponentScore
// ────────────────────────────────────────────────────────────────────────────────
TEST_CASE("accumulateComponentScore routes Text to keywordScore", "[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::Text, 0.5);
REQUIRE(r.keywordScore.has_value());
CHECK(r.keywordScore.value() == Approx(0.5));
}
TEST_CASE("accumulateComponentScore routes Vector to vectorScore", "[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::Vector, 0.3);
REQUIRE(r.vectorScore.has_value());
CHECK(r.vectorScore.value() == Approx(0.3));
}
TEST_CASE("accumulateComponentScore routes EntityVector to vectorScore",
"[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::EntityVector, 0.2);
REQUIRE(r.vectorScore.has_value());
CHECK(r.vectorScore.value() == Approx(0.2));
}
TEST_CASE("accumulateComponentScore routes KnowledgeGraph to kgScore", "[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::KnowledgeGraph, 0.4);
REQUIRE(r.kgScore.has_value());
CHECK(r.kgScore.value() == Approx(0.4));
}
TEST_CASE("accumulateComponentScore routes PathTree to pathScore", "[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::PathTree, 0.15);
REQUIRE(r.pathScore.has_value());
CHECK(r.pathScore.value() == Approx(0.15));
}
TEST_CASE("accumulateComponentScore routes Tag to tagScore", "[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::Tag, 0.1);
REQUIRE(r.tagScore.has_value());
CHECK(r.tagScore.value() == Approx(0.1));
}
TEST_CASE("accumulateComponentScore routes Metadata to tagScore", "[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::Metadata, 0.1);
REQUIRE(r.tagScore.has_value());
CHECK(r.tagScore.value() == Approx(0.1));
}
TEST_CASE("accumulateComponentScore routes Symbol to symbolScore", "[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::Symbol, 0.25);
REQUIRE(r.symbolScore.has_value());
CHECK(r.symbolScore.value() == Approx(0.25));
}
TEST_CASE("accumulateComponentScore accumulates multiple calls", "[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::Vector, 0.3);
accumulateComponentScore(r, ComponentResult::Source::EntityVector, 0.2);
REQUIRE(r.vectorScore.has_value());
CHECK(r.vectorScore.value() == Approx(0.5));
}
TEST_CASE("accumulateComponentScore Unknown source is no-op", "[search][config][catch2]") {
SearchResult r;
accumulateComponentScore(r, ComponentResult::Source::Unknown, 1.0);
CHECK(!r.keywordScore.has_value());
CHECK(!r.vectorScore.has_value());
CHECK(!r.kgScore.has_value());
CHECK(!r.pathScore.has_value());
CHECK(!r.tagScore.has_value());
CHECK(!r.symbolScore.has_value());
}
// ────────────────────────────────────────────────────────────────────────────────
// SearchEngineConfig validity and explicit behavior
// ────────────────────────────────────────────────────────────────────────────────
TEST_CASE("SearchEngineConfig initializes tunables within supported domains",
"[search][config][catch2]") {
const SearchEngineConfig cfg;
const std::array weights = {
cfg.textWeight, cfg.simeonTextWeight, cfg.graphTextWeight, cfg.pathTreeWeight,
cfg.kgWeight, cfg.vectorWeight, cfg.graphVectorWeight, cfg.entityVectorWeight,
cfg.tagWeight, cfg.metadataWeight,
};
for (const float weight : weights) {
CHECK(std::isfinite(weight));
CHECK(weight >= 0.0F);
}
// Do not invent scalar bounds: zero can disable a feature or select an
// inherited value, and routing margins are score differences, not probabilities.
CHECK(std::isfinite(cfg.similarityThreshold));
CHECK(std::isfinite(cfg.topologyAdaptiveProbeScoreGap));
CHECK(std::isfinite(cfg.topologyNarrowMinBoundaryMargin));
CHECK(std::isfinite(cfg.topologyEvidenceWeight));
CHECK(std::isfinite(cfg.rrfK));
CHECK(cfg.rrfK >= 0.0F);
CHECK(std::isfinite(cfg.bm25NormDivisor));
CHECK(std::isfinite(cfg.rerankBlendWeight));
CHECK(cfg.rerankBlendWeight >= 0.0F);
CHECK(cfg.rerankBlendWeight <= 1.0F);
}
TEST_CASE("topologyVectorPolicyToString covers every policy",
"[search][config][topology][catch2]") {
using Policy = SearchEngineConfig::TopologyVectorPolicy;
CHECK(std::string_view(SearchEngineConfig::topologyVectorPolicyToString(Policy::Augment)) ==
"augment");
CHECK(std::string_view(SearchEngineConfig::topologyVectorPolicyToString(Policy::Narrow)) ==
"narrow");
CHECK(std::string_view(SearchEngineConfig::topologyVectorPolicyToString(Policy::Shadow)) ==
"shadow");
CHECK(std::string_view(SearchEngineConfig::topologyVectorPolicyToString(Policy::RouteOnly)) ==
"route_only");
}
TEST_CASE("topology expansion limit resolves explicit and inherited values",
"[search][config][topology][catch2]") {
SearchEngineConfig cfg;
cfg.topologyExpansionOutputLimit = 0;
CHECK(cfg.resolvedTopologyExpansionOutputLimit(37) == 37);
cfg.topologyExpansionOutputLimit = 91;
CHECK(cfg.resolvedTopologyExpansionOutputLimit(37) == 91);
}
TEST_CASE("SearchEngineConfig preserves typed execution policy across tuning",
"[search][config][topology][catch2]") {
SearchEngineConfig tuned;
tuned.textWeight = 0.25F;
SearchEngineConfig configured;
configured.topologyRoutingMode = SearchEngineConfig::TopologyRoutingMode::HybridAssist;
configured.topologyMaxClusters = 3;
configured.topologyMinClusters = 2;
configured.topologyMaxSeedDocuments = 24;
configured.topologyRoutingAnnCandidateLimit = 16;
configured.topologyAdaptiveProbeScoreGap = 0.08F;
configured.topologyNarrowMinBoundaryMargin = 0.12F;
configured.topologyMaxDocs = 48;
configured.topologyExpansionOutputLimit = 96;
configured.topologyMedoidBoost = 0.17F;
configured.topologyRouteScoringMode =
SearchEngineConfig::TopologyRouteScoringMode::SeedCoverage;
configured.topologySparseDenseAlpha = 0.7F;
configured.topologyMinRouteScore = 0.2F;
configured.topologyExpansionSource =
SearchEngineConfig::TopologyExpansionSource::GraphNeighbors;
configured.topologyGraphNeighborMinScore = 0.4F;
configured.topologyGraphNeighborReciprocalOnly = false;
configured.topologyGraphWeightedSeedRanking = true;
configured.topologyGraphVectorSeedProbe = 16;
configured.topologyVectorPolicy = SearchEngineConfig::TopologyVectorPolicy::Narrow;
tuned.applyExecutionPolicyFrom(configured);
CHECK(tuned.textWeight == Approx(0.25F));
CHECK(tuned.topologyRoutingMode == SearchEngineConfig::TopologyRoutingMode::HybridAssist);
CHECK(tuned.topologyMaxClusters == 3);
CHECK(tuned.topologyMinClusters == 2);
CHECK(tuned.topologyMaxSeedDocuments == 24);
CHECK(tuned.topologyRoutingAnnCandidateLimit == 16);
CHECK(tuned.topologyAdaptiveProbeScoreGap == Approx(0.08F));
CHECK(tuned.topologyNarrowMinBoundaryMargin == Approx(0.12F));
CHECK(tuned.topologyMaxDocs == 48);
CHECK(tuned.topologyExpansionOutputLimit == 96);
CHECK(tuned.resolvedTopologyExpansionOutputLimit(32) == 96);
CHECK(tuned.topologyMedoidBoost == Approx(0.17F));
CHECK(tuned.topologyRouteScoringMode ==
SearchEngineConfig::TopologyRouteScoringMode::SeedCoverage);
CHECK(tuned.topologySparseDenseAlpha == Approx(0.7F));
CHECK(tuned.topologyMinRouteScore == Approx(0.2F));
CHECK(tuned.topologyExpansionSource ==
SearchEngineConfig::TopologyExpansionSource::GraphNeighbors);
CHECK(tuned.topologyGraphNeighborMinScore == Approx(0.4F));
CHECK_FALSE(tuned.topologyGraphNeighborReciprocalOnly);
CHECK(tuned.topologyGraphWeightedSeedRanking);
CHECK(tuned.topologyGraphVectorSeedProbe == 16);
CHECK(tuned.topologyVectorPolicy == SearchEngineConfig::TopologyVectorPolicy::Narrow);
}
TEST_CASE("SearchEngineConfig RouteOnly policy is operator-selected and gated",
"[search][config][topology][route-only][catch2]") {
SECTION("defaults never select RouteOnly or open the evaluation gate") {
SearchEngineConfig cfg;
CHECK(cfg.topologyVectorPolicy == SearchEngineConfig::TopologyVectorPolicy::Shadow);
CHECK_FALSE(cfg.allowUncertifiedRouteOnly);
CHECK(std::string_view(SearchEngineConfig::topologyVectorPolicyToString(
SearchEngineConfig::TopologyVectorPolicy::RouteOnly)) == "route_only");
}
SECTION("tuning replaces weights but keeps the policy and the gate") {
SearchEngineConfig tuned;
tuned.textWeight = 0.25F;
SearchEngineConfig configured;
configured.topologyVectorPolicy = SearchEngineConfig::TopologyVectorPolicy::RouteOnly;
configured.allowUncertifiedRouteOnly = true;
tuned.applyExecutionPolicyFrom(configured);
CHECK(tuned.textWeight == Approx(0.25F));
CHECK(tuned.topologyVectorPolicy == SearchEngineConfig::TopologyVectorPolicy::RouteOnly);
CHECK(tuned.allowUncertifiedRouteOnly);
}
SECTION("a closed gate survives tuning too") {
SearchEngineConfig tuned;
tuned.allowUncertifiedRouteOnly = true;
SearchEngineConfig configured;
tuned.applyExecutionPolicyFrom(configured);
CHECK_FALSE(tuned.allowUncertifiedRouteOnly);
CHECK(tuned.topologyVectorPolicy == SearchEngineConfig::TopologyVectorPolicy::Shadow);
}
}
// ────────────────────────────────────────────────────────────────────────────────
// SearchResponse helpers
// ────────────────────────────────────────────────────────────────────────────────
TEST_CASE("SearchResponse hasResults / isComplete", "[search][config][catch2]") {
SearchResponse resp;
CHECK(resp.hasResults() == false);
CHECK(resp.isComplete() == true);
resp.results.push_back(SearchResult{});
CHECK(resp.hasResults() == true);
resp.timedOutComponents.push_back("slow_component");
CHECK(resp.isComplete() == false);
}
Reference in a new issue View git blame Copy permalink