* test(graph): cover --topology-clusters --scope-cwd outside src/include Follow-up to #280. --topology-clusters --scope-cwd still built its scope from <cwd>/src and <cwd>/include, so a corpus laid out any other way had no scoped members: - The scope path set must hold every stored path under the cwd, also when the cwd is reached through a symlink, and not a sibling that shares the prefix. - GraphTopologySupport must build the set from its own metadata repo. The existing cluster-scope case built its path set by hand from the old src/include contract. It now builds the set from /repo, so /repo/tests/ is in scope and a document outside /repo is not. Signed-off-by: trvon <me@trevon.dev> * fix(graph): scope --topology-clusters --scope-cwd to the whole cwd Follow-up to #280. Topology views built their --scope-cwd path set from <cwd>/src and <cwd>/include only, a leftover from the code-symbol graph removed in v0.20. Corpora laid out any other way reported no scoped members. Build the set from the same roots as the --list-type ranges: the cwd, lexical and symlink-resolved. Keep only paths really under a root, since the prefix query matches with LIKE. The topology view no longer prunes build/, packages and IDE paths, so it matches --list-type. GraphTopologySupport now builds the set from its own metadata repo instead of the CLI app context. Remove buildGraphCodeScopePathSet and buildGraphCurrentScopePathSet, which have no other callers, and use one "Scoped to paths under the current directory" note for both views. Signed-off-by: trvon <me@trevon.dev> * test(cli): cover graph --name on a document with a path:file: node Follow-up to #280. yams graph --name <path> tried path:file:<path> node keys before the document lookup. v0.20 ingestion writes path:logical: and path:<snapshot>: keys, so the probe misses on a fresh corpus; after `yams doctor repair --graph` (or NL entity extraction) the key exists, and --name printed a raw traversal of the file node instead of the document's related documents. Smoke cases through the in-process CLI: - --name on a file with a path:file: node returns its hash and its semantic neighbours (red). - A relative --name resolves under the cwd even though a newer copy with the same relative path exists in another checkout. - A bare file name still resolves by name. The portability allowlist entry for this file moves with its getenv line. Signed-off-by: trvon <me@trevon.dev> * fix(cli): resolve graph --name through the document lookup only Follow-up to #280. yams graph --name first traversed path:file:<path> node keys. v0.20 ingestion writes path:logical:<path> and path:<snapshot>:<path>, which only chain to blob versions; semantic neighbours hang off doc:<hash>. On a fresh corpus the probe always missed. After `yams doctor repair --graph` or NL entity extraction it hit and printed a raw file-node traversal instead of the related documents. Drop the probe and go straight to the document lookup, which starts at doc:<hash> since #280. Resolving to path:logical: instead would show the version chain, not related documents; --node-key still traverses any node. Send --name as the path ingestion stores: relative to the client cwd, lexically normalized, symlinks resolved. The daemon resolved relative names against its own cwd and fell back to a suffix match, which picks the newest copy across checkouts. Fall back to the name as given when it does not resolve, so bare file names and suffixes still work. build_graph_file_node_candidates had no other callers; its tests are replaced by tests for buildGraphDocumentNameCandidates. Signed-off-by: trvon <me@trevon.dev> * test(graph): cover --limit on the document graph lookup Follow-up to #280. yams graph <hash> and --name ignored --limit: the daemon capped the graph query at 100 nodes and the same-directory fallback at 20 documents. Smoke cases through the in-process CLI: - --limit 1 returns one related document by name and by hash, and the first of the uncapped list. - --limit 1 caps the same-directory fallback for a document without graph edges. - Without --limit the related lists are unchanged. Signed-off-by: trvon <me@trevon.dev> * fix(graph): pass --limit through the document graph lookup Follow-up to #280. yams graph <hash> and --name sent --depth but not --limit; DocumentService fixed the graph query at 100 nodes and the same-directory fallback at 20 documents. Add GetRequest.graph_limit (proto field 27) and RetrieveDocumentRequest::graphLimit, and pass them from the CLI through the dispatcher. A limit caps the related list, graph neighbours, path history and the same-directory fallback alike. The graph query never asks for fewer than 100 nodes, since path and blob nodes count toward that limit. The CLI sends a limit only when --limit is given; 0 keeps the old caps, so default output is unchanged. The binary GetRequest layout appends the field and still reads payloads without it. Unit tests cover the proto round trip, the dispatcher mapping and the DocumentService caps. Signed-off-by: trvon <me@trevon.dev> --------- Signed-off-by: trvon <me@trevon.dev>
262 lines
10 KiB
C++
262 lines
10 KiB
C++
// Copyright (c) 2025 YAMS Contributors
|
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
|
|
#include <catch2/catch_test_macros.hpp>
|
|
|
|
#include <yams/cli/graph_scope_support.h>
|
|
#include <yams/cli/graph_topology_support.h>
|
|
#include <yams/metadata/connection_pool.h>
|
|
#include <yams/metadata/metadata_repository.h>
|
|
#include <yams/metadata/path_utils.h>
|
|
#include <yams/topology/topology_artifacts.h>
|
|
|
|
#include <chrono>
|
|
#include <filesystem>
|
|
#include <fstream>
|
|
#include <memory>
|
|
|
|
using namespace yams::cli;
|
|
using namespace yams::metadata;
|
|
using namespace yams::topology;
|
|
|
|
namespace {
|
|
|
|
std::filesystem::path makeTempDbPath() {
|
|
auto dir = std::filesystem::temp_directory_path() / "yams_graph_topology_support_test";
|
|
std::filesystem::create_directories(dir);
|
|
return dir /
|
|
("graph_topology_support_" +
|
|
std::to_string(std::chrono::steady_clock::now().time_since_epoch().count()) + ".db");
|
|
}
|
|
|
|
DocumentInfo makeDocument(const std::string& path, const std::string& hash) {
|
|
DocumentInfo info;
|
|
info.filePath = path;
|
|
info.fileName = std::filesystem::path(path).filename().string();
|
|
info.fileExtension = std::filesystem::path(path).extension().string();
|
|
info.fileSize = 128;
|
|
info.sha256Hash = hash;
|
|
info.mimeType = "text/plain";
|
|
info.createdTime = std::chrono::floor<std::chrono::seconds>(std::chrono::system_clock::now());
|
|
info.modifiedTime = info.createdTime;
|
|
info.indexedTime = info.createdTime;
|
|
info.contentExtracted = true;
|
|
info.extractionStatus = ExtractionStatus::Success;
|
|
populatePathDerivedFields(info);
|
|
return info;
|
|
}
|
|
|
|
TopologyArtifactBatch makeSnapshot() {
|
|
TopologyArtifactBatch snapshot;
|
|
snapshot.snapshotId = "snap-1";
|
|
|
|
ClusterArtifact clusterA;
|
|
clusterA.clusterId = "cluster-a";
|
|
clusterA.memberCount = 2;
|
|
snapshot.clusters.push_back(clusterA);
|
|
|
|
ClusterArtifact clusterB;
|
|
clusterB.clusterId = "cluster-b";
|
|
clusterB.memberCount = 1;
|
|
snapshot.clusters.push_back(clusterB);
|
|
|
|
snapshot.memberships.push_back(DocumentClusterMembership{
|
|
.documentHash = "hash-src",
|
|
.clusterId = "cluster-a",
|
|
.clusterLevel = 0,
|
|
.bridgeScore = 0.1,
|
|
.role = DocumentTopologyRole::Core,
|
|
});
|
|
snapshot.memberships.push_back(DocumentClusterMembership{
|
|
.documentHash = "hash-include",
|
|
.clusterId = "cluster-a",
|
|
.clusterLevel = 0,
|
|
.bridgeScore = 0.2,
|
|
.role = DocumentTopologyRole::Bridge,
|
|
});
|
|
snapshot.memberships.push_back(DocumentClusterMembership{
|
|
.documentHash = "hash-test",
|
|
.clusterId = "cluster-b",
|
|
.clusterLevel = 1,
|
|
.bridgeScore = 0.3,
|
|
.role = DocumentTopologyRole::Outlier,
|
|
});
|
|
|
|
return snapshot;
|
|
}
|
|
|
|
struct GraphTopologySupportFixture {
|
|
GraphTopologySupportFixture() {
|
|
dbPath = makeTempDbPath();
|
|
|
|
ConnectionPoolConfig poolConfig;
|
|
poolConfig.minConnections = 1;
|
|
poolConfig.maxConnections = 2;
|
|
pool = std::make_shared<ConnectionPool>(dbPath.string(), poolConfig);
|
|
REQUIRE(pool->initialize().has_value());
|
|
|
|
metadataRepo = std::make_shared<MetadataRepository>(*pool);
|
|
REQUIRE(metadataRepo->insertDocument(makeDocument("/repo/src/main.cpp", "hash-src"))
|
|
.has_value());
|
|
REQUIRE(metadataRepo->insertDocument(makeDocument("/repo/include/main.h", "hash-include"))
|
|
.has_value());
|
|
REQUIRE(metadataRepo->insertDocument(makeDocument("/repo/tests/main_test.cpp", "hash-test"))
|
|
.has_value());
|
|
}
|
|
|
|
~GraphTopologySupportFixture() {
|
|
metadataRepo.reset();
|
|
pool.reset();
|
|
std::error_code ec;
|
|
std::filesystem::remove(dbPath, ec);
|
|
std::filesystem::remove(dbPath.string() + "-shm", ec);
|
|
std::filesystem::remove(dbPath.string() + "-wal", ec);
|
|
}
|
|
|
|
std::filesystem::path dbPath;
|
|
std::shared_ptr<ConnectionPool> pool;
|
|
std::shared_ptr<MetadataRepository> metadataRepo;
|
|
};
|
|
|
|
} // namespace
|
|
|
|
TEST_CASE("GraphTopologySupport labels topology enums", "[cli][graph][topology]") {
|
|
CHECK(topologyInputKindLabel(TopologyInputKind::SemanticNeighborGraph) ==
|
|
"semantic_neighbor_graph");
|
|
CHECK(topologyInputKindLabel(TopologyInputKind::EmbeddingNeighborhood) ==
|
|
"embedding_neighborhood");
|
|
CHECK(topologyInputKindLabel(TopologyInputKind::Hybrid) == "hybrid");
|
|
|
|
CHECK(topologyRoleLabel(DocumentTopologyRole::Core) == "core");
|
|
CHECK(topologyRoleLabel(DocumentTopologyRole::Bridge) == "bridge");
|
|
CHECK(topologyRoleLabel(DocumentTopologyRole::Medoid) == "medoid");
|
|
CHECK(topologyRoleLabel(DocumentTopologyRole::Outlier) == "outlier");
|
|
|
|
CHECK(formatTopologyRoleCounts({{"bridge", 2}, {"core", 3}}) == "core(3), bridge(2)");
|
|
}
|
|
|
|
TEST_CASE("GraphTopologySupport aggregates cluster stats directly", "[cli][graph][topology]") {
|
|
GraphTopologySupportFixture fixture;
|
|
GraphTopologySupport support(nullptr, "", fixture.metadataRepo);
|
|
const auto snapshot = makeSnapshot();
|
|
|
|
const auto statsById = support.buildClusterStatsById(snapshot);
|
|
REQUIRE(statsById.size() == 2);
|
|
REQUIRE(statsById.contains("cluster-a"));
|
|
REQUIRE(statsById.contains("cluster-b"));
|
|
|
|
CHECK(statsById.at("cluster-a").scopedMemberCount == 2);
|
|
CHECK(statsById.at("cluster-a").roleCounts.at("core") == 1);
|
|
CHECK(statsById.at("cluster-a").roleCounts.at("bridge") == 1);
|
|
CHECK(statsById.at("cluster-b").scopedMemberCount == 1);
|
|
CHECK(statsById.at("cluster-b").roleCounts.at("outlier") == 1);
|
|
}
|
|
|
|
TEST_CASE("GraphTopologySupport scopes cluster stats and membership views to the cwd",
|
|
"[cli][graph][topology]") {
|
|
// --scope-cwd covers every path under the cwd, as `--list-type --scope-cwd` does (#280).
|
|
// It used to keep only <cwd>/src and <cwd>/include, so tests/ fell out of scope.
|
|
GraphTopologySupportFixture fixture;
|
|
REQUIRE(fixture.metadataRepo->insertDocument(makeDocument("/elsewhere/notes.md", "hash-out"))
|
|
.has_value());
|
|
GraphTopologySupport support(nullptr, "", fixture.metadataRepo);
|
|
auto snapshot = makeSnapshot();
|
|
snapshot.memberships.push_back(DocumentClusterMembership{
|
|
.documentHash = "hash-out",
|
|
.clusterId = "cluster-b",
|
|
.clusterLevel = 1,
|
|
.bridgeScore = 0.4,
|
|
.role = DocumentTopologyRole::Outlier,
|
|
});
|
|
const std::filesystem::path cwd = "/repo";
|
|
|
|
auto scopeRes = support.buildCurrentScopePathSet(cwd);
|
|
REQUIRE(scopeRes.has_value());
|
|
const auto& scopedPaths = scopeRes.value();
|
|
CHECK(scopedPaths == std::unordered_set<std::string>{"/repo/src/main.cpp",
|
|
"/repo/include/main.h",
|
|
"/repo/tests/main_test.cpp"});
|
|
|
|
const auto statsById = support.buildClusterStatsById(snapshot, scopedPaths, cwd);
|
|
REQUIRE(statsById.contains("cluster-a"));
|
|
REQUIRE(statsById.contains("cluster-b"));
|
|
CHECK(statsById.at("cluster-a").scopedMemberCount == 2);
|
|
CHECK(statsById.at("cluster-b").scopedMemberCount == 1);
|
|
|
|
const auto views =
|
|
support.buildClusterMembershipViews(snapshot, snapshot.clusters.front(), scopedPaths, cwd);
|
|
REQUIRE(views.size() == 2);
|
|
CHECK(views[0].membership != nullptr);
|
|
CHECK(views[0].resolvedPath == "/repo/src/main.cpp");
|
|
CHECK(views[0].inScope);
|
|
CHECK(views[1].membership != nullptr);
|
|
CHECK(views[1].resolvedPath == "/repo/include/main.h");
|
|
CHECK(views[1].inScope);
|
|
|
|
const auto scopedViews =
|
|
support.buildClusterMembershipViews(snapshot, snapshot.clusters[1], scopedPaths, cwd);
|
|
REQUIRE(scopedViews.size() == 2);
|
|
CHECK(scopedViews[0].resolvedPath == "/repo/tests/main_test.cpp");
|
|
CHECK(scopedViews[0].inScope);
|
|
CHECK(scopedViews[1].resolvedPath == "/elsewhere/notes.md");
|
|
CHECK_FALSE(scopedViews[1].inScope);
|
|
|
|
const auto unscopedViews = support.buildClusterMembershipViews(snapshot, snapshot.clusters[1]);
|
|
REQUIRE(unscopedViews.size() == 2);
|
|
CHECK(unscopedViews.front().resolvedPath == "/repo/tests/main_test.cpp");
|
|
CHECK(unscopedViews.front().inScope);
|
|
}
|
|
|
|
TEST_CASE("Graph --scope-cwd path set covers a corpus without src/ or include/",
|
|
"[cli][graph][topology][scope]") {
|
|
// A corpus laid out as <lib>/include and <lib>/src (or not at all) had an empty scope, so
|
|
// `yams graph --topology-clusters --scope-cwd` reported no scoped members.
|
|
GraphTopologySupportFixture fixture;
|
|
const auto base = std::filesystem::path(makeTempDbPath()).replace_extension(".scope");
|
|
std::filesystem::create_directories(base);
|
|
const auto realRoot = std::filesystem::weakly_canonical(base) / "corpus";
|
|
const auto header = realRoot / "simeon" / "include" / "simeon" / "bm25.hpp";
|
|
const auto source = realRoot / "simeon" / "src" / "bm25.cpp";
|
|
const auto sibling = std::filesystem::weakly_canonical(base) / "corpus-other" / "x.md";
|
|
for (const auto& path : {header, source, sibling}) {
|
|
std::filesystem::create_directories(path.parent_path());
|
|
std::ofstream(path) << "x\n";
|
|
}
|
|
REQUIRE(fixture.metadataRepo->insertDocument(makeDocument(header.generic_string(), "h-hdr"))
|
|
.has_value());
|
|
REQUIRE(fixture.metadataRepo->insertDocument(makeDocument(source.generic_string(), "h-src"))
|
|
.has_value());
|
|
REQUIRE(fixture.metadataRepo->insertDocument(makeDocument(sibling.generic_string(), "h-sib"))
|
|
.has_value());
|
|
|
|
const std::unordered_set<std::string> expected{header.generic_string(),
|
|
source.generic_string()};
|
|
auto checkScope = [&](const std::filesystem::path& cwd) {
|
|
auto scoped = buildGraphScopedPathSet(cwd, fixture.metadataRepo);
|
|
REQUIRE(scoped.has_value());
|
|
CHECK(scoped.value() == expected);
|
|
};
|
|
|
|
checkScope(realRoot);
|
|
|
|
// A cwd reached through a symlink still matches the resolved stored paths.
|
|
const auto linkRoot = base / "corpus-link";
|
|
std::error_code linkEc;
|
|
std::filesystem::create_directory_symlink(realRoot, linkRoot, linkEc);
|
|
if (!linkEc) {
|
|
checkScope(linkRoot);
|
|
}
|
|
|
|
std::error_code ec;
|
|
std::filesystem::remove_all(base, ec);
|
|
}
|
|
|
|
TEST_CASE("GraphTopologySupport returns null snapshot and empty path without CLI context",
|
|
"[cli][graph][topology]") {
|
|
GraphTopologySupport support(nullptr, "snap-missing");
|
|
|
|
const auto snapshot = support.loadTopologySnapshot();
|
|
REQUIRE(snapshot.has_value());
|
|
CHECK_FALSE(snapshot.value().has_value());
|
|
CHECK(support.resolveDocumentPathByHash("missing-hash").empty());
|
|
}
|