Startup ordering defects were only observable by waiting minutes on a live
daemon, and the one that aborted it took a live restart loop to find. Give the
startup contracts a deterministic harness that runs in milliseconds.
+ startup_fuzz_support.h drives the real ServiceManagerFsm and the real
DeferralGate with virtual time: legal phase advances, injected failure,
shutdown and terminal reset, idle/busy, pressure/calm, starvation deadlines
and cancellation
+ oracles fail closed on shutdown resurrection without reset, failure-to-ready,
eligibility disagreeing with fsm.isReady(), admission outside the virtual
deadline, and ineligible work touching the admission gate; fuzzed IEEE-754
components must never yield non-finite or out-of-range scores
+ catch2_startup_replay runs named regressions plus 1024 seeded traces per run
and reports the root seed and action offset for replay
+ fuzz_startup builds the same harness for libFuzzer/AFL, with a seed generator
that refuses to overwrite different existing files
- the fuzzing lane no longer defaults to TSan: harnesses link
-fsanitize=fuzzer,address,undefined and the two runtimes cannot share a
process; combining them now fails with an explanation instead of producing a
binary that cannot start
- documented the lane invocation, target naming and the coverage limits,
including that governor call ordering inside the poller is not covered
Scope: contract-valid single-owner schedules. Executor interleavings, late
callbacks, concurrent FSM instances and filesystem/model faults are not modeled.
Signed-off-by: trvon <git@trevon.dev>
Semantic-neighbor maintenance aborted the daemon during startup:
std::clamp<float> asserted '!(__hi < __lo)' because the adaptive cosine
threshold could exceed 1.0 for a legal pair of identical vectors. The same
work was also admitted while the daemon was still initializing, so the
300-second starvation guard forced a pass before readiness.
+ semanticInverseNorm/semanticCosineSimilarity/semanticExplicitThreshold in the
existing embed policy header, shared by the streamed and corpus branches;
non-finite vectors or scores produce no neighbor and legal scores are bounded
before ranking and threshold selection
+ startupMaintenanceEligible() requires ServiceManagerState::Ready, not
stopping and idle, and is evaluated before
ResourceGovernor::admitDeferrable(), so an elapsed deferral deadline can no
longer manufacture missing readiness
+ StartupPhaseTracker (two atomics, no mutex) plus a warn-level init-waiter
line reporting state, phase and phase_elapsed_ms instead of a debug-level
state number; a 53-minute startup previously logged nothing actionable
+ regression through the public backfill and corpus-rebuild APIs using a 64-D
vector whose old float inverse norm rounds its self-cosine above 1
- removed the duplicated inverse-norm and cosine lambdas from EmbeddingService
- renamed __test_setModelProvider/__test_setKgStore to testing_* because they
used reserved identifiers, across 11 files
- re-anchored 13 pre-existing raw-environment allowlist entries to their new
line numbers; no new exception was added
Startup is still blocked by corpus-sized memory-sync scans; this change stops
the abort and names the blocking phase, it does not make startup fast.
Signed-off-by: trvon <git@trevon.dev>
Remove the bare Linux libtsan lookup that mixes GCC's shared runtime with
Clang's instrumented host. Add an ELF linkage test that requires actual
TSan instrumentation and rejects the other compiler's runtime.
Include cstring explicitly for retrieval-session memory operations.
The regression was exercised against the actual instrumented shared object;
no sanitizer instrumentation, interceptors, or checks are disabled.
Signed-off-by: trvon <git@trevon.dev>
* test(search,daemon,storage): assert behaviour instead of frozen literals
Several suites encoded exact internal values that legitimate tuning and
ranking changes are allowed to move: SCIENTIFIC-era profile weights and
rrfK values, corpus-statistic figures, and hardcoded timeout/cancellation
numbers. They failed on every platform after those changes landed, so the
real behaviour regressions they should catch were hidden behind noise.
Replace the frozen literals with contract assertions: weights are finite,
non-negative and normalize, state selection still holds, capability-gated
sources stay zeroed, and hybrid results keep matching canonical hash/path
sets. Deadlines are not widened and the daemon paths they cover are
unchanged.
Also add independent corpus and adapter oracles, deterministic collector
cancellation and late-result gates, and synchronize ServiceManager
shutdown observation, so the tests fail for the behaviour they name.
The portability allowlist keeps its two reviewed raw test-env reads; only
their line numbers moved.
Signed-off-by: trvon <git@trevon.dev>
* ci(tests): summarise Meson failures without flooding the log
A failing test run printed the whole Meson suite output, so the actual
failing test names and their assertion messages were buried and the
uploaded logs were dominated by passing-test chatter.
Add a bounded reporter that extracts the failing test names plus their
surrounding diagnostics, keeps an overall cap on emitted lines, and is
unit-tested against representative Meson output. Wire it into the Tests
workflow so failures stay readable.
Signed-off-by: trvon <git@trevon.dev>
* build(deb): validate candidate identity, mapping and upgrade ordering
The Debian packaging scripts took filenames and versions on trust, so a
misnamed candidate or a wrong package ordering could pass validation while
the real install path was never exercised.
Fail closed on candidate identity, version and architecture, verify the
package-to-source mapping, seed the legacy-only layout, wait on the actual
services, and assert install/upgrade/remove ordering for both Debian and
RPM in an offline container. Cases that cannot be proven on a host are
reported as skipped rather than silently passing.
Signed-off-by: trvon <git@trevon.dev>
---------
Signed-off-by: trvon <git@trevon.dev>
* fix(search): make the final lexical tie-break a strict weak ordering
The final comparator broke near-ties toward the result with less vector
support, and its pairwise |a - b| <= epsilon test was not transitive, which
is undefined behavior for the std::sort/partial_sort callers in
applySearchTopKLimit.
Compare tie buckets floor(score / epsilon) instead, then lexical anchor,
keyword and vector evidence (more first), then doc id. Behavior with the
tie-break off is unchanged. The comparator moves to
final_result_order_internal.h so it can be tested directly.
Signed-off-by: trvon <me@trevon.dev>
(cherry picked from commit ca9b82d6ce9db31b6c9608a7d95b63203649e5a7)
Signed-off-by: trvon <git@trevon.dev>
* fix(search): keep cross-reranked scores above the unreranked tail
applyCrossRerank rewrote the window's scores to a 0..1 min-max blend while
the tail kept raw fused scores. Later whole-list sorts (concept boost,
top-k partial_sort) compared the two scales, so reranked docs could drop
below tail docs or out of the top-k.
Map the blended score back onto the window's original range, with the
floor raised just above the best tail score when needed. The window's
reranked order is unchanged and docTraces.finalScore records the mapped
score.
Signed-off-by: trvon <me@trevon.dev>
(cherry picked from commit d66c6d304ca4d32b943e164b0d7720fc7f865db1)
Signed-off-by: trvon <git@trevon.dev>
* fix(search): judge weak tier-1 queries on the raw bm25 scale
The weak-query test compared the top Text/SimeonText component score with
weakQueryMinTopTextScore, but lexical batches are min-max normalized, so the
top hit is always 1.0. With enough hits every query was strong: the
weak-query vector fanout boost and weak-query topology expansion never ran.
scoreLexicalBatch now also records absoluteLexicalScore, the hit's score on
the existing absolute scale (-bm25 / bm25NormDivisor, same multipliers),
and classifyTier1TextSignal uses it. Thresholds keep their values; with the
default divisor 25, 0.30 is bm25 <= -7.5 and 0.45 is bm25 <= -11.25.
Signed-off-by: trvon <me@trevon.dev>
(cherry picked from commit bc771c5aab1721170fb9e743013ca0efbf048d61)
Signed-off-by: trvon <git@trevon.dev>
* fix(search): restore measured scientific fusion
Give SCIENTIFIC the measured relative Simeon weight of 0.40 after normalizing the existing text/vector pair. The resulting share is 0.285714, matching the benchmark traces. Other profiles remain unchanged.
Keep its one-slot semantic rescue preset instead of raising it to three at the corpus layer. Preserve the existing large-prose recall widening. Cover the preset weights, config propagation, community blend, rich-graph ratios and corpus rescue behavior.
Signed-off-by: trvon <git@trevon.dev>
* fix(search): order non-finite ranking evidence
Treat NaN score/evidence keys as negative infinity and invalid tie epsilon as zero. This keeps comparator equivalence transitive without changing finite-input ordering. Add exhaustive triples covering NaN, infinities and invalid epsilon.
Signed-off-by: trvon <git@trevon.dev>
* docs(search): clarify lexical scoring limits
Distinguish the shared single-hit score conversion from FTS5 BM25 calibration. Note that lexical tie buckets can still reorder numerically separated rerank-window and tail scores. No behavior change.
Signed-off-by: trvon <git@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
Signed-off-by: trvon <git@trevon.dev>
Let RouteOnly and Augment expand every routed hit along its cluster's
walk order. The snapshot cache loader resolves each cluster's
`orderedMembers` to member hashes (`buildTopologyOrderedMemberHashes`,
`TopologyRoutingSnapshot::orderedMemberHashes`); a cluster with
`orderKind` None or a permutation that fails `validateMemberPermutation`
resolves to nothing. The store's `orderUnavailableReason` is carried on
the session result and emitted as `topology_order_unavailable_reason`
only when set.
When `roadWalkWindow` > 0 the session builds a `TopologyRoadWalkIndex`
after the road beam: the selected routes first, then the road-expanded
clusters, keeping only clusters with an order and counting the rest in
`roadWalkClustersUnordered`. The index borrows its member views from the
snapshot it keeps alive; the certificate is never touched.
The vector stage walks after the certified and road groups. Every routed
hit so far (certified route hits and road-beam hits) anchors +-window
positions along each order it belongs to, bounded to that order
(`walkTopologyRoadOrders`). Anchoring on route hits too means the walk
rescues the ordinal neighbours of the semantically closest members even
with `road_hops` = 0; the plan's "around road hits" reads as every hit a
road-era routed query produced. The walked set, deduplicated against the
hits, is one extra routed query whose quota is its own size (ranking
ordinal neighbours by similarity and truncating would reintroduce the
filter the walk bypasses); its rows and distance evaluations land in the
routed diagnostics like road candidates. Shadow and Narrow never walk.
Config: `search.topology.road_walk_window` (default 0, off) on
`SearchEngineConfig::topologyRoadWalkWindow` (copied by
`applyTopologyPolicyFrom`), resolved through
`ConfigResolver::TopologyRoutingPolicy::roadWalkWindow` and applied by
SearchEngineManager. `makeTopologyRoutingOptions` requests the window
only for Augment and a gate-open RouteOnly, like the beam. The window is
excluded from `topologyRoutingPolicyFingerprint` (comment and test).
Defaults leave every stage byte-identical: the PR 2 goldens (692), the
RouteOnly cells (417) and the road beam cells (904) are unchanged.
Debug keys appear only when the window > 0:
`topology_road_walk_window_applied` (0 when nothing was walkable),
`topology_road_walk_anchors`, `topology_road_walk_candidates_added`,
`topology_road_walk_clusters_unordered`.
Bench: YAMS_BENCH_TOPOLOGY_ROAD_WALK_WINDOW writes
`search.topology.road_walk_window`; YAMS_BENCH_TOPOLOGY_ROAD_ORDER and
YAMS_BENCH_TOPOLOGY_ROAD_MAX_PER_CLUSTER write `topology.road_order` and
`topology.road_max_per_cluster` (neither had a bench mapping, so Eval 3
can set the order kind per arm and build roads at all).
Tests: tests/unit/search/search_vector_stage_road_walk_catch2_test.cpp
(new catch2_search_vector_stage_road_walk): two reciprocal chains A
(six members) and B (four) with a road, built with
`TopologyBuildConfig::roadOrder` Time from eventTime so the walk order
differs from the similarity ranking; loader resolution including the
invalid-permutation drop and the cache carry; fingerprint equality for
any window and the option gating per policy; session-level certificate,
beam and work invariance with the index listing routes then road
clusters, walk sets for windows 0/1/2/100 bounded to the cluster, an
unordered cluster counted and skipped, no index without any order, the
order diagnostic through recordTopologyRoutingDebug; RouteOnly window 0
equal to the road beam cell with no walk key, window 1 adding the
immediate order neighbours of all four hits, window 2 the whole
clusters, rows visited growing while router work is unchanged,
certified keys unchanged, unordered B contributing nothing, no order
anywhere equal to the beam cell plus counters, hops 0 walking the
selected route alone; Augment adding the walked documents as rescues;
Shadow and Narrow emitting nothing. ConfigResolver covers the TOML key.
The line-keyed portability allowlist is re-keyed for the two shifted
files (search_engine.cpp, retrieval_quality_bench.cpp); no new raw
environment reads, the bench reads its keys through
yams::config::getenv_optional.
Signed-off-by: trvon <me@trevon.dev>
Give every cluster with at least two members a walk order:
`ClusterArtifact::orderedMembers`, a uint32 permutation over the
hash-sorted `memberDocumentHashes`, plus `orderKind`. Three kinds, each
deterministic with ties resolved to the smaller member hash:
- Time: ascending `TopologyDocumentInput::eventTime`; members without a
time last, by hash.
- Session: grouped by `sessionId`; groups ordered by earliest event time
(groups with no time after those with one), then session id; inside a
group by time then hash; unsessioned members last, by time then hash.
- SimilarityChain: greedy nearest-neighbor tour by cosine over the
members' embeddings, starting from the cluster representative
(`medoid`, else the smallest-hash usable member); members without a
usable embedding last, by hash.
The knob is `TopologyBuildConfig::roadOrder`, resolved from TOML
`topology.road_order` = none|time|session|similarity_chain through the
topology.* resolver block and `TopologyManager::setRoadOrder`, threaded
into the build config like `roadMaxPerCluster`. Orders are computed after
boundary spill so they cover replicas, at full build in every engine and
for the rebuilt clusters of a connected-engine incremental update;
untouched clusters keep theirs (a kind change reaches them on the next
full rebuild, like the representative rule) and `none` clears every
order. Orders are independent of `roadMaxPerCluster`.
Orders persist in the road sidecar, now version 3: after the roads, u32
count, then per ordered cluster u32 cluster-id table index, u32 kind,
u32 length and the permutation as u32 indices. A sidecar without orders
is still written as v2, so roads-only sidecars are byte-identical to the
previous layout; v1 and v2 decode with no orders, and the envelope gains
`member_order_count`. The sidecar is written when the batch has roads or
at least one ordered cluster. On load each order is validated on its own
(`attachTopologyMemberOrders`: cluster exists, kind not none,
permutation covers the members exactly once); an invalid entry drops
only that cluster's order and appends `<token>:<clusterId>` to the new
load-time diagnostic `TopologyArtifactBatch::orderUnavailableReason`.
Roads never lose anything to an order error and vice versa.
The main snapshot codec, the routing construction, representation and
road fingerprints never see orders, so the default `none` leaves the
snapshot payload, the sidecar and every fingerprint byte-identical. No
search consumer yet; the road walk lands separately.
Tests: tests/unit/topology/topology_ordered_members_catch2_test.cpp
(permutation validator tokens, time order with missing times, session
grouping and group tie-break, hand-computed greedy chain from the medoid
on a five-point fixture with fallback start, unusable members and exact
ties, every engine stores valid permutations of every kind including over
spill replicas, default none byte-identical payload/sidecar/fingerprints,
sidecar v3 round trip plus hand-built v2 bytes and corrupt-section
rejection, per-cluster attach validation with reasons, store round trip
with orders-only sidecars, invalid order dropped with reason while roads
and other orders stay, incremental update keeps untouched orders and
recomputes rebuilt ones), plus resolver and manager setter coverage.
Signed-off-by: trvon <me@trevon.dev>
* feat(topology): typed event time and session for topology inputs
Add `eventTime` (epoch seconds, UTC) and `sessionId` to
`TopologyDocumentInput`. The extractor fills them from two documented
metadata keys: `event_time` (integer epoch seconds, or ISO-8601
`YYYY-MM-DD` / `YYYY-MM-DD[T ]HH:MM[:SS[.frac]][Z|±HH:MM]`, naive
datetimes are UTC) takes precedence over the document createdTime; an
unparsable value is skipped with a debug log, never an error, and a
zero createdTime leaves the field unset. `session_id` is copied
verbatim and empty when absent. The keys are only consulted when the
extraction loads metadata; otherwise createdTime is the only source.
`parseTopologyEventTime` is public for tests and future callers; the
calendar math is Hinnant's days_from_civil with range-checked fields.
Extraction stats count the event time source and invalid values.
Nothing downstream reads the fields yet. Artifacts, codecs and both
routing fingerprints are unchanged, which the new test pins.
Tests (tests/unit/topology/topology_input_event_time_catch2_test.cpp):
parser accept/reject tables, metadata precedence over createdTime,
invalid ignored, session_id, includeMetadata=false fallback, and
byte-identical snapshot payload plus equal construction and road
fingerprints with and without the typed fields.
Signed-off-by: trvon <me@trevon.dev>
* bench(longmemeval): event_time and session_id metadata, round docs, haystack sidecar
Every exported corpus document now carries `metadata.event_time`
(ISO-8601, normalized from the LongMemEval `YYYY/MM/DD (Day) HH:MM`
haystack date, first-seen date per session) and `metadata.session_id`.
Document ids, titles, texts, queries and qrels are unchanged in the
default mode.
`--round-docs` adds one document per session round (`<sid>__r<n>`): a
user turn opens a round and the replies that follow join it; a leading
assistant turn is its own round. Round docs share the session's
`event_time`/`session_id` and add `round_index`; qrels stay at the
session level.
`--full-haystack` also writes `haystacks.jsonl`, one line per question
with `haystack_session_ids` and the normalized `haystack_dates`, so
per-question membership survives the global corpus deduplication
(the 2026-10-10 preflight lost it).
The converter is split into `convert` / `write_outputs`; `--self-test`
runs both on a three-session synthetic fixture without a download and
is the regression check for the default output.
Signed-off-by: trvon <me@trevon.dev>
* bench(retrieval): carry BEIR corpus metadata onto ingested documents
`retrieval_quality_bench` ingests the BEIR corpus as a directory, which
cannot carry per-file metadata, so `event_time`, `session_id` and
`round_index` from corpus.jsonl never reached the topology extractor.
`BEIRDocument::metadata` now holds the corpus line's `metadata` object
(strings as is, numbers and booleans stringified; both the generic and
the SciFact loader read it), and `applyBenchmarkDocumentMetadata`
copies it onto the stored documents matched by exact file path through
the metadata repository (`ON CONFLICT DO UPDATE`, so warm caches can
repeat it). The bench runs it after the ingest wait, for cold and
reused corpora alike, and logs how many documents were updated or not
found.
Tests: loader parses and coerces metadata; apply helper writes the
keys, skips documents without metadata, counts missing ones and is
idempotent against a real MetadataRepository.
The portability allowlist entries for the two touched bench files are
re-pointed at their shifted lines; no raw environment read was added.
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
Let the vector stage follow cluster roads from the selected route
clusters. `TopologyRoutingSnapshot::roadAdjacency` is built by the
snapshot cache loader (`buildTopologyRoadAdjacency`) as the union of
both stored orientations of every road, per source sorted by weight
desc then target cluster id asc; it is empty when the batch carries no
roads, whether never built or dropped with `roadUnavailableReason`.
The session runs a beam after the certificate is materialized: for
`roadHops` hops it scores every unvisited road neighbour of the
frontier by road weight times the router's query-centroid similarity
((cosine + 1) / 2), keeps `roadBeamWidth` per hop (score desc, cluster
id asc) and dedupes across hops. Each centroid evaluation is cached per
cluster and counted once in `routeRepresentativeDistanceEvaluations`
and `routeExactRepresentativeDistanceEvaluations`, so routed work stays
comparable with the no-road arm. The result lands only in
`TopologyRoutingSessionResult::roadExpandedClusterIds` and
`roadExpandedDocumentHashGroups` (one member group per cluster,
excluding certificate members). `certificate.allowedDocumentHashes`,
admission and `topologyRoutingPolicyFingerprint` are untouched; the
fingerprint comment and a test pin that the road knobs stay out of it.
`VectorStageRequest` carries the road groups with their own quotas
(`allocateBalancedRouteQuotas` over the road groups) and cap. RouteOnly
and Augment query them after the certified groups; the certified part
of the routed result is identical with roads on or off. Shadow and
Narrow never see road groups, and `makeTopologyRoutingOptions` requests
hops only for Augment and a gate-open RouteOnly, so a gated-off
RouteOnly running as Shadow takes no roads with it.
Config: `search.topology.road_hops` (default 0, off) and
`search.topology.road_beam_width` (default 4, read when hops > 0) on
`SearchEngineConfig` (`topologyRoadHops`, `topologyRoadBeamWidth`,
copied by `applyTopologyPolicyFrom`), resolved through
`ConfigResolver::TopologyRoutingPolicy` and applied by
SearchEngineManager. Defaults leave every stage byte-identical: the PR 2
goldens (692) and the RouteOnly cells (417) are unchanged.
Debug keys appear only when hops > 0: `topology_road_hops_applied`,
`topology_road_clusters_added`, `topology_road_candidates_added` (road
documents the stage appended), `topology_road_work_centroid_evaluations`,
`topology_road_skip_reason` (roads_absent, no_selected_routes,
route_low_confidence) and, under stage trace,
`topology_road_expanded_cluster_ids`.
Bench: YAMS_BENCH_TOPOLOGY_ROAD_HOPS / YAMS_BENCH_TOPOLOGY_ROAD_BEAM_WIDTH
write the two TOML keys.
Tests: tests/unit/search/search_vector_stage_road_beam_catch2_test.cpp
(new catch2_search_vector_stage_road_beam): a four-cluster fixture with
hand-built roads A-B 2.0, A-D 1.0, B-C 1.5 where the query routes to A;
adjacency construction and cache carry; fingerprint equality for hops
0 vs 2 and any beam width; session-level certificate invariance with
hops 0/1/2, beam width 2, early stop and roads absent; RouteOnly hops 0
equals the route-only cell with no road keys, hops 1 gives A u B, hops 2
with beam width 1 gives A u B u C via the heavier road, work counters
grow with hops; Augment hops 1 adds B's documents as rescues; Shadow and
Narrow take no road groups. ConfigResolver covers both TOML keys.
The line-keyed portability allowlist is re-keyed for the two shifted
files (search_engine.cpp, retrieval_quality_bench.cpp); no new raw
environment reads are added, the bench reads its two keys through
yams::config::getenv_optional.
Signed-off-by: trvon <me@trevon.dev>
Maintain cluster roads across incremental rebuilds instead of clearing
them. `ConnectedComponentTopologyEngine::updateArtifacts` now keeps the
roads between clusters the update did not touch, drops every road with
an endpoint in a removed or rebuilt cluster (reused ids included), and
recomputes the rebuilt clusters' roads from the dirty region's raw
neighbor lists against the merged memberships, so a region document's
neighbor in an untouched cluster yields that road in both orientations.
Each source then re-ranks its top-K over the union with PR 6's order and
tie-break (weight desc, target id asc).
Only the region's neighbor lists are available during an update, so a
cross-boundary edge is observed from one side; the out-of-region target
is credited with the mirror observation its own list would have added.
That is exact on a symmetric neighbor graph and the stated approximation
otherwise. An untouched source can only re-rank the roads its last full
build kept, so the result is derived, not full.
Provenance on the batch: `roadsIncrementalPasses` counts passes since
the last full accumulation (0 = full roads) and `roadsFullBuildEpoch`
is the `topologyEpoch` of the snapshot that accumulation produced. A
full build leaves both at 0; an update sets the epoch from `existing`
when its roads were full and carries it forward otherwise. The fields
ride in the road sidecar header, now version 2 (u64 epoch, u32 passes
after the snapshot id); version 1 sidecars decode with both at 0, the
envelope format string is unchanged, and the main v1 payload and the
routing construction fingerprint do not see them. The store copies them
both ways. Roads off (`road_max_per_cluster = 0`, the default) still
yields no roads and zero provenance, so default behavior is unchanged.
`topology_build_utils.h` splits `accumulateClusterRoads` into pair-weight
accumulation (with the optional observed-document set that drives
mirroring), orientation and ranking, and adds `deriveIncrementalRoads`
over those pieces; the full-build path passes no observed set and is
unchanged.
Louvain and k-means `updateArtifacts` rebuild the supplied input from
scratch, so their roads are full roads for that batch with zero passes;
the test states and checks that property for them.
Tests: tests/unit/topology/topology_roads_incremental_catch2_test.cpp
(roads off through an update, full-build provenance, kept/retired/
recomputed roads with mirrored weights on a four-cluster graph, top-K
re-ranking over kept and recomputed roads, derivation without prior
roads, a property test over seeded planted-cluster symmetric graphs
with cross-edge rewiring and member moves comparing incremental roads to
a from-scratch build after each of several dirty regions, a tight-top-K
property for rebuilt sources and derived weights, and the alternate
engines), plus sidecar v2 round trip, hand-built v1 decode, future
version rejection and store round trip of the provenance in
topology_road_sidecar_catch2_test.cpp.
Signed-off-by: trvon <me@trevon.dev>
Persist `TopologyArtifactBatch::roads` in a sidecar knowledge-graph node,
`topology:roads:<snapshotId>` of type `topology_road_snapshot`, instead
of growing the main snapshot codec. The sidecar has its own binary magic
('Y','A','M','R'), version 1, and a Zstd JSON envelope with
`format = "topology_roads_zstd_v1"`. The main snapshot stays a plain v1
payload, byte-identical with roads on or off, so memory-sync replicas on
older peers keep decoding it, and an unknown node type is simply never
read by an old store. `deserializeTopologyBatchCompressed` now rejects an
envelope whose `format` it does not know instead of treating it as legacy
raw JSON.
Header layout (little-endian): u32 magic, u32 version, u32 maxPerCluster,
u32 reserved, string snapshotId, u32 cluster-id table count + strings,
u32 road count, then per road u32 from index, u32 to index, f64 weight.
`TopologyArtifactBatch::roadMaxPerCluster` records the bound the builder
applied so the header can carry it; both engines set it.
The store writes the sidecar right after the snapshot node and before
the `topology:snapshot:latest` pointer, so the pointer remains the commit
point: an injected sidecar failure leaves the previous snapshot current.
Retention deletes each pruned snapshot's sidecar with it, and re-storing
a snapshot id without roads removes its stale sidecar. Loading is
optional and soft: a missing sidecar yields empty roads with no error; an
undecodable or invalid one (snapshot mismatch, endpoint not a cluster,
self loop, negative or non-finite weight, more roads per source than the
declared top-K) drops the roads and records
`TopologyArtifactBatch::roadUnavailableReason`, surfaced on the routing
session and as the `topology_road_unavailable_reason` debug key, while
the snapshot still loads and routes.
`topologyRoadFingerprint(batch)` gives roads their own order-independent
identity for cache provenance; the routing construction fingerprint is
unchanged and stays road-free through the persisted path.
Lean (formal/topology/Yams/Topology/Artifacts.lean): `ClusterRoad`,
`WellFormedRoads` (endpoints in clusters, no self loop, non-negative
weight, bounded per source), `constructionIdentity`, and lemmas that the
construction identity and `WellFormedBatch` are independent of roads and
that an empty road layer is well formed. Checked with lake (lean 4.31.0).
Tests: tests/unit/topology/topology_road_sidecar_catch2_test.cpp (codec
round trip, foreign/truncated payload rejection, main payload identical
on/off, validation reasons, store round trip, missing sidecar, every
corrupt-sidecar case with routing still admitted, write-before-pointer,
retention, fingerprints through the persisted path) and the extended
fingerprint invariance test in topology_roads_catch2_test.cpp.
`road_to` knowledge-graph edges are deferred until a consumer needs them.
Signed-off-by: trvon <me@trevon.dev>
Add cluster roads: undirected cluster-to-cluster links whose weight is
the sum of every raw neighbor observation between members of two
clusters. The shared helper `accumulateClusterRoads` in
topology_build_utils.h reads `TopologyDocumentInput::neighbors` before
the reciprocal/minEdgeScore filter, because the connected engine's
filtered adjacency only carries cross-cluster edges where an oversized
component was split, and k-means never consults the graph.
Each cluster keeps its top-K roads (weight desc, then target cluster id
asc) as `ClusterRoad {fromClusterId, toClusterId, weight}` on
`TopologyArtifactBatch::roads`, sorted by source cluster. Both the
connected engine and buildBatchFromAssignment (kmeans/louvain) call it;
an incremental update clears roads until the next full rebuild.
The knob is `TopologyBuildConfig::roadMaxPerCluster`, resolved from
TOML `topology.road_max_per_cluster` through the topology.* resolver
block and `TopologyManager::setRoadMaxPerCluster`. The default 0
computes no roads; the binary codec and the routing construction
fingerprint ignore the field, so existing snapshots and fingerprints
are unchanged with roads on or off.
Tests: tests/unit/topology/topology_roads_catch2_test.cpp (raw vs
filtered neighbors, top-K and tie-break, order independence, both
orientations, out-of-batch and non-finite scores, kmeans and louvain,
codec bytes and fingerprint unchanged), plus resolver and manager
setter coverage.
Signed-off-by: trvon <me@trevon.dev>
Add TopologyVectorPolicy::RouteOnly: the vector stage runs only the routed
candidate queries (DocumentTopK on Simeon PQ, ExactDocumentComplete on
ExactScan) with no global fill and no admission gate, reports the routed
diagnostics as vector search cost and applies the routed work to the
certificate. Results come solely from the routed set.
RouteOnly serves uncertified routes, so it runs only behind
search.evaluation.allow_uncertified_routes (SearchEngineConfig::
allowUncertifiedRouteOnly). Without the gate the policy downgrades to
Shadow and records topology_vector_policy_downgraded=uncertified.
Vec0 has no routed candidates: the global leg is returned unchanged and
the source reports unavailable.
Every search now records topology_route_certified (the certificate admits
a hard route after the observed work is applied) and
vector_candidate_source_effective (ann | route_only | augment | narrow |
unavailable | none); the sequential path reports ann.
Wiring: search.topology.vector_policy accepts route_only;
ConfigResolver::TopologyRoutingPolicy carries the gate;
SearchEngineManager applies both and warns when route_only is configured
without the gate; applyTopologyPolicyFrom copies the gate so corpus
tuning can never select RouteOnly or open it. Defaults are unchanged.
Bench: YAMS_BENCH_TOPOLOGY_VECTOR_POLICY=route_only writes both TOML keys.
Also fix the stale Narrow/Vec0 doc comment on the policy enum.
Tests: new catch2_search_vector_stage_route_only pins RouteOnly x
{SimeonPqAdc, ExactScan, Vec0} x gate on/off, a certified route and an
executor-backed run; the gate-off cells are compared key for key against
Shadow. ConfigResolver, SearchEngineConfig and SearchTuner tests cover
the two TOML keys and the tuner invariant. The PR 2 goldens are unchanged
(692 assertions).
Signed-off-by: trvon <me@trevon.dev>
* test(search): vector stage completes into its own state on component timeout
Executor-backed SearchEngine with a vector hydration step that blocks until the
test releases it. With a small componentTimeout the search returns without the
vector component; the test then releases the worker and runs a second search
from the same call site, so the second search's locals reuse the first frame's
stack region. A stage that still writes into the first frame races with the
second search (TSan data race) until the stage state is heap-owned by the
posted work. A second case pins that a component meeting its deadline is still
collected with its diagnostics.
Signed-off-by: trvon <me@trevon.dev>
* fix(search): vector-stage lifetime on component timeout
The posted vector work still reached into the search frame after a
componentTimeout: its injected query wrapper captured `this`, workingConfig
and four frame-local relaxed-retry atomics, and the frame adopted the stage
outcome unconditionally after collectIf, even when the component had timed out
and the worker was still running (a data race on the outcome, and a stale write
into whatever next occupies that stack region once the frame returns).
Give the wrapper a heap home: VectorStageState {SearchEngineConfig config;
relaxedRetryEnabled; RelaxedVectorRetryStats relaxedRetry; VectorStageOutcome
outcome} is captured by shared_ptr in the posted lambda and the query wrapper is
built from it by makeQueryVectorWithRelaxedRetry, which holds the repository
and vector database by value, so the work depends on neither the frame nor the
engine. The sequential path uses the same factory bound to frame locals; the
relaxed-retry atomics become a plain struct only the frame thread touches.
collectAndRecord now returns the ComponentStatus. The frame adopts the stage
state only when the component finished (Success or Failed); after TimedOut or
Canceled the late completion lands in state only the worker references and is
dropped with it, so the diagnostics merge and route-work application are
no-ops. Behavior with no timeout is unchanged.
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
Pin the current behaviour of the vector candidate stage in
search_engine.cpp before it is extracted into search_vector_stage.{h,cpp}.
A new catch2_search_vector_stage binary runs the matrix policy
{Shadow, Augment, Narrow commit, Narrow fallback} x backend
{SimeonPqAdc, ExactScan, Vec0L2} with stage trace off and on, plus one
executor-backed fanout case, and compares each cell against explicit
goldens: fused hash/rank/score and vector contribution, raw vector
ComponentResults (trace_component_hits_json), the stage's debug keys
without timings, certificate.work / admission.work via the
topology_route_work_* and admission keys, and the merged vector_search_*
diagnostics. Quirks are pinned as-is: relaxed_vector_primary_hit_count is
last-writer, Vec0 keeps global results unchanged under Shadow/Augment,
Narrow commits only on Simeon PQ because ExactScan and Vec0 never observe
an ANN candidate budget (work_observation_unavailable), and Vec0 leaks a
partial ann_candidate_budget_actual into the unobserved work record. A
hidden [.dump] case regenerates the goldens.
Move the shared topology fixtures (TopologySearchFixture,
buildTwoClusterTopologyBatch, topologyRoutingTestConfig,
runTopologySearch, configureCertifiedTopologyRoute, TopologyVectorEnvPin)
into tests/unit/search/topology_search_test_support.h so both tests use
the same corpus. runTopologySearch takes an optional executor and
configureCertifiedTopologyRoute an options struct; existing call sites are
unchanged. The portability allowlist entry for the raw YAMS_TEST_TMPDIR
read moves with the helper. No product behaviour changes.
Signed-off-by: trvon <me@trevon.dev>
SessionService reads the current session from XDG_STATE_HOME. On a build
host whose user had an auto-created session ("auto: /data") with watch
enabled, every test that booted a ServiceManager inherited it, the session
watcher walked /data during init, and six daemon tests timed out. With the
state dir isolated the same six pass.
The shared test main now points XDG_STATE_HOME at a per-process temp dir
when the caller did not set one, and removes it at exit. A small test checks
the process sees no host session.
Signed-off-by: trvon <me@trevon.dev>
* feat(profiling): Tracy zones on p2p and memory-sync background paths
Zone the memory-sync cycle (publish, sync, reconcile, apply, backfill,
delete-outbox drain), the p2p reconnect cycle, accept loop, per-session
worker and frame read, and the delta send/apply/bootstrap stages. Plot the
retained session count and pending erase intents. No behavior change; the
macros are no-ops without Tracy.
Signed-off-by: trvon <me@trevon.dev>
* test(infra): per-test-case CPU and peak RSS accounting for Catch2 binaries
A Catch2 listener in the shared test main records getrusage (GetProcessTimes
on Windows) CPU per case and prints the five heaviest cases plus the run's
peak RSS to stderr at run end. YAMS_TEST_RUSAGE=1 adds one line per case.
The measurement helpers are header-only and unit tested.
Signed-off-by: trvon <me@trevon.dev>
* fix(vector): make vector_database.cpp compile with Tracy enabled
The Profiling lane did not build: eight YAMS_PLOT sites in
vector_database.cpp used -> on a Result<T>, and two passed an int that is
ambiguous between Tracy's PlotData overloads. Use .value() and int64_t
casts. No behavior change outside TRACY_ENABLE builds.
Signed-off-by: trvon <me@trevon.dev>
* build: enable Tracy for every yams library in the Profiling lane
Define yams_tracy_args/yams_tracy_deps once in the root meson.build
(Tracy found, optional rocprofiler hooks, link check) and apply them to
every library and executable under src/, replacing the per-library copies
in search, topology, vector, daemon and tests/benchmarks. Empty when
enable-profiling is off.
Make YAMS_PLOT accept any numeric type and fix a Result deref in
search_service so the newly instrumented TUs compile with Tracy.
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
TuningManager treated any open non-health connection as activity, so a
client that kept its socket open held the daemon in busy mode and kept
background scaling up. RequestHandler now counts executing non-health
requests with an RAII guard, SocketServer wires the counter into
DaemonStats, and the idle/busy decisions read it.
Tests: request_handler_status_unary, socket_server_teardown, and two slow
real-service tests (tuning_registry_activity, socket_proxy_request_activity).
Signed-off-by: trvon <me@trevon.dev>
- Include <version> before testing __cpp_lib_* macros. Without it every
library gate read 0 in a file that included this header first (verified
with Apple Clang 21 and GCC 16 at -std=c++23).
- Detect [[likely]]/[[unlikely]] with __has_cpp_attribute. __cpp_attributes
is 200809L on every compiler, so YAMS_HAS_LIKELY_UNLIKELY was always 0.
- YAMS_HAS_REFLECTION now requires __cpp_impl_reflection and
__cpp_lib_reflection >= 202506L (P2996R13). __cpp_reflection is not a
standard macro.
- Remove YAMS_HAS_PROFILES and YAMS_PROFILE. __cpp_profiles does not
exist and C++26 did not adopt Profiles; nothing outside the test used it.
- YAMS_CPP_VERSION reads _MSVC_LANG when defined.
- get_feature_summary() reports the language mode instead of tying
"C++23" to constexpr vector/string, which are C++20 features.
- Correct the per-gate toolchain notes (flat_map: libstdc++ 15 / libc++ 20 /
MSVC 19.51; move_only_function: not in libc++; expected: Clang 18 with
libstdc++ reports 0) and drop the std::expected migration guidance:
Result<T> stays.
Tests: core_cpp23_features checks each gate against the compiler's own
feature-test macros with the header included first. New
core_cpp23_features_fallback forces every gate to 0 so the fallback paths,
including the magic_numbers runtime path, compile in every lane.
YAMS_CONSTEXPR_IF_SUPPORTED's inline fallback was invalid on a lambda; the
tests now use it on a function.
Signed-off-by: trvon <me@trevon.dev>
* test(topology): characterize degree representatives and raw BQ sign bits
Pin the current behaviour before adding opt-in representative and BQ
rotation options:
- connected, louvain and kmeans engines on a skewed cluster (dense
clique plus a weak-edge hub): medoid hash, score, extra routing
representatives and roles;
- equal-degree ties go to the smaller document hash;
- the unrotated BinaryQuantizedIndex matches a raw-coordinate sign-bit
reference, with and without a prefix dimension.
Signed-off-by: trvon <me@trevon.dev>
* feat(topology): add an opt-in medoid representative rule
topology.representative_rule = degree (default) | medoid selects how a
cluster's representative document is chosen. Degree keeps the max
within-cluster weighted degree. Medoid picks the member minimising
sum_j sqrt(max(0, 2(1 - cos))) over the other members.
- one shared helper, applyRepresentativeRule(), is called by the
connected, louvain and kmeans engines after their unchanged degree
choice; medoid falls back to it without usable geometry;
- clusters above 256 usable members are searched on a deterministic
hash-ordered stride sample (kMedoidRepresentativeMaxMembers);
- ties within 1e-9 go to the smaller document hash;
- with medoid, farthest-first extra routing representatives are seeded
from the medoid instead of the centroid;
- the rule is stored in the formerly reserved first header byte of the
binary snapshot (0 = degree, so old snapshots keep their meaning) and
extends the construction fingerprint only when it is not degree;
- search debug stats report topology_representative_rule.
Signed-off-by: trvon <me@trevon.dev>
* feat(topology): add an opt-in randomized Hadamard rotation for BQ routing
search.topology.bq_rotation = none (default) | fwht selects the rotation
applied before the centroid BQ shortlist takes sign bits. Raw-coordinate
sign bits make cos(pi * h / d) a poor estimate on anisotropic embeddings;
after x -> (1/sqrt(n)) H D [x; 0] every bit is a random hyperplane.
- RandomizedHadamardRotation in yams/vector: fixed splitmix64 sign
diagonal plus an in-place FWHT, zero-padded to a power of two; Simeon's
FWHT projection works on int32 sketches with row subsampling, so a
small local transform is used instead;
- BinaryQuantizedIndex::build() takes the rotation and seed, rotates
stored vectors before the prefix is taken, and rotates every float
query itself, so centroids and queries always share one seed
(kDefaultBinaryRotationSeed);
- the routing snapshot cache records bqRotation and bqRotationSeed and
rebuilds the route index when either differs;
- the routing policy fingerprint changes only for a non-default rotation;
- search debug stats report topology_route_bq_rotation.
Signed-off-by: trvon <me@trevon.dev>
* test(bench): map representative rule and BQ rotation arm knobs
Harness-only env knobs written into the isolated benchmark config.toml,
so arms go through ConfigResolver like the other topology settings:
- YAMS_BENCH_TOPOLOGY_REPRESENTATIVE_RULE=degree|medoid
-> [topology] representative_rule
- YAMS_BENCH_TOPOLOGY_BQ_ROTATION=none|fwht
-> [search.topology] bq_rotation, next to bq_candidate_limit and
bq_prefix_dim
The debug-file setup entry records the requested values; per-query
search_stats carry the effective topology_representative_rule and
topology_route_bq_rotation. Portability allowlist lines are shifted for
the inserted code.
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
* test(vector): pin vec0 ranking to cosine order on non-unit rows
vec0 ranks by L2 distance, which matches cosine order only for unit
vectors. Insert rows whose L2 order disagrees with cosine order and
assert vec0 returns cosine order, agrees with the exact cosine scan,
skips zero rows and rejects a zero query. Pins a vector-enabled env
so the cases run under the CI test env.
Signed-off-by: trvon <me@trevon.dev>
* fix(vector): rank vec0 by cosine on unit-normalized copies
vec0 ranks by L2 distance, which matches cosine order only when both
sides are unit length. Stored rows reach the vectors table at whatever
magnitude the provider, config, import or sync produced, and the vec0
aux table copied them as-is, so vec0 selected and ordered rows
differently from the Simeon PQ and exact cosine paths.
The vec0 table is a derived index rebuilt from vectors.embedding, so
write the unit direction into it and leave the source blob, which
export/sync and exact scoring read, untouched. Normalize the query the
same way. Zero rows are left out of vec0 and never match; a zero or
non-finite query is rejected as the exact scan rejects it. Returned
rows are ordered by their cosine score with the PQ tie-break.
No schema change. The aux table is rebuilt on first use in each
process (ready dims are in-memory), so existing non-unit rows are
normalized on the next rebuild without a repair step.
Signed-off-by: trvon <me@trevon.dev>
* test(topology): pin the k-means stored centroid to the unit direction
Spherical k-means assigns against unit centroids but stored the shorter
arithmetic mean. Assert the stored centroid is unit length along the
member mean direction, with and without boundary spill. Add a SOAR case
showing that with unit centroids the spill follows angular distance,
and that a short mean centroid reorders the candidates.
Reformat the touched k-means test file to the lint formatter.
Signed-off-by: trvon <me@trevon.dev>
* fix(topology): store the unit k-means centroid
The k-means engine clusters by cosine against normalized centroids,
then stored the unnormalized member mean as centroidEmbedding. Routing
reads the centroid by cosine and does not care, but SOAR boundary spill
measures Euclidean residuals against it. A loose cluster's short mean
adds a radial term to those residuals, so spill candidates were ranked
in a different space from the one the clusters were built and routed
in.
Store the unit mean direction for k-means, which is the converged
Lloyd centroid. Connected-component and Louvain keep the arithmetic
mean, as their tests and contract require. Correct the stale field
comment.
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
* test(wal): pin v1 entry encoding to a Clang-written fixture
store_block_entry_clang_v1.bin is one StoreBlock entry serialized by the
current wal_entry.cpp built with Clang 21, which zero-fills the 4 bytes of
trailing Header padding. The new cases decode and verify it, re-encode the
same logical entry and require identical bytes, and round-trip fresh
entries through serialize/deserialize/verifyChecksum.
GCC 16 leaves that padding uninitialized in stableHeaderBytes(), so these
cases fail there.
Signed-off-by: trvon <me@trevon.dev>
* fix(wal): encode entry header field by field, never via padding
stableHeaderBytes() built the checksummed header with `Header stable{}`
plus member assignments and then memcpy'd sizeof(Header) bytes. That is
aggregate initialization, so the 4 bytes of trailing padding (offsets
44..47) are not guaranteed to be zero. Clang 21 memsets the object first;
GCC 16 at -O0 stores only the members, so stack garbage reached both the
serialized entry and the CRC, and verifyChecksum() failed on GCC builds.
Encode and decode the header field by field into a zero-filled 48-byte
array at the existing offsets, with static_asserts pinning every offset
and size. The checksum is CRC32 over that canonical encoding plus the
payload, which is exactly what Clang builds have always written, so their
WAL files verify unchanged. deserialize() also accepts an entry whose CRC
covers nonzero padding as stored on disk (written by earlier GCC builds)
and normalizes its in-memory checksum. WALFile and WALManager recovery
read headers through the new WALEntry::decodeHeader().
TransactionData (uint64 + uint32) has 4 bytes of trailing padding and was
copied whole into the payload, relying on the same unguaranteed zeroing
(GCC 16 and Clang 21 happen to zero it today); it is now encoded field by
field too. The remaining payload structs are padding-free, which
appendObjectBytes/readObject now enforce with
has_unique_object_representations_v.
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
With the daemon shipped as a systemd user unit, the CLI must not fight
systemd for it:
- Auto-start (DaemonClient::startDaemon): when the unit is installed but
not running and would run exactly the daemon the client expects (same
socket, data dir and config as the default resolution, default
binary), start it with `systemctl --user start` instead of forking a
copy that a later login would race for the socket. Flag or
environment overrides (--socket, --data-dir, YAMS_DAEMON_SOCKET,
YAMS_DATA_DIR, YAMS_CONFIG, YAMS_DAEMON_BIN) keep the fork path.
- `yams daemon start|stop|restart` use `systemctl --user` for an
installed/active unit (start/restart without custom path options,
stop/restart without --socket), falling back to IPC if systemctl
fails.
- `yams daemon install --user` with no options enables the packaged unit
(`systemctl --user enable --now`) instead of writing a copy into
~/.config/systemd/user that would shadow it; with options it still
writes that override and says so. `uninstall --user` reloads the
manager after removing an override and explains how to mask the
globally enabled packaged unit for one account.
The decision is a pure function (userUnitMatchesPaths) with Catch2
coverage; systemctl is only run with whitelisted verbs and only when a
user manager is reachable ($XDG_RUNTIME_DIR set, Linux).
Signed-off-by: trvon <me@trevon.dev>
The packaged service becomes a systemd user unit that runs each user's
own daemon on the CLI's default per-user socket, with that user's
config and data. There is no system daemon left to discover, and the
daemon always runs as the same user as the CLI, so both workarounds for
the system-service design go:
- Revert "feat(cli): find the packaged system daemon socket without
configuration" (client-side /run/yams/yams-daemon.sock discovery, the
systemd refusal in `daemon stop|restart`, the spawn refusal and the
status mismatch suppression, plus its tests).
- Revert "fix(cli): send file content to a daemon that runs as another
user" (peer-UID probe and content upload in `yams add`).
Per-user socket resolution is back to: explicit flag/env/config, then
$XDG_RUNTIME_DIR/yams-daemon.sock (or the /tmp fallback).
Signed-off-by: trvon <me@trevon.dev>
The resolution tests used ELF names (libonnxruntime.so.1) everywhere,
but the policy only treats .dylib files as runtime libraries on macOS,
so the macOS test lane would have failed. Fixtures now use the host's
names (libonnxruntime.1.dylib / libonnxruntime.1.23.0.dylib on macOS)
and build paths with fs::path; the test is not registered on Windows,
where resolution is the DLL search for onnxruntime.dll.
grim (Linux, Debug): 7/7 cases, 34 assertions pass.
Signed-off-by: trvon <me@trevon.dev>
The ONNX plugin dlopen()ed ONNX Runtime from directories next to the
plugin first and only then from a few system prefixes, with no notion of
which copy it got or why, and the shipped packages contained no copy at
all, so on most hosts the plugin could only report "runtime not found".
The runtime resolution moves into plugins/ort_runtime/, built as a
hidden-visibility static library both ONNX-using plugins link (their
export maps still expose only the plugin C ABI). The policy is split out
as pure code (ort_runtime_resolution.{h,cpp}) so it is testable without a
runtime. Candidates are tried in this order and the first compatible one
wins:
1. YAMS_ONNX_RUNTIME_LIB / YAMS_ONNX_RUNTIME_DIR (existing pin, kept),
then the plugin config key runtime_library ([plugins.onnx] /
[plugins.glint], a file or directory)
2. a system copy: the loader search by soname, then the usual library
directories (multiarch dir, /usr/lib64, /usr/lib, /usr/local/lib,
/opt/onnxruntime, Homebrew) for versioned names such as Debian's
libonnxruntime.so.1.23
3. the private copy in <libdir>/yams/onnxruntime/
4. legacy locations next to the plugin (build trees)
A candidate is accepted only if it exports OrtGetApiBase, GetApi() returns
an API table for the ORT_API_VERSION the plugins are compiled against, and
it meets the 1.23 minimum; otherwise it is skipped with a reason. A soname
lookup that returns a copy another plugin already loaded from the bundled
directory is reported as that copy. The choice ("Using system|bundled ONNX
Runtime <ver> from <path> (<why>)") and every skipped candidate go to the
plugin log and, through the libyams_onnx_resource host sink (new
OnnxConcurrencyRegistry::emitHostLog), to the daemon log; plugin health
reports runtime_source. With no compatible copy the plugin reports "ONNX
Runtime unavailable: ..." instead of failing.
Tests: tests/unit/plugins/ort_runtime_resolution_catch2_test.cpp covers
the order, the override pin, configured directories, system-over-bundled
preference, skipping an incompatible system copy, the not-found result
and the already-loaded classification.
Signed-off-by: trvon <me@trevon.dev>
The deb, rpm and Arch packages start yams-daemon.service on
/run/yams/yams-daemon.sock, but clients only looked at the per-user
default (XDG_RUNTIME_DIR, /var/run as root, /tmp/yams-daemon-<uid>.sock).
With no YAMS_DAEMON_SOCKET set, root and normal users silently fell back
to an in-process daemon and a private store in ~/.local/share/yams, so
the installed service was never used.
Client-side resolution now goes: an explicit socket (flag, env, config),
the per-user daemon when its socket exists or a personal data directory
was named (flag, env, core.data_dir from yams init), then the system
socket when it exists and is connectable (root or the yams group), and
finally the per-user default. Daemons keep binding what
resolve_runtime_paths() names; `yams daemon install --user` writes the
per-user socket, never the discovered one.
Because systemd owns the system unit, `yams daemon stop|restart` refuse
it with a systemctl hint and the client never tries to spawn a daemon on
that socket. `daemon status` no longer flags the system corpus as a data
directory mismatch. socket_utils also gains a peer-UID probe used by the
next change.
The two socket default tests now use the per-user resolver so they do
not depend on whether the build host runs the packaged service.
Signed-off-by: trvon <me@trevon.dev>
meson_options.txt had 47 options. Several were read but never set by
any workflow, script or doc, several only fed test-lane timeouts, and
enable-{tsan,asan,ubsan} duplicated b_sanitize. 47 -> 33.
Removed (behavior kept at the previous default everywhere):
- require-fts5 (configure now always errors on a SQLite without FTS5,
the old default), conan-generators-dir, cli11-cmake-path,
onnxruntime-cmake-path (Conan/CMake discovery under the build dir is
unchanged), split-debug.
- unit-timeout, integration-timeout, integration-ui-timeout,
integration-smoke-timeout, integration-daemon-timeout and stress-iters:
replaced by the same constants at their call sites (smoke keeps its
test-timeout-scale multiplier). Only meson/profiles/ci-fast.ini and
nightly-soak.ini set them, and nothing used those profiles; both go.
- enable-integration-services/-smoke and enable-daemon-integration:
folded into enable-integration-tests. Debug builds still register every
integration suite; nothing set the sub-toggles.
Kept, because third_party/symspell's meson.build reads them:
bench-timeout, enable-tsan and enable-asan. YAMS itself now derives all
sanitizer handling (extra flags, TSan runtime links, test timeouts) from
b_sanitize; setup.sh still mirrors its choice into enable-tsan/asan for
the submodule, and the CI ASan lane keeps -Denable-asan=true for the
same reason. enable-ubsan is a deprecated no-op (a warning if it
disagrees with b_sanitize) so build dirs configured with it still
regenerate; the UBSan lane passes only -Db_sanitize=undefined and still
gets -fno-sanitize-recover=all (standalone UBSan aborts; ASan+UBSan stays
recoverable, as before).
Conan: the without_* Boost list was applied only to iOS/Android. YAMS
uses Boost.Asio (header-only) and links only boost_system/boost_thread
on every platform (asio/ssl uses OpenSSL directly; no spawn/context), so
the list now applies everywhere. Debug Boost package on grim: 45 static
libraries / 229 MiB -> 10 / 6.8 MiB (Release: 45 / 30 MiB before).
Verified on grim: ./setup.sh Debug --no-tsan --with-tests reconfigures
an existing build/debug and builds; an existing build/release (configured
with -Denable-tsan/-Denable-asan) regenerates.
Signed-off-by: trvon <me@trevon.dev>
* fix(daemon): never gate memory-sync publish on inbound apply
applyMemorySyncWinners() ran the inbound stages content -> metadata ->
vector -> topology and returned on the first stage failure, before the
node's own outbound backfill. In a mesh where every node holds an inbound
record whose prerequisite only a peer can publish (an embedding whose
content blob has not replicated, an edge whose target node has not), every
node refused to publish, so the prerequisites never arrived: a stable
mutual deadlock with healthy transport and nothing converging.
The outbound publish now runs after the inbound stages whatever their
outcome; only a stop request ends a cycle early. Each inbound stage also
runs even when an earlier one failed. Cross-stage dependencies (metadata
and vectors need local content bytes, edges need their endpoint nodes) are
checked per record against the local stores, so a failed stage shows up
downstream as unapplied records rather than as skipped stages. Stages keep
their prerequisite order so a prerequisite landing in a cycle is visible to
its dependents in that same cycle.
A two-node test drives real ServiceManager apply cycles over a shared
store, each node holding an edge whose target only the other can publish;
it converges in a bounded number of rounds only with this change.
Signed-off-by: trvon <me@trevon.dev>
* fix(memory-sync): defer records whose prerequisite has not replicated
Vector, metadata, and topology apply treated a missing replicated
prerequisite as a hard error and rejected the whole batch: one embedding
whose content blob had not arrived, or one edge whose target node had not,
blocked every other winner in that stage on every cycle. These are
ordering races between independently published records, not failures.
Each adapter now defers such a record instead: it is skipped, the rest of
the batch applies, and its logical key is reported through deferredKeys().
Nothing marks it applied, so it remains a winner in the replicated index
and the next cycle retries it once the prerequisite lands; apply needs no
pending queue of its own and loses nothing across restarts. An edge whose
endpoint is being deleted in the same batch is deferred too rather than
inserted against a stale node id. Malformed records, probe failures, and
store errors still fail the stage.
Content blobs are independent, so the content stage now attempts every
blob and fails afterwards with the first genuine error; a corrupt blob no
longer starves the blobs after it. A blob whose bytes are not hydrated in
the sync cache yet is deferred like the other stages.
Signed-off-by: trvon <me@trevon.dev>
* feat(p2p): report memory-sync apply health in status
The mesh deadlock was invisible: every node reported failed_cycles=0 and a
climbing successful_cycles while nothing converged, because a cycle counts
as successful once reconciliation completes, whatever the apply stages did
afterwards. Deferred records would be just as invisible without a signal.
ServiceManager now keeps an ApplyHealth record of the apply path:
- deferred records per stage, and the age of the oldest one still pending,
with a warning once a deferral passes ten minutes (its prerequisite
likely never arrived). Ages are tracked for at most 4096 keys per stage,
keeping the oldest; counts stay exact.
- hard failures per stage, plus the last failure's stage, message, and age.
- apply_cycles and apply_failed_cycles. failed_cycles keeps meaning
reconciliation failures; a cycle whose apply stage hard-failed is counted
in apply_failed_cycles instead of passing as a plain success.
- publish_skipped_cycles (delete outbox preempted the cycle) and
publish_failed_cycles (an outbound backfill domain failed).
The fields travel through MemorySyncStatus, MemorySyncResponse, and
protobuf fields 22-37 to `yams p2p status` (new key=value tokens on the
first line, an `apply` line with per-stage counts, a last-apply-failure
line, and an `apply` object in --json) and to the memory-sync section of
`yams daemon status`. The p2p status formatter moves behind a header so it
can be tested without a daemon.
Signed-off-by: trvon <me@trevon.dev>
* fix(memory-sync): keep content deferrals when the same scan hits a failure
The content stage scans every blob and fails afterwards with the first
genuine error, but the apply cycle discarded the deferred keys that the
same scan collected whenever it failed. A corrupt blob therefore hid every
blob still waiting for its bytes: the deferred count stayed at its previous
value (0 on the first pass) and the stale-deferral warning never fired for
a prerequisite that was actually missing.
A stage pass now carries both outcomes. The content-blob scan moves into
ContentBlobSyncAdapter::applyWinners, which reports the applied count, the
deferred keys, and the first failure of one complete scan, and
ApplyHealth::recordPass records the failure and replaces the deferred set
together. A stage that returns an error still leaves the deferred set as
last observed, since an aborted scan says nothing about prerequisites.
The tracked-deferral cap already carried every still-deferred tracked key
before admitting new ones, but its carry loop had a cap check that read as
if it could stop early. Drop it (the tracked set never exceeds the cap) and
pin the behaviour with tests that place the oldest key after more than the
cap's worth of new keys and that refill a full tracked set.
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
supportedExtensions() returned its static set by value, so capabilities()
constructed the extension vector from begin() of one temporary and end()
of another. That range is undefined behavior. libstdc++ and libc++ release
iterators happened to walk the first copy to completion, but the MSVC debug
STL rejects it in _Verify_range with a modal "Microsoft Visual C++ Runtime
Library" assertion dialog. HandlerRegistry::selectBest calls capabilities()
for every candidate, so every lines: search and every applyExtractScope call
hung forever on Windows Debug builds.
Return the shared set by const reference and take begin()/end() from that
one instance. New text_basic_handler unit tests pin the shared instance,
the capability list, registry selection and line-range extraction; the old
code also fails them under -D_GLIBCXX_DEBUG with "function requires a valid
iterator range".
Signed-off-by: trvon <me@trevon.dev>
The governor only measured the daemon itself (RSS, own CPU), so it could
not tell that the machine was busy. On a loaded host the daemon still
started background work at full width, and that work ran far less
efficiently: ingesting the same 1899-file corpus cost 41 daemon CPU
seconds on an idle 48-core host and 96 CPU seconds with the host 2x
oversubscribed (stress-ng), and took 4x longer to finish.
Sample a cheap host-wide signal on the existing governor tick, at most
once per second: Linux PSI "some avg10" for cpu, io and memory, falling
back to the one-minute load average per online CPU; on macOS the load
average plus the kernel VM pressure level. A busy host is a flag with a
release band (signals must drop to 75% of the threshold to clear) so
work does not flap around a threshold.
Host load stays a separate dimension from the governor's pressure
level. Folding it into the level would reuse the Warning response,
which blocks model loads and trims foreground concurrency; a busy build
on the same machine should not stop a search from loading its model.
Background scheduling consumes the flag in a follow-up change.
Thresholds are typed config under [tuning.resource]: host_pressure,
host_cpu_pressure_pct (40), host_io_pressure_pct (30),
host_memory_pressure_pct (10) and host_load_per_cpu (1.5). They show in
the runtime tuning status with their provenance.
Signed-off-by: trvon <me@trevon.dev>
YamsDaemon::stop() stopped the SocketServer and destroyed it while the
IOCoordinator threads were still running, and only then stopped and
destroyed the IOCoordinator. SocketServer::stop() waits at most ~2s for
connections to drain, and the handle_connection coroutine frames keep a
raw pointer to the server in their cleanup guard. A connection that
outlived the drain window either resumed on an I/O thread after the
server was freed, or had its frame destroyed together with the
io_context, and both paths decremented the connection counters of the
freed SocketServer (ASAN heap-use-after-free in CleanupGuard). The
ServiceManager-init failure path stopped the server the same way and
left the I/O threads running.
Tear the IPC layer down in one place, teardownSocketServer(): stop the
server (which still releases its acceptors and accept strand while the
io_context runs), stop and join the I/O threads so no handler can
resume, destroy the IOCoordinator so the io_context destroys the pending
handler frames while the server is valid, and only then destroy the
server. All daemon stop and startup-failure paths use it.
The unit test stops the io_context under the connection handlers so
they are deterministically pending when the teardown runs; the daemon
integration test parks the I/O threads past the drain window.
Signed-off-by: trvon <me@trevon.dev>
With 5 x 10 MB rotation the daemon log covered about three hours on the
audited machine, mostly one of a few lines repeated:
- "Failed to submit extraction ...: entity_graph_jobs channel full"
(11k lines) from GraphComponent, echoed per batch by PostIngestQueue's
"KG batch failed", plus "Entity channel full, dropping job";
- the per-document "Title+NL extraction for <hash> in N ms" info line;
- session-watcher scan/indexing warnings retried every pass.
Add yams::common::LogRateLimiter, a small lock-free helper that admits
one message per interval and reports how many it held back ("(N similar
messages suppressed)"), and use it at those sites (60 s for the queue
paths, 10 min for the session watcher, including its first-failure
warning per path). The Title+NL timing line is per-document progress
and moves to debug. The drop counters are unchanged, so status still
shows the full numbers.
Signed-off-by: trvon <me@trevon.dev>
vectors.db had no reclaim path at all. On a real install it had grown to
39.7 GB with 48% of its pages on the freelist (about 19 GB of dead
space), while yams.db already had a startup auto-VACUUM with a size,
free-page-ratio and free-disk check.
Move that decision into a shared sqlite_vacuum helper so both databases
follow the same rules, and let RepairService's background loop check
vectors.db once a day (15 minutes after start, typed config
[daemon.maintenance] vector_vacuum_interval_hours, 0 disables).
VACUUM holds the vectors.db write lock for the whole rewrite, so it only
starts while the daemon is idle: no client connections, resource
pressure below Warning, no queued or in-flight embedding writes, no
VectorIndexCoordinator rebuild or bulk load, and no on-demand repair.
It runs on its own connection with file-backed temp storage and a short
busy timeout, so a live writer makes it back off instead of waiting, and
a progress handler rolls it back as soon as any of those signals change
or shutdown starts. Readers on the owner's connection are unaffected in
WAL mode. Besides the data-dir free-space rule it also requires the temp
directory to hold a copy of the live pages.
Tests cover the thresholds (including the audited 39.7 GB / 48% case),
a real vacuum that shrinks a file, abort and live-writer back-off, the
admission truth table, and RepairService vacuuming a live VectorDatabase
file only when idle, which stays writable afterwards.
Signed-off-by: trvon <me@trevon.dev>
Since v0.7.3 'yams get' has requested compressed transfers
(acceptCompressed), and the daemon has answered with a framed payload: a
CompressionHeader followed by a Zstandard body. Only the MCP server
decoded it; 'yams get' wrote the frame itself to stdout or -o, so users
saw the 'CNRK' header and a zstd frame instead of the document. 'yams cat'
was unaffected because it does not request compression.
- Move the frame format into compression/framed_payload
(encodeFramedPayload/decodeFramedPayload). DocumentService encodes with
it; 'yams get' and MCP decode with it.
- The decoder checks the header, body size and CRC, and the decoded size
and CRC. A payload that fails is an error; MCP used to fall back to
returning the framed bytes as content.
Verified end to end: 'yams get' output (stdout and -o) is byte-identical
to the added file while the transfer stays compressed (51.75x).
Signed-off-by: trvon <me@trevon.dev>
Every store added two metadata rows per snapshot to the document,
snapshot_id:<id> and snapshot_time:<id>, and nothing removed them.
Content-addressed documents are shared across paths and passes, so they
piled up: on one 93 GB database they were 43.1M of 44.0M metadata rows,
one file carried 52,367 snapshots, and 'yams list' timed out hydrating
2.69M rows for its 100 documents. Meanwhile findDocumentsBySnapshot read
only the latest snapshot_id, so a re-stored document dropped out of every
earlier snapshot.
- Migration 42 creates document_snapshots(document_id, snapshot_id,
snapshot_time), keyed on (document_id, snapshot_id). It only creates the
table, so startup stays fast.
- The insert transaction records membership; stores keep only the latest
snapshot_id/snapshot_time keys.
- findDocumentsBySnapshot and file history read membership, falling back
to legacy keys through one shared parser (legacySnapshotEntries).
- getMetadataForDocuments skips legacy keys via idx_metadata_doc_key: on
the 93 GB database, 0.21 s instead of 18.5 s for list's 100 documents.
- RepairService moves legacy keys in the background, 1000 snapshots per
batch, 200 ms apart, only below Warning pressure.
Signed-off-by: trvon <me@trevon.dev>
lzma_compressor.cpp targets the 7-Zip LZMA SDK (LzmaEnc.h/Alloc.h), but
the build only finds xz/liblzma, which does not ship those headers, so
every build (release logs: 'Has header "LzmaEnc.h" ... NO') compiled it
out. No build could read or write LZMA blocks, and its decoder never
bounded output by the expected size.
The policy still chose LZMA for cold/archival and large inactive data;
compression then failed ('Compressor not available') and batch stores
kept those blocks uncompressed. Select Zstandard, which already uses the
archive level for old data.
Remove the compressor, its test, the enable-lzma option and the build
flags that passed it. The LZMA algorithm id stays in the header format,
and compression.lzma_level remains accepted in config (now unused).
SymbolEnricher boosted results by looking up code-symbol nodes that no
longer exist since v0.20, so it ran KG queries for the top results and
never boosted anything. Remove it, the YAMS_SYMBOL_WEIGHT variable, the
symbol_rank_* search stats, and the symbolRank option on the service,
retrieval, MCP and CLI request types. The IPC field stays in the wire
format but is ignored, and --symbol-rank/--no-symbol-rank remain
accepted as hidden no-ops so existing scripts keep working.
Migration 40 dropped the symbol tables and graph edges, but the daemon
could still adopt an external symbol_extractor_v1 plugin (for example a
0.19 plugin left in a user plugin directory) and rebuild them.
Remove the interface header, the ABI adapter, plugin adoption, the
EntityGraphService extract-and-write path, the post-ingest symbol
channel, the write-coordinator symbol ops, and the KG-store symbol
metadata and extraction-state API. The ABI loader and the external host
now refuse a plugin that declares the interface (or the legacy
symbol_extraction capability) with a one-time warning. Status no longer
reports symbol extractors, generated configs no longer write
[plugins.symbol_extraction] (an existing section still parses), and
the benchmark suite drops symbol_extraction_bench.
Tests that exercised the dropped tables or the v39 schema template are
removed; the rest run against the current schema.
BREAKING CHANGE: code-symbol extraction is gone. Removed: the
symbol_extractor_v1 plugin ABI, `yams config grammar` (list, download,
path, auto-enable, auto-disable), the grammar steps in `yams init`, the
meson option `plugin-symbols`, the conan option
`enable_symbol_extraction`, and the Windows installer SymbolPlugin
component. Databases migrate to schema 41; a v0.19 binary refuses to
open them, and symbol data dropped by migration 40 cannot be recovered.
- Add BinaryQuantizer and BinaryQuantizedIndex in include/yams/vector/binary_quantization.h
and src/vector/binary_quantization.cpp implementing 1-bit sign quantization,
packed 64-bit word storage, hardware popcount Hamming distance, and cosine
similarity estimation.
- Integrate centroidBqIndex into SparseRouteIndex, constructing compact BQ
centroids alongside dense ANN indices during SparseGuidedClusterRouter::buildRouteIndex.
- Add bqCandidateLimit to TopologyRouteRequest and tracking metrics (bqUsed,
bqCandidates, bqDistanceEvaluations) to SparseRouteWork.
- Enable fast candidate pre-filtering in SparseGuidedClusterRouter::route with
1-bit Hamming distance scans, providing sub-millisecond coarse candidate
shortlisting and automatic fallback when the dense ANN index is omitted.
- Add Catch2 unit test suites in tests/unit/vector/binary_quantization_catch2_test.cpp
and tests/unit/topology/topology_baseline_catch2_test.cpp.
Signed-off-by: trvon <me@trevon.dev>
- Add StringInterner in include/yams/search/string_interner.h for
reusable, thread-compatible flyweight string deduplication with
heterogeneous string_view lookups.
- Update TopologyRoutingSnapshot in include/yams/search/topology_routing_session.h
to store zero-copy string_view keys for clustersById and membershipsByDocumentHash,
pointing directly into immutable TopologyArtifactBatch strings.
- Add shared_ptr<StringInterner> to TopologyRoutingSnapshot to intern repeated
lookup and dynamically generated strings.
- Eliminate ~75,000 duplicate heap-allocated std::string objects per cached
snapshot, reducing cache footprint by ~70%.
- Add Catch2 tests in tests/unit/search/string_interner_catch2_test.cpp covering
interning, deduplication, lifetime guarantees, and zero-copy pointer preservation
across snapshot upgrades.
Signed-off-by: trvon <me@trevon.dev>
- Add Migration 41 to drop and recreate metadata_value_counts triggers with
'WHEN ... AND key NOT LIKE topology.%' condition, eliminating ~650,000 trigger
executions per topology rebuild in a 72k document corpus.
- Purge legacy continuous float/structural keys from metadata table
(persistence_score, cohesion_score, bridge_score, role, cluster_level,
parent_cluster_id, overlap_cluster_ids_json) in Migration 41.
- In MetadataKgTopologyArtifactStore::storeBatch, replace N individual
getDocumentByHash calls with single batchGetDocumentsByHash and write
only essential snapshot_id and cluster_id to metadata table.
- In MetadataKgTopologyArtifactStore::loadMemberships, add in-memory fast
path serving memberships directly from cached latest snapshot batch.
- Update serializeTopologyBatchCompressed to handle uncompressed tiny batches.
- Add Catch2 unit tests in tests/unit/topology/topology_metadata_store_catch2_test.cpp
validating trigger bypass, metadata pruning, and fast-path membership loading.
Signed-off-by: trvon <me@trevon.dev>
Replace 960+ MB raw JSON snapshot serialization in MetadataKgTopologyArtifactStore with a compact, string-interned binary format compressed with Zstandard.
- Add yams::topology::serializeTopologyBatchBinary and deserializeTopologyBatchBinary with integer length packing and string deduplication.
- Wrap binary payload in a backward-compatible JSON envelope with format: zstd_binary_v1 and base64 encoded payload, maintaining SQLite json_valid(properties) == 1.
- Fall back gracefully to legacy raw JSON deserializer when reading pre-existing uncompressed snapshots.
- Add Catch2 unit test suite verifying binary roundtrip fidelity, compression ratio, backward-compatibility with legacy JSON snapshots, and corrupted payload resilience.
Signed-off-by: trvon <me@trevon.dev>
Remove legacy Tree-sitter symbol extractor plugin (plugins/symbol_extractor_treesitter),
the unused plugin-symbols build option, and associated tests and benchmark targets:
- Drop plugins/symbol_extractor_treesitter/ (grammar_loader, symbol_extractor, plugin_api).
- Remove plugin-symbols Meson option and packaging references.
- Remove dedicated symbol plugin tests and benchmark targets (symbol_extraction_bench,
symbol_extractor_all_langs_catch2_test, symbol_extractor_edge_cases_catch2_test,
symbol_extractor_solidity_catch2_test, symbol_extractor_query_test,
symbol_extractor_validation_test).
- Clean up dead symbol extraction plugin helper and test from hnsw_entity_search_bench.
- Prune obsolete portability allowlist entries.
Signed-off-by: trvon <me@trevon.dev>
Drop legacy Tree-sitter symbol extraction subsystem, associated database
tables, triggers, and AST edge types:
- Migration 40 drops symbol_metadata, document_symbol_extraction_state,
symbol_metadata_fts, associated triggers, and prunes historical bloated
topology snapshots while preserving active snapshot topology,
semantic_neighbor edges, and non-symbol observed_as provenance edges.
Exclude nl_entity:% nodes from kg_nodes / kg_edges pruning to preserve
canonicalized method-type technology entities.
Provides complete downSQL restoring symbol tables, indexes, FTS virtual table,
and sync triggers for clean v39 rollbacks.
- In EntityGraphService, ensure locally resolved documentDbId propagates to
WriteBatch for knowledgeGraphDocumentId to ensure accurate enrichment
token completion, and only populate batch->documentIdToDelete for snapshot
documents with a resolved document database ID.
- In SymbolEnricher and SearchService, restore one-directional symbol-contains-query
matching with non-empty checks to prevent short substrings matching broad queries.
- In tests/unit/metadata/symbol_metadata_schema_test.cpp, migrate test DB up to
version 39 so tests pass against their target version prior to v40 table drops.
- Remove Tree-sitter grammar download and compile workflows from CLI init
- Default plugin-symbols to false.
- Fix macOS daemon client and CLI executable resolution using
_NSGetExecutablePath with candidate fallbacks.
- Add migration validation tests and symbol enrichment unit tests.
Signed-off-by: trvon <me@trevon.dev>
Add the extended Win32 prefix per filesystem operation and only when
the path reaches MAX_PATH, instead of rewriting the whole storage root.
Short paths keep their ordinary spelling, so listing, canonicalisation
and iteration semantics are unchanged.
Derive list() keys lexically from de-prefixed entries, and canonicalise
containment in the extended namespace but compare it in the ordinary one,
so prefixed and plain spellings are never mixed.
Keep the existing publication regression and cover drive/UNC spelling
plus long-key CRUD.
Signed-off-by: trvon <me@trevon.dev>
Prune CI artifacts and stream metadata scans. Preserve relocated CLI
loader dependencies, reject incomparable path roots, and make deadline
and corruption fixtures test their stated contracts.
Keep unresolved Windows publication and hook failures diagnostic-only.
Signed-off-by: trvon <me@trevon.dev>
Scope vector fixture overrides across every disable alias and verify restoration. Prune excluded repository directories before walking them. Exercise CLI serve delegation and distinguish outer spawn failure from missing child diagnostics using isolated stubs.
Signed-off-by: trvon <me@trevon.dev>
