9 commits

Author	SHA1	Message	Date
Trevon	e0d15dc680	fix(p2p): stop quarantining honest writers for concurrent-session races (#245) ... * feat(p2p): log durable writer quarantines and outbound session failures A node that durably quarantined a peer writer left no trace of why: the three p2p quarantine sites returned an error to their session, and the reconnect loop dropped every outbound session error without logging it, so a mesh that stopped receiving one writer showed only a quarantined record count. Outbound sessions are half of all sessions in a full mesh. Each p2p quarantine site now logs the writer, the failed check and the local and frozen counters, and a failed outbound session logs its peer and error, matching the existing inbound failure log. Signed-off-by: trvon <me@trevon.dev> * fix(p2p): stop quarantining honest writers for concurrent-session races Each pair of direct peers normally runs two sessions at once, one initiated from each side. Both freeze a window of the peer's writer at handshake, and whichever applies first advances the receiver's durable prefix under the other. The second session then compared its frozen window with that live prefix and treated the difference as forged history: validateHistoryExtension demanded the window start right after the live commitment ("history extension is not contiguous"), and an empty window demanded the live counter equal the frozen frontier. Both durably quarantined an honest writer, after which the node dropped its documents and refused every later session with it. A 5-node mesh run logged exactly this (local=30 frontier=32 staged=4: a window frozen at 28 arriving after another session had applied 29-30), and the mesh never converged. The receiver now validates a window against durable history instead of a snapshot. Operations at or below the advanced prefix are replays that applyDeltas leaves untouched (a different record under a known operation id is still a fork). The rest must chain from the durable commitment exactly to the authenticated frontier, and an empty window must match the durable commitment at its frontier. validateHistoryExtension returns a typed WriterHistoryViolation for writer faults (fork, forged frontier, gap, unauthenticated record) and an Error for local or transient failures, and only a violation quarantines; an Error aborts the session for a retry on the next cycle. The security property is unchanged: nothing is applied unless a hash chain links durable history, itself verified against the writer's authenticated commitments, to the frontier the writer authenticated in this handshake. Overlapping replays change no state, and forks, forged frontiers, gaps and unauthenticated records still quarantine, now also when they overlap an advanced prefix (new tests cover each). The pre- and post-exchange history checks get the same treatment. A live prefix between the proven handshake prefix and the frozen frontier is consistent, since delta exchange verifies the rest; one beyond the frontier is compared with the durable commitment at the frontier rather than rejected as a rollback. The handshake also clamps the live local-writer window resolver to the frozen snapshot, so concurrent local writes no longer fail the session with "local writer-window resolver violated negotiated bounds". Signed-off-by: trvon <me@trevon.dev> * feat(p2p): report outbound sessions and durable writer quarantines `yams p2p status` showed inbound session failures but not outbound ones, and only a `quarantined` record count that mixes durable writer quarantine with transient rejections (a record whose causal dependency from a third writer has not arrived yet). An operator could not tell a mesh that was retrying from one that had permanently cut off a writer. Status now reports: - outbound_sessions / outbound_failures and the last outbound failure with its peer, counted for both explicit connects and reconnects. Failed sessions in either direction are aborted and retried. - quarantined_writers: writers durably quarantined by this node, read from the replication checkpoint, so it survives restarts. Each quarantine is also logged with its reason. The fields travel through MemorySyncStatus, MemorySyncResponse and protobuf fields 38-42 to the text and JSON status and to `yams daemon status -d`, following the apply-health fields. Signed-off-by: trvon <me@trevon.dev> * fix(memory-sync): verify replayed writer counters against durable history validateHistoryExtension skipped every staged operation at or below the writer's durable prefix as a replay without looking at its record hash, leaving fork detection to applyDeltas. applyDeltas recognizes a fork only by an operation id it still remembers, and that memory is lost on restart and bounded by maxTrackedIdentities. A different, internally valid record for an already-applied counter therefore passed as a harmless replay instead of being quarantined as the fork it is, whether the window only replayed old counters or rode the fork along an honest extension. Before skipping a replayed counter, compare its record hash with the durable history entry committed at that counter (rebuilding the entry like computeHistoryCommitmentAt when it is missing). A mismatch is a WriterHistoryViolation, so the p2p session durably quarantines the writer; an identical replay is still accepted as a no-op. A history entry that cannot be read stays a local Error and only aborts the session. Signed-off-by: trvon <me@trevon.dev> * fix(memory-sync): log every durable writer quarantine where it happens Durable writer quarantine was logged only by its two p2p callers. The duplicate-operation fork path inside applyDeltas and the shared-store scan path quarantined silently, so the documented guarantee that every durable quarantine logs its reason and counters did not hold, and an operator saw a quarantined_writers count with nothing in the log to explain it. quarantineWriter now takes the reason and is the one place that logs a new quarantine: writer, reason, source node, and the writer's local history and version counters. It logs nothing when the writer was already quarantined, so a repeated request is not reported twice. The p2p callers drop their own warnings and fold their session context (frontier, staged count, prefix match) into the reason instead. memory_sync now declares its spdlog dependency explicitly. Signed-off-by: trvon <me@trevon.dev> * fix(cli): show outbound p2p sessions in daemon status The daemon already reports outbound sessions, failures, and the last outbound failure, and `yams p2p status` prints them, but `yams daemon status -d` only gained the quarantined-writers row. An outbound direction that keeps failing is aborted and retried, never quarantined, so the detailed status was the place an operator would miss it. The Memory Sync section now shows an Outbound sessions row (started and failed counts, warning when any failed) for direct-backend nodes, and a Last outbound failure row with the peer, the error, and its age. Signed-off-by: trvon <me@trevon.dev> --------- Signed-off-by: trvon <me@trevon.dev>	2026-10-01 12:18:47 -06:00
Trevon	7be908e5b7	fix(p2p): stop the memory-sync apply/publish deadlock and report apply health (#243) ... * fix(daemon): never gate memory-sync publish on inbound apply applyMemorySyncWinners() ran the inbound stages content -> metadata -> vector -> topology and returned on the first stage failure, before the node's own outbound backfill. In a mesh where every node holds an inbound record whose prerequisite only a peer can publish (an embedding whose content blob has not replicated, an edge whose target node has not), every node refused to publish, so the prerequisites never arrived: a stable mutual deadlock with healthy transport and nothing converging. The outbound publish now runs after the inbound stages whatever their outcome; only a stop request ends a cycle early. Each inbound stage also runs even when an earlier one failed. Cross-stage dependencies (metadata and vectors need local content bytes, edges need their endpoint nodes) are checked per record against the local stores, so a failed stage shows up downstream as unapplied records rather than as skipped stages. Stages keep their prerequisite order so a prerequisite landing in a cycle is visible to its dependents in that same cycle. A two-node test drives real ServiceManager apply cycles over a shared store, each node holding an edge whose target only the other can publish; it converges in a bounded number of rounds only with this change. Signed-off-by: trvon <me@trevon.dev> * fix(memory-sync): defer records whose prerequisite has not replicated Vector, metadata, and topology apply treated a missing replicated prerequisite as a hard error and rejected the whole batch: one embedding whose content blob had not arrived, or one edge whose target node had not, blocked every other winner in that stage on every cycle. These are ordering races between independently published records, not failures. Each adapter now defers such a record instead: it is skipped, the rest of the batch applies, and its logical key is reported through deferredKeys(). Nothing marks it applied, so it remains a winner in the replicated index and the next cycle retries it once the prerequisite lands; apply needs no pending queue of its own and loses nothing across restarts. An edge whose endpoint is being deleted in the same batch is deferred too rather than inserted against a stale node id. Malformed records, probe failures, and store errors still fail the stage. Content blobs are independent, so the content stage now attempts every blob and fails afterwards with the first genuine error; a corrupt blob no longer starves the blobs after it. A blob whose bytes are not hydrated in the sync cache yet is deferred like the other stages. Signed-off-by: trvon <me@trevon.dev> * feat(p2p): report memory-sync apply health in status The mesh deadlock was invisible: every node reported failed_cycles=0 and a climbing successful_cycles while nothing converged, because a cycle counts as successful once reconciliation completes, whatever the apply stages did afterwards. Deferred records would be just as invisible without a signal. ServiceManager now keeps an ApplyHealth record of the apply path: - deferred records per stage, and the age of the oldest one still pending, with a warning once a deferral passes ten minutes (its prerequisite likely never arrived). Ages are tracked for at most 4096 keys per stage, keeping the oldest; counts stay exact. - hard failures per stage, plus the last failure's stage, message, and age. - apply_cycles and apply_failed_cycles. failed_cycles keeps meaning reconciliation failures; a cycle whose apply stage hard-failed is counted in apply_failed_cycles instead of passing as a plain success. - publish_skipped_cycles (delete outbox preempted the cycle) and publish_failed_cycles (an outbound backfill domain failed). The fields travel through MemorySyncStatus, MemorySyncResponse, and protobuf fields 22-37 to `yams p2p status` (new key=value tokens on the first line, an `apply` line with per-stage counts, a last-apply-failure line, and an `apply` object in --json) and to the memory-sync section of `yams daemon status`. The p2p status formatter moves behind a header so it can be tested without a daemon. Signed-off-by: trvon <me@trevon.dev> * fix(memory-sync): keep content deferrals when the same scan hits a failure The content stage scans every blob and fails afterwards with the first genuine error, but the apply cycle discarded the deferred keys that the same scan collected whenever it failed. A corrupt blob therefore hid every blob still waiting for its bytes: the deferred count stayed at its previous value (0 on the first pass) and the stale-deferral warning never fired for a prerequisite that was actually missing. A stage pass now carries both outcomes. The content-blob scan moves into ContentBlobSyncAdapter::applyWinners, which reports the applied count, the deferred keys, and the first failure of one complete scan, and ApplyHealth::recordPass records the failure and replaces the deferred set together. A stage that returns an error still leaves the deferred set as last observed, since an aborted scan says nothing about prerequisites. The tracked-deferral cap already carried every still-deferred tracked key before admitting new ones, but its carry loop had a cap check that read as if it could stop early. Drop it (the tracked set never exceeds the cap) and pin the behaviour with tests that place the oldest key after more than the cap's worth of new keys and that refill a full tracked set. Signed-off-by: trvon <me@trevon.dev> --------- Signed-off-by: trvon <me@trevon.dev>	2026-10-01 12:18:47 -06:00
trvon	1fa2e92510	fix(daemon): show host pressure and deferred work in daemon status ... When the daemon holds background work back for a busy host, a user looking at a stale topology or an unfinished repair scan needs to see why. Status had no host signal at all. Carry the governor's host sample through the metrics snapshot and StatusResponse: source (psi, loadavg), whether the host counts as busy, the PSI cpu/io/memory "some avg10" values or the per-CPU load average, and which deferrable work is currently held back. The proto transport piggybacks the fields on request_counts like the other string status fields; the native serializer appends them as optional tail fields. `yams daemon status` (brief and detailed) shows a Host Load row, e.g. "Busy cpu 72% · io 3% · mem 0% · deferring topology, repair", and `yams status --json` gains a host_pressure object. Document the new [tuning.resource] keys next to the other typed tuning sections. Signed-off-by: trvon <me@trevon.dev>	2026-09-27 20:55:03 -06:00
trvon	86699d80f5	fix(cli): show vector index as pending until vectors initialize ... `yams daemon status` printed "Vector Index: disabled by configuration" whenever the daemon had not attempted vector init yet. That includes every start where vectors are enabled but the metadata integrity check is still running, so a slow start looked like a misconfiguration. The daemon now publishes readiness key vector_db_disabled (only when true, so clients that list every non-ready key do not show it) when vectors are disabled by configuration. The renderer says "disabled / by configuration" only in that case; otherwise it shows "pending" with what it is waiting on (e.g. "waiting for database (Checking Integrity)"). The detailed view's Vector DB row gets the same distinction instead of "Not initialized". Signed-off-by: trvon <me@trevon.dev>	2026-09-27 14:00:25 -06:00
trvon	0b1b162e2a	fix(daemon): report the startup integrity check as its own phase ... Without a trusted clean-shutdown stamp, startup runs a full PRAGMA quick_check before the database is ready. On a 93 GB database that took over 11 minutes, and status showed 'Database: Opening' while the log repeated 'still opening database', so a healthy scan looked like a hang. Add the checking_integrity database phase, set it around the blocking check, render it as 'Checking integrity ... full scan after unclean stop' in 'yams daemon status', and name it in the startup watchdog log. The check itself is unchanged: it still blocks readiness, and still recovers confirmed corruption. Signed-off-by: trvon <me@trevon.dev>	2026-09-26 08:56:51 -06:00
trvon	26a1716400	fix: preserve cancellation across worker restart ... Reassert backend cancellation under the lifecycle lock. Protect fake backend condition-variable predicates against lost wakeups. Fix Windows fixture encoding, shell selection, and rooted-path checks. Keep the unresolved rename failure diagnostic-only. Signed-off-by: trvon <me@trevon.dev>	2026-09-14 00:58:19 -06:00
trvon	09aa7aef80	fix(ci): repair audited platform regressions ... Prune CI artifacts and stream metadata scans. Preserve relocated CLI loader dependencies, reject incomparable path roots, and make deadline and corruption fixtures test their stated contracts. Keep unresolved Windows publication and hook failures diagnostic-only. Signed-off-by: trvon <me@trevon.dev>	2026-09-13 20:48:30 -06:00
Trevon	e4b32e98bd	fix: fence enrichment completion and reconcile promotion prerequisites (#157) ... * test(ci): stabilize main matrix (#112) Signed-off-by: trvon <me@trevon.dev> * ci(release): preview experimental promotion (#111) Signed-off-by: trvon <me@trevon.dev> * ci(release): reconcile draft promotion safely (#113) * ci(release): reconcile draft promotion Create promotion visibility independently of release readiness. Leave existing PRs untouched to preserve concurrent maintainer edits. Validate against current main without masking candidate rejection. Signed-off-by: trvon <me@trevon.dev> * ci: isolate hook Git environments Prevent temporary Git fixtures from modifying the calling worktree. Clear repository-local hook variables before the unchanged build gates, and isolate validator and fixture Git subprocesses from caller overrides. Exercise linked-worktree config, ref and index preservation offline. Signed-off-by: trvon <me@trevon.dev> --------- Signed-off-by: trvon <me@trevon.dev> * fix(ci): reject invalid candidate evidence Explain missing main history without mutating refs or weakening promotion validation. Match experiment boolean and aggregation parsing, respect candidate-limit precedence, and return issues for malformed configuration or metric values. Signed-off-by: trvon <me@trevon.dev> * fix: preserve completion and retrieval contracts Propagate derivation SQL failures, invalidate KG status caches, and avoid unused preview reads. Align explicit hash validation and export standalone client dependencies. Keep secret exceptions fixture-scoped and remove undefined behavior from status fixtures. Signed-off-by: trvon <me@trevon.dev> * fix: fence asynchronous enrichment completion Coordinate graph and NL commits before acknowledging enrichment. Atomically classify embedding reuse, fence repair status writes, and preserve newer vectors on old failures. Upgrade content readiness triggers and keep repository counters coherent. Six affected ASan suites pass; this does not provide restart replay or generation-scoped vector storage. Signed-off-by: trvon <me@trevon.dev> * fix: bound semantic graph output reservation Use retained neighbors for admission and node capacity instead of mixing corpus and pair counts. Preserve single-neighbor output and avoid quadratic reservation. Characterization regression fails three assertions before the fix; full component ASan suite passes afterward. Signed-off-by: trvon <me@trevon.dev> * fix: fence standalone embedding repair completion Mint attempt tokens before reading canonical repair input and publish only current persisted document embeddings. Do not skip stale physical vectors. Make SQLite preparation fault injection connection-local and deterministic. Full component and metadata ASan suites pass. Signed-off-by: trvon <me@trevon.dev> * fix: preserve enrichment failure reporting Admit pending embedding attempts before provider failures while preserving completed results. Route title no-op acknowledgement through the writer so flush reports SQL failures and duplicate jobs can retry. Four affected full ASan suites pass; host configuration hashes unchanged. Signed-off-by: trvon <me@trevon.dev> * test: align explicit hash prefix boundaries Reject five characters and accept the supported six-character minimum. Full hybrid_search suite passes. Separate daemon shutdown sanitizer failure remains under investigation despite a passing targeted rerun. Signed-off-by: trvon <me@trevon.dev> * test: align UI hash validation boundary Signed-off-by: trvon <me@trevon.dev> --------- Signed-off-by: trvon <me@trevon.dev>	2026-09-12 18:53:12 -06:00
Trevon	60855b7b78	[11/14] Separate status presentation and centralize shared query helpers (#151) ... * refactor(daemon): one table for repair operation wire codes The code published under repair_current_operation_code was defined in three places: a name->code chain in RepairService.cpp, an enum->name switch beside it, and a code->label switch in the CLI. The two daemon tables agreed with each other but the CLI copy had already drifted in wording, and nothing tied the three together. metric_keys.h now carries the single code<->name table next to the metric key it describes, with constexpr lookups in both directions. RepairService's name->code chain delegates to it; the enum->name switch stays because it is keyed by the private enum. The CLI copy goes in the next commit with the status renderer. Signed-off-by: trvon <me@trevon.dev> * refactor(cli): extract the daemon status renderer showStatus() in daemon_command.cpp was 1,762 lines: two RPCs, a bootstrap-file fallback, and two ~400/1,100-line render bodies that wrote straight to std::cout from inside the command class, reachable only through a live daemon. The render bodies read nothing from the command except the expected data directory, the explicit-socket flag, and a memory-sync fetch they triggered mid-render. The bodies now live in src/cli/commands/daemon_status_render.cpp as free functions over StatusResponse plus a small DaemonStatusRenderContext (configured data dir, explicit-socket flag, optional MemorySyncResponse) writing to an ostream. The readiness/label helpers that only they used (humanizeToken, classifyReadiness, isEphemeralDataDir, daemonDataDirFromStatus, paintStatus, neutralText) move with them. DaemonCommand keeps the RPCs and the bootstrap fallback; the memory-sync fetch becomes fetchMemorySyncStatus() and runs just before rendering instead of in the middle of it, so the section prints in the same place with the same bounded 3 s timeout. The move is verbatim: after the mechanical substitutions (std::cout -> os, expectedDataDir() -> ctx.configuredDataDir, the memory-sync call, the explicit-socket flag, and the repair label function) the two bodies match the originals character for character modulo whitespace. The CLI's private repair-operation switch is gone; the label now comes from the shared table in metric_keys.h with underscores rendered as spaces, which reproduces the previous wording ("stuck docs", "block refs", "path tree"). tests/unit/cli/daemon_status_render_catch2_test.cpp drives both renderers and the memory-sync section from synthetic responses: overview rows, blockers with init progress, last error, data-dir mismatch (and its suppression under --socket), temporary data dir, the active repair operation label, every detailed section, and the started/not-started memory-sync gate. daemon_command.cpp: 4,082 -> 2,341 lines; daemon_status_render.cpp: 1,642 lines. Signed-off-by: trvon <me@trevon.dev> * refactor(app): one home for hash predicates, path aliasing, and metadata qualifiers Three small helpers had grown parallel copies with quietly different rules: - "does this look like a hash" existed four times: search_service (8-64 hex), document_service (6-63), cat_command (6-64), plus a never-defined utils::looksLikeHash declaration in services.hpp. The 8 vs 6 threshold is deliberate (D3): a bare query token must reach 8 hex characters before it stops being text, while an explicit hash argument may be 6. yams/common/hash_predicates.h now names the two predicates apart (looksLikeHashQueryToken, looksLikePartialHashArgument) over one isHexDigits; document_service's 6..63 bound was deliberate for the name resolver (a full 64-hex name must not resolve as a prefix and delete by hash), so its two resolver sites now guard size() != 64 explicitly, as the partial-hash resolution site already did. - macOS /private aliasing had three incompatible policies: search appended the alias of /var only, grep canonicalized /var and /tmp to the /private spelling, and the delete resolver displayed /private/var as /var. yams/common/fs_utils.h now carries one alias table ({/var,/tmp} <-> /private/...) with three lexical policies over it (otherSpelling, canonicalSpelling, displaySpelling) and the platform-gated wrappers callers use. Declared behavior change, macOS only: search patterns under /tmp now also match their /private/tmp spelling, and the resolver displays /private/tmp/... as /tmp/.... - Inline key=value metadata filters were parsed twice: SearchService's extractStructuredMetadataQuery (exactly one '=', no quoting/grouping characters, deduplicated by key) and SearchEngine's parseMetadataFiltersFromQuery (any '=' token, duplicates kept). The stricter, tested parser moves into yams/search/query_qualifiers.hpp and both call it. Declared behavior change for the hybrid metadata component: tokens like name="x" or a=b=c stay in the query instead of becoming filters, and repeated keys collapse to the last value. Tests: hash_predicates_catch2_test.cpp pins both thresholds and the 64-character bound; path_alias_catch2_test.cpp pins the alias table, the three policies, and the platform gating; query_qualifiers_catch2_test.cpp gains the structured-metadata cases (split, dedupe, malformed tokens, filters-only). search_service, document_service, grep_service, service_utilities, and resolve_submodule pass unchanged. Portability allowlist anchors reflowed for the four files whose lines moved. Signed-off-by: trvon <me@trevon.dev> * fix(cli): flush status before optional collection Render useful primary status before waiting for memory-sync details. Keep both brief and detailed paths behind the tested ordering seam. Signed-off-by: trvon <me@trevon.dev> --------- Signed-off-by: trvon <me@trevon.dev>	2026-09-12 10:09:21 -04:00