* test(daemon): memory_sync catch-up sweeps must go idle on unchanged stores
The vector and topology catch-up sweeps restart a full pass as soon as
one ends, so an idle node reads and re-checks every node, edge, vector,
and committed topology record forever. On a 3-node mesh limited to 2
CPUs each this held every node at ~24% of a core with nothing changing.
New cases drive the production backfill through ServiceManager:
- after a completed pass with no writes, further cycles serve no sweep
item and take no graph-store connection;
- a node, edge, and vector written after a pass are published by the
next pass while unchanged records keep their committed operation;
- a node deleted after the sweeps went idle is retracted with its edge.
Store seams: KnowledgeGraphStore::changeGeneration() and
VectorDatabase::changeGeneration() move on writes and batch commits,
not on reads.
Red: catch2_memory_sync_apply "[idle]" 3 failed (items 52 == 32,
acquisitions 157 == 97); kg_store and vector_smoke do not compile.
Signed-off-by: trvon <me@trevon.dev>
* fix(daemon): run memory_sync catch-up sweeps only when a store changed
The vector and topology catch-up sweeps in publishMemorySyncBackfill
restarted a full pass as soon as one ended. An idle node therefore read
and re-checked every node, edge, vector, and committed topology record
forever, and every published record cost each peer a delta exchange and
an apply. On a 3-node mesh limited to 2 CPUs each this held every node
at ~24% of a core with nothing changing; on a real corpus the first
pass never ends.
Each store now exposes a change generation: KnowledgeGraphStore counts
write transactions (single writes through the write pool and write
batch commits), VectorDatabase counts inserts, updates, and deletes. A
sweep records the generation a pass began at and, once the pass
completes, does not start another until the generation differs. Writes
behind a running pass's cursor move the generation past the pass's own
reading, so the following pass still covers them; deletes move it too,
so the retraction phase still runs after a rebuild drops records. With
nothing changed, a cycle costs each sweep one atomic read.
No config knob: the gate is a property of the stores, not a tunable.
Tests: catch2_memory_sync_apply (14 cases, incl. 3 new [idle] cases),
catch2_kg_store, catch2_vector_submodule.
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
* refactor(tuning): prune dormant runtime knobs
Remove unused tuning controls and adaptive search budget state. Extract testable ONNX slot reconciliation and search telemetry hydration seams, and correct the Apple Silicon sysctl value width.
* refactor(vector): retire dormant backends
Remove unused FAISS, model-infrastructure, VectorIndexManager, and TurboQuant paths while keeping Simeon PQ and float-vector storage.
Migrate unused V2.2 quantized schemas transactionally to V2.3. Databases containing non-default quantized formats or packed-code blobs fail closed with recovery guidance and no mutation.
* refactor(plugins): remove legacy SDK tooling
Remove the Ghidra and SDK submodules, their bespoke tests, and stale scan exclusions. Retain generic PluginProcess coverage against the repository mock fixture.
* build(vector): remove stale FAISS overrides
* refactor(ipc): remove dormant FSM snapshots
* refactor(ingest): unify bounded KG enqueue
* fix(plugins): harden allocation contracts
Keep ABI outputs null or fully initialized when allocations fail, and represent empty object payloads without relying on malloc(0).
* fix(storage): define empty object reads
Return a null buffer with zero length for successful empty-object reads instead of treating malloc(0) as an allocation contract.
* refactor(storage): trim through string views
* test: prune dormant harness helpers
Remove unused readiness and fixture helpers, and keep benchmark client-count doubling in a wide integer domain.
* chore(cpp): initialize analyzer-facing locals
* build(simeon): advance analyzer cleanup
* refactor(config): drop dead daemon client keys
* refactor(ipc): inject daemon client transport
Socket and in-process execution now share DaemonClient request dispatch, so CLI fallbacks do not need alternate service paths.
* refactor(retrieval): route CLI through daemon
Search and grep now use the same daemon handlers over socket or in-process transport. Live grep remains an explicit freshness inspection.
* refactor(graph): move CLI scope into daemon
Graph CLI requests now share daemon handlers across socket and in-process transports. CWD filtering occurs before pagination so scoped totals and pages stay correct.
* build: isolate test-only runtime controls
* refactor(metadata): decouple lock telemetry
* fix(services): return on request timeout
* fix(retrieval): enforce workspace scope
* fix(graph): scope nodes before pagination
* fix(grep): drop binary result lines
* fix(daemon): bound checkpoint wait
* fix(repair): use KG health checks
* fix(storage): always verify compressed CRC
* perf(vector): skip unchanged PQ snapshots
* build(zyp): exclude bench from default build
* build: protect Meson files from formatter
* refactor: remove unreachable production APIs
* refactor(daemon): remove dormant runtime state
* fix(vector): harden daemon provider calls
Timed-out coroutines must own their client and requests. Provider discovery must also fail instead of returning synthetic embeddings in production.
* refactor(storage): remove unusable adapters
* fix(cli): unify in-process add requests
* fix(tsan): preserve async worker progress
* test(cli): isolate daemon-dependent suites
* fix(repair): expose queue backoff progress
* fix(graph): handle qualified C++ symbols
Signed-off-by: trvon <me@trevon.dev>
* refactor(vector)!: remove dead transition APIs
Split Simeon vector and retrieval linkage so consumers inherit only the runtime surface they use.
BREAKING CHANGE: remove unused async/model/config helpers and the undefined vector-backend factory from public headers.
Signed-off-by: trvon <me@trevon.dev>
* fix(vector): harden PQ snapshot lifecycle
Validate and atomically install persisted generations, revalidate after races, and fall back to exact search when a snapshot is stale. Add the exact-scan reference lane and distinguish observed work counters from unavailable backend metrics.
Signed-off-by: trvon <me@trevon.dev>
* feat(topology): expose route coordinates
Persist chart distortion and local-dimension evidence, retain pre-scalarization route axes, and emit detailed trace rows without treating unavailable vector work as zero.
Signed-off-by: trvon <me@trevon.dev>
* feat(xplan): add vector decision gates
Track binary and corpus identity, preserve missing-work semantics, and add repeated PQ routing, crossover, and rerank plans with quality and cost summaries.
Signed-off-by: trvon <me@trevon.dev>
* feat(topology): formalize coordinate routing
Separate chart geometry from route-risk admission and prove candidate-stage recall and strict-focus boundaries. Record the measured atlas limitations and literature basis for the next construction experiment.
Signed-off-by: trvon <me@trevon.dev>
* fix(search): serialize SymSpell readers
SQLite-backed lookups reuse mutable prepared statements, so shared-reader locking is unsafe. Advance the owned SymSpell revision for batched delete insertion and cover concurrent root lookups under TSAN.
Signed-off-by: trvon <me@trevon.dev>
* fix(vector): make mutations atomic
Preserve stored vectors on replacement and commit failures, reject cross-thread writes during explicit transactions, and restore exact reranking for quantized-primary PQ rows.
* fix(search)!: surface SymSpell write errors
Propagate persistent dictionary failures through metadata ingestion, roll back failed batches, and make clear remove stored terms.
BREAKING CHANGE: SymSpellSearch mutations now return Result values.
* feat(topology): formalize candidate expansion
* feat(vector): add coordinate-aware embeddings
* feat(topology): preserve protected relation cover
* feat(search): add graph candidate rescue
* feat(daemon): expose retrieval hydration telemetry
* feat(xplan): add rescue and hydration gates
* refactor(content): dedupe media handlers
Reuse shared graph presenter helpers and centralize repeated audio/video/archive buffer processing, validation, lifecycle, and stats helper logic.
* feat(lifecycle): prune stale corpus data
Keep cleanup preview-first while retention stays opt-in and long prune work no longer holds IPC responses open.
* feat(search): add topology candidate rescue
Preserve high-value routed candidates across vector backends while keeping exact rerank diagnostics explicit.
* feat(search): expose evidence paths
* docs: define mobile host contract
* formal: model bounded candidate rescue
* fix(prune): inspect extensionless signatures
* fix(prune): tolerate stale extensions
---------
Signed-off-by: trvon <me@trevon.dev>
missing paths where feasible, fail fast where support is
not real, and sync docs with live behavior.
- add vector/metadata and buffer extraction coverage
- narrow storage and s3/plugin loader contracts
- refresh pipeline, build, and test docs
