Publish owning data-directory snapshots under a mutex and route bootstrap
and fallback publications through the same synchronization boundary.
Keep polling completion locked through its final notification so shutdown
cannot destroy the condition variable while the notifier still uses it.
Exercise both publication windows with deterministic event-based tests and
use the repository thread-compatibility wrapper.
Preserve Failed in status rendering instead of reporting initialization.
Reanchor the existing metrics environment-read entry; add no exceptions.
This does not close the separate queued-collector and embedding-borrower
lifetime audits or establish universal bounded callback shutdown.
Signed-off-by: trvon <git@trevon.dev>
* test(search,daemon,storage): assert behaviour instead of frozen literals
Several suites encoded exact internal values that legitimate tuning and
ranking changes are allowed to move: SCIENTIFIC-era profile weights and
rrfK values, corpus-statistic figures, and hardcoded timeout/cancellation
numbers. They failed on every platform after those changes landed, so the
real behaviour regressions they should catch were hidden behind noise.
Replace the frozen literals with contract assertions: weights are finite,
non-negative and normalize, state selection still holds, capability-gated
sources stay zeroed, and hybrid results keep matching canonical hash/path
sets. Deadlines are not widened and the daemon paths they cover are
unchanged.
Also add independent corpus and adapter oracles, deterministic collector
cancellation and late-result gates, and synchronize ServiceManager
shutdown observation, so the tests fail for the behaviour they name.
The portability allowlist keeps its two reviewed raw test-env reads; only
their line numbers moved.
Signed-off-by: trvon <git@trevon.dev>
* ci(tests): summarise Meson failures without flooding the log
A failing test run printed the whole Meson suite output, so the actual
failing test names and their assertion messages were buried and the
uploaded logs were dominated by passing-test chatter.
Add a bounded reporter that extracts the failing test names plus their
surrounding diagnostics, keeps an overall cap on emitted lines, and is
unit-tested against representative Meson output. Wire it into the Tests
workflow so failures stay readable.
Signed-off-by: trvon <git@trevon.dev>
* build(deb): validate candidate identity, mapping and upgrade ordering
The Debian packaging scripts took filenames and versions on trust, so a
misnamed candidate or a wrong package ordering could pass validation while
the real install path was never exercised.
Fail closed on candidate identity, version and architecture, verify the
package-to-source mapping, seed the legacy-only layout, wait on the actual
services, and assert install/upgrade/remove ordering for both Debian and
RPM in an offline container. Cases that cannot be proven on a host are
reported as skipped rather than silently passing.
Signed-off-by: trvon <git@trevon.dev>
---------
Signed-off-by: trvon <git@trevon.dev>
Corrupt-DB handling could destroy the only copy of documents:
- removeCorruptDbFiles() deleted every readable yams.db.corrupt-* after a
startup or `repair --orphans --remove-corrupt` salvage, including DBs whose
salvageFromCorruptDb() had failed or only partly copied, and DBs that were
never salvaged because their document count was <= the live DB's.
- A corrupt DB whose count was <= the live DB's was otherwise kept forever
with only a "skipping" log, with no path that ever decided it was safe to
drop.
- quarantineAndRecreate() renamed onto yams.db.corrupt-<second>, so two
quarantines in the same second overwrote the first copy.
Deletion is now decided by evidence, not by which code path ran:
corruptDbSalvageConfirmed() opens the corrupt DB read-only, attaches the
live DB and checks that every document hash in it is present in the live
DB (salvage copies documents keyed by hash). removeSalvagedCorruptDbs()
removes an artifact only when that check succeeds and the artifact is older
than a retention window (quarantine timestamp from its name, else mtime);
unreadable, unconfirmed or young artifacts are kept and reported with the
reason.
Startup salvage no longer deletes anything. RepairService owns cleanup:
runRecoveryArtifactMaintenance() runs from the background loop when
maintenanceAllowed(), every recoveryArtifactSweepInterval (6h), with
corruptDbRetention (7 days) in RepairService::Config. An explicit
`repair --orphans --remove-corrupt` waives the retention window but still
requires confirmation. yams doctor explains the retention policy next to
the artifacts it lists.
The "yams.db" / ".corrupt-" naming is centralized in db_recovery.h
(kMetadataDbFileName, kCorruptDbMarker, corruptDbPrefix(),
isCorruptDbFileName(), listCorruptDbs()) and used by salvage, recovery,
RepairService and the doctor checks instead of four hand-rolled scans.
Signed-off-by: trvon <me@trevon.dev>
A clean `yams daemon restart` never published the clean-shutdown
integrity stamp, so every start paid for a full PRAGMA quick_check
(13-27 min on a 93 GB database). Request handlers offloaded to
WorkCoordinator threads (list, grep, get, search) kept running through
Phase 5: nothing interrupted a long SQLite statement or a long result
loop, both 5 s and 30 s joins expired, the workers were abandoned, and
the stamp was disabled.
Shutdown now tells in-flight work to stop before joining:
- ConnectionPool::interruptActiveConnections() calls sqlite3_interrupt
on every leased connection under the pool mutex, which keeps each
lease open while it is interrupted. DatabaseManager exposes it for
both pools. Phase 4 interrupts once, and Phase 5 re-interrupts on each
join wait slice through a new joinWithTimeout hook, because SQLite
drops an interrupt that lands between two statements.
- ServiceManager owns a shutdown cancellation flag, set at Phase 4
before the interrupt, and publishes it as AppContext::cancellationSignal.
DocumentService::list and GrepService::grep check it between query
and hydration steps and between document batches, and return
OperationCancelled; SearchService checks it on entry and forwards it
as SearchExecutionContext::hostCancellationSignal, which the engine's
existing cancellation checks and the fanout collector now honor.
- The concept-extraction wait no longer blocks without bound when
componentTimeout is 0, and a canceled search stops waiting for the
query embedding / concept tasks. Those tasks now own copies of their
inputs instead of capturing the caller's stack, so abandoning them is
safe.
Interruption stays scoped to shutdown: the flag is only set in Phase 4
and cleared on initialize, and interrupting an idle lease does not fail
its next statement.
Also erase a connection from the leased set when its rollback throws;
it was left behind as a dangling pointer, which the interrupt walk
would have dereferenced.
Signed-off-by: trvon <me@trevon.dev>
A single removal that failed returned before the watcher advanced its
snapshot, so every later pass saw the same files as changed, re-ran
addDirectory on all of them, and retried the removal forever. Removal
used force=false, and a path indexed more than once fails with 'Multiple
documents match', so one such path wedged the watcher permanently. A
live daemon re-indexed 1,793 files every 2 s this way and logged a
warning on each pass.
- Hold back only the paths that fail: a changed file whose indexing
failed stays changed, a removal that failed stays pending, and the
snapshot advances for everything else.
- Remove a deleted path with force: the file is gone, so every document
stored at it is stale.
- Warn once per failing path and retry quietly after that.
- Honor the watched directory's .gitignore (the unused gitignorePatterns
and gitignoreMtime state), and skip .git, so ignored build output is
not tracked, restatted, or removed.
Signed-off-by: trvon <me@trevon.dev>
- config_resolver: several [tuning] defaults derive from core count (e.g.
onnx_max_concurrent is 7 on a 48-thread host), so a reload is now checked
against the values observed before the override instead of != sentinel.
- service_manager: write the P2P trust files with explicit permissions so
a umask of 002 (Debian user-private groups) no longer trips the
group-writable check before the symbolic-link check under test.
Prune CI artifacts and stream metadata scans. Preserve relocated CLI
loader dependencies, reject incomparable path roots, and make deadline
and corruption fixtures test their stated contracts.
Keep unresolved Windows publication and hook failures diagnostic-only.
Signed-off-by: trvon <me@trevon.dev>
Require authenticated direct stores and explicit operator enrollment while hardening private keys and trust files across Unix and Windows. Sign local control artifacts, bind them to a protected store generation, and serialize async initialization shutdown.
Signed-off-by: trvon <git@trevon.dev>
* fix(memory-sync): keep replication responsive
Skip poisoned blobs without wedging backfill and publish immutable status snapshots so daemon IPC does not block on an active reconciliation cycle.
Reconstructed from ff62d3f8, 8e232721, and e60f513d.
Signed-off-by: trvon <git@trevon.dev>
* feat(p2p): add authenticated direct transport
Require TLS 1.3 mutual identity and bind the authenticated certificate, node, corpus, and epoch before replication state is exchanged.
Reconstructed from 96f9db88 and c46de77b.
Signed-off-by: trvon <git@trevon.dev>
* feat(p2p): exchange causal deltas
Persist peer pins and causal resume state before visibility so reconnects continue bounded authenticated replication without becoming remote query proxies.
Reconstructed from 1b8be146 and 41883338.
Signed-off-by: trvon <git@trevon.dev>
* feat(p2p): integrate direct sync services
Wire listener, reconnect, IPC, CLI, peer persistence, and daemon lifecycle around the authenticated direct transport.
Reconstructed from a19aaa44.
Signed-off-by: trvon <git@trevon.dev>
* fix(p2p): apply replicated document state
Index authenticated remote winners, propagate document tombstones, and keep service/test wiring complete across daemon paths.
Reconstructed from 4ca660c6, 443253f1, 3e9db24d, and 4b2ddcc5.
Signed-off-by: trvon <git@trevon.dev>
* test(p2p): register direct sync gates
Replace the superseded internal contract with operator-facing direct-sync guidance and register daemon integration coverage in the standard test layout.
Reconstructed from 360b9115 and 89416ccb.
Signed-off-by: trvon <git@trevon.dev>
* fix(p2p): require bounded peer enrollment
Bound aggregate sessions and worker lifetimes, create owner-only identity keys atomically, and reject unknown inbound peers until operators pin their mTLS identity.
Reconstructed from f91be528, 79de8e48, 4fb1e4b8, dc30a6c0, and b329edd3.
Signed-off-by: trvon <git@trevon.dev>
* fix(p2p): persist quarantine and deletes
Persist full-history writer commitments before invalidating forked state, and order local deletion through a durable authenticated outbox so tombstones survive crashes and reconnects.
Reconstructed from 74ce01e0 and 19888604.
Signed-off-by: trvon <git@trevon.dev>
* feat(p2p): add authenticated cold bootstrap
Install bounded zero-state snapshots only from previously operator-pinned peers, bind them to exact frontiers and full-history commitments, and recover staged journals before replication can resume.
Reconstructed from 8a9f5acf.
Signed-off-by: trvon <git@trevon.dev>
---------
Signed-off-by: trvon <git@trevon.dev>
* refactor(tuning): prune dormant runtime knobs
Remove unused tuning controls and adaptive search budget state. Extract testable ONNX slot reconciliation and search telemetry hydration seams, and correct the Apple Silicon sysctl value width.
* refactor(vector): retire dormant backends
Remove unused FAISS, model-infrastructure, VectorIndexManager, and TurboQuant paths while keeping Simeon PQ and float-vector storage.
Migrate unused V2.2 quantized schemas transactionally to V2.3. Databases containing non-default quantized formats or packed-code blobs fail closed with recovery guidance and no mutation.
* refactor(plugins): remove legacy SDK tooling
Remove the Ghidra and SDK submodules, their bespoke tests, and stale scan exclusions. Retain generic PluginProcess coverage against the repository mock fixture.
* build(vector): remove stale FAISS overrides
* refactor(ipc): remove dormant FSM snapshots
* refactor(ingest): unify bounded KG enqueue
* fix(plugins): harden allocation contracts
Keep ABI outputs null or fully initialized when allocations fail, and represent empty object payloads without relying on malloc(0).
* fix(storage): define empty object reads
Return a null buffer with zero length for successful empty-object reads instead of treating malloc(0) as an allocation contract.
* refactor(storage): trim through string views
* test: prune dormant harness helpers
Remove unused readiness and fixture helpers, and keep benchmark client-count doubling in a wide integer domain.
* chore(cpp): initialize analyzer-facing locals
* build(simeon): advance analyzer cleanup
* refactor(config): drop dead daemon client keys
* refactor(ipc): inject daemon client transport
Socket and in-process execution now share DaemonClient request dispatch, so CLI fallbacks do not need alternate service paths.
* refactor(retrieval): route CLI through daemon
Search and grep now use the same daemon handlers over socket or in-process transport. Live grep remains an explicit freshness inspection.
* refactor(graph): move CLI scope into daemon
Graph CLI requests now share daemon handlers across socket and in-process transports. CWD filtering occurs before pagination so scoped totals and pages stay correct.
* build: isolate test-only runtime controls
* refactor(metadata): decouple lock telemetry
* fix(services): return on request timeout
* fix(retrieval): enforce workspace scope
* fix(graph): scope nodes before pagination
* fix(grep): drop binary result lines
* fix(daemon): bound checkpoint wait
* fix(repair): use KG health checks
* fix(storage): always verify compressed CRC
* perf(vector): skip unchanged PQ snapshots
* build(zyp): exclude bench from default build
* build: protect Meson files from formatter
* refactor: remove unreachable production APIs
* refactor(daemon): remove dormant runtime state
* fix(vector): harden daemon provider calls
Timed-out coroutines must own their client and requests. Provider discovery must also fail instead of returning synthetic embeddings in production.
* refactor(storage): remove unusable adapters
* fix(cli): unify in-process add requests
* fix(tsan): preserve async worker progress
* test(cli): isolate daemon-dependent suites
* fix(repair): expose queue backoff progress
Retry failed session indexing and deletion without advancing fingerprints. Share the lifecycle shutdown budget so restart does not time out during a graceful drain.
* fix(release): repair package repo publish
Arch repo metadata and R2 uploads need to be reproducible from CI, with retry hardening for transient Cloudflare failures.
* fix(daemon): stop shutdown log storm
Init waiter must treat ShuttingDown as terminal so daemon shutdown tests do not flood Meson logs while the daemon is stopping.
* refactor(cli): delegate serve to MCP binary
Keep yams serve UX while removing MCP server linkage from yams-cli. The standalone MCP binary owns stdio serving and exits on stdin EOF.
* refactor(cli): move daemon path helpers
Keep CLI path resolution out of YamsDaemon so later link cuts can separate CLI and daemon internals. Full yams_daemon removal still needs a socket-only daemon-client split.
* refactor(daemon): split embedded client
* perf(search): add lean-guided benchmark gates
Add Lean topology/search models, ingestion ablation docs, and focused regression tests for topology and compression threshold behavior.
Extract search observability, keep benchmark-only ablation controls, and make retrieval quality benchmarks default to Simeon while preserving explicit ONNX opt-in.
* refactor(config): move daemon policies
* refactor(daemon): extract plugin runtime
* refactor(daemon): untangle service coordinators
Untangle CLI/MCP from daemon service coordinators and the daemon static lib.
- MetadataRepository uses a daemon-neutral tree-diff callback. The
GraphComponent wrapper moves to src/daemon/components/metadata_graph_bridge.cpp,
owned by the daemon library.
- WriteCoordinator moves to a dedicated build-only yams_write_coordinator
static lib. Drop ResourceGovernor coupling inside the writer loop; batch
sizing still respects config.maxBatchSize.
- AppContext gains enqueuePostIngest callback. ServiceManager wires it to
its own enqueuePostIngest; app services no longer call ServiceManager
directly.
- Embedding repair uses BulkIngestLease/BeginBulkIngestCallback instead of
concrete VectorIndexCoordinator::BulkScope. Daemon dispatchers provide a
lease factory backed by the real coordinator.
- CLI daemon_command uses config helpers instead of YamsDaemon::getXDGRuntimeDir.
- Remove explicit dependency('yams_daemon') from src/cli and tools/yams-mcp.
- Declare missing yams_extraction/yams_ingest deps on yams_app_services so
app services link cleanly without daemon.
Validated:
meson compile -C build/release -j4 yams-daemon yams-cli yams-mcp-server
ninja -C build/release -t commands tools/yams-cli/yams-cli tools/yams-mcp/yams-mcp-server | grep -c libyams_daemon.a -> 0
yams-cli --version, yams-mcp-server init, yams serve --help smoke tests
Size drop on macOS release: yams-cli 41M -> 32M, yams-mcp-server 37M -> 22M.
* fix(cli): keep stdin add local fallback
No embedded daemon now. When yams add - buffers stdin and daemon IPC is missing, store via local app services instead of failing socket_missing.
Validated:
meson compile -C build/debug -j4 yams-cli catch2_cli_commands
MESON_BUILD_ROOT=/Users/trevon/work/tools/yams/build/debug build/debug/tests/catch2_cli_commands 'AddCommand - reads from piped stdin and stores content'
MESON_BUILD_ROOT=/Users/trevon/work/tools/yams/build/debug build/debug/tests/catch2_cli_commands
meson compile -C build/release -j4 yams-cli yams-mcp-server
meson compile -C build/asan-api -j2 yams-cli catch2_cli_commands
MESON_BUILD_ROOT=/Users/trevon/work/tools/yams/build/asan-api ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 build/asan-api/tests/catch2_cli_commands 'AddCommand - reads from piped stdin and stores content'
ninja -C build/release -t commands tools/yams-cli/yams-cli tools/yams-mcp/yams-mcp-server | grep libyams_daemon.a -> no output
* feat(bench): add xplan multi-arm experiment harness
Unified plan runner, ablation axes, REPORT/ablation docs, and
KPI snapshot so optimization loops stop living in shell matrices.
* feat(formal): expand topology/search Lean contracts
Boundary models for search provenance, construction gates,
and cluster/route objectives so Lean guides correctness while
xplan ranks performance levers.
* feat(search,topology): construction purity + graph expansion path
Raise topology build defaults (min edge 0.25, max component 64) and
split oversized CC pieces so hybrid assist is quality-positive on
scifact. Add GraphNeighbors expansion with medoid-anchor fallback,
structured helpers instead of nested session logic, and BEIR-default
xplan plans. Search engine extraction (fanout/classic pipeline) rides
along as the integration surface for routing.
* feat(topology): add KMeans engine and SGC identity contracts
Register kmeans in the factory with deterministic spherical k-means
over embeddings, and enforce SGC document-identity invariants in
debug builds with matching unit coverage.
* feat(daemon): topology readiness gates and selectable reranker
Drive topologyReady from published epoch/freshness/rebuild/dirty
ingest state (not just KG drain), surface those fields in search
stats, and wire onnx|simeon|auto|none cross-reranker selection.
* chore(search): drop meta_path tuner knobs
Meta-path routing is gone; remove dead TunedParams fields and the
ConfigResolver test for [search.meta_path].
* feat(bench): add topology_purity_validate plan
Repeats=3 validation matrix for construction-purity winners vs
topo_off so MRR deltas can clear run-to-run noise.
* chore: format add_command and config include order
Mechanical clang-format / include reorder only; no behavior change.
* feat(bench): decision-grade xplan loop for search engine
Document search-engine measurement in AGENTS; harden xplan with
repeats=3 plans, quality×latency primaries, mechanism metrics from
debug.jsonl, multi-rep log promotion, and archive superseded topology
plans.
* feat(graph): denser product semantic_neighbor topK
Raise default YAMS_GRAPH_SEMANTIC_TOPK fallback from 4 to 8 so product
graphs leave fewer degree-0 docs for GraphNeighbors expansion.
* feat(search): extract topology assist and cross-rerank seams
Pull topology assist into search_topology_stage (seed merge, skip
reasons, optional vector seed probe default 0). Extract cross_rerank
compute from search_engine. Wire probe via typed config/ConfigResolver.
* feat(bench): product-path search engine ablation plans
Add topology-off component, vector-weight, and tiered-execution xplan
plans (repeats=3) and wire them into the AGENTS measurement loop table.
* fix(ci): repair pre-push fallbacks
* feat(bench): lock-free mid-run builds + multi-corpus gates
xplan skips meson rebuild when the quality binary exists (flock +
YAMS_BENCH_FORCE_BUILD override) to stop multi-arm link races. Add
graph_vector weight and nfcorpus transfer-gate plans; document in AGENTS.
* fix(daemon): keep SearchEngine alive via shared_ptr cache
getCachedEngine previously published a raw pointer that metrics polling
used through rebuild, racing destructor of SearchEngine::Impl (TSAN).
Store and return a shared_ptr snapshot so diagnostic readers pin the
engine for the duration of getConfig/health checks.
Also adds the cautious vectorWeight 0.30→0.20 multi-corpus xplan plan.
* feat(search): route vectors through cached topology
Build and reuse validated topology snapshots, rank routed members by query similarity, and narrow the vector leg to the accepted document set with explicit work diagnostics.
Remove the retired HDBSCAN engine and vendor dependency, preserve typed topology policy through search tuning, and cover the runtime/configuration contracts.
* feat(bench): consolidate search decision plans
Enforce deterministic experiment identity and declared baselines, expose routed/global vector work metrics, and register the focused benchmark contracts.
Promote the compact repeats=3 routing-budget gate, archive superseded topology mechanism screens, and add multi-corpus product calibration plans.
* feat(search): refine topology vector routing
* fix(search): propagate daemon readiness
* feat(bench): measure generalized memory health
* fix(cli): honor in-process fallbacks
* ci: harden smolvm debug builds
* perf(search): route topology through filtered ANN
* feat(bench): measure read-write pressure
* perf(metadata): index symbol lookups on read pool
* fix(daemon): isolate startup reads and maintenance
* feat(bench): replay isolated generalized corpora
* feat(search): add evidence routing pipeline
Replace the blocking smolvm pre-push path with host ASan/TSan validation, harden sanitizer-sensitive smoke tests, and add recovery cleanup for malformed SQLite WAL sidecars.
* refactor(search): unify topology evidence path
Remove classic and sidecar fusion lanes so topology augments the recall-preserving ANN and lexical candidate pool through one product path.
* refactor(bench): prune superseded search lanes
Measure the topology-plus-ANN product path with stable experiment identity and remove obsolete sidecar and tiered controls.
* docs(search): retire stale experiment notes
Keep the benchmark handoff focused on the topology-plus-ANN product lane and its active decision plans.
* refactor(search): prune dead search surfaces
Remove unused fusion, reranker, policy, and router abstractions.
Bound topology materialization and keep document trace payloads opt-in.
* fix(cli): clear build warnings
* fix(xplan): reject stale run identity
* refactor(search)!: remove legacy query stack
Replace the unused AST/ranker/post-processor surface with focused facet aggregation used by product search.
BREAKING CHANGE: removes query parser, result ranker, search filter, and SearchResults container APIs from yams_search.
* refactor(search): type value stage traces
Remove synthetic ComponentResult markers for embedding and concept stages. Preserve failure and timeout states instead of overwriting them with fake successful results.
* refactor(search): isolate benchmark env overlays
Move test-only search environment parsing behind an injectable internal adapter. Keep all existing overlay keys and tuner pin semantics while restoring SearchEngineBuilder to composition.
* refactor(search): unify topology routing options
Snapshot topology policy once at the assist boundary and pass it through routing sessions and diagnostics. Remove repeated scalar copying without changing routing behavior or metric keys.
* fix(search): unify tuner config ownership
Use one complete config-to-params overlay for runtime seeding and no-tuner policy resolution. Reseed attached tuners on setConfig so reported and executed configuration cannot diverge.
* fix(search): snapshot tuner state per query
Read tuned config, params, and state under one lock before routing. Add a TSAN correlation test so adaptive observation cannot produce a torn logical query snapshot.
* fix(search): enforce topology routing contract
Generated configs must preserve hybrid-assist shadow routing, while daemon-backed search must abstain when published topology artifacts are stale. Keep standalone validation and global ANN recall intact.
* fix(daemon): let SQLite reclaim its file
Directly resizing a live database from sampled page counts can truncate pages allocated by concurrent writers. VACUUM must remain the only file-reclamation mechanism.
* fix(metadata): tolerate missing symbol FTS
Readiness lookups can race an unavailable optional FTS migration. Retain the indexed fast path, but retry against the base table when prepare or cached-statement execution proves the capability absent.
* build(fuzz): drop removed query parser target
The optional AFL build still referenced the deleted legacy query parser API and could no longer compile.
* fix(bench): include writes in drain oracle
A zero post-ingest queue is not durable drain while coordinated writes remain queued or in flight.
* fix(daemon): vacuum padded WAL databases safely
* perf(daemon): coalesce metadata writes
* refactor(metadata): split repository ops
* fix(metadata): harden repository invariants
* feat: test coverage and continued refactor work in metadata code
* fix(graph): handle path-like explore queries
- prefer file path symbol lookup for graph explore\n- sanitize alias FTS queries before MATCH
* fix(daemon): quiesce shutdown background work
- Interrupt pending DB acquires before worker shutdown
- Wait for tracked background coroutines and timers to drain
* fix(daemon): cancel repair work on shutdown
Blocked embed and post-ingest queues could keep salvage and\nstuck-doc repair alive during shutdown. Consume cancellation in\nrepair retry loops and lock the integrated ServiceManager path\nwith regressions so shutdown fails fast instead of drifting.
* perf(metadata): reduce sqlite lock contention
* perf(metadata): flatten batch content retries
* fix(daemon): align shutdown wait budgets
A low shutdown timeout override could expire before ServiceManager\nfinished its 5s+30s WorkCoordinator join path. Share the join\nbudget constants, clamp graceful lifecycle waits to that floor,\nand lock the behavior with a daemon regression.
* perf(metadata): bound feedback write waits
* perf(metadata): prep batch content before tx
* fix(daemon): honor no-embeddings tasks
Post-ingest queue tasks dropped the per-task noEmbeddings flag,\nso focused extraction tests still emitted embed jobs and Windows\nCI exercised extra async paths. Preserve the flag through the bus,\nskip embed dispatch when requested, and wake the extraction poller\nimmediately after successful queueing.
* perf(daemon): batch fallback metadata writes
* feat(perf,docs): optimizations for metadata writes, prompt updates and test improvements
* perf(metadata): flatten insert retries
* fix(metadata): clear CI portability blockers
Use the shared portable ScopedEnvVar in metadata search-query helper tests, add mutators for test reuse, and normalize Statement::bindAll integral dispatch so long long arguments bind consistently across platforms. Add focused regression coverage for the bindAll long long path.
* perf(metadata): batch writes and steady bench
* fix(daemon): skip inactive post-ingest pollers
* perf(metadata): tighten tag and path caches
* perf(metadata): cache small batch upserts
* fix(daemon): harden embedded host teardown
* feat(docs): updating docs policies and language
* perf(metadata): tune doc update retries
* perf(metadata): bound download retries
* perf(metadata): bound download metadata waits
* bench(metadata): scope post-ingest content path
* fix(daemon): export embedded test hooks
* fix(daemon): harden test helper linkage
* fix(build): add missing standard includes
* test(storage): add crash-replay and handle-failure coverage for ReferenceCounter
Adds 6 test cases:
- Recovery after forced close preserves committed data
- Uncommitted transaction rolled back after forced close
- Corrupted database file rejected on open
- Consistent stats after interrupted commit recovery
- Multiple PENDING transactions survive force close
- WAL checkpoint survives forced close
* fix(ci): remove ccache — conflicts with OpenBLAS ASM as direct compiler
* fix(ci): use explicit CC/CXX paths to prevent Conan from using ccache directly
ccache was picked up by Conan/CMake as the compiler itself (/usr/bin/ccache)
rather than as a transparent wrapper, causing OpenBLAS ASM compilation to fail
with 'ccache: invalid option -- D'. Setting explicit compiler paths prevents this.
* fix(ci): add YAMS_SKIP_CCACHE to avoid OpenBLAS ASM ccache conflict
setup.sh wraps CC/CXX with ccache for faster incremental builds, but
this causes Conan/CMake to use ccache directly as the ASM compiler for
OpenBLAS, where it fails on -D flags. Add an opt-out gate in setup.sh
and set YAMS_SKIP_CCACHE=1 in .build.yml for sourcehut CI VMs.
* fix(ci): add libomp-dev, libfaiss-dev, libopenblas-dev for FAISS build
* fix(ci): limit meson compile to -j1 to prevent OOM on sourcehut VMs
* fix(build): restore hooks and shrink srht
Compile the storage library with YAMS_TESTING in test builds so
debug-only hooks link again.
Include the POSIX chmod header used by the storage utility
Catch2 coverage cases.
Prune SourceHut fallback stage archives to runtime files so
branch artifacts stay under the upload limit.
* refactor(storage): drop dead code, add coverage
* refactor(cli): polish grep/search graph UX
* fix(daemon): stabilize shutdown and CI paths
- force vector-backed tests to opt out of CI-wide disable flags
- reset reference prune statements between deletes
- make optional-plugin and POSIX-only test assumptions portable
- extract ServiceManager open/shutdown helpers for safer WAL recovery work
* test(daemon): add db vacuum coverage
* test(daemon): add WAL startup recovery
- cover stale WAL startup recovery in ServiceManager
- cover repeated restart cycles with seeded WAL sidecars
- assert daemon recovery reporting for corrupt data-dir DBs
- verify oversized metadata DBs shrink after WAL recovery shutdown
* refactor(grep): consolidate indexed content scan
* fix(grep): remove pre-scan candidate truncation
* refactor(runtime): split startup, harden storage
* feat(bench): add manifest retrieval corpus
* fix(bench): run fusion experiment on simeon
* fix(cli): sharpen search graph hints
* fix(build): refresh version metadata
* fix(ci): stabilize sync search and srht packaging
* fix(ci): unblock windows and shrink artifacts
* fix(ci): stabilize linux runtime checks
* fix(build): shrink srht manifest
