Commit graph

7 commits

Author SHA1 Message Date
Trevon
7076e4b283
feat(daemon): shutdown hardening + test helper linkage fixes (#44)
* perf(daemon): coalesce metadata writes

* refactor(metadata): split repository ops

* fix(metadata): harden repository invariants

* feat: test coverage and continued refactor work in metadata code

* fix(graph): handle path-like explore queries

- prefer file path symbol lookup for graph explore\n- sanitize alias FTS queries before MATCH

* fix(daemon): quiesce shutdown background work

- Interrupt pending DB acquires before worker shutdown
- Wait for tracked background coroutines and timers to drain

* fix(daemon): cancel repair work on shutdown

Blocked embed and post-ingest queues could keep salvage and\nstuck-doc repair alive during shutdown. Consume cancellation in\nrepair retry loops and lock the integrated ServiceManager path\nwith regressions so shutdown fails fast instead of drifting.

* perf(metadata): reduce sqlite lock contention

* perf(metadata): flatten batch content retries

* fix(daemon): align shutdown wait budgets

A low shutdown timeout override could expire before ServiceManager\nfinished its 5s+30s WorkCoordinator join path. Share the join\nbudget constants, clamp graceful lifecycle waits to that floor,\nand lock the behavior with a daemon regression.

* perf(metadata): bound feedback write waits

* perf(metadata): prep batch content before tx

* fix(daemon): honor no-embeddings tasks

Post-ingest queue tasks dropped the per-task noEmbeddings flag,\nso focused extraction tests still emitted embed jobs and Windows\nCI exercised extra async paths. Preserve the flag through the bus,\nskip embed dispatch when requested, and wake the extraction poller\nimmediately after successful queueing.

* perf(daemon): batch fallback metadata writes

* feat(perf,docs): optimizations for metadata writes, prompt updates and test improvements

* perf(metadata): flatten insert retries

* fix(metadata): clear CI portability blockers

Use the shared portable ScopedEnvVar in metadata search-query helper tests, add mutators for test reuse, and normalize Statement::bindAll integral dispatch so long long arguments bind consistently across platforms. Add focused regression coverage for the bindAll long long path.

* perf(metadata): batch writes and steady bench

* fix(daemon): skip inactive post-ingest pollers

* perf(metadata): tighten tag and path caches

* perf(metadata): cache small batch upserts

* fix(daemon): harden embedded host teardown

* feat(docs): updating docs policies and language

* perf(metadata): tune doc update retries

* perf(metadata): bound download retries

* perf(metadata): bound download metadata waits

* bench(metadata): scope post-ingest content path

* fix(daemon): export embedded test hooks

* fix(daemon): harden test helper linkage

* fix(build): add missing standard includes
2026-06-08 16:29:44 -04:00
Trevon
08322369cb
Search/grep UX, runtime hardening, and CI stabilization (#39)
* test(storage): add crash-replay and handle-failure coverage for ReferenceCounter

Adds 6 test cases:
- Recovery after forced close preserves committed data
- Uncommitted transaction rolled back after forced close
- Corrupted database file rejected on open
- Consistent stats after interrupted commit recovery
- Multiple PENDING transactions survive force close
- WAL checkpoint survives forced close

* fix(ci): remove ccache — conflicts with OpenBLAS ASM as direct compiler

* fix(ci): use explicit CC/CXX paths to prevent Conan from using ccache directly

ccache was picked up by Conan/CMake as the compiler itself (/usr/bin/ccache)
rather than as a transparent wrapper, causing OpenBLAS ASM compilation to fail
with 'ccache: invalid option -- D'. Setting explicit compiler paths prevents this.

* fix(ci): add YAMS_SKIP_CCACHE to avoid OpenBLAS ASM ccache conflict

setup.sh wraps CC/CXX with ccache for faster incremental builds, but
this causes Conan/CMake to use ccache directly as the ASM compiler for
OpenBLAS, where it fails on -D flags. Add an opt-out gate in setup.sh
and set YAMS_SKIP_CCACHE=1 in .build.yml for sourcehut CI VMs.

* fix(ci): add libomp-dev, libfaiss-dev, libopenblas-dev for FAISS build

* fix(ci): limit meson compile to -j1 to prevent OOM on sourcehut VMs

* fix(build): restore hooks and shrink srht

Compile the storage library with YAMS_TESTING in test builds so
debug-only hooks link again.

Include the POSIX chmod header used by the storage utility
Catch2 coverage cases.

Prune SourceHut fallback stage archives to runtime files so
branch artifacts stay under the upload limit.

* refactor(storage): drop dead code, add coverage

* refactor(cli): polish grep/search graph UX

* fix(daemon): stabilize shutdown and CI paths

- force vector-backed tests to opt out of CI-wide disable flags
- reset reference prune statements between deletes
- make optional-plugin and POSIX-only test assumptions portable
- extract ServiceManager open/shutdown helpers for safer WAL recovery work

* test(daemon): add db vacuum coverage

* test(daemon): add WAL startup recovery

- cover stale WAL startup recovery in ServiceManager
- cover repeated restart cycles with seeded WAL sidecars
- assert daemon recovery reporting for corrupt data-dir DBs
- verify oversized metadata DBs shrink after WAL recovery shutdown

* refactor(grep): consolidate indexed content scan

* fix(grep): remove pre-scan candidate truncation

* refactor(runtime): split startup, harden storage

* feat(bench): add manifest retrieval corpus

* fix(bench): run fusion experiment on simeon

* fix(cli): sharpen search graph hints

* fix(build): refresh version metadata

* fix(ci): stabilize sync search and srht packaging

* fix(ci): unblock windows and shrink artifacts

* fix(ci): stabilize linux runtime checks

* fix(build): shrink srht manifest
2026-05-27 12:15:17 -04:00
trvon
8772cc8e54
fix(docs,ci): addressing breaking changes in zig v0.16.x in Dockerfile, docs update and code cleanup 2026-04-25 11:27:18 -04:00
trvon
9c23fcf869
feat: large commit, new tune command, search tuning and improved corpus learning with persistent values 2026-04-15 17:10:58 -04:00
trvon
e261420351
fix: suspected deadlock fixes in repair, bugs addressed in request logic. Continued triage from infer output and test failures 2026-04-11 18:45:22 -04:00
trvon
fc284f3b37
fix: resolving TSAN issues and expanding graph building to use gliner data 2026-02-19 14:52:01 -05:00
trevon
7b24b26373
graph traversal improvements and continued test migration 2025-12-17 22:26:26 -05:00