Commit graph

50 commits

Author SHA1 Message Date
Trevon
08322369cb
Search/grep UX, runtime hardening, and CI stabilization (#39)
* test(storage): add crash-replay and handle-failure coverage for ReferenceCounter

Adds 6 test cases:
- Recovery after forced close preserves committed data
- Uncommitted transaction rolled back after forced close
- Corrupted database file rejected on open
- Consistent stats after interrupted commit recovery
- Multiple PENDING transactions survive force close
- WAL checkpoint survives forced close

* fix(ci): remove ccache — conflicts with OpenBLAS ASM as direct compiler

* fix(ci): use explicit CC/CXX paths to prevent Conan from using ccache directly

ccache was picked up by Conan/CMake as the compiler itself (/usr/bin/ccache)
rather than as a transparent wrapper, causing OpenBLAS ASM compilation to fail
with 'ccache: invalid option -- D'. Setting explicit compiler paths prevents this.

* fix(ci): add YAMS_SKIP_CCACHE to avoid OpenBLAS ASM ccache conflict

setup.sh wraps CC/CXX with ccache for faster incremental builds, but
this causes Conan/CMake to use ccache directly as the ASM compiler for
OpenBLAS, where it fails on -D flags. Add an opt-out gate in setup.sh
and set YAMS_SKIP_CCACHE=1 in .build.yml for sourcehut CI VMs.

* fix(ci): add libomp-dev, libfaiss-dev, libopenblas-dev for FAISS build

* fix(ci): limit meson compile to -j1 to prevent OOM on sourcehut VMs

* fix(build): restore hooks and shrink srht

Compile the storage library with YAMS_TESTING in test builds so
debug-only hooks link again.

Include the POSIX chmod header used by the storage utility
Catch2 coverage cases.

Prune SourceHut fallback stage archives to runtime files so
branch artifacts stay under the upload limit.

* refactor(storage): drop dead code, add coverage

* refactor(cli): polish grep/search graph UX

* fix(daemon): stabilize shutdown and CI paths

- force vector-backed tests to opt out of CI-wide disable flags
- reset reference prune statements between deletes
- make optional-plugin and POSIX-only test assumptions portable
- extract ServiceManager open/shutdown helpers for safer WAL recovery work

* test(daemon): add db vacuum coverage

* test(daemon): add WAL startup recovery

- cover stale WAL startup recovery in ServiceManager
- cover repeated restart cycles with seeded WAL sidecars
- assert daemon recovery reporting for corrupt data-dir DBs
- verify oversized metadata DBs shrink after WAL recovery shutdown

* refactor(grep): consolidate indexed content scan

* fix(grep): remove pre-scan candidate truncation

* refactor(runtime): split startup, harden storage

* feat(bench): add manifest retrieval corpus

* fix(bench): run fusion experiment on simeon

* fix(cli): sharpen search graph hints

* fix(build): refresh version metadata

* fix(ci): stabilize sync search and srht packaging

* fix(ci): unblock windows and shrink artifacts

* fix(ci): stabilize linux runtime checks

* fix(build): shrink srht manifest
2026-05-27 12:15:17 -04:00
trvon
9b2c1b8631
docs: benchmark updates, small regression fixes in multiclient communication, ci fixes for windows build for pinned boost resolution, additional writer coordinator migrations 2026-04-30 14:35:21 -04:00
trvon
84015b39b7
build(ci,tests): cleaning build system and updates related to ci test failures and compat 2026-04-25 14:48:44 -04:00
trvon
d7ab4844a1
fix: small fixes for benchmarks and documentation update 2026-04-17 22:32:04 -04:00
trvon
8777a27f3c
feat: build improvements, docs updated and optimizations for cli latency 2026-04-09 22:43:02 -04:00
trvon
e6e0b39fd8
feat: additional optimizations and improvements for package validation and ingestion speed 2026-04-08 12:51:04 -04:00
trvon
e09778b7fe
fix: improvements for embedding generation (embedding gemma specific issue) and Dockerfile regression fix 2026-04-07 15:54:52 -04:00
trvon
09c8e0f959
feat: profile for building mobile bindings 2026-04-06 13:28:34 -04:00
trvon
970f253380
fix: cleaning up macos build detection logic 2026-04-02 15:43:33 -04:00
trvon
147f5ca052
feat: moving changelogs to archive, implementing TurboQuant then will integrate in vector engine 2026-03-25 08:15:26 -04:00
trvon
0261b90a7b
refactor(build): centralize dependency discovery and trim Meson duplication 2026-03-23 15:41:13 -04:00
trvon
d86ac665cd
fix: updating conanfile for onnxruntime to respect system version, updated tests for windows compat, update vectors to enforce wal check point so files does not explode in size 2026-02-21 12:20:50 -05:00
trvon
6a90f8c934
feat: test improvements, ASAN fixes, and onnx model performance improvements 2026-02-17 08:59:53 -05:00
trvon
1d7b15d41d
feat: merge and site update 2026-02-14 11:59:36 -05:00
trvon
a0fd82e25c fix: Fixing issues that break windows release build. This should allow new msvc version Working on debug builds. 2026-02-10 15:03:39 -08:00
trvon
e0612298d3
fix: fixing rocm builds, removing dead code and some optimizations 2026-02-05 18:33:54 -05:00
trvon
357720407b
fixing rocm gpu build logic 2026-02-05 12:07:10 -05:00
trvon
8835d6704e
fix: removing mobile bindings from builds 2026-01-31 17:15:25 -05:00
trvon
bbe5217101
ci improvements for faster builds 2026-01-21 15:53:19 -05:00
trvon
1abb382ec7
bug fixes for rocm and test improvements for windows 2026-01-21 12:13:19 -05:00
trvon
cb256f5066
fixing windows debug setup and adding rocm build support 2026-01-21 10:39:07 -05:00
trvon
b3fbff0457
fixing model logic to reduce pool size when using cpu 2026-01-20 13:05:42 -05:00
trvon
6e9221c8b2 fixing for building libsql 2026-01-20 01:05:57 +00:00
trvon
0dd0a87132
adding compile support for onnxruntime 2026-01-17 16:05:20 -05:00
trvon
ae52598a58
batching improvements. using libsql experimental concurrency for insert. still testing search regressions. 2026-01-16 16:45:43 -05:00
trvon
83bd1b7fc1
updating entity interface and adding new plugin for entity extraction 2026-01-08 15:06:58 -05:00
trvon
4321f29679
profiling improvements, ft5s bug fixes for retrieval, and benchmark optimizations 2026-01-06 14:57:13 -05:00
trvon
c7c48e8934
removed hnsw vector manager. search optimizations 2026-01-05 23:39:18 -05:00
trvon
b912ea0b83
compression bug fix 2026-01-03 22:38:28 -05:00
trvon
bc71d36534
migrated pdf plugin, daemon shutdown logic fix 2025-12-31 14:46:07 -05:00
trevon
48dc0dfa89
removing gtest migrated tests and continuing with test migration 2025-12-16 10:50:18 -05:00
trevon
ebe81823db
improvements for daemon lifecycle, tree repair, and mcp regression 2025-11-26 14:22:35 -05:00
trevon
e6991b4813
Merge branch 'main' of https://github.com/trvon/yams 2025-11-25 10:41:19 -05:00
trvon
5b23afa99d
adding dedicate graph component for graph operations, updating socket stale logic and fixing CI issues in release. 2025-11-16 13:31:04 -05:00
trvon
41bb390be9
hot fix v0.7.9 2025-11-15 11:02:30 -05:00
trvon
b06e9335b9
staging v0.7.8. running benchmarks on updated search engine and improving code for testing/benchmarking 2025-11-13 10:27:10 -05:00
trvon
12fc1b8a50
resolved concurrency bugs in daemon. added fuzzing work in progress per issue #8 2025-11-11 10:10:07 -05:00
trvon
4702a345ad
ci fixes, test updates and documentation imrpovements 2025-11-07 13:20:58 -05:00
trvon
eba47216a3
search engine improvements 2025-11-07 11:08:32 -05:00
trvon
a7c14f54cc
Bug fixes for prune command and test improvements. 2025-11-06 09:13:29 -05:00
trvon
0cd1bd0b4c
stabilizations. removing http as I am not using it. Focusing on streamlining build and system 2025-11-05 10:52:27 -05:00
trvon
97cfe741e5
clang tidy fixes and macos crash fix 2025-11-04 18:31:58 -05:00
trvon
7d9dcd986d
additional ci fixes for docker and meson setup 2025-11-01 16:46:31 -04:00
trvon
166d8c520d
staging v0.7.7 2025-11-01 16:02:12 -04:00
trvon
ed7f10fb36
CI updates 2025-10-30 00:10:03 -04:00
trvon
bbf23bb190
test migrations to catch2, service manager + socket server refactor for better startup performance. search bug fixes 2025-10-29 23:08:09 -04:00
trvon
6971daf41c
staging docs 2025-10-23 23:01:29 -04:00
trvon
8d23211af2
improvements for ci, setup.sh and updates to new symbol extraction plugin 2025-10-21 19:38:45 -04:00
trvon
43289f8aee
fixing setup for macos 2025-10-20 16:32:17 -04:00
trvon
b6ca31dfb0
adding symbol extraction work, simplying build with ./setup.sh, updated plugin tests with new meson shard and other goodies 2025-10-20 16:11:31 -04:00