Commit graph

28 commits

Author SHA1 Message Date
trvon
d67d5d63ac
Remediate Fold SAST findings
Centralize duplicated helpers and harden numeric, timing, IPC, SQLite retry, config, plugin, metrics, and async content-store paths identified during Fold SAST triage.\n\nAdd targeted tests for shared utilities and preserve per-task YAMS checkpoints.
2026-05-27 22:41:44 -04:00
Trevon
bfe7cd94ca
test: expand storage-readiness coverage — 48.1%→59.8% broad, 74.3% narrow (#38)
* fix(daemon): avoid socket cancel on stop

Signal active sockets with shutdown only so pending I/O wakes without entering Boost.Asio's reactor cancel path during teardown.

* fix(metadata): open read pools read-only

* ci: fix coverage report generation

* fix(metadata): tag unscoped pool holders

* test(daemon): avoid ipc teardown takeover

* ci: skip plugin coverage objects

* fix(daemon): repair completed FTS5 ghosts

* build(vector): guard unusable FAISS headers

* ci: use clang gcov for coverage

* test(daemon): stabilize arm CI flakes

* Harden search and compression numeric config

* fix(daemon): reconcile orphan vector embeddings

* fix: address vector cleanup review comments

* chore(main): release 0.16.1

* Harden search ranking divisors

* Harden snippet distance and command registration

* test(storage): add readiness coverage gate

Document the distributed sync readiness surface and add direct adapter, refcount, and GC failure-path coverage.

* test(storage): cover strict resolver fallback

* fix(search): tighten retrieval ranking

Honor component weights in default RRF, over-fetch vector chunks before document aggregation, and persist topology centroids for sparse-guided routing.

* fix(search): route weak queries through topology

* fix(search): improve rerank and weak queries

Wire provider-backed cross reranking, add direct Simeon candidates for weak lexical pools, and report stage-level retrieval metrics.

* fix(storage): honor explicit plugin names

Add readiness coverage for factory, URL backend, and compressed storage failure paths.

* fix(storage): avoid remote size listing

* fix(api): rollback failed content adds

* test(storage): cover engine readiness

* test(storage): cover compressed readiness

* fix(core): harden delete and recovery paths

Route prune and local delete fallback through DocumentService so content, metadata, vectors, and KG cleanup stay consistent.

Treat transient SQLite quick_check lock failures as ResourceBusy, refresh IPC activity on streaming writes, and preserve reranker config in tuner seeding.

* test(storage): cover URL backend readiness

* fix(api): harden content rollback

* feat(storage): add list seam, URL listing, and write-edge coverage

* feat(api): reconcile ambiguous remote manifest writes

* feat(storage): add deep compressed scan validation

* test(storage): add plugin loader error path coverage

* fix(gc): prune reference rows after collection

* test(api,storage): add health, compact, gc, and verify edge coverage

* docs: update storage-readiness coverage to 71.8%

* test(metadata): add connection pool shutdown-while-leased coverage

* test(metadata): add tree builder snapshot root stability coverage

* test(vector): add before-init and after-close operation rejection coverage

* test(metadata): add database post-close, open-failure, and cache-clear coverage

* test(integrity): add P2P wrong-hash rejection + batch matrix resolution

* docs: audit and resolve all matrix rows — 18 fully covered

* test(metadata): add failed-migration and rollback coverage

* test(storage): add getBlockSize-invalid-key, empty-verify, PUT/DELETE retry coverage

* feat(storage): add file-open and rename failure hooks + ContentStore retrieve edge coverage

* docs: add broad-surface coverage with per-subsystem breakdown

* test(integrity): add chunk validator manifest validation coverage

* test(vector): add batch insert, delete, and update operation coverage

* docs: update integrity subsystem coverage to 46.4%

* docs: consolidate coverage report — clean matrix, remove boilerplate

* docs: update coverage with complete-suite numbers — 59.8% broad, 74.3% narrow

* test(storage): add URLBackend HEAD exists with 2xx coverage

* bump submodules

* fix: address PR review — curl_slist leak, ec ordering, CAS rollback, production hooks

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-25 22:58:15 -04:00
trvon
39576186ee
test: instrumentation expansions for ingestion speed experiments and tuning improvements 2026-04-26 19:55:29 -04:00
trvon
f825bc3cf5
feat: ci/test regressions updates, reinstated onnxruntime (simeon will still be default), expanding topology instrumentation in search, newsletter update with throughput comparison 2026-04-26 17:03:37 -04:00
trvon
7d29f96df5
fix(ci): test fixes 2026-04-26 01:02:36 -04:00
trvon
8417340289
feat: extending instrumentation for memory and cluster profiling. COntinue UX improvements for grep and search cli commands and mcp 2026-04-24 08:31:16 -04:00
trvon
926ef8a640
feat: memory optimizations, and topology tuning improvements 2026-04-23 20:06:45 -04:00
trvon
bdcff5b636
fix: HNSW and topology rebuilds are now batched in repair. Cleaning up from previous migration work 2026-04-21 13:58:10 -04:00
trvon
e44c28b2ff
feat: extractor reranker and helpers from search engine. Establishing dual paths for simeon and onnxruntime to present heavy vs light system trade offs 2026-04-21 11:35:31 -04:00
trvon
6c2c31f2bc
fix: cli clean up of old comments and duplicate code. search improvements as we migrate yams to simeon. 2026-04-21 10:20:01 -04:00
trvon
e4a8639ef6
feat: tuning topology system to work better with simeon via configuration 2026-04-19 20:10:46 -04:00
trvon
33821d93f1
feat: test hardening, added benchmark for multiclient and improve stability to support up to 16 clients before performance degrades, improved parallization and request handling in daemon ingestion pipeline 2026-02-16 15:46:16 -05:00
trvon
8f38ee76be
feat: updating metrics for search and retrieval for future learning system. Additional tuning for search performance 2026-02-13 21:09:26 -05:00
trvon
a44c2fb4af
feat: implementing embedding chunking strategy and two staged ingestion that processes then distributes document data 2026-02-12 00:46:04 -05:00
trvon
b071112836
feat: test coverage improvements. Scaling bug addresses 2026-02-04 20:20:18 -05:00
trvon
79ea83eac8
fix: test fixes annd windows build downgrade to c++20 2026-02-02 02:15:09 -05:00
trvon
0de5bdf466
bug fixes and additional tuning fixes 2026-01-26 19:30:38 -05:00
trvon
e309d96414
bug fixes for daemon, ui fixes for grep and search 2026-01-25 22:52:25 -05:00
trvon
5af411d45c
test improvements 2026-01-23 23:00:56 -05:00
trvon
f654c26e92
grep and build optimizations, windows build fixes 2026-01-23 09:39:06 -05:00
trvon
c7fb151f3d
skipping v0.7.11 and going straight to v0.8.0. Staging pre-release 2026-01-22 14:21:32 -05:00
trvon
2f84e362b7
concurrency limits fixed 2026-01-22 01:02:43 -05:00
trvon
5445567041
improving tune advisor hardware calculation for all components 2026-01-20 15:30:38 -05:00
trvon
aacd7af41f
running linter 2025-12-13 16:27:07 -05:00
trevon
0ab2bb2b36
ingestion optimizations, continued test migrations 2025-12-12 12:50:56 -05:00
trevon
e47142343e
updating tests / completing catch2 migration. plugin trust fixes 2025-12-12 10:58:48 -05:00
trevon
798b736156
checkpoint. working on plugin system external (sdk supported plugin) loading, updated graph implementation for binary extraction via ghidra plugin 2025-12-04 21:12:40 -05:00
trevon
c4e8099f07
improvements for daemon service management 2025-12-02 12:03:40 -05:00