Commit graph

19 commits

Author SHA1 Message Date
Trevon
d398556ee5
fix(tests): replace frozen expectations and harden release gates (#323)
* test(search,daemon,storage): assert behaviour instead of frozen literals

Several suites encoded exact internal values that legitimate tuning and
ranking changes are allowed to move: SCIENTIFIC-era profile weights and
rrfK values, corpus-statistic figures, and hardcoded timeout/cancellation
numbers. They failed on every platform after those changes landed, so the
real behaviour regressions they should catch were hidden behind noise.

Replace the frozen literals with contract assertions: weights are finite,
non-negative and normalize, state selection still holds, capability-gated
sources stay zeroed, and hybrid results keep matching canonical hash/path
sets. Deadlines are not widened and the daemon paths they cover are
unchanged.

Also add independent corpus and adapter oracles, deterministic collector
cancellation and late-result gates, and synchronize ServiceManager
shutdown observation, so the tests fail for the behaviour they name.

The portability allowlist keeps its two reviewed raw test-env reads; only
their line numbers moved.

Signed-off-by: trvon <git@trevon.dev>

* ci(tests): summarise Meson failures without flooding the log

A failing test run printed the whole Meson suite output, so the actual
failing test names and their assertion messages were buried and the
uploaded logs were dominated by passing-test chatter.

Add a bounded reporter that extracts the failing test names plus their
surrounding diagnostics, keeps an overall cap on emitted lines, and is
unit-tested against representative Meson output. Wire it into the Tests
workflow so failures stay readable.

Signed-off-by: trvon <git@trevon.dev>

* build(deb): validate candidate identity, mapping and upgrade ordering

The Debian packaging scripts took filenames and versions on trust, so a
misnamed candidate or a wrong package ordering could pass validation while
the real install path was never exercised.

Fail closed on candidate identity, version and architecture, verify the
package-to-source mapping, seed the legacy-only layout, wait on the actual
services, and assert install/upgrade/remove ordering for both Debian and
RPM in an offline container. Cases that cannot be proven on a host are
reported as skipped rather than silently passing.

Signed-off-by: trvon <git@trevon.dev>

---------

Signed-off-by: trvon <git@trevon.dev>
2026-10-19 19:16:57 -06:00
trvon
93f2439471
build(plugins): export only the plugin C ABI; hide onnx_resource internals
Plugins and libyams_onnx_resource were linked with default visibility, so
their dynamic symbol tables exported everything they contained, including
statically linked third-party code. The S3 plugin exported 11,147
symbols (OpenSSL, libcurl, zlib, spdlog, nlohmann_json); the ONNX
resource library exported 895 (850 of them spdlog). Every loader uses
RTLD_LOCAL, so this was not a live interposition bug, but the export
table is the ABI and anything that binds RTLD_GLOBAL (or a future host)
could resolve into a plugin's private copies.

Plugins:
- plugins/exports/yams_plugin.map (ELF version script, global
  yams_plugin_*, local *) and yams_plugin.exp (ld64
  -exported_symbols_list) are applied to every plugin (s3, onnx, glint,
  zyp) with gnu_symbol_visibility: 'hidden' and --exclude-libs,ALL.
- The S3 legacy factory (yams_plugin_create/destroy_object_storage) is
  marked YAMS_PLUGIN_API so it stays exported under hidden visibility.

libyams_onnx_resource:
- Builds with hidden visibility; YAMS_ONNX_RESOURCE_API now expands to
  visibility("default") on GCC/Clang, so only OnnxConcurrencyRegistry is
  exported.
- It no longer links spdlog. Its messages reached the host log only
  because the executable's spdlog interposed the library's copy; with the
  copy hidden, the registry would have logged through a private default
  logger to stdout. The registry now takes a host log sink
  (OnnxConcurrencyRegistry::setLogSink); TuningManager and
  AbiModelProviderAdapter install a spdlog forwarder from the header-only
  onnx_registry_spdlog_sink.h. Without a sink, messages are dropped.

Tests (red first on grim, Debug):
- tests/plugins/plugin_export_surface_catch2_test.cpp dlopens the built
  S3 and zyp plugins and asserts the entry points resolve while libcurl,
  OpenSSL, zlib and zyp internals do not: 2/2 cases failed (11 of 31
  assertions) before, pass after.
- resource_governance_test gains a case for the log sink routing (did
  not compile before; passes after). `meson test --suite plugins`: 5/5.

Measured on grim (release, stripped staged payload):

  file                          bytes before  after       exports before  after
  libyams_object_storage_s3.so   7,854,064   6,240,112          11,147     10
  yams_zyp.so                       80,112      63,728             103      8
  libyams_onnx_resource.so         477,472      30,840             895     20
  bin/yams-cli                  33,712,056  33,859,512              80     39
  bin/yams-daemon               28,389,784  28,537,304              98     39

The executables now carry their own spdlog code paths instead of
exporting them for the resource library to interpose (+0.4%), and export
41 fewer symbols each. check_install_surface turns GREEN; budgets are
reset to the measured payload plus ~20%.

Signed-off-by: trvon <me@trevon.dev>
2026-10-01 18:51:34 -06:00
trvon
ee51abcb1a
refactor(symbols): remove in-tree tree-sitter symbol extractor plugin and tests
Remove legacy Tree-sitter symbol extractor plugin (plugins/symbol_extractor_treesitter),
the unused plugin-symbols build option, and associated tests and benchmark targets:
- Drop plugins/symbol_extractor_treesitter/ (grammar_loader, symbol_extractor, plugin_api).
- Remove plugin-symbols Meson option and packaging references.
- Remove dedicated symbol plugin tests and benchmark targets (symbol_extraction_bench,
  symbol_extractor_all_langs_catch2_test, symbol_extractor_edge_cases_catch2_test,
  symbol_extractor_solidity_catch2_test, symbol_extractor_query_test,
  symbol_extractor_validation_test).
- Clean up dead symbol extraction plugin helper and test from hnsw_entity_search_bench.
- Prune obsolete portability allowlist entries.

Signed-off-by: trvon <me@trevon.dev>
2026-09-22 07:37:30 -06:00
trvon
4e566f6387
fix: harden buffer and parser contracts
Validate Simeon buffer boundaries and replace ZYP reverse pointer arithmetic with bounded indexing. Keep release SIMD kernels unchecked while asserting contracts in debug builds.
2026-07-29 19:00:10 -04:00
trvon
8e8b0ec716
fix(test): harden sanitizer merge blockers 2026-06-29 11:34:02 -04:00
Trevon
b603738f44
Stabilize sanitizer lanes and daemon concurrency (#40)
* fix(repair): keep RPC progress alive

* fix(daemon): propagate DB open failure to FSM and add shutdown cancellation

* fix(db): classify FTS5 inverted-index errors as repairable, not transient locks

* fix(daemon): clean up stale corrupt files after salvage and suppress optional readiness keys

* test(db_recovery): update FTS5 classification test for new transient-lock exclusion

* test(gc): add garbage collector unit tests for collection, dry-run, scheduling

* test(repair): expand repair manager tests with hash-verified recovery and source ordering

* fix(gc): dry-run mode no longer increments blocksDeleted/bytesReclaimed

* fix(mock,mcp): add atomic bool for lock consistency, move field assignments

* style: fix unnecessary-copy, dead-store, and const-refable findings from Infer

* fix(daemon): increase default shutdown timeout from 15s to 120s

* fix(daemon): demote non-critical warnings to info (FTS5, VACUUM, WorkCoordinator timeout)

* fix(db): re-enable WAL auto-checkpoint to prevent unbounded WAL growth

* test(pool): add WAL auto-checkpoint and concurrency coverage for connection pool

* test(pool): add exhaustion, stats, lifecycle, and read-only connection pool tests

* test(pool): add tagging, WAL journal, reuse, and busy timeout tests

* test(metrics): relax session snippet assertion to tolerate stub behavior

* fix(test): stabilize sanitizer triage failures

* fix(build): wire version and zyp deps

* fix(daemon): close TSAN race clusters

* fix(test): guard optional zyp dep

* fix(test): address PR review findings
2026-06-01 12:14:43 -04:00
trvon
1fd57b2e3c
test: expand coverage and harden ci 2026-05-20 10:37:54 -04:00
trvon
4617bf4d8f
feat: further removing hnsw references and centralizing on simeon pq implementation 2026-04-22 01:10:57 -04:00
trvon
e2c8ba043b
fix: removing gtest and continued cleanup work of test suite 2026-04-14 12:33:17 -04:00
trvon
52b6ef293c
feat: improving daemon command socket handling and helpful logs 2026-03-19 13:41:13 -04:00
trvon
508f3c26d5
feat: submodule commits and test expansions 2026-03-06 16:52:52 -05:00
trvon
099bbef70d
feat: large update. Documentation update, config and plugin hardning and improvements related to loading trust. Exposed trust state in the doctor command with hints on improvements. Bug fixes and test improvements to reduce flakiness and TSAN issues 2026-02-26 11:21:29 -05:00
trvon
83bd1b7fc1
updating entity interface and adding new plugin for entity extraction 2026-01-08 15:06:58 -05:00
trvon
0557c9b33b
test improvements and linter run 2025-12-31 12:51:28 -05:00
trevon
3b58ac1743
test stabilizations for windows. search metadata/tags improvements 2025-11-26 21:45:50 -05:00
trvon
166d8c520d
staging v0.7.7 2025-11-01 16:02:12 -04:00
trvon
d19a5d7b64
bug fixes and optimizations for v0.7.7. Finalizing tests, benchmarks and bug fixes prior to release 2025-10-24 12:49:00 -04:00
trvon
8d23211af2
improvements for ci, setup.sh and updates to new symbol extraction plugin 2025-10-21 19:38:45 -04:00
trvon
b6ca31dfb0
adding symbol extraction work, simplying build with ./setup.sh, updated plugin tests with new meson shard and other goodies 2025-10-20 16:11:31 -04:00