2d01ba314c* fix(core): add assertions and harden daemon lifecycle * fix(daemon): tighten shutdown lifecycle invariants * fix(test): add PressureLimitedPoller capability-sleep shutdown test * refactor(vector): harden backend contracts Split the vector backend surface into narrower contracts and move entity read paths to Result<> so callers can distinguish backend errors from empty results. Also fixes the TurboQuant lazy-init race, makes VectorDatabase error state thread-safe, wires eager index preparation, and adds regression plus memory-scaling benchmark coverage. * fix(vector): address audit findings plus hardening Fix the five confirmed bugs: coroutine lifetime hazard in embedding_generator (await_with_timeout outliving captured state), VectorSystemManager init retry latch poisoning, incremental HNSW persistence race, vec0 registry key collisions, and simeon PMI output_dim/config() drift. Add invariant hardening: PostIngestQueue stage validation + numChunks assertion, VectorSystemManager success postcondition, VectorDatabase TurboQuant non-null assertions on all quantize/ dequantize paths, and sqlite_vec_backend Simeon PQ range and divisibility preconditions. Also adds regression coverage for init-retry and lock-busy paths and a locked node snapshot helper for safe incremental HNSW persistence. * fix(test): add PressureLimitedPoller wake-timer shutdown tests * chore(vector): resolve clang-tidy hygiene across queue, vdb, and sqlite-vec Replace unprotected std::getenv calls with locked getenvCopy helpers, add debug-level logging to previously-silent catch blocks in PostIngestQueue, VectorDatabase, and SqliteVecBackend, wrap PostIngestQueue timer-detach allocations in try-catch, and fix unsigned-widening / optional-deref / unused-include hygiene. Verified with LSP diagnostics and rebuilds of catch2_vector_submodule and catch2_daemon_component_submodule. * fix(daemon): harden PressureLimitedPoller inFlightCounter null check * fix(daemon): harden PostIngestQueue wait_until deadline checks * fix(daemon): clean up recovery sentinels regardless of salvage count Previously the .recovered-* sentinel and .corrupt-* file cleanup was gated on documentsSalvaged > 0, leaving quarantine artifacts to accumulate on repeated crash/recovery cycles where the corrupt DB had 0 salvageable docs. Move cleanup outside the salvage-count guard so stale quarantine artifacts are always removed after examination. * feat(vec0): restore persisted ANN index on daemon restart After vec0 rebuild, probe for vectors_<dim>_hnsw_meta shadow tables and load a previously-persisted HNSW index. When found, the ANN index is set directly on the vec0 table, skipping the O(n log n) rebuild that was previously paid on every daemon restart. Removes the stub restoreVec0AnnIndexIfPresentUnlocked() in favor of inline restoration in rebuildVec0DimUnlocked(), which has the vec0 table already created and ready for ann_index assignment. * fix: enable DCHECK by default and assert metadata counter consistency * fix(services): assert document deletion ordering content-before-metadata * fix(test): add fork-based assertion validation tests * fix(test): expand assertion tests for POSTCONDITION, DCHECK, UNREACHABLE * fix(ci): enable -Denable-dcheck=true in all test CI lanes * fix(test): address assertion review findings * refactor(daemon): isolate poller config assertion checks * refactor(services): reduce deletion and poller complexity * style(ci): normalize test workflow spacing * refactor(services): extract list query helpers * refactor(services): extract list hydration helpers * refactor(daemon): extract metrics snapshot helpers * refactor(daemon): extract metrics counter snapshot * refactor(daemon): extract metrics resource snapshot * feat(graph): add daemon-backed explore Wire graph explore through daemon IPC, keep CLI rendering stable, and fall back locally when daemon transport or protocol skew fails. Also add graph context service/tests, improve graph follow-up hints, and keep release builds working with the current JSON library. * fix(services): prevent prune delete metadata orphans * fix(cli): honor data-dir in graph explore - bypass daemon explore when the CLI pins a data dir so\n temp fixtures do not resolve against a long-lived global DB\n- guard snippet truncation and ratio math to remove Fold-\n reported underflow and div-by-zero paths\n- clamp zero-based symbol lines before rendering graph\n context * fix(daemon): make lifecycle stop completion safe * fix(daemon): gate db-open startup against shutdown * fix(metadata): serialize FTS indexed counter updates * test(daemon): simulate vector lock contention cross-process * test(daemon): isolate post-ingest identity batch * style(daemon): normalize test include order * refactor(graph): split cli graph helpers Keep graph command paths smaller and easier to evolve. Lock graph explore/topology contracts into focused presenter, dispatcher, and support tests so follow-up refactors keep the current IPC and hot-path behavior intact. * fix(config): harden expand_tilde and fix Windows XDG state path doubling - expand_tilde: handle bare ~ and ~/ explicitly to avoid std::out_of_range on substr(2) for single-char input - getXDGStateHome: on Windows, go up two levels from %LOCALAPPDATA%\yams\state to %LOCALAPPDATA% so callers' xdg/yams append doesn't double the yams segment * fix(static-analysis): add logging to empty catch blocks and fix nosemgrep syntax * fix(static-analysis): strip trailing text from nosemgrep comments * fix(doctor): add logging to daemon_check empty catch blocks * fix(db_repair): add logging to empty catch blocks * fix(chunking): add precondition checks for rabin polynomial and windowSize * fix(safety): add null precondition checks for Fold proved null-deref sites * fix(daemon): widen init-failure dispatch, add null/dbz preconditions InitializationFailedEvent races with shutdown — async init fails after teardown begins. Widen canDispatch to accept Ready/ShuttingDown. Add static_assert on SnapshotRing N>0. Add YAMS_PRECONDITION null guards on hex/stages/data/bytes. * ci(ubsan): use restricted build targets to avoid runner timeout * fix(daemon): detach wake timers on Windows stop path to prevent SIGSEGV * fix(config): cast parseTomlU32 to size_t for batch fields * fix(ci): use static helper for Windows timer leak, restrict ubsan tests
65 lines
7.4 KiB
Meson
65 lines
7.4 KiB
Meson
option('build-cli', type: 'boolean', value: true, description: 'Build CLI tools')
|
||
option('build-mcp-server', type: 'boolean', value: true, description: 'Build MCP server')
|
||
option('wasi', type: 'boolean', value: false, description: 'Build a WASI-friendly subset (no daemon IPC, no dynamic plugins, limited features)')
|
||
option('enable-lzma', type: 'feature', value: 'auto', description: 'Enable LZMA via liblzma (xz)')
|
||
option('enable-onnx', type: 'feature', value: 'auto', description: 'ONNX Runtime integration in vector library (auto-detect)')
|
||
option('require-fts5', type: 'boolean', value: true, description: 'Fail configure if sqlite3 lacks FTS5')
|
||
option('require-sqlite-vec', type: 'boolean', value: true, description: 'Require sqlite-vec-cpp submodule for static vector DB; when true and not found, Meson will error out (recommended). Set to false only if providing a runtime module via YAMS_SQLITE_VEC_MODULE.')
|
||
option('database-backend', type: 'combo', choices: ['libsql', 'sqlite'], value: 'libsql', description: 'Database backend: libsql (default, MVCC concurrent writes) or sqlite (standard SQLite3)')
|
||
option('conan-generators-dir', type: 'string', value: '', description: 'Optional path (relative to source root) to Conan generator outputs; when empty Meson searches under the build directory (e.g. build/debug/conan).')
|
||
option('cli11-cmake-path', type: 'string', value: '', description: 'Optional path to CLI11 CMake package directory (e.g., /opt/homebrew/Cellar/cli11/2.4.1/lib/cmake/CLI11). When set, Meson will add this to cmake_module_path for CLI11 discovery.')
|
||
option('onnxruntime-cmake-path', type: 'string', value: '', description: 'Optional path to onnxruntime CMake package dir (e.g., /opt/homebrew/opt/onnxruntime/lib/cmake/onnxruntime). Used to augment cmake_module_path for ORT discovery in Meson.')
|
||
option('protoc-path', type: 'string', value: '', description: 'Optional path to a build-machine protoc binary for protobuf code generation, primarily for cross-builds.')
|
||
option('build-plugins', type: 'boolean', value: true, description: 'Build plugins under plugins/')
|
||
option('plugin-s3', type: 'boolean', value: true, description: 'Build object_storage_s3 plugin')
|
||
option('plugin-onnx', type: 'boolean', value: true, description: 'Build ONNX plugin (requires onnxruntime and vector lib)')
|
||
option('build-tests', type: 'boolean', value: false, description: 'Build unit/integration tests under tests/')
|
||
option('build-fuzzers', type: 'boolean', value: false, description: 'Build fuzzing harnesses under tools/fuzzing/')
|
||
option('build-benchmarks', type: 'boolean', value: false, description: 'Build benchmark utilities and executables under src/benchmarks')
|
||
option('plugin-symbols', type: 'boolean', value: true, description: 'Build Tree-sitter symbol extractor plugin')
|
||
option('plugin-zyp', type: 'boolean', value: true, description: 'Build zyp PDF extractor plugin (requires Zig 0.15+)')
|
||
option('plugin-glint', type: 'boolean', value: false, description: 'Build Glint NL entity extractor plugin (GLiNER-based)')
|
||
option('yams-version', type: 'string', value: '', description: 'Override runtime/display version (falls back to project version when empty). A generated header will expose this.')
|
||
option('unit-timeout', type: 'integer', value: 1200, description: 'Default timeout for selected unit/mobile test targets')
|
||
option('test-timeout-scale', type: 'integer', value: 1, description: 'Multiplier for daemon startup/wait timeouts in test code (e.g., 2 gives 2× the default waits)')
|
||
option('bench-timeout', type: 'integer', value: 90, description: 'Timeout (seconds) for benchmark/stress-oriented test suite')
|
||
option('integration-timeout', type: 'integer', value: 600, description: 'Timeout (seconds) for integration_services suite (doubled for CI stability)')
|
||
option('integration-ui-timeout', type: 'integer', value: 1200, description: 'Timeout (seconds) for UI/service integration batches')
|
||
option('integration-smoke-timeout', type: 'integer', value: 600, description: 'Timeout (seconds) for integration_smoke suite (doubled for CI stability)')
|
||
option('integration-daemon-timeout', type: 'integer', value: 1200, description: 'Timeout (seconds) for integration_daemon suite (doubled for CI stability)')
|
||
option('stress-iters', type: 'integer', value: 100, description: 'Default iteration count for StressTail tests (propagated via YAMS_STRESS_ITERS env)')
|
||
|
||
# Test registration toggles (CI-friendly defaults)
|
||
option('enable-integration-tests', type: 'boolean', value: false, description: 'Register integration smoke, CLI, daemon, search, and services tests')
|
||
option('enable-integration-services', type: 'boolean', value: false, description: 'Register integration services test binary')
|
||
option('enable-integration-smoke', type: 'boolean', value: false, description: 'Register integration smoke test binary')
|
||
option('enable-daemon-integration', type: 'boolean', value: false, description: 'Register daemon integration test binary (e2e with running daemon)')
|
||
option('enable-mobile-tests', type: 'boolean', value: false, description: 'Register mobile ABI tests')
|
||
option('enable-mobile-bindings', type: 'boolean', value: false, description: 'Build mobile C ABI shared library (yams_mobile)')
|
||
option('enable-vector-tests', type: 'boolean', value: false, description: 'Include vector/embedding tests (requires ONNX runtime and model downloads)')
|
||
option('enable-bench-tests', type: 'boolean', value: false, description: 'Register bench/isolated performance-heavy unit tests')
|
||
option('enable-profiling', type: 'boolean', value: false, description: 'Enable Tracy profiling instrumentation')
|
||
option('enable-rocprofiler-tracy', type: 'feature', value: 'auto', description: 'Link Tracy client against ROCm rocprofiler (auto-detect ROCm; disabled otherwise)')
|
||
option('enable-stress-tests', type: 'boolean', value: false, description: 'Register stress test binary')
|
||
option('enable-dcheck', type: 'boolean', value: true, description: 'Enable YAMS_DCHECK even when NDEBUG is set (enabled by default; disable with -Denable-dcheck=false for max-speed shipping builds)')
|
||
|
||
# libc++ hardening modes (only effective when using libc++)
|
||
option('libcxx-hardening', type: 'combo', choices: ['none', 'fast', 'extensive', 'debug'], value: 'none', description: 'libc++ hardening mode: none (default), fast (minimal overhead), extensive (more checks), debug (maximum validation)')
|
||
|
||
# Thread sanitizer (for detecting data races in debug builds)
|
||
option('enable-tsan', type: 'boolean', value: false, description: 'Enable ThreadSanitizer for race detection (debug builds only, significant performance impact)')
|
||
|
||
# Address sanitizer (for detecting memory errors in debug builds)
|
||
option('enable-asan', type: 'boolean', value: false, description: 'Enable AddressSanitizer for memory error detection (debug builds only, significant performance impact)')
|
||
|
||
# Undefined behavior sanitizer (for detecting UB in debug builds)
|
||
|
||
# FAISS backend for online vector index updates (optional)
|
||
option('enable-faiss', type: 'boolean', value: true,
|
||
description: 'Build with FAISS HNSW vector backend (for online/incremental index updates)')
|
||
option('enable-ubsan', type: 'boolean', value: false, description: 'Enable UndefinedBehaviorSanitizer (debug builds only; significant performance impact)')
|
||
|
||
option('enable-re2', type: 'feature', value: 'auto', description: 'Use Google RE2 regex engine for grep (10-50x faster than std::regex; auto-detect)')
|
||
|
||
# C++20 Modules (experimental)
|
||
option('enable-modules', type: 'boolean', value: false, description: 'Enable C++20 module compilation (experimental, requires GCC 13+/Clang 16+/MSVC 17.4+)')
|