Commit graph

734 commits

Author SHA1 Message Date
trvon
fb7e76bb39
test(policy): gate configuration authority 2026-08-05 20:53:19 -04:00
trvon
88a77addd5
refactor(bench): type run configuration 2026-08-05 20:53:18 -04:00
trvon
30618eb113
refactor(config): consolidate runtime tuning 2026-08-05 20:53:18 -04:00
trvon
9c9e19199e
refactor(test): consolidate daemon harness 2026-08-05 20:53:18 -04:00
trvon
424d91f771
refactor(metadata): extract content transaction seams 2026-08-05 20:53:18 -04:00
trvon
4d7f15dd5c
refactor(bench): isolate benchmark controls 2026-08-05 20:53:18 -04:00
trvon
8b501a344e
refactor(config): consolidate embedding policy 2026-08-05 20:53:18 -04:00
trvon
bdd7281077
fix(mcp): restore WASI final linkage
Keep config and native subsystems out of the minimal archive and export the profile definition so consumer ABI matches.
2026-08-05 20:53:17 -04:00
trvon
e85339ac30
refactor(config): unify flat TOML lookups 2026-08-05 20:53:17 -04:00
trvon
5066fc0893
test: isolate process environment 2026-08-04 21:31:39 -04:00
trvon
34a799d10e
refactor(config): unify environment policy 2026-08-04 14:35:10 -04:00
trvon
72c8d21054
refactor(config): centralize runtime paths 2026-08-04 13:10:40 -04:00
trvon
8ed1cc6696
fix(daemon): propagate startup failures 2026-08-04 11:48:38 -04:00
trvon
1a2d0f905b
fix(daemon): preserve ingestion admission 2026-08-03 17:52:58 -04:00
trvon
3f017dd47a
bench(ingestion): validate Simeon lifecycle 2026-08-03 17:52:58 -04:00
trvon
fbb8fcb915
perf(vector): persist reusable warm starts 2026-08-03 17:52:58 -04:00
trvon
b319f0de6f
perf(daemon): trust clean metadata restarts 2026-08-03 17:52:58 -04:00
trvon
745818e67d
perf(topology): streamline artifact construction
Reuse admitted relations and dense splitter state while preserving deterministic identities, centroids, and the 64-document safety cap.

Add corpus-shaped benchmarks so construction regressions stay measurable.
2026-08-03 17:52:58 -04:00
trvon
d03ffd07b3
refactor(plugins): centralize lifecycle policy 2026-08-03 17:52:58 -04:00
trvon
77c4bdff40
test: isolate integration state 2026-08-03 15:40:16 -04:00
trvon
f2c051cf27
fix(daemon): clarify IPC response mismatches 2026-08-01 11:28:04 -04:00
trvon
9ff5aa7ba1
perf(storage): slice-by-8 CRC32 on all crc paths, 13.7x faster
Profiling running daemon: bit-by-bit CRC32 top CPU consumer (2289
samples/15s during grep). Same algorithm/polynomial, byte-identical
output (oracle test 0..512 len), stored checksums stay valid.

Applies slice-by-8 everywhere crc ran:
- yams::common::crc32 (+ crc32Update seeded variant for mid-stream)
- compression::calculateCRC32/updateCRC32 delegate to it
- storage-local duplicate copy removed (dead scope)
- WAL entry checksum copy removed

Also restructures sizeof arithmetic in WAL size math so opengrep does
not misread it as pointer arithmetic.
2026-08-01 00:12:32 -04:00
trvon
4c3eb3d018
refactor: dedup retry helpers + test fixtures
- isTransientMetadataError + MetadataTelemetry: 2 copies (grep/search)
  -> shared service_utils.hpp
- grep_service local string_contains -> shared features::string_contains
- ArtifactGuard: 5 identical copies in corruption tests -> shared
  tests/common/sqlite_corruption.h

retryMetadataOp loops stay split (async coroutine vs sync sleep - real
difference). annotateResultRelations/applyFileTypeFilters/formatSnippet
deferred: behavior parity question, not mechanical dedup.
2026-08-01 00:12:32 -04:00
trvon
728a304f29
refactor(cli): share time filters between get/list
Same 6-field time filter logic copy-pasted in GetCommand and ListCommand
(~72 lines each). Extract to yams::cli::applyDocTimeFilters + DocTimeFilters,
wire both commands, add unit test.
2026-08-01 00:12:31 -04:00
trvon
3b1c86f490
refactor: deduplicate trim/escapeRegex/hasWildcard helpers
The same string helpers were copy-pasted across CLI and service code:
trim/trimCopy in 5 files, escapeRegex in 3, hasWildcard in 2, and the
regex meta-char table in 2. Move canonical versions to
yams::common::string_utils.h (trimCopy, hasWildcard, escapeRegex,
isRegexMetaChar) and wire all call sites to them; daemon_helpers'
trim_copy now delegates. Add a Catch2 test for the shared helpers.

Kept isTransientMetadataError and annotateResultRelations untouched:
they have semantically different implementations and need a separate
review, not a mechanical merge.
2026-08-01 00:12:31 -04:00
trvon
dcb6f71aba
fix(storage): retry reference counter init on transient DB lock
The add-sandbox smoke test (and CLI startup generally) can fail with
'database is locked' when a prior async batch commit holds the reference
database longer than ReferenceDB's short retry budget, which CI runners
hit consistently while local runs pass. Retry the whole init (schema +
migrations) with a bounded, growing backoff (up to ~12s total) so
transient contention no longer fails startup.

Adds a Catch2 test that holds a write lock across init and releases it
mid-retry.
2026-08-01 00:12:31 -04:00
trvon
4750aa59c9
fix(test): use compat jthread shim in deadline test
Xcode 16.4's Apple libc++ lacks std::jthread; the deadline test used
it directly, breaking the macos-arm64 build. Use the repo's existing
yams::compat::jthread shim (native alias when available, std::thread
fallback otherwise).
2026-07-31 18:29:14 -04:00
trvon
3a470cb7e8
merge: portability fixes (MSVC + Apple libc++) from fix/portability-ci-findings 2026-07-31 16:43:06 -04:00
trvon
25bce22f15
fix(build): restore MSVC and Apple libc++ portability
- replace GCC/Clang-only __builtin_{add,sub,mul}_overflow calls with
  yams/core/checked_arithmetic.h helpers (addOverflow/subOverflow/
  mulOverflow/narrowOverflow); MSVC lacks these builtins, breaking the
  Windows build introduced by #61
- port the reviewed helper + unit tests from bugfix/review-findings
- detect floating-point std::from_chars availability at compile time
  (some Apple libc++ versions only provide the integral overload) and
  fall back to a strict strtod parser; template indirection keeps the
  missing overload from being diagnosed in the discarded branch
- add config_parse_utils and checked_arithmetic test targets
2026-07-31 16:40:15 -04:00
trvon
8cc1533fa8
fix(services): saturate add deadline budget 2026-07-31 14:42:49 -04:00
Trevon
0dd3985db4
refactor: simplify runtime paths and harden retrieval (#61)
* refactor(tuning): prune dormant runtime knobs

Remove unused tuning controls and adaptive search budget state. Extract testable ONNX slot reconciliation and search telemetry hydration seams, and correct the Apple Silicon sysctl value width.

* refactor(vector): retire dormant backends

Remove unused FAISS, model-infrastructure, VectorIndexManager, and TurboQuant paths while keeping Simeon PQ and float-vector storage.

Migrate unused V2.2 quantized schemas transactionally to V2.3. Databases containing non-default quantized formats or packed-code blobs fail closed with recovery guidance and no mutation.

* refactor(plugins): remove legacy SDK tooling

Remove the Ghidra and SDK submodules, their bespoke tests, and stale scan exclusions. Retain generic PluginProcess coverage against the repository mock fixture.

* build(vector): remove stale FAISS overrides

* refactor(ipc): remove dormant FSM snapshots

* refactor(ingest): unify bounded KG enqueue

* fix(plugins): harden allocation contracts

Keep ABI outputs null or fully initialized when allocations fail, and represent empty object payloads without relying on malloc(0).

* fix(storage): define empty object reads

Return a null buffer with zero length for successful empty-object reads instead of treating malloc(0) as an allocation contract.

* refactor(storage): trim through string views

* test: prune dormant harness helpers

Remove unused readiness and fixture helpers, and keep benchmark client-count doubling in a wide integer domain.

* chore(cpp): initialize analyzer-facing locals

* build(simeon): advance analyzer cleanup

* refactor(config): drop dead daemon client keys

* refactor(ipc): inject daemon client transport

Socket and in-process execution now share DaemonClient request dispatch, so CLI fallbacks do not need alternate service paths.

* refactor(retrieval): route CLI through daemon

Search and grep now use the same daemon handlers over socket or in-process transport. Live grep remains an explicit freshness inspection.

* refactor(graph): move CLI scope into daemon

Graph CLI requests now share daemon handlers across socket and in-process transports. CWD filtering occurs before pagination so scoped totals and pages stay correct.

* build: isolate test-only runtime controls

* refactor(metadata): decouple lock telemetry

* fix(services): return on request timeout

* fix(retrieval): enforce workspace scope

* fix(graph): scope nodes before pagination

* fix(grep): drop binary result lines

* fix(daemon): bound checkpoint wait

* fix(repair): use KG health checks

* fix(storage): always verify compressed CRC

* perf(vector): skip unchanged PQ snapshots

* build(zyp): exclude bench from default build

* build: protect Meson files from formatter

* refactor: remove unreachable production APIs

* refactor(daemon): remove dormant runtime state

* fix(vector): harden daemon provider calls

Timed-out coroutines must own their client and requests. Provider discovery must also fail instead of returning synthetic embeddings in production.

* refactor(storage): remove unusable adapters

* fix(cli): unify in-process add requests

* fix(tsan): preserve async worker progress

* test(cli): isolate daemon-dependent suites

* fix(repair): expose queue backoff progress
2026-07-31 14:16:45 -04:00
trvon
0c424ee421
docs: repair benchmark documentation references 2026-07-29 22:48:22 -04:00
trvon
e290deb944
test(daemon): isolate sanitizer lifecycle fixtures 2026-07-29 22:30:32 -04:00
trvon
0b3317b757
fix(graph): propagate workspace scope through daemon 2026-07-29 22:29:56 -04:00
trvon
9447f70c58
refactor: deduplicate retrieval helpers
Share behavior-preserving helpers across daemon transport, MCP retrieval, and topology construction.
2026-07-29 19:04:56 -04:00
trvon
f53d063a50
fix(daemon): make watcher and restart reliable
Retry failed session indexing and deletion without advancing fingerprints. Share the lifecycle shutdown budget so restart does not time out during a graceful drain.
2026-07-29 19:01:22 -04:00
trvon
70d02362b6
fix(graph): scope exploration to workspace
Canonicalize source paths and strip snapshot suffixes before containment checks. Keep cross-repository exploration available through --global.
2026-07-29 19:00:49 -04:00
trvon
4e566f6387
fix: harden buffer and parser contracts
Validate Simeon buffer boundaries and replace ZYP reverse pointer arithmetic with bounded indexing. Keep release SIMD kernels unchecked while asserting contracts in debug builds.
2026-07-29 19:00:10 -04:00
trvon
a23038f589
refactor: remove dead code and duplicate helpers
Make FAISS index ownership explicit and remove unused Rabin tables.
2026-07-29 16:17:10 -04:00
trvon
9855c7bff8
fix: synchronize daemon service publication 2026-07-29 14:12:06 -04:00
trvon
7a64babc5e
docs: reduce documentation surface
Keep maintained entry points and executable contracts. Remove historical guides and stale references that drift from command help and source.
2026-07-29 13:02:15 -04:00
trvon
91fb1a32d9
fix(rm): make deletion safe and observable 2026-07-29 10:46:44 -04:00
trvon
dabb424596
fix(daemon): verify lifecycle process identity 2026-07-29 10:35:30 -04:00
trvon
2345462016
fix(graph): scope explore to working tree
Apply scope before result budgets. Preserve wire compatibility and fall back locally when older daemons return out-of-scope data.
2026-07-29 02:35:04 -04:00
trvon
b31de9ce99
fix(storage): harden manifest and graph cleanup
Decode canonical and legacy binary manifests strictly, including protobuf builds. Add indexed, transactional document graph cleanup with typed forced-deletion errors, and resolve the associated arithmetic, narrowing, pagination, backoff, and nullable-coordinator findings.
2026-07-29 02:35:04 -04:00
trvon
10ff33532c
fix(search): cancel disconnected requests
Retire timed-out one-shot sockets and propagate connection cancellation through daemon search execution. Stop scheduling later search stages after cancellation while safely draining in-flight work, and reject KG ingest counts that exceed the wire format.
2026-07-28 23:49:14 -04:00
trvon
8ad4ab29e0
fix(search): correct code navigation
Honor root-anchored and slash-qualified ignore patterns during live fallback. Center hydrated snippets on query matches and preserve symbol ranking through the daemon boundary.
2026-07-28 22:33:31 -04:00
trvon
11b02411e6
fix(graph): improve agent navigation
Rank natural-language graph matches by multi-term file coverage, repair stale in-bounds snippet locations, and replace affected-test label scans with bounded batch traversal. Correct the YAMS-first prompt to route relationship questions through the matching graph operation.
2026-07-28 20:30:48 -04:00
trvon
cee3adc1cc
fix(release): harden runtime boundaries
Refuse to write configuration when cryptographic API-key generation fails. Remove dead MCP and config split residue, make numeric parsing locale-independent, and synchronize poller coalescing coverage on the coroutine boundary.
2026-07-28 19:17:45 -04:00
trvon
ed8492bb65
perf(ingest): bound store batching
Separate storage intake from downstream transaction sizing so pressure caps and measured defaults can evolve independently.
2026-07-28 10:43:58 -04:00