Commit graph

29 commits

Author SHA1 Message Date
Trevon
4e97b31fef
chore(warnings): clean up compiler warnings on GCC, Clang and Apple Clang (#279)
Some checks failed
Fuzz Smoke / libFuzzer smoke (push) Has been cancelled
* chore(warnings): default-initialize aggregate members left out of brace-init

GCC -Wmissing-field-initializers fired about 1300 times where tests and
src build structs with a short positional brace list. Give the omitted
members an empty default member initializer. Value-initialization from
{} is what aggregate init already did for them, so behavior is
unchanged.

Signed-off-by: trvon <me@trevon.dev>

* chore(warnings): remove or annotate unused parameters, functions and variables

- init_command, s3_signer test, request_handler_lifetime test: comment out
  the unused parameter name.
- video_content_handler: path is only read under YAMS_HAVE_FFPROBE.
- vector_memory_scaling_bench: main() takes no arguments.
- daemon_status_render: delete unused neutralText helper.
- DaemonMetrics: mallocStackLoggingNoCompactEnabled is only called in the
  __APPLE__ block, so define it there.
- SocketServer: einval_streak only matters in the __APPLE__ EINVAL path.
- onnx_plugin_harness test, vector_backend_engine_compare bench,
  work_coordinator test: mark platform-conditional or busy-loop
  variables [[maybe_unused]].

Signed-off-by: trvon <git@trevon.dev>

* chore(warnings): fix sign-compare, range-loop, nodiscard and dangling-ref in tests

- search_admission_controller: count started workers in std::uint32_t so
  the comparison with the uint32 cap is same-signed.
- p2p_manager: iterate the invalid-host list by const reference.
- integrity_verifier: discard the nodiscard results the tests do not
  inspect with static_cast<void>.
- post_ingest_queue: findNode/findEdge take std::string_view so no
  temporary std::string is created for the call (GCC
  -Wdangling-reference false positive on the returned reference).
- topology_ablation_bench: the doc comment ended a line with a backslash,
  which continues the comment (-Wcomment); show the env export on its own
  line.

Signed-off-by: trvon <git@trevon.dev>

* chore(warnings): drop redundant std::move of a const captured json

The coroutine lambda is not mutable, so the captured optional<json> is
const and std::move(*progressToken) already selected the copy assignment.
Drop the std::move so the code states what actually happens.

Signed-off-by: trvon <git@trevon.dev>

* chore(warnings): mark __int128 use in db_integrity_stamp as an extension

-Wpedantic flags __int128. Use a __extension__ typedef, the GCC/Clang
idiom for an intentional extension, instead of the bare keyword.

Signed-off-by: trvon <git@trevon.dev>

* chore(warnings): remove always-true unsigned >= 0 comparisons

GCC -Wtype-limits flagged 51 test assertions of the form
CHECK(unsignedValue >= 0) that can never fail, plus a dead
`totalCount.value() < 0` clause in the graph listByType handler (the
count is std::size_t). In the tests keep the evaluation as a
static_cast<void> so the accessors are still exercised; in the handler
drop the dead clause and keep the range check.

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>
Signed-off-by: trvon <git@trevon.dev>
2026-10-08 14:30:28 -06:00
Trevon
e4b32e98bd
fix: fence enrichment completion and reconcile promotion prerequisites (#157)
Some checks failed
Secret scan / gitleaks (push) Failing after 8s
Mirror source to Forgejo / mirror (push) Has been skipped
Release Candidate / Validate promotion and preview release (push) Has been cancelled
Tests / Static Analysis (push) Has been cancelled
Tests / Warm Conan & ccache (linux-arm64) (push) Has been cancelled
Tests / Warm Conan & ccache (macos-arm64) (push) Has been cancelled
Tests / Warm Conan & ccache (linux-x86_64) (push) Has been cancelled
Tests / Warm Conan & ccache (macos-x86_64) (push) Has been cancelled
Tests / Warm Conan & ccache (windows-x86_64) (push) Has been cancelled
Tests / warm (push) Has been cancelled
Tests / Tests (linux-arm64) (push) Has been cancelled
Tests / Tests (macos-arm64) (push) Has been cancelled
Tests / Tests (linux-x86_64-asan) (push) Has been cancelled
Tests / Tests (linux-x86_64-ubsan) (push) Has been cancelled
Tests / Tests (linux-x86_64) (push) Has been cancelled
Tests / Tests (windows-x86_64) (push) Has been cancelled
* test(ci): stabilize main matrix (#112)

Signed-off-by: trvon <me@trevon.dev>

* ci(release): preview experimental promotion (#111)

Signed-off-by: trvon <me@trevon.dev>

* ci(release): reconcile draft promotion safely (#113)

* ci(release): reconcile draft promotion

Create promotion visibility independently of release readiness.
Leave existing PRs untouched to preserve concurrent maintainer edits.
Validate against current main without masking candidate rejection.

Signed-off-by: trvon <me@trevon.dev>

* ci: isolate hook Git environments

Prevent temporary Git fixtures from modifying the calling worktree.
Clear repository-local hook variables before the unchanged build gates,
and isolate validator and fixture Git subprocesses from caller overrides.
Exercise linked-worktree config, ref and index preservation offline.

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>

* fix(ci): reject invalid candidate evidence

Explain missing main history without mutating refs or weakening promotion validation. Match experiment boolean and aggregation parsing, respect candidate-limit precedence, and return issues for malformed configuration or metric values.

Signed-off-by: trvon <me@trevon.dev>

* fix: preserve completion and retrieval contracts

Propagate derivation SQL failures, invalidate KG status caches, and avoid unused preview reads. Align explicit hash validation and export standalone client dependencies. Keep secret exceptions fixture-scoped and remove undefined behavior from status fixtures.

Signed-off-by: trvon <me@trevon.dev>

* fix: fence asynchronous enrichment completion

Coordinate graph and NL commits before acknowledging enrichment. Atomically classify embedding reuse, fence repair status writes, and preserve newer vectors on old failures. Upgrade content readiness triggers and keep repository counters coherent. Six affected ASan suites pass; this does not provide restart replay or generation-scoped vector storage.

Signed-off-by: trvon <me@trevon.dev>

* fix: bound semantic graph output reservation

Use retained neighbors for admission and node capacity instead of mixing corpus and pair counts. Preserve single-neighbor output and avoid quadratic reservation. Characterization regression fails three assertions before the fix; full component ASan suite passes afterward.

Signed-off-by: trvon <me@trevon.dev>

* fix: fence standalone embedding repair completion

Mint attempt tokens before reading canonical repair input and publish only current persisted document embeddings. Do not skip stale physical vectors. Make SQLite preparation fault injection connection-local and deterministic. Full component and metadata ASan suites pass.

Signed-off-by: trvon <me@trevon.dev>

* fix: preserve enrichment failure reporting

Admit pending embedding attempts before provider failures while preserving completed results. Route title no-op acknowledgement through the writer so flush reports SQL failures and duplicate jobs can retry. Four affected full ASan suites pass; host configuration hashes unchanged.

Signed-off-by: trvon <me@trevon.dev>

* test: align explicit hash prefix boundaries

Reject five characters and accept the supported six-character minimum. Full hybrid_search suite passes. Separate daemon shutdown sanitizer failure remains under investigation despite a passing targeted rerun.

Signed-off-by: trvon <me@trevon.dev>

* test: align UI hash validation boundary

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>
2026-09-12 18:53:12 -06:00
Trevon
9883a4382b
refactor(vector): decouple embedding generator policy (#83)
* refactor(vector): add injectable executor for the embedding daemon bridge

await_with_timeout spawned coroutines unconditionally on the daemon
global IO context, coupling the vector-layer embedding generator to
daemon globals. EmbeddingConfig gains an optional
boost::asio::any_io_executor executorOverride; the bridge uses it when
present and keeps the global executor as the default — identical
behavior for all existing callers (embedding_service, model providers,
CLI all construct without an override).

Lead 1 plan step (b) of the embedding_generator vector->daemon
decomposition (Brick audit follow-up).

Validation: full ninja build green; meson test yams:service_manager,
vector_submodule, yams:embedding_lifecycle, yams:embedding_recovery —
all green.

Signed-off-by: trvon <me@trevon.dev>

* refactor(vector): split DaemonBackend into own TU

Keep daemon client/IPC headers out of the vector-layer generator TU;
EmbeddingGenerator builds the backend via createDaemonBackend factory
on the existing IEmbeddingBackend seam. Behavior-preserving.

Brick audit lead 1: 5 file-level cycles hubbed on
embedding_generator.cpp.

Signed-off-by: trvon <me@trevon.dev>

* revert(vector): keep bridge executor out of public EmbeddingConfig

Pre-push gate caught it: the config layer includes the vector header
transitively (ConfigResolver.h -> document_chunker.h ->
embedding_generator.h) and has no boost asio include path, so
<boost/asio/any_io_executor.hpp> in the public header breaks the
config build.

Executor selection stays in embedding_daemon_backend.cpp (daemon-facing
TU by design); per-instance injection needs the config->vector include
direction fixed first — recorded as follow-up.

Signed-off-by: trvon <me@trevon.dev>

* refactor(vector): trust resolved backend policy

Remove EmbeddingGenerator's ambient ConfigResolver read. Callers now own
backend-policy resolution; EmbeddingService marks its already-resolved
runtime policy unconditionally. Keep TuneAdvisor only for the process-wide
concurrency gate, where instance-local policy would not coordinate admission.

Tests pin typed backend authority against conflicting environment state.

Signed-off-by: trvon <me@trevon.dev>

* chore(vector): remove stale config include

The generator no longer reads config helpers after backend policy moved to
callers, so drop the misleading DaemonBackend comment and unused include.

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>
2026-09-02 10:12:46 -04:00
trvon
b7eaea8263
test: move stress cases to slow suite
Stress-tagged cases ran in the default unit lane because only ~[slow]
was filtered. Tag them [slow] and add ~[slow]/[slow] splits to the five
binaries (mcp_submodule, work_coordinator, daemon_background,
plugin_infrastructure, hybrid_search) so they run in the slow suite.
2026-08-12 20:36:16 -06:00
Trevon
0dd3985db4
refactor: simplify runtime paths and harden retrieval (#61)
* refactor(tuning): prune dormant runtime knobs

Remove unused tuning controls and adaptive search budget state. Extract testable ONNX slot reconciliation and search telemetry hydration seams, and correct the Apple Silicon sysctl value width.

* refactor(vector): retire dormant backends

Remove unused FAISS, model-infrastructure, VectorIndexManager, and TurboQuant paths while keeping Simeon PQ and float-vector storage.

Migrate unused V2.2 quantized schemas transactionally to V2.3. Databases containing non-default quantized formats or packed-code blobs fail closed with recovery guidance and no mutation.

* refactor(plugins): remove legacy SDK tooling

Remove the Ghidra and SDK submodules, their bespoke tests, and stale scan exclusions. Retain generic PluginProcess coverage against the repository mock fixture.

* build(vector): remove stale FAISS overrides

* refactor(ipc): remove dormant FSM snapshots

* refactor(ingest): unify bounded KG enqueue

* fix(plugins): harden allocation contracts

Keep ABI outputs null or fully initialized when allocations fail, and represent empty object payloads without relying on malloc(0).

* fix(storage): define empty object reads

Return a null buffer with zero length for successful empty-object reads instead of treating malloc(0) as an allocation contract.

* refactor(storage): trim through string views

* test: prune dormant harness helpers

Remove unused readiness and fixture helpers, and keep benchmark client-count doubling in a wide integer domain.

* chore(cpp): initialize analyzer-facing locals

* build(simeon): advance analyzer cleanup

* refactor(config): drop dead daemon client keys

* refactor(ipc): inject daemon client transport

Socket and in-process execution now share DaemonClient request dispatch, so CLI fallbacks do not need alternate service paths.

* refactor(retrieval): route CLI through daemon

Search and grep now use the same daemon handlers over socket or in-process transport. Live grep remains an explicit freshness inspection.

* refactor(graph): move CLI scope into daemon

Graph CLI requests now share daemon handlers across socket and in-process transports. CWD filtering occurs before pagination so scoped totals and pages stay correct.

* build: isolate test-only runtime controls

* refactor(metadata): decouple lock telemetry

* fix(services): return on request timeout

* fix(retrieval): enforce workspace scope

* fix(graph): scope nodes before pagination

* fix(grep): drop binary result lines

* fix(daemon): bound checkpoint wait

* fix(repair): use KG health checks

* fix(storage): always verify compressed CRC

* perf(vector): skip unchanged PQ snapshots

* build(zyp): exclude bench from default build

* build: protect Meson files from formatter

* refactor: remove unreachable production APIs

* refactor(daemon): remove dormant runtime state

* fix(vector): harden daemon provider calls

Timed-out coroutines must own their client and requests. Provider discovery must also fail instead of returning synthetic embeddings in production.

* refactor(storage): remove unusable adapters

* fix(cli): unify in-process add requests

* fix(tsan): preserve async worker progress

* test(cli): isolate daemon-dependent suites

* fix(repair): expose queue backoff progress
2026-07-31 14:16:45 -04:00
trvon
4df44e0c2f
fix(tests): use portable getpid shim 2026-05-20 15:54:13 -04:00
trvon
a930776655
feat: cleaning up test suite 2026-04-14 10:25:54 -04:00
trvon
a3fd21766c
feat: reliability improvements for daemon lifecycle, submodule bumps, plugin improvements and search tuning optimizations based on benchmarks 2026-03-09 19:38:15 -04:00
trvon
33821d93f1
feat: test hardening, added benchmark for multiclient and improve stability to support up to 16 clients before performance degrades, improved parallization and request handling in daemon ingestion pipeline 2026-02-16 15:46:16 -05:00
trvon
c0e17835ac
search optimizations 2026-01-22 23:33:10 -05:00
trvon
633c3ce1cc
gliner term expansion for better search and search model ranking update 2026-01-22 23:17:40 -05:00
trvon
dabb828e7a
stagging changes for release of v0.7.11 . Working on improving retrieval 2026-01-19 18:59:06 -05:00
trvon
67a3c521e0
search engine tests, hnsw bug fix and test fixes 2026-01-17 22:13:08 -05:00
trvon
b078329bc8
search regressions fixes. removed bktree for symspell 2026-01-07 23:43:12 -05:00
trvon
c7c48e8934
removed hnsw vector manager. search optimizations 2026-01-05 23:39:18 -05:00
trvon
82eea28210
removing duplicate hnsw and leverage sqlite vec cpp library updates 2026-01-05 10:16:32 -05:00
trvon
b3d44b9e98
removing deprecated search executor. continuing work for rag benchmarking 2026-01-04 18:29:09 -05:00
trvon
f2e864f051
test fixes and bug fixes 2025-12-26 16:57:13 -05:00
trvon
c7237ecec5
UI improvements for list and maintenance requests improvements 2025-12-23 23:12:35 -05:00
trvon
f129e0f81e
test updates, mcp graph tool, search bug fixes, grep UI updates 2025-12-22 19:38:09 -05:00
trevon
606712b5f1
test improvements and stablizations 2025-12-06 18:27:25 -05:00
trvon
069c9116f0
bug fixes, linter run and improvements staging v0.7.10 2025-12-03 15:11:09 -05:00
trevon
2ec7ecff5a
addressing search regressions, crashes and linux build errors 2025-12-03 10:42:27 -05:00
trevon
d24a961e2e
checkpoint system, removed hybrid search engine after its been deprecated, build updates 2025-12-02 15:21:31 -05:00
trvon
c16939f482
linter 2025-11-28 20:30:41 -05:00
trevon
1de9e31a58
cleaning up tests and exposing windows packaging and builds in ci 2025-11-27 10:54:09 -05:00
trvon
a7c14f54cc
Bug fixes for prune command and test improvements. 2025-11-06 09:13:29 -05:00
trvon
166d8c520d
staging v0.7.7 2025-11-01 16:02:12 -04:00
trvon
bbf23bb190
test migrations to catch2, service manager + socket server refactor for better startup performance. search bug fixes 2025-10-29 23:08:09 -04:00