Commit graph

19 commits

Author SHA1 Message Date
Trevon
4e97b31fef
chore(warnings): clean up compiler warnings on GCC, Clang and Apple Clang (#279)
Some checks failed
Fuzz Smoke / libFuzzer smoke (push) Has been cancelled
* chore(warnings): default-initialize aggregate members left out of brace-init

GCC -Wmissing-field-initializers fired about 1300 times where tests and
src build structs with a short positional brace list. Give the omitted
members an empty default member initializer. Value-initialization from
{} is what aggregate init already did for them, so behavior is
unchanged.

Signed-off-by: trvon <me@trevon.dev>

* chore(warnings): remove or annotate unused parameters, functions and variables

- init_command, s3_signer test, request_handler_lifetime test: comment out
  the unused parameter name.
- video_content_handler: path is only read under YAMS_HAVE_FFPROBE.
- vector_memory_scaling_bench: main() takes no arguments.
- daemon_status_render: delete unused neutralText helper.
- DaemonMetrics: mallocStackLoggingNoCompactEnabled is only called in the
  __APPLE__ block, so define it there.
- SocketServer: einval_streak only matters in the __APPLE__ EINVAL path.
- onnx_plugin_harness test, vector_backend_engine_compare bench,
  work_coordinator test: mark platform-conditional or busy-loop
  variables [[maybe_unused]].

Signed-off-by: trvon <git@trevon.dev>

* chore(warnings): fix sign-compare, range-loop, nodiscard and dangling-ref in tests

- search_admission_controller: count started workers in std::uint32_t so
  the comparison with the uint32 cap is same-signed.
- p2p_manager: iterate the invalid-host list by const reference.
- integrity_verifier: discard the nodiscard results the tests do not
  inspect with static_cast<void>.
- post_ingest_queue: findNode/findEdge take std::string_view so no
  temporary std::string is created for the call (GCC
  -Wdangling-reference false positive on the returned reference).
- topology_ablation_bench: the doc comment ended a line with a backslash,
  which continues the comment (-Wcomment); show the env export on its own
  line.

Signed-off-by: trvon <git@trevon.dev>

* chore(warnings): drop redundant std::move of a const captured json

The coroutine lambda is not mutable, so the captured optional<json> is
const and std::move(*progressToken) already selected the copy assignment.
Drop the std::move so the code states what actually happens.

Signed-off-by: trvon <git@trevon.dev>

* chore(warnings): mark __int128 use in db_integrity_stamp as an extension

-Wpedantic flags __int128. Use a __extension__ typedef, the GCC/Clang
idiom for an intentional extension, instead of the bare keyword.

Signed-off-by: trvon <git@trevon.dev>

* chore(warnings): remove always-true unsigned >= 0 comparisons

GCC -Wtype-limits flagged 51 test assertions of the form
CHECK(unsignedValue >= 0) that can never fail, plus a dead
`totalCount.value() < 0` clause in the graph listByType handler (the
count is std::size_t). In the tests keep the evaluation as a
static_cast<void> so the accessors are still exercised; in the handler
drop the dead clause and keep the range check.

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>
Signed-off-by: trvon <git@trevon.dev>
2026-10-08 14:30:28 -06:00
trvon
fb38c7c091
fix(client): decode compressed transfers for every client
Compressed GetResponse payloads were decoded only by 'yams get' (since
#214) and the MCP server, each with its own copy of the decoder. Any other
client of the daemon (library users, future commands) got the framed
bytes. In-process callers of DocumentService::retrieve defaulted to
acceptCompressed=true, so the daemon's CatRequest handler and the mobile
bindings would also have returned a frame instead of the document.

- DaemonClient::get decodes a compressed response once it is complete and
  returns the document; the compression fields stay for display, and a
  payload that does not decode is an error. The transfer stays compressed
  on the wire.
- RetrieveDocumentRequest::acceptCompressed defaults to false; only
  callers whose receiver decodes should opt in.
- Remove the now-redundant decoders from 'yams get' and the MCP get
  handler.
- The integration test now requires a compressed request to deliver the
  stored document, identical to a plain one.

Signed-off-by: trvon <me@trevon.dev>
2026-09-27 08:11:17 -06:00
trvon
73e5d3831c
fix(cli): decode compressed content in yams get
Since v0.7.3 'yams get' has requested compressed transfers
(acceptCompressed), and the daemon has answered with a framed payload: a
CompressionHeader followed by a Zstandard body. Only the MCP server
decoded it; 'yams get' wrote the frame itself to stdout or -o, so users
saw the 'CNRK' header and a zstd frame instead of the document. 'yams cat'
was unaffected because it does not request compression.

- Move the frame format into compression/framed_payload
  (encodeFramedPayload/decodeFramedPayload). DocumentService encodes with
  it; 'yams get' and MCP decode with it.
- The decoder checks the header, body size and CRC, and the decoded size
  and CRC. A payload that fails is an error; MCP used to fall back to
  returning the framed bytes as content.

Verified end to end: 'yams get' output (stdout and -o) is byte-identical
to the added file while the transfer stays compressed (51.75x).

Signed-off-by: trvon <me@trevon.dev>
2026-09-26 21:10:00 -06:00
Trevon
e4b32e98bd
fix: fence enrichment completion and reconcile promotion prerequisites (#157)
Some checks failed
Secret scan / gitleaks (push) Failing after 8s
Mirror source to Forgejo / mirror (push) Has been skipped
Release Candidate / Validate promotion and preview release (push) Has been cancelled
Tests / Static Analysis (push) Has been cancelled
Tests / Warm Conan & ccache (linux-arm64) (push) Has been cancelled
Tests / Warm Conan & ccache (macos-arm64) (push) Has been cancelled
Tests / Warm Conan & ccache (linux-x86_64) (push) Has been cancelled
Tests / Warm Conan & ccache (macos-x86_64) (push) Has been cancelled
Tests / Warm Conan & ccache (windows-x86_64) (push) Has been cancelled
Tests / warm (push) Has been cancelled
Tests / Tests (linux-arm64) (push) Has been cancelled
Tests / Tests (macos-arm64) (push) Has been cancelled
Tests / Tests (linux-x86_64-asan) (push) Has been cancelled
Tests / Tests (linux-x86_64-ubsan) (push) Has been cancelled
Tests / Tests (linux-x86_64) (push) Has been cancelled
Tests / Tests (windows-x86_64) (push) Has been cancelled
* test(ci): stabilize main matrix (#112)

Signed-off-by: trvon <me@trevon.dev>

* ci(release): preview experimental promotion (#111)

Signed-off-by: trvon <me@trevon.dev>

* ci(release): reconcile draft promotion safely (#113)

* ci(release): reconcile draft promotion

Create promotion visibility independently of release readiness.
Leave existing PRs untouched to preserve concurrent maintainer edits.
Validate against current main without masking candidate rejection.

Signed-off-by: trvon <me@trevon.dev>

* ci: isolate hook Git environments

Prevent temporary Git fixtures from modifying the calling worktree.
Clear repository-local hook variables before the unchanged build gates,
and isolate validator and fixture Git subprocesses from caller overrides.
Exercise linked-worktree config, ref and index preservation offline.

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>

* fix(ci): reject invalid candidate evidence

Explain missing main history without mutating refs or weakening promotion validation. Match experiment boolean and aggregation parsing, respect candidate-limit precedence, and return issues for malformed configuration or metric values.

Signed-off-by: trvon <me@trevon.dev>

* fix: preserve completion and retrieval contracts

Propagate derivation SQL failures, invalidate KG status caches, and avoid unused preview reads. Align explicit hash validation and export standalone client dependencies. Keep secret exceptions fixture-scoped and remove undefined behavior from status fixtures.

Signed-off-by: trvon <me@trevon.dev>

* fix: fence asynchronous enrichment completion

Coordinate graph and NL commits before acknowledging enrichment. Atomically classify embedding reuse, fence repair status writes, and preserve newer vectors on old failures. Upgrade content readiness triggers and keep repository counters coherent. Six affected ASan suites pass; this does not provide restart replay or generation-scoped vector storage.

Signed-off-by: trvon <me@trevon.dev>

* fix: bound semantic graph output reservation

Use retained neighbors for admission and node capacity instead of mixing corpus and pair counts. Preserve single-neighbor output and avoid quadratic reservation. Characterization regression fails three assertions before the fix; full component ASan suite passes afterward.

Signed-off-by: trvon <me@trevon.dev>

* fix: fence standalone embedding repair completion

Mint attempt tokens before reading canonical repair input and publish only current persisted document embeddings. Do not skip stale physical vectors. Make SQLite preparation fault injection connection-local and deterministic. Full component and metadata ASan suites pass.

Signed-off-by: trvon <me@trevon.dev>

* fix: preserve enrichment failure reporting

Admit pending embedding attempts before provider failures while preserving completed results. Route title no-op acknowledgement through the writer so flush reports SQL failures and duplicate jobs can retry. Four affected full ASan suites pass; host configuration hashes unchanged.

Signed-off-by: trvon <me@trevon.dev>

* test: align explicit hash prefix boundaries

Reject five characters and accept the supported six-character minimum. Full hybrid_search suite passes. Separate daemon shutdown sanitizer failure remains under investigation despite a passing targeted rerun.

Signed-off-by: trvon <me@trevon.dev>

* test: align UI hash validation boundary

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>
2026-09-12 18:53:12 -06:00
trvon
5066fc0893
test: isolate process environment 2026-08-04 21:31:39 -04:00
trvon
91fb1a32d9
fix(rm): make deletion safe and observable 2026-07-29 10:46:44 -04:00
trvon
b2ede10e54
fix: stabilize CI release gates
Fix DB vacuum, compression gates, fuzzy search, temp dirs.
   Repair release packaging: Bash 3.2, Arch containers,
   package validation defaults, artifact names, matrix metadata.

   Local tests + release act lane pass.
2026-07-03 22:40:05 -04:00
Trevon
08322369cb
Search/grep UX, runtime hardening, and CI stabilization (#39)
* test(storage): add crash-replay and handle-failure coverage for ReferenceCounter

Adds 6 test cases:
- Recovery after forced close preserves committed data
- Uncommitted transaction rolled back after forced close
- Corrupted database file rejected on open
- Consistent stats after interrupted commit recovery
- Multiple PENDING transactions survive force close
- WAL checkpoint survives forced close

* fix(ci): remove ccache — conflicts with OpenBLAS ASM as direct compiler

* fix(ci): use explicit CC/CXX paths to prevent Conan from using ccache directly

ccache was picked up by Conan/CMake as the compiler itself (/usr/bin/ccache)
rather than as a transparent wrapper, causing OpenBLAS ASM compilation to fail
with 'ccache: invalid option -- D'. Setting explicit compiler paths prevents this.

* fix(ci): add YAMS_SKIP_CCACHE to avoid OpenBLAS ASM ccache conflict

setup.sh wraps CC/CXX with ccache for faster incremental builds, but
this causes Conan/CMake to use ccache directly as the ASM compiler for
OpenBLAS, where it fails on -D flags. Add an opt-out gate in setup.sh
and set YAMS_SKIP_CCACHE=1 in .build.yml for sourcehut CI VMs.

* fix(ci): add libomp-dev, libfaiss-dev, libopenblas-dev for FAISS build

* fix(ci): limit meson compile to -j1 to prevent OOM on sourcehut VMs

* fix(build): restore hooks and shrink srht

Compile the storage library with YAMS_TESTING in test builds so
debug-only hooks link again.

Include the POSIX chmod header used by the storage utility
Catch2 coverage cases.

Prune SourceHut fallback stage archives to runtime files so
branch artifacts stay under the upload limit.

* refactor(storage): drop dead code, add coverage

* refactor(cli): polish grep/search graph UX

* fix(daemon): stabilize shutdown and CI paths

- force vector-backed tests to opt out of CI-wide disable flags
- reset reference prune statements between deletes
- make optional-plugin and POSIX-only test assumptions portable
- extract ServiceManager open/shutdown helpers for safer WAL recovery work

* test(daemon): add db vacuum coverage

* test(daemon): add WAL startup recovery

- cover stale WAL startup recovery in ServiceManager
- cover repeated restart cycles with seeded WAL sidecars
- assert daemon recovery reporting for corrupt data-dir DBs
- verify oversized metadata DBs shrink after WAL recovery shutdown

* refactor(grep): consolidate indexed content scan

* fix(grep): remove pre-scan candidate truncation

* refactor(runtime): split startup, harden storage

* feat(bench): add manifest retrieval corpus

* fix(bench): run fusion experiment on simeon

* fix(cli): sharpen search graph hints

* fix(build): refresh version metadata

* fix(ci): stabilize sync search and srht packaging

* fix(ci): unblock windows and shrink artifacts

* fix(ci): stabilize linux runtime checks

* fix(build): shrink srht manifest
2026-05-27 12:15:17 -04:00
trvon
77cab7f6e9
fix(ci): stabilize PR checks
Signed-off-by: trvon <me@trevon.dev>
2026-05-21 16:18:08 -04:00
trvon
1315843e89
fix(ci): stabilize tests and coverage 2026-05-21 09:26:43 -04:00
trvon
4df44e0c2f
fix(tests): use portable getpid shim 2026-05-20 15:54:13 -04:00
trvon
a930776655
feat: cleaning up test suite 2026-04-14 10:25:54 -04:00
trvon
49a106aeef
fix: cleaning up ENV leakage from debugging 2026-04-02 16:58:33 -04:00
trvon
9c201afae7
feat: search improvements, test correctness fixes and bug fixes 2026-03-23 13:59:54 -04:00
trvon
d353e53f26
feat: search optimizations (expanding re-ranking work to recovery items that are not strong candidates based on how they are retrieved). Test improvements and TSAN/ASAN fixes. 2026-03-23 09:15:05 -04:00
trvon
2fd9164bda
fix: tsan fixes, test instrumentation improvements and improved sql handler logic 2026-03-22 20:41:48 -04:00
trvon
9c496e674e
fix: correctness fixes for stale sqlite handles (apart of previous work), improvements in test suite and continue work for search optimizations 2026-03-22 20:06:29 -04:00
trvon
9091a9fa5d
fix: ci fixes, coverage improvements and test improvements and bugs fixes 2026-03-22 17:27:34 -04:00
trvon
15e049daee
feat: test suite improvements. Migrating remaining tests to catch2. Improving hnsw for usage in benchmarks 2026-03-22 07:56:54 -04:00