Commit graph

59 commits

Author SHA1 Message Date
trvon
8250b061fc
build: release buildtype with split debug symbols
Release used buildtype=debugoptimized (-O2 -g, strip=false) and relied on
scripts/prune-runtime-install.sh to strip the staged payload, discarding
the debug info. Crash reports from shipped binaries could not be
symbolicated and the build type label did not match what shipped.

- setup.sh/setup.ps1 Release and the Meson project default use
  buildtype=release. setup.sh adds -Ddebug=true for Release so -O3 code
  keeps DWARF for the split, and passes -Db_pie=true explicitly (also a
  project default) so reconfigured build dirs pick it up. Windows keeps
  release without -g so no PDBs land in the MSI stage. Debug/Fuzzing are
  unchanged (-O0 -g, assertions on).
- scripts/split-debug-symbols.sh replaces prune-runtime-install.sh: ELF
  files get objcopy --only-keep-debug (compressed) into <debug-dir>/
  <path>.debug plus .build-id/xx/yyyy.debug links, strip
  --strip-unneeded and --add-gnu-debuglink; Mach-O files get dsymutil
  dSYMs and strip -S -x. Without a debug dir it only strips.
- The release job splits into $BUILD_DIR/debug-symbols and uploads it as
  a debug-symbols-<suffix> workflow artifact (30 days) instead of
  shipping or dropping it. build-deb.sh, the Dockerfile, PKGBUILD and
  build-arch-pkg.sh call the split script in strip-only mode.

Measured on grim (Linux x86_64, clang 21, stripped staged payload):

  file                         debugoptimized  release -O3   delta
  bin/yams-cli                     35,528,968   36,733,304   +3.4%
  bin/yams-daemon                  29,731,312   30,685,792   +3.2%
  bin/yams-mcp-server              19,504,928   19,865,472   +1.8%
  libyams_onnx_resource.so            477,280      481,488   +0.9%
  libyams_object_storage_s3.so      8,226,608    8,239,016   +0.2%
  yams_zyp.so                          79,936       80,040   +0.1%

Debug symbols: 309 MiB (compressed .debug files); gdb resolves
`info line main` on the stripped daemon through the .build-id tree.
check_install_surface: no debug sections or .symtab remain; the
remaining violations are the export leaks fixed later in this series.

Signed-off-by: trvon <me@trevon.dev>
2026-10-01 16:53:23 -06:00
trvon
66e59d305f
build: make the deprecated ONNX Runtime integration opt-in
Default builds no longer pull onnxruntime: conan enable_onnx and the conan
profiles default to False, meson enable-onnx defaults to disabled and skips
onnxruntime discovery entirely, and setup.sh/setup.ps1 only enable ONNX
(and the ONNX-backed Glint plugin) with --with-onnx / -WithOnnx.
YAMS_DISABLE_ONNX=true still forces it off.

This removes a build blocker on hosts where conan tries to build
onnxruntime from source (e.g. ROCm/MIGraphX on Linux). Release, Docker, and
Arch packaging pass the flag explicitly, so shipped artifacts are unchanged
until ONNX is dropped.
2026-09-22 18:56:22 -06:00
Trevon
0dd3985db4
refactor: simplify runtime paths and harden retrieval (#61)
* refactor(tuning): prune dormant runtime knobs

Remove unused tuning controls and adaptive search budget state. Extract testable ONNX slot reconciliation and search telemetry hydration seams, and correct the Apple Silicon sysctl value width.

* refactor(vector): retire dormant backends

Remove unused FAISS, model-infrastructure, VectorIndexManager, and TurboQuant paths while keeping Simeon PQ and float-vector storage.

Migrate unused V2.2 quantized schemas transactionally to V2.3. Databases containing non-default quantized formats or packed-code blobs fail closed with recovery guidance and no mutation.

* refactor(plugins): remove legacy SDK tooling

Remove the Ghidra and SDK submodules, their bespoke tests, and stale scan exclusions. Retain generic PluginProcess coverage against the repository mock fixture.

* build(vector): remove stale FAISS overrides

* refactor(ipc): remove dormant FSM snapshots

* refactor(ingest): unify bounded KG enqueue

* fix(plugins): harden allocation contracts

Keep ABI outputs null or fully initialized when allocations fail, and represent empty object payloads without relying on malloc(0).

* fix(storage): define empty object reads

Return a null buffer with zero length for successful empty-object reads instead of treating malloc(0) as an allocation contract.

* refactor(storage): trim through string views

* test: prune dormant harness helpers

Remove unused readiness and fixture helpers, and keep benchmark client-count doubling in a wide integer domain.

* chore(cpp): initialize analyzer-facing locals

* build(simeon): advance analyzer cleanup

* refactor(config): drop dead daemon client keys

* refactor(ipc): inject daemon client transport

Socket and in-process execution now share DaemonClient request dispatch, so CLI fallbacks do not need alternate service paths.

* refactor(retrieval): route CLI through daemon

Search and grep now use the same daemon handlers over socket or in-process transport. Live grep remains an explicit freshness inspection.

* refactor(graph): move CLI scope into daemon

Graph CLI requests now share daemon handlers across socket and in-process transports. CWD filtering occurs before pagination so scoped totals and pages stay correct.

* build: isolate test-only runtime controls

* refactor(metadata): decouple lock telemetry

* fix(services): return on request timeout

* fix(retrieval): enforce workspace scope

* fix(graph): scope nodes before pagination

* fix(grep): drop binary result lines

* fix(daemon): bound checkpoint wait

* fix(repair): use KG health checks

* fix(storage): always verify compressed CRC

* perf(vector): skip unchanged PQ snapshots

* build(zyp): exclude bench from default build

* build: protect Meson files from formatter

* refactor: remove unreachable production APIs

* refactor(daemon): remove dormant runtime state

* fix(vector): harden daemon provider calls

Timed-out coroutines must own their client and requests. Provider discovery must also fail instead of returning synthetic embeddings in production.

* refactor(storage): remove unusable adapters

* fix(cli): unify in-process add requests

* fix(tsan): preserve async worker progress

* test(cli): isolate daemon-dependent suites

* fix(repair): expose queue backoff progress
2026-07-31 14:16:45 -04:00
trvon
597aed4fc9
fix(docker): add wget for Zig, disable faiss
openblas arm64 build fails in Docker containers. YAMS_DISABLE_FAISS=1
env var skips faiss (and its openblas transitive dep). Zig download
now uses wget -c for resume on flaky ziglang.org connections.
2026-05-06 15:10:03 -04:00
trvon
8f5b0d0eb3
fix(docker): add gfortran for openblas arm64 build 2026-05-06 09:08:44 -04:00
trvon
18b10dc444
fix: addressing build regressions where fiass needed openmp installed 2026-05-05 19:50:16 -04:00
trvon
8772cc8e54
fix(docs,ci): addressing breaking changes in zig v0.16.x in Dockerfile, docs update and code cleanup 2026-04-25 11:27:18 -04:00
trvon
127bc9e9e0
fix: fixing leakage of build artifacts in release, unifying repair path, adding cache to tuning path and bumping simeon version 2026-04-23 10:20:25 -04:00
trvon
e09778b7fe
fix: improvements for embedding generation (embedding gemma specific issue) and Dockerfile regression fix 2026-04-07 15:54:52 -04:00
trvon
211596aade
feat: expanding graph utility in search, fixing Dockerfile build logic and improving search tracing logic for graph performance 2026-03-14 19:56:41 -04:00
trvon
280d31ca63
Merge branch 'fix/dco-clean' 2026-02-24 23:07:05 -05:00
trvon
a177027320
fix: house cleaning 2026-02-09 21:12:26 -05:00
trvon
1a2178af13
improving UX of metrics 2026-01-27 22:38:34 -05:00
trvon
6a714aab62
Dockerfile build fix and model improvements 2026-01-22 14:50:30 -05:00
trvon
a8a6559377
dockerfile and ci fixes with improvements for model loading logic 2026-01-19 22:28:30 -05:00
trvon
cc75a5504f
benchmark fixing and pool allocation improvements 2026-01-19 21:26:04 -05:00
trvon
6e9221c8b2 fixing for building libsql 2026-01-20 01:05:57 +00:00
trvon
866ee12efc
zig versioning fix 2025-12-31 15:18:17 -05:00
trvon
9246ab5813
ci fixes 2025-12-31 15:04:51 -05:00
trvon
bc71d36534
migrated pdf plugin, daemon shutdown logic fix 2025-12-31 14:46:07 -05:00
trevon
16d225599d
fixing plugin load logic for external plugin, test compilation issues and adding entity integration test 2025-12-20 11:19:37 -05:00
trevon
7ad3f6f784
graph optimizations and cleanup. CI improvements for submodules 2025-12-18 13:04:57 -05:00
trevon
e47142343e
updating tests / completing catch2 migration. plugin trust fixes 2025-12-12 10:58:48 -05:00
trvon
07218a145e
plugin fixes for windows 2025-12-10 01:04:20 -05:00
trevon
e41c06d6bc
improvements for lifecycle status and github action fixes 2025-11-26 00:28:30 -05:00
trvon
d989da66dd
fixing prune command and tests fixes 2025-11-06 12:19:08 -05:00
trvon
0cd1bd0b4c
stabilizations. removing http as I am not using it. Focusing on streamlining build and system 2025-11-05 10:52:27 -05:00
trvon
7d9dcd986d
additional ci fixes for docker and meson setup 2025-11-01 16:46:31 -04:00
trvon
d19a5d7b64
bug fixes and optimizations for v0.7.7. Finalizing tests, benchmarks and bug fixes prior to release 2025-10-24 12:49:00 -04:00
trvon
8d23211af2
improvements for ci, setup.sh and updates to new symbol extraction plugin 2025-10-21 19:38:45 -04:00
trvon
b25a9f690d
v0.7.5 2025-10-13 20:07:48 -04:00
trvon
83fa1ad1ba
v0.7.5 staging and license updates 2025-10-13 17:07:31 -04:00
trvon
3ad7417b1e
ci and other fixes 2025-10-11 14:11:20 -04:00
trvon
b92603a4d3
linux compat fixes 2025-10-07 11:34:38 -04:00
Trevon
19fb4fee75
updating, stagging fixes, then will address ci 2025-09-27 04:39:58 -04:00
Trevon
d35267df3b
ci fixes for github 2025-09-25 14:14:17 -04:00
Trevon
43eddf27ff
staging plugin improvements and ci fixes 2025-09-23 10:54:24 -04:00
Trevon
6ba8d16d0b
ci updates for builds 2025-09-22 16:12:45 -04:00
Trevon
71fe994dbf
platform specific fixes for ci 2025-09-22 15:41:57 -04:00
Trevon
7247888a06
mcp compat updates and ci improvements 2025-09-22 15:27:42 -04:00
Trevon
b91176fdc3
ci improvements 2025-09-22 12:22:34 -04:00
Trevon
0d8d59c9aa
removing yams-repo for builds 2025-09-22 11:55:32 -04:00
Trevon
5c696fab05
ci improvements 2025-09-22 11:39:09 -04:00
Trevon
0a434ff29c
updating contributing for new meson build system. 2025-09-21 22:19:20 -04:00
Trevon
43699f99a3
v0.7.0-prerelease 2025-09-20 22:55:34 -04:00
Trevon
9a04cfa48c
v0.6.20 ci updates 2025-09-14 14:37:00 -04:00
Trevon
31e1aa5b56
v0.6.14 2025-09-12 23:37:14 -04:00
Trevon
c9f1e2b372
v0.6.3 usability and ci patches 2025-09-04 09:32:39 -04:00
Trevon
7c92f09d21
v0.5.7 2025-08-29 06:12:50 -04:00
Trevon
a9cad7699f
v0.4.4 2025-08-21 06:05:46 +00:00