Commit graph

14 commits

Author SHA1 Message Date
trvon
ca944de51b
build: trim Meson options and build only the Boost libraries YAMS links
meson_options.txt had 47 options. Several were read but never set by
any workflow, script or doc, several only fed test-lane timeouts, and
enable-{tsan,asan,ubsan} duplicated b_sanitize. 47 -> 33.

Removed (behavior kept at the previous default everywhere):
- require-fts5 (configure now always errors on a SQLite without FTS5,
  the old default), conan-generators-dir, cli11-cmake-path,
  onnxruntime-cmake-path (Conan/CMake discovery under the build dir is
  unchanged), split-debug.
- unit-timeout, integration-timeout, integration-ui-timeout,
  integration-smoke-timeout, integration-daemon-timeout and stress-iters:
  replaced by the same constants at their call sites (smoke keeps its
  test-timeout-scale multiplier). Only meson/profiles/ci-fast.ini and
  nightly-soak.ini set them, and nothing used those profiles; both go.
- enable-integration-services/-smoke and enable-daemon-integration:
  folded into enable-integration-tests. Debug builds still register every
  integration suite; nothing set the sub-toggles.

Kept, because third_party/symspell's meson.build reads them:
bench-timeout, enable-tsan and enable-asan. YAMS itself now derives all
sanitizer handling (extra flags, TSan runtime links, test timeouts) from
b_sanitize; setup.sh still mirrors its choice into enable-tsan/asan for
the submodule, and the CI ASan lane keeps -Denable-asan=true for the
same reason. enable-ubsan is a deprecated no-op (a warning if it
disagrees with b_sanitize) so build dirs configured with it still
regenerate; the UBSan lane passes only -Db_sanitize=undefined and still
gets -fno-sanitize-recover=all (standalone UBSan aborts; ASan+UBSan stays
recoverable, as before).

Conan: the without_* Boost list was applied only to iOS/Android. YAMS
uses Boost.Asio (header-only) and links only boost_system/boost_thread
on every platform (asio/ssl uses OpenSSL directly; no spawn/context), so
the list now applies everywhere. Debug Boost package on grim: 45 static
libraries / 229 MiB -> 10 / 6.8 MiB (Release: 45 / 30 MiB before).

Verified on grim: ./setup.sh Debug --no-tsan --with-tests reconfigures
an existing build/debug and builds; an existing build/release (configured
with -Denable-tsan/-Denable-asan) regenerates.

Signed-off-by: trvon <me@trevon.dev>
2026-10-01 18:52:34 -06:00
trvon
c597f99d44
fix(mobile): port bindings to current service APIs
With -Denable-mobile-bindings=true, libyams_mobile no longer compiled:
- diff/cat called IDocumentService::retrieveDocument, which was replaced
  by retrieve(); they now fetch stored bytes through retrieve() with
  acceptCompressed left false, so content comes back uncompressed, and
  map service errors through map_error_code instead of NOT_FOUND.
- repair awaited DaemonClient::call() as a future and read response
  fields that no longer exist; it now goes through daemon_call() and
  folds RepairResponse::operationResults into the C result.
- YAMS_PROFILE_ZONE is not a defined macro; use YAMS_ZONE_SCOPED_N.
- doctor built its version with std::string(char); emit JSON instead.
- yams_mobile_string_view_destroy, _cat_result_destroy and
  _restore_result_destroy were declared but never defined.

The repair/diff/cat/restore/model/doctor declarations sat outside the
extern "C" block and inside #ifdef __cplusplus, so C and FFI hosts could
not see them and C++ builds exported mangled names. Move them into the
C block.

Daemon-mode contexts handed their own worker_pool executor to the
process-wide connection pool keyed by socket path. The pool outlived
yams_mobile_context_destroy, so a second context reused sockets bound to
a freed thread pool (ASAN heap-use-after-free in AsioConnection) and the
daemon was left with connections that never drained, which then tripped
a use-after-free in daemon shutdown. Contexts now use single-use
connections and disconnect the client before joining the worker pool.

Tests: add an embedded cat/diff test (exact bytes, line diff, identical
docs, not-found mapping) and a daemon test that creates and destroys two
contexts and checks the server returns to its baseline connection count.

This proves the Linux x86_64 host build and host-side behavior only. It
does not show that an Android .so or iOS XCFramework loads on a device.

Signed-off-by: trvon <me@trevon.dev>
2026-09-27 20:11:49 -06:00
Trevon
946944f79c
[10/14] Move large inline implementations into translation units (#150)
* test(memory-sync): give the CLI fixture the shared corpus scope it needs

94e948c6 made createMemorySyncService fail closed unless
memory_sync.corpus_scope is "shared" and updated the service-manager
test, but not the p2p CLI fixture, whose eight cases have failed at
construction ever since ("replication requires
memory_sync.corpus_scope=shared"). The fixture now sets the scope; the
cases run again and pass.

Signed-off-by: trvon <me@trevon.dev>

* refactor(memory_sync): move MemorySyncLoop bodies into a translation unit

include/yams/memory_sync/memory_sync.h was a 3,527-line header-only
class: 120 inline method definitions, 2,900 lines of implementation,
pulled into every translation unit that includes daemon.h or the p2p
protocol header. The memory-sync subsystem had no source directory at
all.

MemorySyncLoop's bodies now live in src/memory_sync/memory_sync.cpp,
compiled once into a new yams_memory_sync static library that follows
the yams_wal pattern. The header keeps the declarations, the nested
types, the constructor, one-line accessors, and the YAMS_TESTING hooks
(those must stay inline because the library is not built with the
testing define). The daemon library and its declared dependency link
the new library, as do the five memory-sync test binaries.

Zero semantic change: the move is mechanical (a scripted split that
copies each body verbatim, drops default arguments and static from the
definitions, and qualifies nested return types). All seven memory-sync
test binaries and the service-manager suite pass unchanged.

memory_sync.h: 3,527 -> 569 lines.

Signed-off-by: trvon <me@trevon.dev>

* refactor(daemon): move TuneAdvisor method bodies out of the header

TuneAdvisor.h was 3,345 lines, almost all of it the bodies of ~300
static accessors and setters, included by 60 translation units. Each
of them recompiled the whole policy surface on every touch, and any
change to one knob rebuilt most of the daemon.

The bodies now live in src/daemon/components/TuneAdvisor.cpp (which
previously held only the system-memory probe). The header keeps the
declarations, the static inline override slots (unchanged, so the
ODR story for process-wide tuning state is exactly what it was), the
YAMS_TESTING hooks, and the two methods whose bodies are
preprocessor-conditional on YAMS_TESTING; those must stay inline
because the library is not built with the testing define and moving
them would silently switch tests to the production branch.

Mechanical move by the same scripted split used for memory_sync.h:
bodies copied verbatim, static and default arguments dropped from the
definitions, nested return types qualified. The one template
(readConfiguredOverridesSnapshot) stays in the header because three
other translation units instantiate it.

The header is included well outside src/daemon (vector, repair, app
services, MCP, the CLI doctor, the ONNX plugin), and those targets do
not link yams_daemon. They used to get the bodies for free from the
header, so the move needs a home they can all link: the new
yams_tune_advisor static library (src/daemon/tune_advisor/meson.build)
builds TuneAdvisor.cpp alone, and yams_daemon, yams_vector,
yams_repair, yams_app_services, yams_mcp, yams_cli, the ONNX plugin,
and the standalone snapshot test depend on it.

Executables compiled with -DYAMS_TESTING link a second build of the
same file, yams_tune_advisor_testing, ahead of the production archive.
Under that define compatibilityEnvironment() reads the process
environment live instead of the startup snapshot, which every
env-driven tuning test relies on; before the split each such TU
instantiated the inline bodies itself with the define. Meson emits
dependencies in list order and the first archive defining a symbol
wins, so the order is load-bearing: test_deps, the benchmark
dependency lists, and the mobile ABI test all put the testing archive
first, and tests/meson.build says so. The testing archive is built
under the same predicate that enters tests/ (a plain debug configure
included), so `meson setup --buildtype=debug` keeps working. The
standalone snapshot test keeps the production build on purpose.

No behavior change; a full debug compile is clean and the tuning
suites pass unchanged.

TuneAdvisor.h: 3,345 -> 1,170 lines.

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>
2026-09-12 10:09:20 -04:00
Trevon
4228a0c5e9
chore: housekeeping — rm dead code, consolidate MCP tools, add tests, clean docs (#50)
* implemented stubbed IGraphContextService methods

* feat: wire graph communication updates in ipc, mcp and cli

* fix: fix call graph recall and add profiling coverage to graph calls sites

* fix: fixes for work where we added agent navigation to graph tooling

* fix(corpus): clean FTS and KG orphans on document delete

* fix(repair): clean orphans, honor cancellation, surface stats

* fix(rm): order content removal before metadata/KG delete. keep manifest while content is still referenced

* fix(daemon): seed built-in content extractor for readiness

* chore(cli): remove dead stubs and unwired helpers

* feat(search): named-collection scoping + provable relevance gate

* fix(refactor): rm dead stubs, inline simd, add proto/metrics tests

* chore: rm dead interfaces/enums, consolidate guards, add tests

* refactor: rm dr vaporware, collapse download handlers, clean list flags

* docs: rm stale testing investigation logs

* fix: address Copilot review findings from PR #50

- relationship_ops: return existing id when INSERT OR IGNORE skips
- document_crud_ops: propagate FTS5 cleanup error (was silent skip)
- builtin_text_content_extractor: unsigned char cast for tolower, add cctype
- mcp_server download_jobs: validate job_id for status/cancel actions
- list_command tests: unsigned char cast for isspace, add cctype include
2026-06-16 19:37:24 -04:00
trvon
a7ea3da82a
updating boost variable exposure 2026-05-03 17:03:34 -04:00
trvon
94c8e6a9db
feat(linter): running linter on codebase 2026-04-20 16:13:30 -04:00
trvon
09c8e0f959
feat: profile for building mobile bindings 2026-04-06 13:28:34 -04:00
trvon
8da76c9c47
feat: expanding mobile abi compiled support 2026-04-06 10:25:10 -04:00
trvon
1962ff6a90
fix: fix issues with mobile bindings and improving abi alignment 2026-04-06 09:11:33 -04:00
trvon
35400002c9
feat: mobile binding improvements 2026-04-06 06:43:24 -04:00
trvon
c4ae922b7a
feat: mobile bindings expansion, update mcp for graph operations and download tool hardening 2026-02-15 17:50:14 -05:00
trvon
b071112836
feat: test coverage improvements. Scaling bug addresses 2026-02-04 20:20:18 -05:00
trvon
8835d6704e
fix: removing mobile bindings from builds 2026-01-31 17:15:25 -05:00
trvon
e04effe2d2
fixing conan file so ft5s builds 2025-10-04 21:12:19 -04:00