Commit graph

17 commits

Author SHA1 Message Date
trvon
7755f3e2b2
fix(onnx): keep one CoreML model cache per ONNX Runtime version
The ONNX plugin cached compiled CoreML models directly in the model
directory. With the plugin now able to run on either the system or the
bundled ONNX Runtime, a cache compiled by one version was reused by the
other: on macOS, after a run on Homebrew's 1.30 the bundled 1.23 failed
every embedding batch with 'Non-zero status code returned while running
..._CoreML_... node'. The default cache is now
<model dir>/coreml-ort-<runtime version>; YAMS_COREML_CACHE_DIR still
overrides it.

Verified on macOS arm64: a run on the system runtime creates
coreml-ort-1.30.0, a run with Homebrew's copy hidden uses the bundled
runtime, creates coreml-ort-1.23.0 and embeds new documents without
CoreML errors.

Signed-off-by: trvon <me@trevon.dev>
2026-10-01 23:39:41 -06:00
trvon
fe07469620
feat(plugins): prefer a system ONNX Runtime, fall back to a bundled copy
The ONNX plugin dlopen()ed ONNX Runtime from directories next to the
plugin first and only then from a few system prefixes, with no notion of
which copy it got or why, and the shipped packages contained no copy at
all, so on most hosts the plugin could only report "runtime not found".

The runtime resolution moves into plugins/ort_runtime/, built as a
hidden-visibility static library both ONNX-using plugins link (their
export maps still expose only the plugin C ABI). The policy is split out
as pure code (ort_runtime_resolution.{h,cpp}) so it is testable without a
runtime. Candidates are tried in this order and the first compatible one
wins:

  1. YAMS_ONNX_RUNTIME_LIB / YAMS_ONNX_RUNTIME_DIR (existing pin, kept),
     then the plugin config key runtime_library ([plugins.onnx] /
     [plugins.glint], a file or directory)
  2. a system copy: the loader search by soname, then the usual library
     directories (multiarch dir, /usr/lib64, /usr/lib, /usr/local/lib,
     /opt/onnxruntime, Homebrew) for versioned names such as Debian's
     libonnxruntime.so.1.23
  3. the private copy in <libdir>/yams/onnxruntime/
  4. legacy locations next to the plugin (build trees)

A candidate is accepted only if it exports OrtGetApiBase, GetApi() returns
an API table for the ORT_API_VERSION the plugins are compiled against, and
it meets the 1.23 minimum; otherwise it is skipped with a reason. A soname
lookup that returns a copy another plugin already loaded from the bundled
directory is reported as that copy. The choice ("Using system|bundled ONNX
Runtime <ver> from <path> (<why>)") and every skipped candidate go to the
plugin log and, through the libyams_onnx_resource host sink (new
OnnxConcurrencyRegistry::emitHostLog), to the daemon log; plugin health
reports runtime_source. With no compatible copy the plugin reports "ONNX
Runtime unavailable: ..." instead of failing.

Tests: tests/unit/plugins/ort_runtime_resolution_catch2_test.cpp covers
the order, the override pin, configured directories, system-over-bundled
preference, skipping an incompatible system copy, the not-found result
and the already-loaded classification.

Signed-off-by: trvon <me@trevon.dev>
2026-10-01 22:42:15 -06:00
trvon
2d0df863b7
fix: improved plugin locking around onnx session state and ownership improvements 2026-04-12 00:33:08 -04:00
trvon
d5967deedf
fix: make Gemma CoreML fallback automatic 2026-04-02 13:56:42 -04:00
trvon
86ec015eea
feat: massive commit. Download logic updates with expanding controls (stop, start, resume etc.) from daemon control. This is used in my archive openclaw agent. This commit also includes work to harden onnxruntime abi access. this should allow loading newer abi's when builds are pinned to 1.23.0. 2026-03-21 17:12:47 -04:00
trvon
a9bad18fa4
feat: adding additional tests, making the aggressive profile scaling better 2026-02-17 19:15:26 -05:00
trvon
8f38ee76be
feat: updating metrics for search and retrieval for future learning system. Additional tuning for search performance 2026-02-13 21:09:26 -05:00
trvon
b40b60f054
fix: bug fixes for resource limiting and onnxruntime model shapes for embeddings 2026-02-12 01:52:33 -05:00
trvon
44a94eabe1
feat: fuzzing and config expansion. Moving to optimizing ingestion. Adding tunable embedding (chunk size etc.) 2026-02-11 23:21:26 -05:00
trvon
04b55be44e
fix: apply tuning logic for profiles to GPU 2026-02-10 15:49:26 -05:00
trvon
2e966c4f1b
fix: windows build fixes 2026-02-09 23:26:33 -05:00
trvon
7764a0c3c9
fix: ROCM/MIGraphX onnxruntime test validate improvements 2026-02-09 21:22:15 -05:00
trvon
fd32c519c5
fix: staging ROCM improvements 2026-02-09 20:07:52 -05:00
trvon
c83110319d
feat: benchmark, model testing / enhancements, insertion performance updates 2026-02-08 18:33:29 -05:00
trvon
a3522b8b22
fix: expose gpu usage, add command imrpovments and other bug fixes 2026-02-08 14:04:57 -05:00
trvon
567ee32a99
fix: fixing macos coreml detection 2026-02-06 10:16:35 -05:00
trvon
df1348f3c7
fix: tuning improvements and better tests. Now working address memory issue of huge queue when ingesting allot of files 2026-02-06 04:31:09 -05:00