Commit graph

8 commits

Author SHA1 Message Date
trvon
fe07469620
feat(plugins): prefer a system ONNX Runtime, fall back to a bundled copy
The ONNX plugin dlopen()ed ONNX Runtime from directories next to the
plugin first and only then from a few system prefixes, with no notion of
which copy it got or why, and the shipped packages contained no copy at
all, so on most hosts the plugin could only report "runtime not found".

The runtime resolution moves into plugins/ort_runtime/, built as a
hidden-visibility static library both ONNX-using plugins link (their
export maps still expose only the plugin C ABI). The policy is split out
as pure code (ort_runtime_resolution.{h,cpp}) so it is testable without a
runtime. Candidates are tried in this order and the first compatible one
wins:

  1. YAMS_ONNX_RUNTIME_LIB / YAMS_ONNX_RUNTIME_DIR (existing pin, kept),
     then the plugin config key runtime_library ([plugins.onnx] /
     [plugins.glint], a file or directory)
  2. a system copy: the loader search by soname, then the usual library
     directories (multiarch dir, /usr/lib64, /usr/lib, /usr/local/lib,
     /opt/onnxruntime, Homebrew) for versioned names such as Debian's
     libonnxruntime.so.1.23
  3. the private copy in <libdir>/yams/onnxruntime/
  4. legacy locations next to the plugin (build trees)

A candidate is accepted only if it exports OrtGetApiBase, GetApi() returns
an API table for the ORT_API_VERSION the plugins are compiled against, and
it meets the 1.23 minimum; otherwise it is skipped with a reason. A soname
lookup that returns a copy another plugin already loaded from the bundled
directory is reported as that copy. The choice ("Using system|bundled ONNX
Runtime <ver> from <path> (<why>)") and every skipped candidate go to the
plugin log and, through the libyams_onnx_resource host sink (new
OnnxConcurrencyRegistry::emitHostLog), to the daemon log; plugin health
reports runtime_source. With no compatible copy the plugin reports "ONNX
Runtime unavailable: ..." instead of failing.

Tests: tests/unit/plugins/ort_runtime_resolution_catch2_test.cpp covers
the order, the override pin, configured directories, system-over-bundled
preference, skipping an incompatible system copy, the not-found result
and the already-loaded classification.

Signed-off-by: trvon <me@trevon.dev>
2026-10-01 22:42:15 -06:00
trvon
d5967deedf
fix: make Gemma CoreML fallback automatic 2026-04-02 13:56:42 -04:00
trvon
a0b4915bcd
fix: bypass broken Gemma CoreML path and restore ANN/TurboQuant search 2026-04-01 21:09:22 -04:00
trvon
86ec015eea
feat: massive commit. Download logic updates with expanding controls (stop, start, resume etc.) from daemon control. This is used in my archive openclaw agent. This commit also includes work to harden onnxruntime abi access. this should allow loading newer abi's when builds are pinned to 1.23.0. 2026-03-21 17:12:47 -04:00
trvon
cc28b8a343
feat: instrumented embedding for deeper insight into hotspots. Fixed CoreML scaling that resulted in high ram usage due to VRAM calculation. Expanded testing for embedding and draining of work 2026-02-11 12:07:08 -05:00
trvon
c83110319d
feat: benchmark, model testing / enhancements, insertion performance updates 2026-02-08 18:33:29 -05:00
trvon
a3522b8b22
fix: expose gpu usage, add command imrpovments and other bug fixes 2026-02-08 14:04:57 -05:00
trvon
567ee32a99
fix: fixing macos coreml detection 2026-02-06 10:16:35 -05:00