27 KiB
27 KiB
Changelog
All notable changes to YAMS (Yet Another Memory System) will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Archived Changelogs
- v0.6.x archive: docs/changelogs/v0.6.md
- v0.5.x archive: docs/changelogs/v0.5.md
- v0.4.x archive: docs/changelogs/v0.4.md
- v0.3.x archive: docs/changelogs/v0.3.md
- v0.2.x archive: docs/changelogs/v0.2.md
- v0.1.x archive: docs/changelogs/v0.1.md
Known Issues
- Daemon may show broken connection with rapid CLI/MCP usage
- Embeddings generation consumes all of daemon IPC bandwidth. This will become immediately apparent after the onnx plugin is loaded with
yams plugin load onnx. The system will attempt to generate all missing embeddings. - We have noticed high CPU usage of the daemon when idling. We will continue to investigate and optimize this issue.
[v0.6.36] - 2025-09-16
Fixed
- Adding homebrew formula as module
[v0.6.35] - 2025-09-16
Added
- CLI doctor: proactively triggers a plugin scan when neither typed providers nor
plugins_jsonare available, then briefly waits so results appear. Matchesyams plugin listbehavior for faster diagnosis. - CLI add: auto-detects directory paths and enables
--recursiveautomatically (LLM/UX friendly) with an informational log.
Changed
- Doctor plugin listing now prefers daemon typed providers from
status.providersand falls back togetStats().additionalStats["plugins_json"]for parity withyams plugin list. Retains macOSotool -Lhint for degraded plugins. - Directory ingestion: daemon dispatcher treats any directory path as directory ingestion (forces recursive=true) even if clients omit the flag.
Fixed
- ServiceManager: corrected placement of
env_truthy()(was nested in another function) and added<cctype>include to fix macOS build failures. - DocumentService: early, clear error when a directory path is sent to the single-file
store()path (preventsfile_size: Is a directory).
[v0.6.34] - 2025-09-16
Fixed
- CI improvements and fixes for release assets
- Fixing regression from LLD cmake linking
[v0.6.33] - 2025-09-16
Fixed
- MCP server
- Improved download logic to improve retrieving content from a retrieved files
- CI/CD
- Sourcehut: Improving build packaging step
- Github Actions: Improving test.yml configuration step
[v0.6.32] - 2025-09-16
Fixed
- Fixing linking errors seen on ubuntu 24.04
- CI/CD
- Sourcehut: Fixing how packaging is exposed
- Github Actions: Fixing test dependency setup and release package steps
[v0.6.31] - 2025-09-16
Fixed
- CI/CD
- GitHub Actions: tests workflow YAML fixed (indentation of
timeout-minutesunder the job). - Docker workflow: ARM64 builds now use QEMU + Buildx on
ubuntu-latest; nativeubuntu-24.04-armrunners no longer required for ARM publishing. Multi-arch manifest unchanged.
- GitHub Actions: tests workflow YAML fixed (indentation of
[v0.6.30] - 2025-08-16
Fixed
- Fixed regressions in CLI / MCP UI performance from the introduction of the daemon
- ARM Build: Boost
find_packagefailure resolved via corrected Conan toolchain path and added generator directory visibility during CMake configure.
Added
- Packaging: Introduced Homebrew formula templating (
packaging/homebrew/yams.rb.template) with placeholder substitution for version + sha256 during stable releases. - Release workflow: Generates and commits
latest.jsonmanifest plus Homebrew formula in a single pass; adds commit step guarded to stable channel. - Homebrew: Added
livecheckblock for future tap/live version detection. - Services
- RetrievalService parity for CLI and MCP (get/list/grep) with shared facade.
- DocumentIngestionService used by CLI add and MCP add for daemon‑first ingestion.
- Daemon: Native chunked get protocol handlers (GetInit/GetChunk/GetEnd) backed by in‑memory RetrievalSessionManager, enabling efficient large content retrieval and capped buffers.
Changed
- CI/Build
- Unpinned Conan across tests and release workflows (allow latest compatible Conan 2.x) while retaining reproducibility via profile + lock hashing.
- Updated
CMakePresets.jsontoolchain paths (removed incorrect nestedbuild/Debug/build/Releasesegments) enabling correct Boost discovery on ARM. - Extended test job timeout to 50 min (ARM warm builds) and added Conan/Boost diagnostics aiding cache hit analysis.
- Replaced heredoc inline formula generation with template-driven substitution reducing workflow maintenance surface and diff noise.
- CLI + MCP: Switched get/list/grep to RetrievalService; add paths to DocumentIngestionService to eliminate bespoke logic and timeouts/hangs divergences.
- CLI: Guarded
cfg.dataDirassignment behindhasExplicitDataDir()throughout commands per refactor plan. Centralized DaemonClient data‑dir resolution in default constructor (env → configcore.data_dir→ XDG/HOME →./yams_data). - RetrievalService:
getChunkedBufferprefers the native chunked protocol and falls back to unaryGetonly forNotImplemented/Timeout/NetworkError. Improved name‑smart get to resolve via filename when hybrid search returns a path (macOS path aliasing resilience).
[v0.6.29] - 2025-09-14
CI Bump
- Fixing github actions by removing conan version pin and fixing test.yml
[v0.6.28] - 2025-09-14
CI Bump
- Fixing github actions macos key signing regression
[v0.6.27] - 2025-09-14
Changed
- CI/build: Nightly packaging auto-enables on
refs/heads/main(no time-based scheduler needed). Improved repo-root detection (yams/vs.) across all tasks, and broadened artifact globs to support both layouts. - CI/build: Hardened packaging tasks — CPack RPM now guarded by
rpmbuildavailability and won’t fail the job if RPM tooling is missing; DEB packaging remains enabled. Addedset -eto packaging steps for clearer early exits while keeping RPM optional. - Knowledge Graph: Added unit tests for alias exact/fuzzy resolution, neighbor retrieval, and doc-entity roundtrips (
kg_store_alias_and_entities_test.cpp). - Search + KG: Added SimpleKGScorer end-to-end test validating entity and structural scores and explanations (
kg_scorer_simple_test.cpp).
[v0.6.26]
CI Bump
- Sourcehut debian image updates for packaging on fedora
- Github CI improvements for testing and release builds
[v0.6.25] - 2025-09-14
CI Bump
- Fixing github actions cache and release naming
[v0.6.24] - 2025-09-14
CI Bump
- Fixing build matrix used in github actions to fix artifact uploading.
[v0.6.23] - 2025-09-14
CI Bump
- Increasing test timeouts and fixing packing errors with sourcehut builds
[v0.6.22] - 2025-09-14
CI Bump
- Fixing macos env error
[v0.6.21] - 2025-09-14
Fixes
- Fixing prefix for sourcehut builds
- Adding cache for Github CI
[v0.6.20] - 2025-09-13
Added
- Asynchronous post‑ingest pipeline (daemon): decouples heavy work from
add/add_directory.- New
PostIngestQueueperforms extraction → full‑text index → knowledge graph upserts in the background. - Queue metrics (threads, queued, processed, failed) with EMAs for latency and throughput are exposed via
status.
- New
- Knowledge Graph enrichment beyond tags:
- Lightweight analyzers extract URLs, emails, and optional file paths and add
MENTIONSedges. - Caps and toggles are controlled via
TuneAdvisor.
- Lightweight analyzers extract URLs, emails, and optional file paths and add
- Batch KG operations:
KnowledgeGraphStore::upsertNodes(std::vector<KGNode>)used in the post‑ingest worker to avoid read‑after‑write races.- New
KnowledgeGraphStore::addEdgesUnique(std::vector<KGEdge>)batches edge inserts with on‑conflict de‑duplication on(src_node_id, dst_node_id, relation).
- TuneAdvisor controls (code‑level, no env):
kgBatchNodesEnabled,kgBatchEdgesEnabled(default true)- Analyzer toggles:
analyzerUrls,analyzerEmails,analyzerFilePaths(file paths off by default) maxEntitiesPerDoc(default 32)
- MCP observability and parity:
- New
statustool that returns daemon readiness and counters (includes post‑ingest and MCP worker pools). - New
yams://statusresource for symmetry withyams://stats. - Minimal
doctortool summarizes degraded subsystems and suggests actions.
- New
Changed
addandadd_directorydelegate extraction/index/graph work to the post‑ingest queue;DocumentServicehonorsdeferExtractionto keep adds fast.- Post‑ingest worker now batches KG node upserts and uses unique batch edge inserts to reduce DB round‑trips.
- Vector/Readiness metrics mirrored into
status.requestCountsfor MCP/CLI parity.
Fixed
- Daemon readiness consistency across CLI/UI:
DaemonMetricsnow derives booleanreadyfrom the lifecycle state (authoritative), not deprecatedfullyReady(). Preventsdoctorshowing "NOT READY, state=Ready".- Normalized status strings to lowercase ("ready", "degraded", etc.) in the status path to match serializer/client expectations and avoid case-related derivation bugs.
- Stats
not_readyflag now reflects lifecycle readiness (Ready/Degraded →false) instead of alwaystrue, soyams statswon’t unnecessarily fall back to local mode. - Bootstrap status JSON (
yams-daemon.status.json) now writes a lowercaseoverallfield for consistency with IPC lifecycle strings.
- Stats daemon reachability:
- Increased
yams statsshort-mode timeout from 1.5s → 5s and set explicit connect/header/body timeouts to avoid premature local fallback, aligning withstatus/doctorbehavior. not_readyheuristic is now lifecycle-driven (Ready/Degraded → false), so stats no longer assumes unready by default.- Temporary mitigation:
yams statscurrently forces local-only output while we investigate a daemon GetStats streaming/header path issue that can stall and cause slow fallbacks. The daemon path will be re-enabled once the bug is fixed.
- Increased
[v0.6.19] - 2025-09-13
Fixed
- MCP stdio framing correctness and async send:
- Fixed double-encoding on outbound async path:
outboundDrainAsyncnow uses a framed string sender instead of passing a serialized JSON string tosend(const json&). - Removed extra CRLF after framed payload; framed responses now send exactly
Content-Lengthbytes after the blank line (LSP/MCP compliant). - Added
StdioTransport::sendFramedSerialized(const std::string&)for pre-serialized payloads used by the outbound drain.
- Fixed double-encoding on outbound async path:
- MCP prompts/get schema correctness for MCP Inspector:
- Roles now use only
assistantanduser(removedsystem). contentis a single object{ "type": "text", "text": ... }, not an array.
- Roles now use only
- macOS/Linux build/link improvements:
- Resolved yams-daemon undefined symbols on Linux by including IPC OBJECT files in the daemon static lib and linking
yams_ipc_proto. - Fixed yams-mcp-server linking on macOS by avoiding IPC OBJECTs that pull daemon-only symbols; explicitly link IPC/client/proto libs instead.
- Resolved yams-daemon undefined symbols on Linux by including IPC OBJECT files in the daemon static lib and linking
Changed
- StdioTransport testability: in test builds (
YAMS_TESTING), receive() usescin.rdbuf()->in_avail()to support reliable stringstream-driven tests. - Default server send now includes
Content-Type: application/vscode-jsonrpc; charset=utf-8and a trailing CRLF when using header framing to maximize client compatibility. - ONNX plugin dependency hygiene: Removed unintended PDFium bundling logic from the ONNX plugin CMake (it never linked or required PDFium). Added a configure-time guard warning if
pdfium::pdfiumbecomes linked accidentally to keep plugin responsibilities isolated. PDF extraction remains solely in thepdf_extractorplugin.
Added
- ONNX/GenAI path (vector library):
- Scaffolded a minimal GenAI adapter in the vector layer and made GenAI the default when ONNX is enabled; falls back to raw ONNX Runtime when GenAI is unavailable.
Daemon & Plugins
- Plugin autoload diagnostics: adoption logs now include the plugin file path; environment
YAMS_PLUGIN_DIRis prioritized over system directories to avoid stale installs.
[v0.6.18] - 2025-09-13
Hot fix
- CI Bump : CMake and conan update for
[v0.6.17] - 2025-09-13
Hot fix
- CI Bump : Resolving compile compatibiliy
Changed
- Build: Removed legacy
-fcoroutines-tsflag for modern Clang toolchains (C++20 coroutines are enabled by default). Prevents build failures on distributions where the flag is rejected. - Build: Fixed macOS Conan host profile parsing error by removing unsupported
[env]section; libc++ enforcement now handled via workflow sanitization. - Build: Added defensive scrub in
CMakeLists.txtto strip any externally injected-fcoroutines-tsoccurrences from cached flags.
[v0.6.16] - 2025-09-13
Hot fix
- CI Bump : Fixing syntax
[v0.6.15] - 2025-09-13
Hot fix
- CI Bump : Updating clang version in host.ninja for builds
[v0.6.14] - 2025-09-12
Added
- CLI Doctor:
yams doctor dedupe— detects (and optionally deletes) duplicate documents grouped by--mode(path|name|hash) with keep strategies (--strategy keep-newest|keep-oldest|keep-largest), dry-run by default, and safety flag--forcefor hash mismatches.yams doctor embeddings clear-degraded— attempts to clear the embedding degraded state by loading a preferred/available model (interactive confirmation, best-effort).
- Status + CLI plugin diagnostics:
- StatusResponse now carries typed
providersand askippedPluginsarray (path + reason). yams plugins listprefers typed providers and supports--verboseto print a “Skipped plugins” section with clear reasons (preflight/dlopen/symbols/name-policy).
- StatusResponse now carries typed
- Config and init:
- New
[daemon].plugin_name_policywith valuesrelaxed(default) orspec(requirelibyams_*_plugin.*). examples/config.v2.tomldocuments the field;yams init --enable-pluginsnow writesplugin_dirand setsplugin_name_policy = "spec"for canonical naming by default.- New
[daemon].auto_repair_batch_sizedefaulted to 16; daemon applies it when starting the RepairCoordinator.
- New
- macOS plugin robustness:
- ONNX plugin CMake now bundles ONNX Runtime dylibs (existing) and PDFium dylib into local subdirs and injects
@loader_path/onnxruntimeand@loader_path/pdfiuminto rpaths. SIP‑safe, no DYLD tweaks.
- ONNX plugin CMake now bundles ONNX Runtime dylibs (existing) and PDFium dylib into local subdirs and injects
Changed
- Safer ABI plugin scanning:
scanTarget()no longerdlopen()s candidates (prevents constructor crashes); macOSdlopen_preflight()is performed inload()and failures are handled gracefully.- Filename filter accepts
libyams_*_plugin.*andyams_*_plugin.*; strict policy can be enforced via config/env.
- CLI
yams plugins listauto‑starts the daemon, triggers a scan, waits briefly for readiness, and prints typed providers. Helpful guidance is printed if no info found.
Fixed
- macOS crash on plugin scan/load:
- Eliminated segfaults by avoiding
dlopen()during scan; added preflight + robust error logging in load path; fixeddlerror()usage (read once, avoid null concatenation).
- Eliminated segfaults by avoiding
- Repair throttling and backpressure:
- RepairCoordinator honors config batch size and TuneAdvisor env knobs (
YAMS_REPAIR_MAX_BATCH,YAMS_REPAIR_TOKENS_IDLE/BUSY,YAMS_REPAIR_MAX_BATCHES_PER_SEC, etc.) and pauses token issuance when hitting per‑second caps to keep the daemon responsive.
- RepairCoordinator honors config batch size and TuneAdvisor env knobs (
- Linux build/linking:
yams-daemonnow links explicitly againstyams::daemon_ipcandyams::daemon_client, resolving undefined symbol errors (ConnectionFsm/RequestHandler/ResourceTuner) on Ubuntu/ELF toolchains.
- Test/CI stability:
run_syncrewritten to use a promise +co_spawn(detached)andco_await std::move(aw)to avoid move/copy pitfalls with Boost.Asio awaitables.
- Examples/config:
examples/config.v2.tomlupdated withplugin_name_policyand daemon plugin notes; defaults align with init.
Notes
- If your macOS ONNX plugin previously failed with
libpdfium.dylibnot found, rebuild/reinstall the plugin; the new CMake will bundle PDFium into a localpdfium/folder and adjust rpaths to@loader_path/pdfium. - Background repair can be tuned at runtime via environment without restarting; see the
TuneAdvisorvariables above for recommended settings during heavy foreground load.
[v0.6.13] - 2025-09-11
Hotfixes
- Build stabilizations and improvements from MacOS testing
[v0.6.12] - 2025-09-11
Added
- CLI:
yams graph— read‑only graph viewer that mirrorsget --graph.- Usage:
yams graph <hash> [--depth 1-5]oryams graph --name "<path>" [--depth N]. - Shows related documents via the knowledge graph without fetching content.
- Usage:
- Config: added
[daemon]section toexamples/config.v2.tomlwithenable = true,auto_load_plugins = true, optionalplugin_dir, and notes aboutYAMS_PLUGIN_DIRdiscovery. - CLI/build: restored Hot/Cold mode helpers and include wiring for list/grep/retrieval.
yams doctor- Yams plugin and daemon health are now under one command
- Doctor checks for vector DB vs model embedding dimension mismatch with clear fix steps.
- Doctor warnings for missing companion files (config.json, tokenizer.json) next to model.onnx.
- Plugin System
- Generic C‑ABI plugin loader with trust policy (scan/load/unload; trust list/add/remove)
- IPC handlers for plugin scan/load/unload/trust; CLI
yams pluginsupports list/info/scan/load/unload/trust - DR CLI gating:
yams drsubcommands (status/agent/help) now require plugins and print a guidance message; see docs/PLUGINS.md - Spec docs: docs/spec/plugin_spec.md, WIT drafts (object_storage_v1.wit, dr_provider_v1.wit), JSON Schemas (manifest/object_storage_v1/dr_provider_v1)
- External examples scaffold under docs/examples/plugins and guide at docs/guide/external_dr_plugins.md for out‑of‑tree GPL plugins (S3/R2)
- CLI model tooling
yams model provider: shows active model provider status (loaded models) and the preferred model from config for transparency.
- Daemon plugin UX
- Autoload from trusted + standard directories on startup (disable with
YAMS_AUTOLOAD_PLUGINS=0). - Trust‐add autoload: newly trusted directories/files are scanned and loaded immediately (best‑effort).
- Load by name: resolves logical names by scanning plugin descriptors across default dirs (e.g.,
yams plugin load onnx). - External plugin host (metadata only): supports
.yams-plugin.jsonlayouts for non‑ABI external integrations with trust and listing.
- Autoload from trusted + standard directories on startup (disable with
- Embedding provider safeguard
- When a plugin model_provider_v1 is adopted, the daemon auto‑selects
embeddings.preferred_model = "all-MiniLM-L6-v2"if the user hasn’t set one (or if it’s set to a non‑ONNX default). This nudges embeddings to the ONNX path by default.
- When a plugin model_provider_v1 is adopted, the daemon auto‑selects
- Status proto v3: extended
StatusResponseover protobuf to carry readiness (per-subsystem), init progress,overall_status, and request counts. Serializer/deserializer updated with back‑compat casting and safe parsing. - Streaming progress events for long ops:
EmbedEventandModelLoadEventover IPC. - Startup hints: daemon logs a hint to run
yams repair --embeddingson new/migrated data dirs and a hint to useyams stats --verbosefor monitoring.
Changed
- Documentation: general improvements and clarifications.
- CI: improved build reliability and semantic version enforcement.
yams doctorsummary now includes a Knowledge Graph section; when the graph is empty it recommends:yams doctor repair --graph.- PDF Extraction: The built-in PDF text extraction logic has been refactored into a standalone plugin (
pdf_extractor) that implements the newcontent_extractor_v1andsearch_provider_v1interfaces. - Model download command now uses the unified downloader for all artifacts (model + companions).
- Improved repo/URL inference; clearer errors list exact URLs attempted.
- ONNX model provider: batched ONNX inference for efficiency; expanded config parsing (hidden_size, max_position_embeddings).
- Init: default v2 config now includes a
[daemon]section withenable = trueandauto_load_plugins = trueso plugin adoption (e.g., ONNX) works out‑of‑the‑box afteryams initwhen plugins are present in standard directories. - CLI: added DR command (gated) and extended plugin command with install/enable/disable/verify stubs (no‑ops for now)
- Stats default view becomes adaptive: when service‑specific metrics or recommendations are relevant, the non‑verbose path includes the Recommendations and Service Status sections automatically; the compact STATS footer remains.
- Recommendation text updated to be actionable: suggests
yams repair --embeddingsinstead of a vague “run indexing”. - Doctor now performs smarter PID checks: prints the configured PID file and also checks a stable fallback in
/tmp, reporting active/stale states explicitly. - CLI daemon alignment:
get,grep, andlistnow initializeDaemonClientwith the CLI data directory and standardized timeouts (header 30s, body 120s). The CLI also seedsYAMS_STORAGEandYAMS_DATA_DIRso daemon auto-start binds to the same storage as the CLI (parity withsearch). - MCP server storage selection: server now resolves and sets
DaemonClientdataDirfrom environment/config (priority:YAMS_STORAGE→YAMS_DATA_DIR→~/.config/yams/config.toml→XDG_DATA_HOME/HOME) and seeds the same env vars for consistent daemon storage across processes. - MCP tools:
gettool schema now defaultsinclude_contenttotrueso content is returned by default unless explicitly disabled. - Embeddings: streaming and stability improvements
- Client streams batch embeddings by default when batch > 1 and retries transient failures with exponential backoff and dynamic sub‑batching (min 4).
- Added detailed ONNX per‑item inference progress logs and total timing in ModelManager::runInference for better diagnosis of hangs.
- Increased default daemon client timeouts to 120s; configurable via environment:
YAMS_REQUEST_TIMEOUT_MS,YAMS_HEADER_TIMEOUT_MS,YAMS_BODY_TIMEOUT_MS. - Doctor/repair now surfaces EmbeddingEvent progress to stdout during
yams doctor repair --embeddingsso users see live progress. - Daemon respects client streaming hints for embedding requests; emits progress chunks and a final response.
- Daemon logs batch embedding processing time (ms).
- Daemon mux tuning centralized via TuneAdvisor (header-only):
- New server-side knobs:
YAMS_SERVER_MAX_INFLIGHT,YAMS_SERVER_QUEUE_FRAMES_CAP,YAMS_SERVER_QUEUE_BYTES_CAP,YAMS_SERVER_WRITER_BUDGET_BYTES. - Existing knobs:
YAMS_WRITER_BUDGET_BYTES,YAMS_MAX_MUX_BYTES,YAMS_WORKER_THREADS,YAMS_CHUNK_SIZE. - SocketServer now honors
TuneAdvisor::chunkSize()for streaming chunk size (no 32 KiB cap). - RequestHandler reads server settings via TuneAdvisor for consistent behavior.
- New server-side knobs:
Fixed
- Knowledge Graph repair:
yams doctor repair --graphnow populates nodes/entities.- Implemented SQLite KG node upsert/lookups; previously stubbed methods caused
updated=0(no-op) during repair even with existing tags/metadata.
- Implemented SQLite KG node upsert/lookups; previously stubbed methods caused
- Plugin list over protobuf
GetStatsResponseJSON now embedsplugins_json; serializer/deserializer preserve it soyams plugin listshows loaded plugins reliably.
- Framing/transport for plugin ops
- Resolved “Frame build failed” during
plugin scan/loadby aligning CLI and daemon payloads on the protobuf Envelope path.
- Resolved “Frame build failed” during
- Build/link fixes
- Link
nlohmann_json::nlohmann_jsonintoyams_ipc_proto(proto serializer uses JSON). - Link
-ldlfor CLI doctor on Linux (dlopen indoctor plugin). - Fixed unterminated string literals in plugin CLI output.
- macOS: CMake configuration and linking fixes for Darwin toolchain (guard LINK_GROUP to Linux; use -force_load where appropriate).
- Link
- External plugin host
- Corrected trust file newline writes, regex escapes, and Result usage in load(); ensured proper namespace scoping to fix build.
Notes
- When
[embeddings].enable = false, the daemon disables the model provider and plugin auto‑loading to reduce startup overhead; enable embeddings to allow autoload to adopt a provider (e.g., ONNX).
Known Issues
- Embeddings with
nomic-embed-text-v1.5(ONNX) may time out or fail under the daemon's streaming path on some systems.- Symptoms:
yams repair --embeddingsreports "Embedding stream failed: Read timeout" and logs showEmbedDocuments dispatch: model='nomic-embed-text-v1.5'. - Status: under investigation in the experimental branch (plugin/SDK path).
- Workarounds:
- Prefer other ONNX models:
all-MiniLM-L6-v2(fast) orall-mpnet-base-v2(higher quality):yams config embeddings model all-MiniLM-L6-v2(or pass--model all-MiniLM-L6-v2to commands)yams daemon restart
- Raise client timeouts for long embedding runs:
export YAMS_CLIENT_HEADER_TIMEOUT_MS=60000; export YAMS_CLIENT_BODY_TIMEOUT_MS=300000
- Reduce batch pressure:
yams config embeddings tune balancedoryams config embeddings batch_size 8then restart daemon. - Preload the model and smoke test a single embed:
yams model load all-MiniLM-L6-v2yams embed --model all-MiniLM-L6-v2 "hello world"
- Ensure the ONNX plugin is loaded and onnxruntime resolves:
yams plugin load onnx;ldd /usr/local/lib/yams/plugins/libyams_onnx_plugin.so
- Prefer other ONNX models:
- Symptoms:
- MIME detection on add: single‑file and directory adds now perform MIME detection (magic → extension fallback) when not provided; MIME is stored in both content metadata and DB.
- Embeddings generation robustness:
- Batch UTF‑8 sanitization prevents protobuf “invalid UTF‑8” errors.
- Default embedding targets restricted to text‑like MIME; binary types (e.g., PDFs/images) are skipped unless explicitly included via
--include-mime. - Daemon model preload is attempted only when the model provider is ready to avoid spurious read timeouts; Hybrid backend continues via local fallback.
- Auto-repair during degraded state: RepairCoordinator now allows small background batches while the daemon is not fully ready, limited by a configurable active‑connections threshold; also gates work using live
activeConnectionsinstead of assuming idle. - MCP server: store by path could return "File not found" due to fragile path resolution.
handleStoreDocumentnow normalizes paths robustly:- Strips
file://scheme, expands~, resolves relative paths against$PWDand processcurrent_path(), and appliesweakly_canonicalbest‑effort. - Prevents spurious failures on repeated adds and differing working directories.
- Strips
- Retrieval via daemon appearing to return no content or “not found” while
searchworked. CLIgetand MCPget/catnow reliably hit the same storage assearch, resolving mismatches caused by unsetdataDiror differing environments.