# Changelog Archive: v0.6.x Series

## [v0.6.11] - 2025-09-06

### Fixed
- MCP server: add_directory and get_by_name could appear to hang when the daemon was not ready or when directories were passed without recursive. The MCP handlers now:
  - Check daemon readiness up front and fall back to local services (IndexingService/DocumentService) when unavailable, matching CLI behavior.
  - Fast‑fail directory adds without `--recursive` with a clear error (parity with CLI).
  - Resolve name→hash via DocumentService in get_by_name before attempting streamed transfers; this avoids unnecessary GetInit/GetChunk paths.
  - Reduce overly long default timeouts for directory indexing to avoid perceived hangs (header 30s, body 300s).
- Daemon status -d could hang while models were loading. Status path no longer calls blocking provider APIs (e.g., `getLoadedModels()`), and the defensive objects-dir scan is capped by files/time to ensure immediate responses.
- Embeddings via daemon IPC were unreliable because server handlers were missing. Added request handling on the daemon side for generate embedding(s), load/unload model, and model status; client RPCs now complete as expected.
- ONNX plugin discovery: plugin loader now detects installed plugins using a compile‑time `YAMS_INSTALL_PREFIX` macro; resolves cases where the ONNX provider failed to load after install.
- Extraction backlog: RepairCoordinator now performs a one‑time backlog enqueue on startup (when idle) so existing documents without embeddings get scheduled; prevents "extraction_pending" from staying high indefinitely.
- Build (Darwin/Clang): resolved "no type named 'stop_source' in namespace 'std'" in `SocketServer` by using the existing `yams::compat::stop_token` and removing the unnecessary `<stop_token>` include.
- Daemon components: fixed mismatched forward declaration of `StateComponent` (class vs struct) to silence `-Wmismatched-tags`.
- IPC RequestHandler logging: avoided evaluated operand in `typeid` usage to silence `-Wpotentially-evaluated-expression`.
- Extraction: corrected a dangling `std::string_view` when parsing range tokens in `format_handler.cpp`.

### Changed
- MCP add_directory: `recursive` now defaults to `true` in the MCP tool schema for parity with CLI directory indexing. Timeouts tuned (30s/300s) and clearer error messages on misuse.
- Keep-hot semantics for embeddings: when `[embeddings].keep_model_hot = true` (i.e., `lazyLoading=false`), the ONNX model pool now pre‑creates a session for hot models (preCreateResources=true) so embeddings are immediately usable after preload.
- Status detail now reports provider presence without enumerating models to avoid lock contention; `onnx_models_loaded` is "unknown" in this non-blocking mode.
- Tests: enable model provider and plugin auto‑loading in daemon unit tests (with lazy loading, no forced preload) to exercise real provider paths instead of bypassing them.
- Build (SourceHut): `.build.yml` adds a Boost fallback install when `BoostConfig.cmake` is missing, ensuring `find_package(Boost CONFIG)` succeeds with Conan CMakeDeps output.
- Build/CMake: define `YAMS_INSTALL_PREFIX` for `yams_daemon` so runtime plugin discovery includes `$prefix/lib/yams/plugins`.
- Embedding service: unified on compat `jthread`/`stop_token`; simplifies cross‑platform behavior and cooperative stop.
- PDF extractor: locally suppressed deprecated `std::wstring_convert/std::codecvt` warnings under Clang to reduce build noise while a modern replacement is planned.
- Storage plugin loader: marked unused parameters to quiet `-Wunused-parameter` without changing behavior.

### Notes
- Large `extraction_pending` values reflect missing embeddings, not just text extraction. With the backlog enqueue and ONNX provider discovery fixes, pending counts should drop as embeddings are generated. Ensure the ONNX plugin builds (onnxruntime present) or set `YAMS_PLUGIN_DIR` to the plugin output directory during development/CI.

## [v0.6.10] - 2025-09-06

## CI bump
- macOS build failures: replaced direct `std::jthread`/`std::stop_token` usages with our portability shim (`yams::compat`) across daemon, services, and embedding service; resolves libc++ gaps on hosted macOS.
- GitHub Actions release workflow: corrected heredoc in summary generation to avoid "unexpected EOF"; now writes Python output to a temp file and reads it safely.

## [v0.6.9] - 2025-09-05

### Notes
- Upgrading daemon is recommended: older daemons (v1) will log a one‑time warning "Daemon protocol v1 < client v2" and some fields will be supplemented client‑side.

### Known Issues
- Daemon startup may remain in "Initializing" when embeddings are configured to keep the model hot (preload) and the ONNX stack cannot complete early model resolution on some systems. Workarounds:
  - Set `[embeddings].keep_model_hot = false` and `[embeddings].preload_on_startup = false` in `~/.config/yams/config.toml` to use lazy loading (model loads on first use).
  - Or export `YAMS_DISABLE_MODEL_PRELOAD=1` before starting the daemon.
  - Status derives from the lifecycle FSM; optional subsystems (models) should not gate readiness in recent builds. If you still see "Initializing", ensure you are running the updated daemon and ONNX plugin.

### Changed
- Status handler hardened to a minimal, safe snapshot; FSM/MUX metrics gated by lifecycle (Ready/Degraded) to avoid init races.
- Linux CPU proxy: read `Threads:` from `/proc/self/status` (robust) instead of parsing `/proc/self/stat` field 20.
- Stats defaults: `yams stats` now begins with the System Health section before compact counters.
- `yams stats`: prefers daemon JSON values to avoid zeros; local fallback includes vector DB size.
- `yams status`: supplements from stats JSON when talking to older daemons (v1) to avoid zeros; shows services summary and waiting components when not ready.
- Service detectors hardened:
  - SearchExecutor now includes a reason when unavailable: `database_not_ready | metadata_repo_not_ready | not_initialized`.
  - ONNX models status reports loaded count with clear guidance to download a preferred model.
- CLI search defaults to hybrid and now auto-retries with fuzzy when strict/hybrid returns zero results for better “true hybrid” behavior.
- Service metadata search path now falls back to fuzzy when full-text returns no results.
 - Model CLI help updated with subcommands and `--url` usage; clearer guidance to avoid confusion with the top-level `download` command.
 - Model CLI guidance: after downloads, success output includes `yams config embeddings model <name>` and related steps; `yams model list` and `yams model info` now mirror a one-line configuration hint.

### Added
- IPC Protocol v2:
  - StatusResponse now carries runtime fields: running, ready, uptime_seconds, requests_processed, active_connections, memory_mb, cpu_pct, version.
  - GetStatsResponse carries numeric fields alongside JSON: total_documents, total_size, indexed_documents, vector_index_size, compression_ratio.
- Daemon stats JSON: explicit `not_ready` flag; includes `durations_ms`, `top_slowest`, and latency percentiles (`latency_p50_ms`, `latency_p95_ms`).
- Bootstrap status file: `~/.local/state/yams/yams-daemon.status.json` with readiness, progress, durations, and top_slowest for pre‑IPC visibility.
- CLI doctor/status: shows top 3 slowest components with elapsed ms when available (from bootstrap JSON).
- CLI UI (retro): compact text for status and stats; `yams stats -v` shows System Health and detailed sections; `stats vectors` prints compact block.
- CLI daemon status (-d):
  - One-line services summary: `SVC  : ✓ Content | ✓ Repo | ✓ Search | ⚠ (0) Models`.
  - WAIT line during initialization listing not-ready services with progress (e.g., `WAIT : search_engine (70%), model_provider (0%)`).
- ServiceManager:
  - Preferred model preload on startup: uses configured `embeddings.preloadModels` (first entry) when no models are loaded; falls back to `all-MiniLM-L6-v2`.
  - Sanity warning when `SearchExecutor` is not initialized despite database and metadata repo ready.
- FSM metrics emission (server): counts payload writes/bytes sent, header reads/bytes received; exposed in Status when Ready/Degraded.
- LatencyRegistry (server): lightweight histogram; p50/p95 emitted via stats JSON.
- Protocol compatibility guard: one‑time warning when daemon/client protocol versions differ.
- SocketServer::setDispatcher(RequestDispatcher*) for safe future hot‑rebinds (not used by default).
- Integration test: daemon_status_stats_integration_test covers Status/Stats proto presence and basic FSM exposure.
- MCP list: added `paths_only` parameter to the MCP list tool; forwarded to daemon `ListRequest.pathsOnly` to engage the hot path (no snippet/metadata hydration).
- MCP grep: introduced `fast_first` option that returns a quick semantic suggestions burst when requested.
- MCP server: startup flags to set hot/cold modes consistently with CLI:
  - `--list-mode` → sets `YAMS_LIST_MODE`
  - `--grep-mode` → sets `YAMS_GREP_MODE`
  - `--retrieval-mode` → sets `YAMS_RETRIEVAL_MODE`
 - CLI model management:
   - `--url` override to download any model by name from a custom URL.
   - Subcommands: `yams model list|download|info|check` (aliases to flags).
   - `yams model check` shows ONNX runtime support and plugin directory status, plus autodiscovers installed models under `~/.yams/models`.
 - Model autodiscovery: `yams model list` now lists locally installed models found at `~/.yams/models/<name>/model.onnx`.
 - Daemon model selection: honors `embeddings.preferred_model` from `~/.config/yams/config.toml` (or XDG config) to preload and prefer that model at runtime.
 - Model path resolution: honors `embeddings.model_path` as the models root (with `~` expansion) and resolves name-based models in priority order: configured root → `~/.yams/models` → `models/` → `/usr/local/share/yams/models`; full paths are used as-is.
 - Docs: PROMPT updated to show multi-path `yams add src/ include/ ...` examples.

### Fixed
- Build: resolved C++ signature mismatch for `EmbeddingService::runRepair` by aligning the implementation with header declarations and providing a proper non-jthread legacy runner.
- MCP schemas updated for list (`paths_only`) and grep (`fast_first`) to reflect new behaviors.
- ONNX plugin: CMake fixes to ensure dynamic plugin loads cleanly:
  - Disable IPO/LTO on `yams_onnx_plugin` (INTERPROCEDURAL_OPTIMIZATION FALSE) to prevent LTO symbol internalization and tooling noise.
  - Ensure exported C symbols have default visibility (`C_VISIBILITY_PRESET/CXX_VISIBILITY_PRESET default`, `VISIBILITY_INLINES_HIDDEN OFF`) so `getProviderName`/`createOnnxProvider` remain visible.
  - On ELF, link with `-Wl,-z,defs` to catch unresolved symbols at link time.
  - Resolves daemon warning: "Plugin missing getProviderName function: .../libyams_onnx_plugin.so".
 - Retrieval now accepts `sha256:<hex>` hashes (normalized before validation) in `DocumentService` `retrieve` and `cat`.
 - Model downloads provide clearer errors when offline (curl exit code mapping for host resolution/connect failures).

## [v0.6.8] - 2025-09-04

### Changed
- Data directory resolution uses a consistent precedence across CLI and daemon: configuration file > environment (`YAMS_STORAGE`/`YAMS_DATA_DIR`) > CLI flag. This avoids CLI defaults masking configured storage roots.
- MCP search defaults hardened: runs in hybrid mode with fuzzy matching enabled by default and a similarity threshold of 0.7 when the client doesn’t provide options.

### Added
- MCP get-by-name and cat now include a fuzzy fallback: if direct lookup fails, a hybrid fuzzy search runs and returns the strongest single match, then retrieves by hash (preferred) or normalized path.
- URL-aware naming: post-index for MCP downloads now sets the document name from the URL basename (query stripped), improving name-based retrieval. MCP get-by-name also accepts full URLs and normalizes them to the basename automatically.

### Fixed
- macOS build compatibility and warnings:
  - `std::jthread/std::stop_token` portability: embedding service falls back to `std::thread` with an atomic stop on platforms lacking libc++ support.
  - Third‑party noise reduced: mark `hnswlib` includes as SYSTEM and silence `sqlite-vec.c`; fence cast-qual warnings under Clang for HNSW.
  - Resolved shadowing and unused variable/parameter warnings across content handlers (PDF, audio, image, binary, archive) and daemon client helpers.
- Name propagation for downloads: downloaded artifacts are indexed with a human-friendly name, making `get --name <file>` work reliably after MCP/CLI downloads.

## [v0.6.7] - 2025-09-04

### CI bump
- SourceHut build fixes
- GitHub CI updates

## [v0.6.6] - 2025-09-04

### CI bump
- SourceHut build fixes
- Github CI updates

### Fixed
- Search regression from adding query results. Querys will now be supported by default in the fallback path after expressions are extracted

### Known Issues
- Data directory path resolutions needs to be audited

## [v0.6.5] - 2025-09-04

### Changed
- Release workflow: migrate to Conan 2 profiles using a dedicated host profile (`conan/profiles/host.jinja`) that enforces `compiler.cppstd=20` and sets `compiler.libcxx=libc++` on macOS.
- CMake (Darwin): replace GNU ld flags (`--start-group/--end-group`, `--whole-archive/--no-whole-archive`) with `-Wl,-force_load` for specific archives in CLI, MCP server, and daemon.
- CI (act): install build tools when missing in containers (cmake, ninja, build-essential/pkg-config) to allow local act runs of the release job.

### Added
- Search ergonomics:
  - `-q, --query` flagged alias to safely pass queries that start with '-' (e.g., `yams search -q "--start-group --whole-archive"`).
  - `--stdin` and `--query-file <path>` to read the query from stdin or a file (`--query-file -` also reads stdin).
  - Helpful "Query is required" guidance when no query is provided, with tips to use `-q/--query`, `--` to stop option parsing, or stdin/file inputs.

### Fixes
- macOS linking failures due to GNU-only linker flags appearing in executable link lines; now prevented and defensively stripped if injected elsewhere.
- SourceHut `.build.yml`: use the Conan host profile and disable ONNX (`-o enable_onnx=False`) to avoid upstream onnxruntime 1.18 build errors with GCC 15; preserves successful release builds on Arch.

## [v0.6.4] - 2025-01-05

### Known Issues
- Daemon not showing correct storage details

### Added
- Streaming metrics: Track and expose `stream_total_streams`, `stream_batches_emitted`, `stream_keepalives`, and average `stream_ttfb_avg_ms` via `yams stats` (JSON and technical details in text mode).
- Grep performance: line-by-line streaming scanner (no full-file buffering) and literal fast-path for `--literal` without word-bounds/case-folding.
- Add performance: parallel directory traversal/processing with bounded workers.
- Session helpers: `yams session pin|list|unpin|warm` for hot data management (feature is experimental and may not work reliably).

### Changed
- List paths-only mode now avoids snippet/metadata hydration end-to-end for faster responses; honors `pathsOnly` through daemon to services.
- Retrieval (cat/get) prefers extracted text (hot) when available before falling back to CAS (cold), respecting per-document `force_cold`.
- Grep hot/cold race: daemon emits hot first-burst and cancels cold; batch/first-burst tuning supported via env for testing.
- Delete command now mirrors `rm` ergonomics:
  - `rm` alias retained; `-f` is an alias for `--force`; `-r` is an alias for `--recursive`
  - Positional targets supported (names/paths/patterns); when a single target is provided with `-r`, it is treated as a directory
  - Multiple positional targets are treated as a names list
  - Selector requirement relaxed when positional targets are used; mode (directory/pattern/name) is inferred heuristically
- List paths-only mode now avoids snippet/metadata hydration end-to-end for faster responses; honors `pathsOnly` through daemon to services.
- Retrieval (cat/get) prefers extracted text (hot) when available before falling back to CAS (cold), respecting per-document `force_cold`.
- Grep hot/cold race: daemon emits hot first-burst and cancels cold; batch/first-burst tuning supported via env for testing.
- CMake: Platform-aware linker selection.
  - Removed hard-coded `-fuse-ld=lld` from presets.
  - Toolchain now attempts `-fuse-ld=lld` only on non-Apple platforms when supported, gated by `YAMS_PREFER_LLD` (default ON).
  - macOS Release builds use `-Wl,-dead_strip` instead of GNU `--gc-sections`.
- Release workflow (macOS): Align Conan arch with target:
  - `macos-arm64` uses `-s arch=armv8`
  - `macos-x86_64` uses `-s arch=x86_64`

### Fixed
- Streaming search finalization: avoid infinite keepalive loop on empty results; keepalive cadence configurable via `YAMS_KEEPALIVE_MS`.
- Stats command: `yams stats help` now prints a concise system metrics guide.
- MCP server timeouts with some clients:
  - Async handlers now run detached (no premature task destruction), preventing “Context server request timeout”
  - Server sends `notifications/ready` after client `notifications/initialized` for better client compatibility
  - Stdio framing hardened to consume trailing CR/LF after payloads to avoid parse glitches across clients
- Streaming search finalization: avoid infinite keepalive loop on empty results; keepalive cadence configurable via `YAMS_KEEPALIVE_MS`.
- Stats command: `yams stats help` now prints a concise system metrics guide.
- Docker: ARM64 build now mirrors AMD64 by using Buildx with proper tag suffixes (-arm64) and pushes versioned and latest tags. Multi-arch manifest creation no longer fails with "not found ... -arm64".
- Release workflow (macOS/Linux): Conan 2 profile updates use correct keys (compiler.cppstd, compiler.libcxx) via `conan profile update`; removed brittle sed/grep edits that broke on macOS.
- Release workflow (Windows): Corrected Conan 2 profile update syntax (`compiler.cppstd=20`).

## [v0.6.3] - 2025-01-04

### Changes
- CI version bump for source hut and github action builds
- `yams add` supports multiple files
- Adding small optimizations to decompression

### Known Issues
- MCP server usage my vary in successful returns
- [v0.6.2] CLI performance still degraded, will be addressed in subsequent release

## [v0.6.2] - 2025-01-04

### Hot fixes
- CI version bump
- MCP server patch (tested working in zed and lm studio but not working in goose or jan)

## [v0.6.1] - 2025-01-03

### Hot fixes
- Docker and sourcehut build file updates
- Daemon liveness checks on startup bug fix where daemon did not signal start
- Added more aggressive parallelization for search now that it holds all resources (will make tunable)

### Known Issues
- MCP server init not working with some clients
- CLI performance has been degraded as it creates socket to daemon each request, working on session mode
- `yams stats -v` not working as expected

## [v0.6.0] - 2025-01-03

### Repository
- **Will move all future development work to experimental branch so that main and releases become more stable**
  - I apologize for recent instabilities

### Added
- Server multiplexing: fair round-robin writer with per-turn byte budget and backpressure caps (default ON).
- Status: exposes multiplexing metrics (active handlers, queued bytes, writer budget).
- Cancel control frame: request type added and server-side cancel scaffolding (per-request contexts).
- **Development Changes**
  - Updated tasks for vscode and zed tasks
  - Enforcing clang-tidy warnings as errors
  - Attempting to stabilize build system as plugin system is implemeneted per roadmap
- **Daemon Logging Rotation**
  - Use rotating file sink to preserve logs across crashes
  - Log rotation info: `Log rotation enabled: /path/to/logfile.log (max 10MB x 5 files)`

- **Connection State Machine**
  - Deterministic `ConnectionFsm` for IPC connection lifecycle management
  - Clean state transitions: Disconnected → Connecting → Connected → ReadingHeader → ReadingPayload → WritingHeader → StreamingChunks → Error/Closed
  - FSM metrics collection gated by daemon log level (debug/trace only)
  - Coordination with `RepairFsm` for safe data integrity operations
  - Guardrails for readable/writable operations based on FSM state
  - Header-first streaming with chunked transfer support

- **Protobuf IPC Migration**
  - Complete migration from custom binary serialization to Protocol Buffers
  - `ProtoSerializer` as the single payload codec for client/server
  - Comprehensive `ipc_envelope.proto` schema with oneof for all request/response types
  - Transport framing (header, CRC, streaming) remains unchanged
  - MAX_MESSAGE_SIZE enforcement in encode/decode paths
  - Older non-protobuf clients are incompatible after this release

### Changed
- **CLI Commands**
  - All CLI commands migrated to async-first pattern with `run_sync` bridge
  - Removed deprecated `daemon_first()` and `daemon_first_pooled()` helpers
  - Commands now use `async_daemon_first()` with proper timeout handling
  - Consistent error handling and retry logic across all commands

- **MCP Server**
  - All handlers converted to async (`yams::Task<Result<T>>`)
  - Improved concurrency and reduced blocking operations

- **Daemon Architecture**
  - Socket server runs in-process, eliminating need for external IPC server
  - `state_.readiness.ipcServerReady` now properly reflects socket server status
  - Graceful shutdown sequence: socket server stops before services
  - Better resource lifecycle management and error propagation

### Fixed
- **Async Infrastructure**
  - Removed spin-wait bridges that caused CPU waste
  - Fixed head-of-line blocking in request processing
  - Proper cancellation handling for in-flight operations
  - Eliminated `Task::get()` usage from production code paths

### Removed
- **Deprecated Synchronous APIs**
  - `AsioClientPool::roundtrip()`, `status()`, `ping()`, `call()` (sync versions)
  - `PooledRequestManager::execute()` (now returns NotImplemented error)
  - `YAMS_ASYNC_CLIENT` environment variable (async is now mandatory)
  - Legacy `MessageSerializer` and custom binary serialization code
  - All synchronous daemon helper functions
  - Forward declarations of `BinarySerializer`/`BinaryDeserializer` remain in `ipc_protocol.h` for cleanup in v0.7.0