Trevon 7076e4b283

feat(daemon): shutdown hardening + test helper linkage fixes (#44 )

* perf(daemon): coalesce metadata writes

* refactor(metadata): split repository ops

* fix(metadata): harden repository invariants

* feat: test coverage and continued refactor work in metadata code

* fix(graph): handle path-like explore queries

- prefer file path symbol lookup for graph explore\n- sanitize alias FTS queries before MATCH

* fix(daemon): quiesce shutdown background work

- Interrupt pending DB acquires before worker shutdown
- Wait for tracked background coroutines and timers to drain

* fix(daemon): cancel repair work on shutdown

Blocked embed and post-ingest queues could keep salvage and\nstuck-doc repair alive during shutdown. Consume cancellation in\nrepair retry loops and lock the integrated ServiceManager path\nwith regressions so shutdown fails fast instead of drifting.

* perf(metadata): reduce sqlite lock contention

* perf(metadata): flatten batch content retries

* fix(daemon): align shutdown wait budgets

A low shutdown timeout override could expire before ServiceManager\nfinished its 5s+30s WorkCoordinator join path. Share the join\nbudget constants, clamp graceful lifecycle waits to that floor,\nand lock the behavior with a daemon regression.

* perf(metadata): bound feedback write waits

* perf(metadata): prep batch content before tx

* fix(daemon): honor no-embeddings tasks

Post-ingest queue tasks dropped the per-task noEmbeddings flag,\nso focused extraction tests still emitted embed jobs and Windows\nCI exercised extra async paths. Preserve the flag through the bus,\nskip embed dispatch when requested, and wake the extraction poller\nimmediately after successful queueing.

* perf(daemon): batch fallback metadata writes

* feat(perf,docs): optimizations for metadata writes, prompt updates and test improvements

* perf(metadata): flatten insert retries

* fix(metadata): clear CI portability blockers

Use the shared portable ScopedEnvVar in metadata search-query helper tests, add mutators for test reuse, and normalize Statement::bindAll integral dispatch so long long arguments bind consistently across platforms. Add focused regression coverage for the bindAll long long path.

* perf(metadata): batch writes and steady bench

* fix(daemon): skip inactive post-ingest pollers

* perf(metadata): tighten tag and path caches

* perf(metadata): cache small batch upserts

* fix(daemon): harden embedded host teardown

* feat(docs): updating docs policies and language

* perf(metadata): tune doc update retries

* perf(metadata): bound download retries

* perf(metadata): bound download metadata waits

* bench(metadata): scope post-ingest content path

* fix(daemon): export embedded test hooks

* fix(daemon): harden test helper linkage

* fix(build): add missing standard includes

2026-06-08 16:29:44 -04:00

6.4 KiB

Raw Permalink Blame History

YAMS — Yet Another Memory System

Persistent memory for LLMs and apps. Content-addressed storage with dedupe, compression, full-text and vector search.

license language github builds last commit

Warning

Experimental — not production ready. Expect bugs and breaking changes until 1.0.

Features

SHA-256 content-addressed storage with block-level dedupe (Rabin chunking)
Full-text search (SQLite FTS5) + semantic vector search (embeddings)
Tree-sitter symbol extraction for 18 languages (list)
Snapshot management with Merkle tree diffs and rename detection
WAL-backed durability, high-throughput I/O, thread-safe
CLI, MCP server, and C-ABI plugins (ONNX/GLiNER/ColBERT, S3 storage, PDF via ZYP)
Interactive relevance tuning through CLI tuning and doctor workflows

Documentation

Topic	Link
Install	docs/user_guide/installation.md
CLI reference	docs/user_guide/cli.md
MCP server	docs/user_guide/mcp.md
Embeddings	docs/user_guide/embeddings.md
Plugins	docs/PLUGINS.md
Build from source	docs/BUILD.md
Architecture	docs/architecture/
Benchmarks	docs/benchmarks/README.md
Changelog	docs/changelogs/
Roadmap	docs/roadmap.md

Community and contributing

Questions, bug reports, docs fixes, tests, and feature ideas are all welcome.

If you'd like to contribute, start with:

CONTRIBUTING.md for workflow and expectations
CODE_OF_CONDUCT.md for community standards
SECURITY.md for responsible vulnerability reporting
.github/ISSUE_TEMPLATE/ for issue templates

If you're unsure whether something is worth opening, it usually is.

Install

Supported: Linux x86_64/ARM64, macOS x86_64/ARM64, Windows x86_64.

# macOS
brew install trvon/yams/yams

# Docker
docker pull ghcr.io/trvon/yams:latest

# Debian/Ubuntu, Fedora/RHEL, Windows: see installation guide

Full install matrix and package repos: docs/user_guide/installation.md.

Build from source

./setup.sh Release          # Linux/macOS (auto-detects toolchain, runs Conan + Meson)
meson compile -C build/release

./setup.ps1 Release         # Windows
meson compile -C build/release

Requires a C++20 toolchain (GCC 13+, Clang 16+, or MSVC 2022+ recommended), Meson, Ninja, CMake, pkg-config, and Conan. See docs/BUILD.md.

Quick Start

yams init                                # interactive; use --auto for headless
yams add ./README.md --tags docs
yams add src/ --recursive --include="*.cpp,*.h" --tags code

yams search "config file" --limit 5
yams grep "TODO" --include="*.cpp"

yams list --limit 20
yams get <hash> -o ./output.bin

Shell completions: yams completion bash|zsh|fish|powershell. Install instructions: docs/user_guide/cli.md#cmd-completion.

MCP Server

YAMS ships an MCP server over stdio (JSON-RPC) for AI assistants.

yams serve

{
  "mcpServers": {
    "yams": { "command": "yams", "args": ["serve"] }
  }
}

Tool reference and MCP client setup: docs/user_guide/mcp.md.

Plugins

yams plugin list                                  # loaded plugins
yams plugin trust add ~/.local/lib/yams/plugins   # trust a directory
yams plugin health                                # status
yams doctor plugin onnx                           # diagnose

Plugin architecture, trust model, and bundled plugins (ONNX, S3, ZYP, GLiNER, symbol extractor): docs/PLUGINS.md.

GPU acceleration (ONNX)

Platform	Provider	Hardware
macOS	CoreML	Apple Silicon Neural Engine + GPU
Linux	CUDA	NVIDIA GPUs
Linux	MIGraphX	AMD GPUs (ROCm)
Windows	DirectML	Any DirectX 12 GPU (NVIDIA, AMD, Intel)

Simeon backend

YAMS uses simeon by default for both dense vector embeddings and lexical (BM25) search — no model download required. Set embeddings.backend = "simeon" in your TOML config (or leave it unset; simeon is the default). Fine-grained knobs live under [embeddings.simeon].

Full config reference and dim tradeoffs: docs/user_guide/embeddings.md.

Troubleshooting

yams doctor              # full diagnostics
yams stats --verbose     # storage statistics
yams repair --all        # repair common issues

Build issues: docs/BUILD.md. Empty yams plugin list? Add a trust path: yams plugin trust add ~/.local/lib/yams/plugins.

Cite

@misc{yams,
  author = {Trevon Williams},
  title = {YAMS: Content-addressable storage with semantic search},
  year = {2025},
  publisher = {GitHub},
  url = {https://github.com/trvon/yams}
}

6.4 KiB Raw Permalink Blame History