Persistent memory for LLMs and apps. Content-addressed storage with dedupe, compression, full-text and vector search.
- C++ 93.1%
- Python 2.4%
- Shell 1.6%
- Meson 1.6%
- Lean 0.7%
- Other 0.6%
ae15bf317a* fix(daemon): harden model state ownership Keep ModelPoolConfig single-defined and serialize telemetry snapshots to avoid ODR drift and torn concurrent reads. * refactor(daemon): consolidate ingest pollers Require the wake-timer contract so every post-ingest stage shares one event-driven poller configuration. * refactor(daemon): unify repair execution Share admission, operation ordering, cancellation, progress, and result aggregation between synchronous and asynchronous repair paths. * refactor(daemon): unify runtime tuning Resolve startup and reload settings through one typed path so parsing, precedence, and range validation cannot drift. * refactor(daemon)!: remove dead IPC surfaces Delete unreferenced framing, cancellation, and compatibility paths while keeping the active streaming protocol intact. BREAKING CHANGE: callers using connectAsync or legacy MessageFramer and FrameReader helpers must migrate to connect, frame_message_into, feed, and get_frame. * refactor(daemon): hide test hooks Preserve test-library linkage without exposing daemon-only probes through production headers or external symbols. * fix(daemon): avoid replaying successful adds A fresh-connection transport retry reuses the response handler. Clear its stale EOF error when the retry returns a valid add response so the outer retry loop does not enqueue the document again. * bench(ingest): enforce lossless pipeline runs Separate admission, storage, drain, enrichment, and searchability timing. Decision-grade plans now reject drops, duplicate processing, incomplete pipelines, and experiment identity drift. * bench(search): expose cold seed ingest timings Pair generalized-memory quality arms with admission, storage, drain, searchability, and enrichment timing from their shared cold seed. Keep the observation distinct from repeated throughput measurements. * perf(ingest): batch deferred content indexing Directory ingestion already schedules content work through post-ingest. Avoid duplicate inline indexing and coalesce normal-priority trickles within a bounded window. * feat(metadata): expose insert writer metrics Separate queue, batch-apply, and transaction time so ingestion runs can attribute connection pressure without changing write semantics. * fix(daemon): retry transient refused connects The daemon socket can be bound before its listener is ready during startup. Reuse the existing bounded retry budget for that refusal window. * bench(ingest): attribute submission path costs Decision-grade runs need stable coalescing identity and separate queue, apply, transaction, and estimated connection-wait costs. * perf(ingest): batch durable document storage Commit chunks, manifests, references, and metadata in bounded batches while preserving publication only after durable reference success. * perf(ingest): coalesce bulk post-processing Use a 16-item bulk cap while keeping latency-sensitive daemon ingestion on the existing four-item priority quota. * bench(ingest): record post-ingest batch size Include the effective cap in metrics and experiment identity so runs with different batching policies cannot be compared as one system. * perf(ingest): batch queued storage waves Preserve single-document fallback semantics and execute inline when the coordinator has one worker to avoid self-deadlock. * perf(ingest): route bulk enrichment batches Keep quota-sized interactive groups on the RPC lane while sending larger lossless groups through bounded coalescing. * test(daemon): initialize task fixtures * perf(ingest): batch post-ingest dispatch * perf(ingest): widen directory storage waves Borrow from a full wave when one document would otherwise fall through to single-document storage. * perf(storage): bound batch writer fanout Reuse a fixed worker set so batch size no longer creates one thread and payload copy per object. * perf(ingest): align storage batch drains Dispatch one configured batch through storage so intermediate waves do not fragment downstream metadata work. * fix(ingest): commit metadata state atomically Fold first-version, extracted-title, and embedding-completion state into existing transactions. Read committed extraction state so duplicate-content tasks cannot double-count corpus counters. * bench(ingest): measure production submission Drive pipelined files through DocumentIngestionService. Export dispatch, fingerprint, and enqueue costs through xplan. * fix(xplan): preserve native failures Report child signals and bounded stderr before readiness checks so assertions are not masked as missing traces. * fix(daemon): preserve embedding success hashes * fix(daemon): serialize pooled socket shutdown * fix(daemon): harden batch publication shutdown * fix(storage): use compat jthread shim * fix(daemon): avoid poller template shadowing * fix(search): keep vector pipeline test enabled * fix: harden shutdown and fallback paths Demote expected shutdown SQL interruptions, ignore readonly FTS integrity noise, avoid stopped-executor socket hangs, and preserve metadata writer fallback outcomes. |
||
|---|---|---|
| .act | ||
| .devcontainer | ||
| .githooks | ||
| .github | ||
| .vscode | ||
| conan | ||
| docker/local-ci | ||
| docs | ||
| examples | ||
| external | ||
| formal/topology | ||
| homebrew-yams@c51677b0eb | ||
| include | ||
| meson | ||
| packaging | ||
| plugins | ||
| public | ||
| scripts | ||
| spec | ||
| sql | ||
| src | ||
| subprojects | ||
| tests | ||
| third_party | ||
| tools | ||
| .actrc | ||
| .build.yml | ||
| .clang-format | ||
| .clang-tidy | ||
| .clangd | ||
| .cppcheck-suppressions | ||
| .dockerignore | ||
| .gitignore | ||
| .gitmodules | ||
| .infer-suppress | ||
| .opengrepignore | ||
| .quality-deadcode-ignore | ||
| .release-please-manifest.json | ||
| AGENTS.md | ||
| bench_profile.json | ||
| CHANGELOG.md | ||
| CITATION.cff | ||
| CMakeLists.txt | ||
| CODE_OF_CONDUCT.md | ||
| commitlint.config.js | ||
| conanfile.py | ||
| CONTRIBUTING.md | ||
| CppProperties.json | ||
| Dockerfile | ||
| GOVERNANCE.md | ||
| LICENSE | ||
| MAINTAINERS.md | ||
| meson.build | ||
| meson_options.txt | ||
| mkdocs.yml | ||
| NOTICE | ||
| README.md | ||
| release-please-config.json | ||
| SECURITY.md | ||
| setup.ps1 | ||
| setup.sh | ||
| spdlog.pc | ||
| tsan.suppressions | ||
YAMS — Yet Another Memory System
Persistent memory for LLMs and apps. Content-addressed storage with dedupe, compression, full-text and vector search.
Warning
Experimental — not production ready. Expect bugs and breaking changes until 1.0.
Features
- SHA-256 content-addressed storage with block-level dedupe (Rabin chunking)
- Full-text search (SQLite FTS5) + semantic vector search (embeddings)
- Tree-sitter symbol extraction for 18 languages (list)
- Snapshot management with Merkle tree diffs and rename detection
- WAL-backed durability, high-throughput I/O, thread-safe
- CLI, MCP server, and C-ABI plugins (ONNX/GLiNER/ColBERT, S3 storage, PDF via ZYP)
- Interactive relevance tuning through CLI tuning and doctor workflows
Documentation
| Topic | Link |
|---|---|
| Install | docs/user_guide/installation.md |
| CLI reference | docs/user_guide/cli.md |
| MCP server | docs/user_guide/mcp.md |
| Embeddings | docs/user_guide/embeddings.md |
| Plugins | docs/PLUGINS.md |
| Build from source | docs/BUILD.md |
| Architecture | docs/architecture/ |
| Benchmarks | docs/benchmarks/README.md |
| Changelog | docs/changelogs/ |
| Roadmap | docs/roadmap.md |
Links
- SourceHut: https://sr.ht/~trvon/yams/
- GitHub mirror: https://github.com/trvon/yams
- Docs site: https://yamsmemory.ai
- Discord: https://discord.gg/rTBmRHdTEc
- License: GPL-3.0-or-later
Community and contributing
Questions, bug reports, docs fixes, tests, and feature ideas are all welcome.
If you'd like to contribute, start with:
CONTRIBUTING.mdfor workflow and expectationsCODE_OF_CONDUCT.mdfor community standardsSECURITY.mdfor responsible vulnerability reporting.github/ISSUE_TEMPLATE/for issue templates
If you're unsure whether something is worth opening, it usually is.
Install
Supported: Linux x86_64/ARM64, macOS x86_64/ARM64, Windows x86_64.
# macOS
brew install trvon/yams/yams
# Docker
docker pull ghcr.io/trvon/yams:latest
# Debian/Ubuntu, Fedora/RHEL, Windows: see installation guide
Full install matrix and package repos: docs/user_guide/installation.md.
Build from source
./setup.sh Release # Linux/macOS (auto-detects toolchain, runs Conan + Meson)
meson compile -C build/release
./setup.ps1 Release # Windows
meson compile -C build/release
Requires a C++20 toolchain (GCC 13+, Clang 16+, or MSVC 2022+ recommended), Meson, Ninja, CMake, pkg-config, and Conan. See docs/BUILD.md.
Quick Start
yams init # interactive; use --auto for headless
yams add ./README.md --tags docs
yams add src/ --recursive --include="*.cpp,*.h" --tags code
yams search "config file" --limit 5
yams grep "TODO" --include="*.cpp"
yams list --limit 20
yams get <hash> -o ./output.bin
Shell completions: yams completion bash|zsh|fish|powershell. Install instructions: docs/user_guide/cli.md#cmd-completion.
MCP Server
YAMS ships an MCP server over stdio (JSON-RPC) for AI assistants.
yams serve
{
"mcpServers": {
"yams": { "command": "yams", "args": ["serve"] }
}
}
Tool reference and MCP client setup: docs/user_guide/mcp.md.
Plugins
yams plugin list # loaded plugins
yams plugin trust add ~/.local/lib/yams/plugins # trust a directory
yams plugin health # status
yams doctor plugin onnx # diagnose
Plugin architecture, trust model, and bundled plugins (ONNX, S3, ZYP, GLiNER, symbol extractor): docs/PLUGINS.md.
GPU acceleration (ONNX)
| Platform | Provider | Hardware |
|---|---|---|
| macOS | CoreML | Apple Silicon Neural Engine + GPU |
| Linux | CUDA | NVIDIA GPUs |
| Linux | MIGraphX | AMD GPUs (ROCm) |
| Windows | DirectML | Any DirectX 12 GPU (NVIDIA, AMD, Intel) |
Auto-detected at build. Override with YAMS_ONNX_GPU=auto|cuda|coreml|directml|migraphx|none. Details: plugins/onnx/README.md.
Simeon backend
YAMS uses simeon by default for both dense vector embeddings and lexical (BM25) search — no model download required. Set embeddings.backend = "simeon" in your TOML config (or leave it unset; simeon is the default). Fine-grained knobs live under [embeddings.simeon].
Full config reference and dim tradeoffs: docs/user_guide/embeddings.md.
Troubleshooting
yams doctor # full diagnostics
yams stats --verbose # storage statistics
yams repair --all # repair common issues
Build issues: docs/BUILD.md. Empty yams plugin list? Add a trust path: yams plugin trust add ~/.local/lib/yams/plugins.
Cite
@misc{yams,
author = {Trevon Williams},
title = {YAMS: Content-addressable storage with semantic search},
year = {2025},
publisher = {GitHub},
url = {https://github.com/trvon/yams}
}