090a5ca69c* perf(search, metadata): add KG write buffer, score cache, adaptive budgets * perf(indexing): wire KGWriteBuffer into indexing service Converts per-document addEdgesUnique calls into batched flush via KGWriteBuffer. 500-doc ingest: 500 SQL round-trips → 1 batch. Verified: 14.2× edge insert gap (14.8K→195K edges/s) in kg_edge bench. Tests: 10/10 Catch2 pass (105 assertions). * perf(storage): add fsync durability, parallel batch, read integrity * test(storage): add stress, crash-sim, cold-cache, batch benchmarks * fix: CI optimizations and work coordinator shutdown fix * fix(testing): stabilize TSAN coverage gate * fix: harden KG flush, score cache, and benches - keep KG buffers intact when batch commit fails - count flushed edges per batch instead of lifetime totals - guard the Simeon score cache and clear it on rebuild - fall back to fsync when F_FULLFSYNC fails on macOS - remove interactive sudo from cold-cache benchmarks
7.4 KiB
7.4 KiB
Building YAMS
Cross-platform build for Linux, macOS, and Windows. Canonical build graph is Meson; setup.sh / setup.ps1 handle toolchain detection, Conan deps, and Meson configure in one step.
Quick start
# Linux / macOS
./setup.sh Release
meson compile -C build/release
# Windows
./setup.ps1 Release
meson compile -C build/release
If setup.sh / setup.ps1 reports that Conan supplied Meson/Ninja for the build directory, use the generated local wrapper (build/.../mesonw on Unix, mesonw.ps1 on Windows) for subsequent meson compile|install|test.
Run ./setup.sh --help (or setup.ps1 -?) for the full flag set.
Prerequisites
| Platform | Compiler (min / recommended) | Notes |
|---|---|---|
| Linux / macOS | GCC 11+ / 13+ | C++20 |
| Linux / macOS | Clang 14+ / 16+ | C++20 |
| Windows | MSVC 193+ (VS 2022) | "Desktop development with C++" workload |
Build tools: Meson, Ninja, CMake 3.23+, pkg-config (Linux/macOS), Conan (default dep path), Python 3.10+.
System libraries
# Ubuntu / Debian
sudo apt-get install -y build-essential cmake meson ninja-build pkg-config libomp-dev libfaiss-dev \
libssl-dev libcurl4-openssl-dev libsqlite3-dev protobuf-compiler libprotobuf-dev zlib1g-dev
# Fedora
sudo dnf install -y gcc-c++ cmake meson ninja-build pkg-config \
openssl-devel libcurl-devel sqlite-devel protobuf-devel zlib-devel
# macOS
brew install cmake meson ninja pkg-config openssl sqlite protobuf
Build options
setup.sh environment variables
| Variable | Default | Purpose |
|---|---|---|
YAMS_OFFLINE |
false | Use only local Conan cache; disable wrap downloads |
YAMS_USE_SYSTEM_DEPS |
false | Skip Conan; use system pkg-config / CMake prefixes |
YAMS_COMPILER |
auto | Force gcc or clang |
YAMS_CPPSTD |
20 | C++ standard (20 or 23) |
YAMS_DISABLE_ONNX |
false | Disable ONNX plugin |
YAMS_DISABLE_SYMBOL_EXTRACTION |
false | Disable Tree-sitter symbols |
YAMS_ONNX_GPU |
auto | auto|cuda|coreml|directml|migraphx|none |
YAMS_INSTALL_PREFIX |
/usr/local | Install location |
YAMS_PKG_CONFIG_PATH |
unset | Extra host pkg-config path |
YAMS_CMAKE_PREFIX_PATH |
unset | Extra host CMake prefix |
YAMS_MESON_NATIVE_FILE |
unset | Override Meson native file |
YAMS_MESON_CROSS_FILE |
unset | Override Meson cross file |
SOURCE_DATE_EPOCH |
unset | Reproducible build timestamp |
Canonical sources: setup.sh, setup.ps1.
Meson options
Commonly used flags — see meson_options.txt for the full list:
| Option | Default | Purpose |
|---|---|---|
build-cli |
true | Build CLI binary |
build-mcp-server |
true | Build MCP server |
build-tests |
false | Build test suite |
enable-onnx |
enabled | ONNX embedding models |
plugin-symbols |
true | Tree-sitter symbol plugin |
plugin-s3 |
true | S3 object storage plugin |
meson configure build/release -Dbuild-tests=true
Dependency versions live in conanfile.py.
Offline and system-deps builds
# Local Conan cache only
./setup.sh Release --offline
# System dependencies, no Conan
YAMS_USE_SYSTEM_DEPS=true YAMS_OFFLINE=true \
YAMS_PKG_CONFIG_PATH=/opt/yams-deps/lib/pkgconfig \
YAMS_CMAKE_PREFIX_PATH=/opt/yams-deps \
./setup.sh Release --system-deps --offline
# CMake wrapper (delegates to Meson)
cmake -S . -B build/cmake-bootstrap \
-DYAMS_PROFILE=Release -DYAMS_OFFLINE=ON -DYAMS_USE_SYSTEM_DEPS=ON
cmake --build build/cmake-bootstrap
Cross builds: set YAMS_MESON_CROSS_FILE, YAMS_BUILD_PKG_CONFIG_PATH, and YAMS_PKG_CONFIG_PATH.
Manual Meson setup
Skip setup.sh and drive Meson directly when you need custom profiles:
conan install . -of build/release -s build_type=Release -s compiler.cppstd=20 --build=missing
meson setup build/release \
--native-file build/release/build-release/conan/conan_meson_native.ini \
--buildtype=debugoptimized -Db_ndebug=true
meson compile -C build/release
Windows requires the local ONNX Runtime Conan recipe once:
conan export conan/onnxruntime --name=onnxruntime --version=1.23.0
conan install . -of build\release -pr:h conan/profiles/host-windows-msvc -pr:b default `
-s build_type=Release --build=missing
meson setup build\release --native-file build\release\build-release\conan\conan_meson_native.ini `
--buildtype=debugoptimized -Db_ndebug=true
meson compile -C build\release
Testing
./setup.sh Debug
meson configure build/debug -Dbuild-tests=true
meson compile -C build/debug
meson test -C build/debug --print-errorlogs
For test-suite shaping guidance (fast correctness lane vs explicit slow stress /
migration / diagnostics suites), see docs/developer/testing.md.
Troubleshooting
| Symptom | Fix |
|---|---|
Clang: cannot find -lstdc++ |
sudo apt install libstdc++-13-dev or YAMS_COMPILER=gcc ./setup.sh Release |
| Windows Boost build failures (VS 2025) | Install v143 toolset, conan remove 'boost/*' -c, rerun setup |
| Windows: missing onnxruntime recipe | conan export conan/onnxruntime --name=onnxruntime --version=1.23.0 |
| Missing system packages | pkg-config --list-all | grep -E "(sqlite|openssl|protobuf)" |
Advanced
# Coverage (manual)
./setup.sh Debug --coverage && meson test -C build/debug --suite unit && gcovr --html -o build/debug/coverage.html
# Or use the tracked local pre-push gate
git config core.hooksPath .githooks
YAMS_COVERAGE_INCLUDE_INTEGRATION=1 .githooks/pre-push
# LTO
CXXFLAGS="-flto" LDFLAGS="-flto" ./setup.sh Release
# Native tuning (local benchmarks only — do not ship)
CXXFLAGS="-march=native" ./setup.sh Release