33243d8472* fix(storage): bound filesystem listPage residency listPage() walked the whole op store and retained every matching key before slicing one page. At 1025 records and a 512-record page it held 1025 keys where 513 is the working set, and the retained set grew with the store. Traverse once into a max-heap holding the smallest eligible limit+1 keys. The walk itself stays complete, so a traversal error still returns an error instead of a partial page. Exclusive lexical cursor, ordering, Windows key decoding and the SIZE_MAX limit stay as they were. Add a YAMS_TESTING-only observation seam that counts real walks, entries visited and peak retained keys, so the traversal cost can be measured instead of inferred. The new traversal shifts two reviewed getenv allowlist line numbers. Signed-off-by: trvon <git@trevon.dev> * fix(memory-sync): keep backfill progress past an oversized vector A record above the resident limit failed publication, the domain cursor never advanced, and every later cycle retried the same row. Valid vectors behind it never reconciled, so one oversized value could pin a domain indefinitely. The daemon log showed this repeating every ~100 s. Classify the row only after a publish fails with InvalidArgument and the same serialized record is independently confirmed to exceed the immutable limit min(maxValueBytes, maxCacheBytes). Skip the row for this pass, keep the local copy, and revisit it when a new source generation or service limit appears. A generic backend InvalidArgument stays retryable, and healthy rows still serialize only once. Signed-off-by: trvon <git@trevon.dev> * fix(memory-sync): reuse a bounded complete listing across publishes Every store invalidated the cached listing pages, so a publish followed by syncFully walked the whole op store again, once per 512-record page. On the profiled daemon that re-walk was the hot path: 99.6% of one core, 87.9% of it system time, with listPage at 55% of user samples. Keep a complete key-only prefix snapshot, built only from normal contiguous pages starting at an empty cursor on the concrete filesystem backend. The snapshot is capped by key count and key bytes, served only in lexical order, and dropped as incomplete unless the walk reached the end. Any mutation invalidates it before the backend call; a successful store or remove then applies the exact key to matching snapshots, and a failure drops them because the write may have committed. Initialising the async path disables cache reuse permanently rather than racing a future. Cache maintenance is best effort: it runs after the backend result is known and any allocation failure clears the cache and keeps the committed result. Opaque cursors are untouched, so only the built-in filesystem backend gets snapshot treatment. Active publishes now cost the same number of walks as a warm sweep instead of four, six and nine at 511, 513 and 1025 records. Signed-off-by: trvon <git@trevon.dev> * test(bench): measure memory-sync listing cost across arms Add a standalone harness that drives a real filesystem op store through MemorySyncLoop, one scenario per process so no cache state survives a repeat, and reports wall, user, system and peak RSS per run. It uses production APIs only, so the same source builds against a baseline and a candidate tree and neither arm needs a measurement seam ported into it. The driver seeds one corpus per size, interleaves the two arms inside each repeat, and writes results.jsonl plus a median report. Enumeration counts are not re-derived here; the unit suite owns those. Signed-off-by: trvon <git@trevon.dev> --------- Signed-off-by: trvon <git@trevon.dev>
597 lines
20 KiB
Meson
597 lines
20 KiB
Meson
# Build standalone benchmark executables. These are not registered as Meson
|
|
# tests; CI invokes them directly to generate JSON/MD reports.
|
|
|
|
fs = import('fs')
|
|
# Benchmarks compile with -DYAMS_TESTING; link TuneAdvisor's testing bodies first (see
|
|
# tests/meson.build).
|
|
tune_advisor_testing_link = [dependency('yams_tune_advisor_testing')]
|
|
|
|
# Google Benchmark dependency (library + main entry point)
|
|
benchmark_dep = dependency('benchmark', required: false)
|
|
benchmark_main_dep = dependency('benchmark', modules: ['benchmark::benchmark_main'], required: false)
|
|
if not benchmark_dep.found()
|
|
benchmark_dep = dependency('google-benchmark', required: false)
|
|
benchmark_main_dep = dependency('google-benchmark', modules: ['benchmark::benchmark_main'], required: false)
|
|
endif
|
|
|
|
api_bench = executable('yams_api_benchmarks', files('api_benchmarks.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [test_deps, dependency('yams_benchmarks')] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
search_bench = executable('yams_search_benchmarks', files('search_benchmarks.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [test_deps, dependency('yams_benchmarks')] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
core_bench = executable('yams_core_benchmarks', files('core_benchmarks.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [test_deps, dependency('yams_benchmarks')] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
topology_build_bench = executable('topology_build_bench', files('topology_build_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: tune_advisor_testing_link + [
|
|
dependency('yams_topology'),
|
|
dependency('nlohmann_json'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
alias_target('bench_topology_build', topology_build_bench)
|
|
|
|
grep_bench = executable('yams_grep_benchmarks', files('grep_benchmarks.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [test_deps, dependency('yams_benchmarks')] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
vector_backend_engine_compare = executable('vector_backend_engine_compare',
|
|
files('vector_backend_engine_compare.cpp'),
|
|
include_directories: incs,
|
|
dependencies: tune_advisor_testing_link + [
|
|
dependency('yams_vector'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_vector_backend_engine_compare', vector_backend_engine_compare)
|
|
|
|
vector_memory_scaling_bench = executable('vector_memory_scaling_bench',
|
|
files('vector_memory_scaling_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: tune_advisor_testing_link + [
|
|
dependency('yams_vector'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_vector_memory_scaling', vector_memory_scaling_bench)
|
|
|
|
vector_persistence_warm_start_bench = executable('vector_persistence_warm_start_bench',
|
|
files('vector_persistence_warm_start_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: tune_advisor_testing_link + [
|
|
dependency('yams_vector'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_vector_persistence_warm_start', vector_persistence_warm_start_bench)
|
|
|
|
# Large-scale ingestion benchmark (requires google-benchmark and provides its own main)
|
|
if benchmark_dep.found()
|
|
large_scale_ingestion_bench_deps = tune_advisor_testing_link + [
|
|
test_deps,
|
|
dependency('yams_benchmarks'),
|
|
benchmark_dep,
|
|
] + yams_tracy_deps
|
|
|
|
large_scale_ingestion_bench = executable('yams_large_scale_ingestion_bench',
|
|
files('large_scale_ingestion_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: large_scale_ingestion_bench_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_large_scale_ingestion', large_scale_ingestion_bench)
|
|
else
|
|
message('Google Benchmark not found; skipping large_scale_ingestion_bench')
|
|
endif
|
|
|
|
if benchmark_dep.found()
|
|
storage_bench_deps = [test_deps, dependency('yams_benchmarks'), benchmark_dep] + yams_tracy_deps
|
|
|
|
storage_bench = executable('yams_storage_throughput_benchmarks',
|
|
files('storage_throughput_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: storage_bench_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
else
|
|
message('Google Benchmark not found; skipping storage throughput benchmark')
|
|
endif
|
|
|
|
if benchmark_dep.found()
|
|
retrieval_bench_deps = [test_deps, dependency('yams_benchmarks')] + yams_tracy_deps + [benchmark_dep]
|
|
|
|
retrieval_bench = executable('yams_retrieval_service_benchmarks',
|
|
files('retrieval_service_benchmarks.cpp'),
|
|
include_directories: incs,
|
|
dependencies: retrieval_bench_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
else
|
|
message('Google Benchmark not found; skipping retrieval service benchmark')
|
|
endif
|
|
|
|
if benchmark_dep.found()
|
|
metadata_query_bench_deps = tune_advisor_testing_link + [dependency('yams_metadata'), dependency('yams_search'), dependency('yams_daemon'), dependency('spdlog')] + yams_tracy_deps + [benchmark_dep]
|
|
|
|
metadata_query_bench = executable('metadata_path_query_bench', files('metadata_path_query_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: metadata_query_bench_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
# Tree-based list with filters benchmark (v0.7.5)
|
|
tree_list_filter_bench = executable('tree_list_filter_bench', files('tree_list_filter_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: metadata_query_bench_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_tree_list_filter', tree_list_filter_bench)
|
|
|
|
# Common dependencies for search-related benchmarks
|
|
search_bench_deps = tune_advisor_testing_link + [
|
|
dependency('yams_metadata'),
|
|
dependency('yams_search'),
|
|
dependency('yams_vector'),
|
|
dependency('yams_daemon'),
|
|
dependency('yams_app_services'),
|
|
dependency('spdlog')
|
|
] + yams_tracy_deps + [benchmark_dep]
|
|
|
|
# RAG retrieval quality benchmark
|
|
retrieval_quality_bench_deps = search_bench_deps + [test_deps, dependency('libcurl'), dependency('nlohmann_json')]
|
|
retrieval_quality_bench_rpath = []
|
|
if host_machine.system() == 'darwin'
|
|
retrieval_quality_bench_rpath += ['/opt/homebrew/lib']
|
|
endif
|
|
|
|
retrieval_quality_bench = executable('retrieval_quality_bench',
|
|
files(
|
|
'search/retrieval_quality_bench.cpp',
|
|
'search/retrieval_benchmark_support.cpp',
|
|
),
|
|
include_directories: incs,
|
|
dependencies: retrieval_quality_bench_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
build_rpath: ':'.join(retrieval_quality_bench_rpath),
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_retrieval_quality', retrieval_quality_bench)
|
|
|
|
# Per-arm retrieval-quality bench: scores each SimeonLexicalBackend bandit arm
|
|
# directly over a judged corpus (nDCG/Recall/MRR), deterministic (no bandit
|
|
# exploration). Drives the strategy-router keep/enable/remove decision.
|
|
simeon_arm_quality_bench = executable('simeon_arm_quality_bench',
|
|
files(
|
|
'search/simeon_arm_quality_bench.cpp',
|
|
),
|
|
include_directories: incs,
|
|
dependencies: search_bench_deps + [test_deps, dependency('nlohmann_json')],
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_simeon_arm_quality', simeon_arm_quality_bench)
|
|
|
|
entity_extraction_bench = executable('entity_extraction_bench',
|
|
files('entity_extraction_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [benchmark_main_dep, benchmark_dep, dependency('spdlog')] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_entity_extraction', entity_extraction_bench)
|
|
|
|
fts5_indexing_bench = executable('fts5_indexing_bench',
|
|
files('fts5_indexing_bench.cpp'),
|
|
include_directories: incs,
|
|
# Keep project/Conan dependencies before benchmark. On macOS runners,
|
|
# benchmark may resolve from Homebrew and add /opt/homebrew/include ahead
|
|
# of Conan's fmt; spdlog then sees the wrong fmt headers and fails to
|
|
# compile fmt::basic_format_string references.
|
|
dependencies: [
|
|
dependency('yams_metadata'),
|
|
dependency('yams_search'),
|
|
dependency('yams_daemon'),
|
|
dependency('spdlog'),
|
|
benchmark_main_dep,
|
|
benchmark_dep,
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_fts5_indexing', fts5_indexing_bench)
|
|
|
|
# KG edge/entity insert microbenchmark. Covers addEdgesUnique, addEdges,
|
|
# upsertNodes, and WriteBatch throughput with varying batch sizes.
|
|
# Used as a baseline for the KGWriteBuffer optimization (#2).
|
|
kg_edge_insert_bench = executable('kg_edge_insert_bench',
|
|
files('kg_edge_insert_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
dependency('yams_metadata'),
|
|
dependency('spdlog'),
|
|
benchmark_main_dep,
|
|
benchmark_dep,
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_kg_edge_insert', kg_edge_insert_bench)
|
|
|
|
else
|
|
message('Google Benchmark not found; skipping service and metadata benchmark executables')
|
|
endif
|
|
|
|
# PBI-004: End-to-end ingestion pipeline benchmark. It provides its own main
|
|
# and must remain buildable when the optional Google Benchmark dependency is absent.
|
|
ingestion_e2e_bench = executable('ingestion_e2e_bench',
|
|
files('search/ingestion_e2e_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
test_deps,
|
|
dependency('yams_metadata'),
|
|
dependency('yams_search'),
|
|
dependency('yams_vector'),
|
|
dependency('yams_daemon'),
|
|
dependency('yams_app_services'),
|
|
dependency('spdlog'),
|
|
dependency('nlohmann_json'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_ingestion_e2e', ingestion_e2e_bench)
|
|
|
|
# KPI 4: repair ability (standalone main, no Google Benchmark dependency)
|
|
repair_ability_bench = executable('repair_ability_bench',
|
|
files('repair_ability_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
test_deps,
|
|
dependency('nlohmann_json'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
alias_target('bench_repair_ability', repair_ability_bench)
|
|
|
|
ingestion_bench = executable('ingestion_throughput_bench', files('ingestion_throughput_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
boost_dep,
|
|
dependency('yams_benchmarks'),
|
|
dependency('yams_api'),
|
|
dependency('yams_app_services'),
|
|
dependency('yams_metadata'),
|
|
dependency('yams_daemon'),
|
|
dependency('yams_storage_engine'),
|
|
dependency('nlohmann_json'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_ingest', ingestion_bench)
|
|
|
|
# PBI-043 Tree-diff benchmarks
|
|
tree_diff_bench = executable('tree_diff_benchmarks', files('tree_diff_benchmarks.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
dependency('yams_benchmarks'),
|
|
dependency('yams_metadata'),
|
|
dependency('yams_storage_engine'),
|
|
dependency('nlohmann_json'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_tree_diff', tree_diff_bench)
|
|
|
|
tree_build_bench = executable('tree_build_bench', files('tree_build_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
dependency('yams_benchmarks'),
|
|
dependency('yams_metadata'),
|
|
dependency('yams_storage_engine'),
|
|
dependency('nlohmann_json'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_tree_build', tree_build_bench)
|
|
|
|
ipc_stream_bench = executable('ipc_stream_bench', files('ipc_streaming_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
dependency('yams_benchmarks'),
|
|
dependency('yams_daemon'),
|
|
dependency('nlohmann_json'),
|
|
dependency('tl-expected'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_ipc_stream', ipc_stream_bench)
|
|
|
|
# Socket accept and metrics performance benchmark (Catch2-based)
|
|
_catch2_with_main_dep_bench = dependency('catch2-with-main', required: false)
|
|
if _catch2_with_main_dep_bench.found()
|
|
daemon_socket_accept_bench = executable('daemon_socket_accept_bench', files('daemon_socket_accept_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
_catch2_with_main_dep_bench,
|
|
test_deps,
|
|
dependency('yams_benchmarks'),
|
|
dependency('yams_daemon'),
|
|
dependency('nlohmann_json'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_socket_accept', daemon_socket_accept_bench)
|
|
else
|
|
message('GTest not found; skipping daemon_socket_accept_bench.')
|
|
endif
|
|
|
|
# Catch2-based benchmarks. Only built when Catch2 is available (build_tests=true in conan).
|
|
catch2_with_main_dep = dependency('catch2-with-main', required: false)
|
|
if catch2_with_main_dep.found()
|
|
write_coordinator_bench = executable('write_coordinator_bench',
|
|
files('write_coordinator_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
catch2_with_main_dep,
|
|
test_deps,
|
|
dependency('nlohmann_json'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
write_coordinator_bench_env = {
|
|
'YAMS_TESTING': '1',
|
|
'YAMS_DISABLE_VECTORS': '1',
|
|
'YAMS_DISABLE_SESSION_WATCHER': '1',
|
|
'YAMS_SKIP_MODEL_LOADING': '1',
|
|
'LD_LIBRARY_PATH': _test_env_ld_library_path,
|
|
}
|
|
|
|
test('write_coordinator_bench', write_coordinator_bench,
|
|
args: ['--allow-running-no-tests'],
|
|
suite: ['bench', 'write-coordinator'],
|
|
is_parallel: false,
|
|
timeout: 600,
|
|
env: write_coordinator_bench_env)
|
|
|
|
simeon_lexical_score_bench = executable('simeon_lexical_score_bench',
|
|
files('simeon_lexical_score_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
catch2_with_main_dep,
|
|
test_deps,
|
|
dependency('nlohmann_json'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
alias_target('bench_simeon_lexical_score', simeon_lexical_score_bench)
|
|
|
|
test('simeon_lexical_score_bench', simeon_lexical_score_bench,
|
|
args: ['--allow-running-no-tests'],
|
|
suite: ['bench', 'simeon'],
|
|
is_parallel: false,
|
|
timeout: 600,
|
|
env: write_coordinator_bench_env)
|
|
|
|
alias_target('bench_write_coordinator', write_coordinator_bench)
|
|
|
|
metadata_contention_bench = executable('metadata_contention_bench',
|
|
files('metadata_contention_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
catch2_with_main_dep,
|
|
test_deps,
|
|
dependency('nlohmann_json'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
metadata_contention_bench_env = {
|
|
'YAMS_TESTING': '1',
|
|
'YAMS_DISABLE_VECTORS': '1',
|
|
'YAMS_DISABLE_SESSION_WATCHER': '1',
|
|
'YAMS_SKIP_MODEL_LOADING': '1',
|
|
'LD_LIBRARY_PATH': _test_env_ld_library_path,
|
|
}
|
|
|
|
test('metadata_contention_bench', metadata_contention_bench,
|
|
args: ['--allow-running-no-tests'],
|
|
suite: ['bench', 'metadata-contention'],
|
|
is_parallel: false,
|
|
timeout: 600,
|
|
env: metadata_contention_bench_env)
|
|
|
|
alias_target('bench_metadata_contention', metadata_contention_bench)
|
|
|
|
# HNSW Entity Search benchmark (Catch2-based)
|
|
# Benchmarks HNSW search performance with various corpus sizes, dimensions, and k values
|
|
hnsw_entity_search_bench = executable('hnsw_entity_search_bench',
|
|
files('hnsw_entity_search_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
catch2_with_main_dep,
|
|
dependency('nlohmann_json'),
|
|
dependency('yams_vector'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
# Register as Catch2 test (not a standard test, only run manually or in nightly CI)
|
|
# Tests are tagged [!benchmark] so won't run by default
|
|
test('hnsw_entity_search_bench', hnsw_entity_search_bench,
|
|
args: ['--allow-running-no-tests'],
|
|
suite: ['bench', 'hnsw'],
|
|
is_parallel: false,
|
|
timeout: 300,
|
|
env: {
|
|
'YAMS_SKIP_MODEL_LOADING': '1',
|
|
'LD_LIBRARY_PATH': _test_env_ld_library_path,
|
|
})
|
|
|
|
alias_target('bench_hnsw', hnsw_entity_search_bench)
|
|
|
|
# Multi-client ingestion & concurrent access benchmark (Catch2-based)
|
|
# Measures ingestion throughput, latency, and fairness under multi-client load.
|
|
# Tagged [!benchmark] so won't run by default - invoke manually or in nightly CI.
|
|
multi_client_ingestion_bench = executable('multi_client_ingestion_bench',
|
|
files('multi_client_ingestion_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
catch2_with_main_dep,
|
|
test_deps,
|
|
dependency('nlohmann_json'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
# Single-client pipelining benchmark: in-flight window scaling on one
|
|
# client process. Quantifies the solo-vs-concurrent per-client throughput
|
|
# gap found in the 2026-06 scaling curve. Tagged [!benchmark].
|
|
single_client_pipeline_bench = executable('single_client_pipeline_bench',
|
|
files('single_client_pipeline_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
catch2_with_main_dep,
|
|
test_deps,
|
|
dependency('nlohmann_json'),
|
|
dependency('spdlog'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
alias_target('bench_single_client_pipeline', single_client_pipeline_bench)
|
|
|
|
multi_client_bench_env = {
|
|
'YAMS_TESTING': '1',
|
|
'YAMS_DISABLE_VECTORS': '1',
|
|
'YAMS_DISABLE_SESSION_WATCHER': '1',
|
|
'YAMS_SKIP_MODEL_LOADING': '1',
|
|
'YAMS_TEST_SAFE_SINGLE_INSTANCE': '1',
|
|
'LD_LIBRARY_PATH': _test_env_ld_library_path,
|
|
}
|
|
|
|
if get_option('build-cli') and tests_has_cli
|
|
multi_client_bench_env += { 'YAMS_BENCH_CLI_BIN': yams_cli_exe.full_path() }
|
|
endif
|
|
|
|
test('multi_client_ingestion_bench', multi_client_ingestion_bench,
|
|
args: ['--allow-running-no-tests'],
|
|
suite: ['bench', 'multi-client'],
|
|
is_parallel: false,
|
|
timeout: 600,
|
|
env: multi_client_bench_env)
|
|
|
|
alias_target('bench_multi_client', multi_client_ingestion_bench)
|
|
|
|
# Topology ablation bench (Catch2-based, component-level)
|
|
# Sweeps router limit and role-score boosts on a synthetic topology batch.
|
|
# Topology ablation benchmark.
|
|
topology_ablation_bench = executable('topology_ablation_bench',
|
|
files('topology_ablation_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
catch2_with_main_dep,
|
|
dependency('yams_topology'),
|
|
dependency('nlohmann_json'),
|
|
] + yams_tracy_deps,
|
|
cpp_args: ['-DYAMS_TESTING=1'] + yams_tracy_args,
|
|
install: false,
|
|
)
|
|
|
|
test('topology_ablation_bench', topology_ablation_bench,
|
|
args: ['--allow-running-no-tests'],
|
|
suite: ['bench', 'topology-ablation'],
|
|
is_parallel: false,
|
|
timeout: 300,
|
|
env: {
|
|
'YAMS_TESTING': '1',
|
|
'LD_LIBRARY_PATH': _test_env_ld_library_path,
|
|
})
|
|
|
|
alias_target('bench_topology_ablation', topology_ablation_bench)
|
|
|
|
# Memory-sync listing cost: one scenario per process, driven by
|
|
# scripts/run_memory_sync_listing_ab.py. Not registered as a Meson test.
|
|
memory_sync_listing_bench = executable('memory_sync_listing_bench',
|
|
files('memory_sync_listing_bench.cpp'),
|
|
include_directories: incs,
|
|
dependencies: [
|
|
dependency('nlohmann_json'),
|
|
dependency('yams_storage_engine'),
|
|
dependency('yams_crypto'),
|
|
dependency('yams_memory_sync'),
|
|
dependency('yams_core'),
|
|
],
|
|
cpp_args: ['-DYAMS_TESTING=1'],
|
|
build_rpath: ':'.join(_test_rpaths),
|
|
install: false,
|
|
)
|
|
|
|
alias_target('bench_memory_sync_listing', memory_sync_listing_bench)
|
|
else
|
|
message('Catch2 not found, skipping hnsw_entity_search_bench and multi_client_ingestion_bench')
|
|
endif
|