yams/tests/unit/daemon
Trevon ae15bf317a
perf(ingest): harden daemon batching and shutdown (#58)
* fix(daemon): harden model state ownership

Keep ModelPoolConfig single-defined and serialize telemetry snapshots to avoid ODR drift and torn concurrent reads.

* refactor(daemon): consolidate ingest pollers

Require the wake-timer contract so every post-ingest stage shares one event-driven poller configuration.

* refactor(daemon): unify repair execution

Share admission, operation ordering, cancellation, progress, and result aggregation between synchronous and asynchronous repair paths.

* refactor(daemon): unify runtime tuning

Resolve startup and reload settings through one typed path so parsing, precedence, and range validation cannot drift.

* refactor(daemon)!: remove dead IPC surfaces

Delete unreferenced framing, cancellation, and compatibility paths while keeping the active streaming protocol intact.

BREAKING CHANGE: callers using connectAsync or legacy MessageFramer and FrameReader helpers must migrate to connect, frame_message_into, feed, and get_frame.

* refactor(daemon): hide test hooks

Preserve test-library linkage without exposing daemon-only probes through production headers or external symbols.

* fix(daemon): avoid replaying successful adds

A fresh-connection transport retry reuses the response handler. Clear its stale EOF error when the retry returns a valid add response so the outer retry loop does not enqueue the document again.

* bench(ingest): enforce lossless pipeline runs

Separate admission, storage, drain, enrichment, and searchability timing. Decision-grade plans now reject drops, duplicate processing, incomplete pipelines, and experiment identity drift.

* bench(search): expose cold seed ingest timings

Pair generalized-memory quality arms with admission, storage, drain, searchability, and enrichment timing from their shared cold seed. Keep the observation distinct from repeated throughput measurements.

* perf(ingest): batch deferred content indexing

Directory ingestion already schedules content work through post-ingest. Avoid duplicate inline indexing and coalesce normal-priority trickles within a bounded window.

* feat(metadata): expose insert writer metrics

Separate queue, batch-apply, and transaction time so ingestion runs can attribute connection pressure without changing write semantics.

* fix(daemon): retry transient refused connects

The daemon socket can be bound before its listener is ready during startup. Reuse the existing bounded retry budget for that refusal window.

* bench(ingest): attribute submission path costs

Decision-grade runs need stable coalescing identity and separate queue, apply, transaction, and estimated connection-wait costs.

* perf(ingest): batch durable document storage

Commit chunks, manifests, references, and metadata in bounded batches while preserving publication only after durable reference success.

* perf(ingest): coalesce bulk post-processing

Use a 16-item bulk cap while keeping latency-sensitive daemon ingestion on the existing four-item priority quota.

* bench(ingest): record post-ingest batch size

Include the effective cap in metrics and experiment identity so runs with different batching policies cannot be compared as one system.

* perf(ingest): batch queued storage waves

Preserve single-document fallback semantics and execute inline when the coordinator has one worker to avoid self-deadlock.

* perf(ingest): route bulk enrichment batches

Keep quota-sized interactive groups on the RPC lane while sending larger lossless groups through bounded coalescing.

* test(daemon): initialize task fixtures

* perf(ingest): batch post-ingest dispatch

* perf(ingest): widen directory storage waves

Borrow from a full wave when one document would otherwise fall through to single-document storage.

* perf(storage): bound batch writer fanout

Reuse a fixed worker set so batch size no longer creates one thread and payload copy per object.

* perf(ingest): align storage batch drains

Dispatch one configured batch through storage so intermediate waves do not fragment downstream metadata work.

* fix(ingest): commit metadata state atomically

Fold first-version, extracted-title, and embedding-completion state
into existing transactions.

Read committed extraction state so duplicate-content tasks cannot
double-count corpus counters.

* bench(ingest): measure production submission

Drive pipelined files through DocumentIngestionService.
Export dispatch, fingerprint, and enqueue costs through xplan.

* fix(xplan): preserve native failures

Report child signals and bounded stderr before readiness checks so
assertions are not masked as missing traces.

* fix(daemon): preserve embedding success hashes

* fix(daemon): serialize pooled socket shutdown

* fix(daemon): harden batch publication shutdown

* fix(storage): use compat jthread shim

* fix(daemon): avoid poller template shadowing

* fix(search): keep vector pipeline test enabled

* fix: harden shutdown and fallback paths

Demote expected shutdown SQL interruptions, ignore readonly FTS integrity noise, avoid stopped-executor socket hangs, and preserve metadata writer fallback outcomes.
2026-07-17 07:04:30 -04:00
..
components perf(ingest): harden daemon batching and shutdown (#58) 2026-07-17 07:04:30 -04:00
abi_model_provider_dimension_catch2_test.cpp fix(security): harden fold-flagged guards 2026-05-17 23:50:32 -04:00
admission_policy_catch2_test.cpp feat: search / retrieval optimizations for performance and accuracy 2026-04-09 21:34:43 -04:00
asio_connection_pool_catch2_test.cpp perf(ingest): harden daemon batching and shutdown (#58) 2026-07-17 07:04:30 -04:00
asio_transport_exhaustion_catch2_test.cpp feat: checkpoint for updated unit test for daemon communication / protobuf errors. Expanding untested surface with optimizations for communication and ingestion. Additional work for ci apt / rpm install improvements, systemd configuration for installs and improved validation testing of packaging logic to reduce gpg errors. 2026-04-08 11:39:15 -04:00
asio_transport_inflight_close_ipc_prefix_catch2_test.cpp fix: download logic improvements and ci test fixes. Stagging download fixes seperate from search features 2026-05-04 13:29:17 -04:00
daemon_background_processing_test.cpp perf(ingest): harden daemon batching and shutdown (#58) 2026-07-17 07:04:30 -04:00
daemon_catch2_test.cpp fix(ci): stablize ci 2026-07-02 21:34:18 -04:00
daemon_client_list_status_catch2_test.cpp perf(ingest): harden daemon batching and shutdown (#58) 2026-07-17 07:04:30 -04:00
daemon_components_test.cpp feat: cleaning up request path and fixing checkpoint logic underload 2026-04-24 18:42:00 -04:00
daemon_fsm_suite_test.cpp fix(daemon): tolerate FSM shutdown reset race 2026-06-12 16:48:31 -04:00
daemon_idle_control_bench_catch2_test.cpp feat: Correctness fixes for flaky tests, improvements for linux build logic based on clion change. Improvement of tuning / governor daemon logic to simplify scaling logic 2026-04-09 11:45:57 -04:00
daemon_metrics_catch2_test.cpp chore: housekeeping — rm dead code, consolidate MCP tools, add tests, clean docs (#50) 2026-06-16 19:37:24 -04:00
daemon_metrics_status_test.cpp fix(testing): stabilize TSAN coverage, harden daemon shutdown, and speed up KG/storage paths (#53) 2026-06-26 17:57:02 -04:00
daemon_protocol_test.cpp perf(ingest): harden daemon batching and shutdown (#58) 2026-07-17 07:04:30 -04:00
daemon_warm_bench_catch2_test.cpp fix: windows test fixes 2026-02-20 22:51:41 -05:00
db_recovery_test.cpp test(metadata): add corruption fixture + suites: page smash (header 2026-07-01 20:11:03 -04:00
db_recovery_vacuum_catch2_test.cpp Search/grep UX, runtime hardening, and CI stabilization (#39) 2026-05-27 12:15:17 -04:00
db_salvage_catch2_test.cpp test(metadata): add corruption fixture + suites: page smash (header 2026-07-01 20:11:03 -04:00
dispatch_response_mapper_catch2_test.cpp fix: improvement of graph node source and retrieval logic and sandbox fixes in add command 2026-02-25 12:23:25 -05:00
dynamic_cap_sentinel_catch2_test.cpp fix(daemon): WAL flush on shutdown, corrupt-DB salvage, test hardening, ASAN fix (#27) 2026-05-09 23:03:23 -04:00
embedding_lifecycle_manager_catch2_test.cpp fix: HNSW and topology rebuilds are now batched in repair. Cleaning up from previous migration work 2026-04-21 13:58:10 -04:00
embedding_provider_recovery_catch2_test.cpp fix: license header file updates, mcp improvements, windows build fixes 2026-01-30 17:23:33 -05:00
entity_graph_service_catch2_test.cpp fix: license header file updates, mcp improvements, windows build fixes 2026-01-30 17:23:33 -05:00
external_plugin_host_test.cpp feat: security fixes for plugin install and adding additional UTF-8 string sanitization 2026-02-28 10:17:03 -05:00
frame_reader_catch2_test.cpp feat: checkpoint for updated unit test for daemon communication / protobuf errors. Expanding untested surface with optimizations for communication and ingestion. Additional work for ci apt / rpm install improvements, systemd configuration for installs and improved validation testing of packaging logic to reduce gpg errors. 2026-04-08 11:39:15 -04:00
graph_component_catch2_test.cpp fix(testing): stabilize TSAN coverage, harden daemon shutdown, and speed up KG/storage paths (#53) 2026-06-26 17:57:02 -04:00
health_check_isolation_catch2_test.cpp feat: adding perf optimizations for search and grep based on previous commit. Improving model logic for future embeddinggemma and improved colbert support. Re-ranking fixes for search to improve retrieval accuracy. 2026-03-11 14:15:33 -04:00
idle_zero_concurrency_catch2_test.cpp fix: correctness fix for abalation test, fixes for daemon response (regression is present underload/repair) 2026-04-20 13:20:24 -04:00
in_process_transport_catch2_test.cpp feat(daemon): shutdown hardening + test helper linkage fixes (#44) 2026-06-08 16:29:44 -04:00
ingest_metrics_publisher_catch2_test.cpp feat: extracted public and categorized methods from service manager. 2026-04-16 10:39:10 -04:00
metrics_consistency_catch2_test.cpp fix: testing improvements and bug fixes in daemon logic for lifecycle management 2026-02-17 15:55:29 -05:00
model_provider_registry_catch2_test.cpp feat: tuning topology system to work better with simeon via configuration 2026-04-19 20:10:46 -04:00
multiplex_client_catch2_test.cpp feat: optimizations based on profile runs in hashing and work coordinator. test updates and correctness fixes 2026-02-24 21:06:43 -05:00
onnx_embedding_diagnostic_test.cpp experimental → main: MCP timeout fix + doctor integrity checks + search/perf improvements (#31) 2026-05-14 23:29:44 -04:00
onnx_gpu_provider_catch2_test.cpp fix: make Gemma CoreML fallback automatic 2026-04-02 13:56:42 -04:00
onnx_model_pool_catch2_test.cpp fix: download logic improvements and ci test fixes. Stagging download fixes seperate from search features 2026-05-04 13:29:17 -04:00
onnx_model_provider_config_test.cpp feat: massive commit. Download logic updates with expanding controls (stop, start, resume etc.) from daemon control. This is used in my archive openclaw agent. This commit also includes work to harden onnxruntime abi access. this should allow loading newer abi's when builds are pinned to 1.23.0. 2026-03-21 17:12:47 -04:00
onnx_plugin_harness_test.cpp feat: removing experiments and unifying compression in embeddings storage. 2026-04-21 18:13:37 -04:00
physical_walk_ttl_test.cpp feat(daemon): adaptive physical-walk TTL + statvfs fast-path, hot-tier getenv→TuningSnapshot, RepairScheduling/RepairTuning slice extractions 2026-04-28 15:39:11 -04:00
plugin_content_extractor_adapter_test.cpp fix: license header file updates, mcp improvements, windows build fixes 2026-01-30 17:23:33 -05:00
plugin_infrastructure_test.cpp feat: cleaning up test suite 2026-04-14 10:25:54 -04:00
plugin_manager_config_test.cpp fix: license header file updates, mcp improvements, windows build fixes 2026-01-30 17:23:33 -05:00
post_ingest_dynamic_cap_recovery_catch2_test.cpp feat: adding perf optimizations for search and grep based on previous commit. Improving model logic for future embeddinggemma and improved colbert support. Re-ranking fixes for search to improve retrieval accuracy. 2026-03-11 14:15:33 -04:00
proto_serializer_catch2_test.cpp chore: housekeeping — rm dead code, consolidate MCP tools, add tests, clean docs (#50) 2026-06-16 19:37:24 -04:00
repair_binary_filter_catch2_test.cpp fix: license header file updates, mcp improvements, windows build fixes 2026-01-30 17:23:33 -05:00
repair_hold_hints_test.cpp optimizations: throughput optimizations, bug fixes, docs updates and benchmark cleanup 2026-04-26 21:43:00 -04:00
repair_scheduling_adapter_catch2_test.cpp feat: repair logic improvements, expanding config options to disable rebuilding/repair for memory constrained systems 2026-04-24 10:53:15 -04:00
repair_service_post_ingest_channel_catch2_test.cpp feat(daemon): shutdown hardening + test helper linkage fixes (#44) 2026-06-08 16:29:44 -04:00
request_handler_admission_reuse_catch2_test.cpp feat: search / retrieval optimizations for performance and accuracy 2026-04-09 21:34:43 -04:00
request_handler_io_edge_catch2_test.cpp fix: fixing leakage of build artifacts in release, unifying repair path, adding cache to tuning path and bumping simeon version 2026-04-23 10:20:25 -04:00
request_handler_lifetime_catch2_test.cpp feat: optimizations based on profile runs in hashing and work coordinator. test updates and correctness fixes 2026-02-24 21:06:43 -05:00
request_handler_repair_cancel_catch2_test.cpp test: expand storage-readiness coverage — 48.1%→59.8% broad, 74.3% narrow (#38) 2026-05-25 22:58:15 -04:00
request_handler_status_unary_catch2_test.cpp feat: recovery test improvements 2026-02-09 23:27:12 -05:00
request_handler_writer_drain_cleanup_catch2_test.cpp fix: windows test fixes 2026-02-20 22:51:41 -05:00
resource_governance_test.cpp perf: cleaning up search experiments, moving to new routing system for search for larger corpus improvements, repair system now triggers semantic / graph repair 2026-04-29 20:36:02 -04:00
sandbox_detection_catch2_test.cpp fix(asan): Addressing asan test regressions and suite failures from recent changes 2026-05-01 09:18:39 -04:00
search_admission_controller_catch2_test.cpp feat: extracted public and categorized methods from service manager. 2026-04-16 10:39:10 -04:00
service_manager_catch2_test.cpp feat(search): make topology-assisted retrieval the default (#57) 2026-07-13 20:26:43 -04:00
service_manager_provider_degraded_catch2_test.cpp feat: test fixes, metric expansion, build improvements for gliner 2026-04-23 08:48:33 -04:00
service_manager_shutdown_ownership_catch2_test.cpp fix: license header file updates, mcp improvements, windows build fixes 2026-01-30 17:23:33 -05:00
simeon_model_provider_catch2_test.cpp feat: bumping simeon, improving graph building, clustering, configuration tuning of said system and updating benchmark files. Additional fixes for daemon readiness and status return underload. 2026-04-20 09:31:45 -04:00
status_readiness_vector_usable_catch2_test.cpp feat: adding instrumentation and config options for build and repair system. Bumping simeon and improving topology system with experimental findings 2026-04-24 17:38:24 -04:00
storage_preflight_test.cpp fix(ci): stablize ci 2026-07-02 21:34:18 -04:00
streaming_handlers_catch2_test.cpp cleaning up code 2026-01-21 16:54:16 -05:00
streaming_processor_catch2_test.cpp ci improvements. streaming processor bug and test fixes 2025-12-28 17:13:54 -05:00
timing_audit_catch2_test.cpp feat: cleaning up test suite 2026-04-14 10:25:54 -04:00
topology_manager_catch2_test.cpp feat: extracted public and categorized methods from service manager. 2026-04-16 10:39:10 -04:00
tune_advisor_snapshot_test.cpp test: instrumentation expansions for ingestion speed experiments and tuning improvements 2026-04-26 19:55:29 -04:00
tune_advisor_threshold_test.cpp fix: fixing leakage of build artifacts in release, unifying repair path, adding cache to tuning path and bumping simeon version 2026-04-23 10:20:25 -04:00
tuning_allocation_catch2_test.cpp feat: cleaning up test suite 2026-04-14 10:25:54 -04:00
tuning_loop_wakeup_catch2_test.cpp feat: cleaning up test suite 2026-04-14 10:25:54 -04:00
tuning_reconciliation_catch2_test.cpp feat: cleaning up test suite 2026-04-14 10:25:54 -04:00
vector_index_coordinator_bulk_scope_catch2_test.cpp feat: continued extraction from serach engine and vector improvements. This allows for better testing and experimentation 2026-04-22 12:24:28 -04:00
vector_index_coordinator_coalesce_catch2_test.cpp fix: text fixes, connection handingly fixes and continued refactor work 2026-04-22 13:20:18 -04:00
vector_index_coordinator_telemetry_catch2_test.cpp perf(ingest): harden daemon batching and shutdown (#58) 2026-07-17 07:04:30 -04:00
work_coordinator_test.cpp fix(test): prevent stack-use-after-scope in work-coordinator coroutine 2026-06-28 22:27:14 -04:00
write_coordinator_catch2_test.cpp feat(search): make topology-assisted retrieval the default (#57) 2026-07-13 20:26:43 -04:00