* feat(storage): expose typed disk pressure policy
Signed-off-by: trvon <git@trevon.dev>
* fix(mcp): preserve embedded stdio buffers
Move process-global stream setup to the MCP executable so embedded transports retain caller-owned redirection. Add a regression for the failing construction order.
* refactor(daemon): scope shared P2P protected-file accessors for cross-TU use
The protected-file accessors (readProtectedP2pPrivateKey,
readProtectedP2pTrustFile, writeExclusiveP2pPrivateKey) are shared by the
P2P identity-key paths and the memory-sync bootstrap. Keep their
platform-specific definitions in p2p_integration.cpp unchanged and scope
them into yams::daemon::service_manager_detail, declared by the private,
non-installed header p2p_protected_file.h. Qualify the internal
callsites. No behavior change.
Prepares moving ServiceManager::initializeMemorySync into the
memory-sync translation unit (Brick audit cohesion note).
Signed-off-by: trvon <me@trevon.dev>
* refactor(daemon): place initializeMemorySync with memory-sync TU
Brick structural audit of PR #79 flagged that
ServiceManager::initializeMemorySync lives in p2p_integration.cpp while
the rest of the memory-sync surface lives in memory_sync.cpp. Move it —
together with its exclusive helper ensureAuthenticatedDirectStore — into
memory_sync.cpp.
The shared protected-file accessors remain defined in
p2p_integration.cpp (see the preceding service_manager_detail commit);
moved function bodies are unchanged and ServiceManager.h is untouched.
Brick evidence (experimental pre-79 af609a1e vs post-79 4d942793):
ServiceManager.cpp functions 84 -> 59 with 25 moved and 0 vanished;
ServiceManager.cpp outgoing call/import edges 1292 -> 954, distinct
targets 59 -> 41.
Validation: full ninja build green; meson test yams:service_manager,
yams:service_manager_shutdown, yams:service_manager_degraded — 3/3 OK.
Signed-off-by: trvon <me@trevon.dev>
* refactor(vector): break simeon_embedding_backend.h include cycle with embedding_generator.h
simeon_embedding_backend.h only declares functions taking EmbeddingConfig
by reference, yet included the full embedding_generator.h definition,
forcing every consumer of the simeon backend header to transitively pull
the generator header (and feeding the embedding_generator <->
simeon_embedding_backend 2-cycle reported by the Brick structural survey).
Replace the include with forward declarations for IEmbeddingBackend and
EmbeddingConfig. Consumers that use the full definitions
(simeon_embedding_backend.cpp, search_engine.cpp,
ingestion_throughput_bench.cpp) now include embedding_generator.h
directly.
Brick caveat: the tool's lexical lane cannot see the header-include edge
removal (0.65-confidence interface-dispatch misattribution keeps a
file-level cycle in its graph); the include-cycle break is verified at
source level. Brick survey pre/post artifacts kept in
worktree build/brick/architecture-iter{,-after}.json.
Validation: full ninja build green; meson test yams:vector_submodule,
yams:simeon_model_provider, yams:embedding_lifecycle,
yams:embedding_recovery, yams:service_manager — 5/5 OK.
Signed-off-by: trvon <me@trevon.dev>
* refactor(vector): extract computeCosineSimilarity from VectorDatabase into vector_utils
The pure cosine-similarity helper was stranded as a static member of the
VectorDatabase storage class, forcing sqlite_vec_backend.cpp to include
vector_database.h for a pure math function (Brick deferred lead #2 from
the PR #79 audit loop).
Move the implementation to vector_utils as a free function
(yams::vector::vector_utils::computeCosineSimilarity, semantics copied
verbatim: 0.0 on size mismatch or zero norm), keep
VectorDatabase::computeCosineSimilarity as a forwarding static for the
external caller in doctor dedupe, and switch the four backend callsites
to the free function. sqlite_vec_backend.cpp no longer includes
vector_database.h.
Validation: full ninja build green; meson test vector_submodule (incl.
simeon_backend_catch2_test) and yams:service_manager — green.
Signed-off-by: trvon <me@trevon.dev>
* refactor(daemon): extract client transport types to break sandbox probe header cycle
ClientTransportMode and ClientConfig moved verbatim from daemon_client.h
into a new shared header client_config.h (Brick deferred lead #3 from the
PR #79 audit loop: daemon_client <-> sandbox_detection type-level cycle).
sandbox_detection.h now depends only on client_config.h; its
implementation adds the explicit daemon_client.h include it needs for
the DaemonClient discovery statics. All existing consumers are
unaffected: daemon_client.h includes client_config.h so ClientConfig
remains visible transitively.
Remaining runtime dependency (sandbox_detection.cpp calling
DaemonClient::resolveSocketPathConfigFirst / isDaemonRunning statics,
which forward to socket_utils and a file-local ping helper) is recorded
as the follow-up for full decoupling; the header-level cycle is broken.
Validation: full ninja build green; meson test yams:service_manager,
yams:multiplex_client, yams:jsonrpc_client, vector_submodule — all green.
Signed-off-by: trvon <me@trevon.dev>
* test: refresh portability allowlist line anchors after include shifts
The Brick-audit include additions (search_engine.cpp,
ingestion_throughput_bench.cpp) and removal (sqlite_vec_backend.cpp)
shifted line numbers, making seven portability allowlist entries stale
(same reviewed raw getenv/path usages, new line numbers). Refresh the
anchors; no code or policy change.
Signed-off-by: trvon <me@trevon.dev>
* fix(daemon): include <fstream> for all POSIX /proc probe paths
The /proc readers compile in the non-Windows/non-Apple branch (BSD and
other POSIX targets), not just under __linux__; guard the <fstream>
include with the same condition so those targets compile.
Copilot review finding on PR #80 (src/daemon/components/DaemonMetrics.cpp:7).
Signed-off-by: trvon <me@trevon.dev>
* refactor(daemon): apply storage.disk_pressure outside applyMemorySync
ConfigResolver::applyMemorySync also applied [storage.disk_pressure],
coupling unrelated config parsing to the memory_sync apply path and
producing a misleading refusal message for invalid disk-pressure
policies. Move the call to its natural site in daemon_main before the
memory_sync apply, with its own accurate refusal message. Same startup
semantics (exit 1 on invalid policy), correct attribution.
Copilot review finding on PR #80 (src/daemon/components/ConfigResolver.cpp:1658).
Signed-off-by: trvon <me@trevon.dev>
* test(daemon): update applyMemorySync no-op expectation after disk-pressure decoupling
applyMemorySync no longer applies [storage.disk_pressure] (moved to its
own call site in daemon_main), so a section set without memory_sync
entries is now a no-op rather than a disk-pressure rejection. The
invalid-disk-pressure rejection is still asserted directly against
applyStorageDiskPressure in the preceding checks.
Caught by the pre-push CI gate (asan+tsan: daemon_component_submodule).
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <git@trevon.dev>
Signed-off-by: trvon <me@trevon.dev>
Compile the direct-P2P test-only machinery out of production builds
(YAMS_DAEMON_TEST_HOOKS_ENABLED now wraps the reconnect-loop hook storage,
invoker, and call site, matching the already-guarded setter). Replace the
single shared writer key in p2p_manager tests with one distinct key pair per
writer identity so origin-binding regressions (a record claiming origin X but
signed by Y's key) fail verification. Extract the copy-pasted bytes/text/TempDir
helpers into tests/common/p2p_test_helpers.h and reuse them across the
transport/delta/manager suites. Replace a timing-sensitive fixed sleep in the
transport session-expiry test with a bounded poll that accepts both deadline
manifestations, and gate the memory-sync-service concurrent-lifecycle stress
case behind YAMS_RUN_STRESS_TESTS=1 per the existing FTS5 pattern. Deduplicate
the byte-identical commitment-JSON serialization inside MemorySyncLoop
(coldBootstrapRoot / persistReplicationCheckpoint) into one static helper.
Signed-off-by: trvon <git@trevon.dev>
Keep `p2p status` text output machine-parseable by removing ANSI severity
colorization, and stop connect/disconnect/forget from throwing on malformed
daemon responses (safe .value defaults, uint64 counters). Bound handshake and
delta timeouts to kP2pMaxOperationTimeout up front, and ceiling direct
sync_interval_ms to the 5-minute reconnect bound so invalid config fails early
instead of aborting daemon startup. Degrade memory-sync status to peerCount=0
on a transient peer-registry read instead of failing the whole status IPC. Add
an explicit `p2p forget` operation (proto FORGET=9 through dispatcher, manager,
and registry) so operators can rotate a peer's pinned key while preserving
fail-closed enrollment.
Signed-off-by: trvon <git@trevon.dev>
Bound pre-authentication JSON nesting depth to 32 so a pathologically nested
control frame cannot overflow the session-thread stack before any trust
decision. Raise the control-frame cap to 512 KiB to cover the worst-case
authenticated state frame (~300 KiB across 256 writers x 256-byte ids). Gate
MemorySyncLoop::applyDeltas through the resource governor so a peer cannot
flood deltas on the direct replication path. Account delta/bootstrap wire bytes
using the exact received frame size instead of a compact re-serialization.
Signed-off-by: trvon <git@trevon.dev>
Add libFuzzer/AFL++ harnesses for the direct-P2P handshake/state/history and
delta/bootstrap control parsers. Fuzzers invoke the same production parser
seams as the live wire path, so coverage cannot drift from real validation. A
protocol-v3 seed exercises exact-version rejection after the v4 upgrade. Bound
noninteractive AFL++ campaigns with AFL_FUZZ_SECONDS and register focused
fuzz-smoke tests without duplicating broad protocol/delta registrations.
Clarify that peer/cycle status is local health telemetry rather than global
convergence, and document the remaining decision-grade live-performance gate.
Signed-off-by: trvon <git@trevon.dev>
Require authenticated direct stores and explicit operator enrollment while hardening private keys and trust files across Unix and Windows. Sign local control artifacts, bind them to a protected store generation, and serialize async initialization shutdown.
Signed-off-by: trvon <git@trevon.dev>
* fix(memory-sync): keep replication responsive
Skip poisoned blobs without wedging backfill and publish immutable status snapshots so daemon IPC does not block on an active reconciliation cycle.
Reconstructed from ff62d3f8, 8e232721, and e60f513d.
Signed-off-by: trvon <git@trevon.dev>
* feat(p2p): add authenticated direct transport
Require TLS 1.3 mutual identity and bind the authenticated certificate, node, corpus, and epoch before replication state is exchanged.
Reconstructed from 96f9db88 and c46de77b.
Signed-off-by: trvon <git@trevon.dev>
* feat(p2p): exchange causal deltas
Persist peer pins and causal resume state before visibility so reconnects continue bounded authenticated replication without becoming remote query proxies.
Reconstructed from 1b8be146 and 41883338.
Signed-off-by: trvon <git@trevon.dev>
* feat(p2p): integrate direct sync services
Wire listener, reconnect, IPC, CLI, peer persistence, and daemon lifecycle around the authenticated direct transport.
Reconstructed from a19aaa44.
Signed-off-by: trvon <git@trevon.dev>
* fix(p2p): apply replicated document state
Index authenticated remote winners, propagate document tombstones, and keep service/test wiring complete across daemon paths.
Reconstructed from 4ca660c6, 443253f1, 3e9db24d, and 4b2ddcc5.
Signed-off-by: trvon <git@trevon.dev>
* test(p2p): register direct sync gates
Replace the superseded internal contract with operator-facing direct-sync guidance and register daemon integration coverage in the standard test layout.
Reconstructed from 360b9115 and 89416ccb.
Signed-off-by: trvon <git@trevon.dev>
* fix(p2p): require bounded peer enrollment
Bound aggregate sessions and worker lifetimes, create owner-only identity keys atomically, and reject unknown inbound peers until operators pin their mTLS identity.
Reconstructed from f91be528, 79de8e48, 4fb1e4b8, dc30a6c0, and b329edd3.
Signed-off-by: trvon <git@trevon.dev>
* fix(p2p): persist quarantine and deletes
Persist full-history writer commitments before invalidating forked state, and order local deletion through a durable authenticated outbox so tombstones survive crashes and reconnects.
Reconstructed from 74ce01e0 and 19888604.
Signed-off-by: trvon <git@trevon.dev>
* feat(p2p): add authenticated cold bootstrap
Install bounded zero-state snapshots only from previously operator-pinned peers, bind them to exact frontiers and full-history commitments, and recover staged journals before replication can resume.
Reconstructed from 8a9f5acf.
Signed-off-by: trvon <git@trevon.dev>
---------
Signed-off-by: trvon <git@trevon.dev>
Deduplicate unary results, verify direct blobs before writing, and surface stream finalization failures instead of reporting partial output as successful.
Signed-off-by: trvon <git@trevon.dev>
The child forked by 'Daemon does not terminate unverified data-dir lock
holder' built an fs::path and called json::dump(), both of which allocate;
a fork in a multithreaded process can leave the malloc lock held, so the
child could fail before signaling readiness. Pre-compute the lock path in
the parent, format the payload with snprintf into a stack buffer, signal a
failure byte on open/flock error, and retry the parent's read on EINTR.
Portability allowlist line numbers updated for the shift.
Stress-tagged cases ran in the default unit lane because only ~[slow]
was filtered. Tag them [slow] and add ~[slow]/[slow] splits to the five
binaries (mcp_submodule, work_coordinator, daemon_background,
plugin_infrastructure, hybrid_search) so they run in the slow suite.
Rebuild macOS TSan outputs so compiler instrumentation and compiler-rt come from one Apple toolchain generation. Preserve daemon fixture dependencies until ServiceManager teardown so sanitizer lanes do not exercise a test-only use-after-free.
* refactor(tuning): prune dormant runtime knobs
Remove unused tuning controls and adaptive search budget state. Extract testable ONNX slot reconciliation and search telemetry hydration seams, and correct the Apple Silicon sysctl value width.
* refactor(vector): retire dormant backends
Remove unused FAISS, model-infrastructure, VectorIndexManager, and TurboQuant paths while keeping Simeon PQ and float-vector storage.
Migrate unused V2.2 quantized schemas transactionally to V2.3. Databases containing non-default quantized formats or packed-code blobs fail closed with recovery guidance and no mutation.
* refactor(plugins): remove legacy SDK tooling
Remove the Ghidra and SDK submodules, their bespoke tests, and stale scan exclusions. Retain generic PluginProcess coverage against the repository mock fixture.
* build(vector): remove stale FAISS overrides
* refactor(ipc): remove dormant FSM snapshots
* refactor(ingest): unify bounded KG enqueue
* fix(plugins): harden allocation contracts
Keep ABI outputs null or fully initialized when allocations fail, and represent empty object payloads without relying on malloc(0).
* fix(storage): define empty object reads
Return a null buffer with zero length for successful empty-object reads instead of treating malloc(0) as an allocation contract.
* refactor(storage): trim through string views
* test: prune dormant harness helpers
Remove unused readiness and fixture helpers, and keep benchmark client-count doubling in a wide integer domain.
* chore(cpp): initialize analyzer-facing locals
* build(simeon): advance analyzer cleanup
* refactor(config): drop dead daemon client keys
* refactor(ipc): inject daemon client transport
Socket and in-process execution now share DaemonClient request dispatch, so CLI fallbacks do not need alternate service paths.
* refactor(retrieval): route CLI through daemon
Search and grep now use the same daemon handlers over socket or in-process transport. Live grep remains an explicit freshness inspection.
* refactor(graph): move CLI scope into daemon
Graph CLI requests now share daemon handlers across socket and in-process transports. CWD filtering occurs before pagination so scoped totals and pages stay correct.
* build: isolate test-only runtime controls
* refactor(metadata): decouple lock telemetry
* fix(services): return on request timeout
* fix(retrieval): enforce workspace scope
* fix(graph): scope nodes before pagination
* fix(grep): drop binary result lines
* fix(daemon): bound checkpoint wait
* fix(repair): use KG health checks
* fix(storage): always verify compressed CRC
* perf(vector): skip unchanged PQ snapshots
* build(zyp): exclude bench from default build
* build: protect Meson files from formatter
* refactor: remove unreachable production APIs
* refactor(daemon): remove dormant runtime state
* fix(vector): harden daemon provider calls
Timed-out coroutines must own their client and requests. Provider discovery must also fail instead of returning synthetic embeddings in production.
* refactor(storage): remove unusable adapters
* fix(cli): unify in-process add requests
* fix(tsan): preserve async worker progress
* test(cli): isolate daemon-dependent suites
* fix(repair): expose queue backoff progress
