A promoted model (promoted.sbm, written only by `model promote` after the
gate) can now request real inspections. Only two, both additive:
- memory_scan: through the daemon's memory-scan queue;
- deep_rescan: the root's image re-scanned on demand; new matches are
recorded as indicators of kind model_inspection (findings titled "YARA
match found by a deeper rescan the inspection model requested") and
positive labels.
watch_subtree has no mechanism and rescan_on_next_rules is the retro-hunt's
job, so both are masked out of acting decisions (kExecutableActions), and
replay's policy evaluation uses the same action set.
Guardrails:
- InspectionExecutor (bounded queue, one low-priority worker, submit()
never blocks); WorkClass::model_inspect lets the governor defer it, and
a deferred decision is dropped.
- Each action rechecks that the promoted model is still active at the
decision's generation (demoting or replacing it revokes queued work),
that the root's node id and raw image path still match (decisions carry
root_image, not the feature template), and skips protected processes.
A deep rescan discards its result if the process, image or rules changed
during the scan.
- An acting decision is durably written before it is dispatched; every
attempt becomes an inspection_outcome record (queued, scanned, skipped,
deferred, revoked, queue_full, failed).
- Source invariant R10: no file may couple learning types to the
exec-guard, OpenSnitch or policy APIs (daemon_app.cpp and main.cpp, the
wiring points, are reviewed by hand). Models never block, contain, kill
or create findings directly.
- Config model_act and model_lambda; status model block reports acting
stats; timings.model_inspection.
docs/evaluation/stage4-execution-gate.md amends the gate's method (not its
thresholds): executable action set, greedy promoted decisions, budgets as
estimated inspection seconds by scan profile.
Also: TerminalSafeBuf's destructor no longer lets an exception escape;
the health message clamps huge durations; replay timestamp overflow; the
Docker test build runs with -j2.
Tests: test_inspection_executor (every skip and execute path, revocation,
queue limits); shadow and replay acting cases; R10 fixture; fault injection
and invariant updates. Validation reported by the session that wrote it:
48/48 local, 46/46 Docker plus privileged live checks, 20 fuzz targets
(9.5M runs), three paired KPI runs without regressions (no acting model
was loaded, so active-inspection cost is not measured yet).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The daemon now scores live windows with the shadow model and logs the
inspection it would request. Nothing is executed.
- learn/shadow: a live replay engine (same windows and rarity as
`strawberry replay`) fed every telemetry record on the writer thread.
When a window closes, the model's value v gives each action a benefit
v x reach - lambda x CPU cost; the best positive one is chosen if the
hour's budget (10/30/120 CPU-s by scan profile) allows, else none.
With probability 0.02 a uniformly random other action is taken instead,
and every decision records its propensity for off-policy evaluation.
Decisions are inspection_action telemetry records.
- Replay gains a live mode (no ranking bookkeeping, so memory stays
bounded) and a window-closed callback.
- Daemon: model_shadow (default on) loads <model_dir>/shadow.sbm when it
appears or changes, checked every minute; an invalid file is reported
and not used. Status gains a `model` block (kernel, generation,
decisions by action, exploration, would-be spend, budget refusals).
- learn_sources in src/meson.build: tests that compile the daemon from
sources take the learning library whole.
Limits, documented: decisions come when a window closes (a few minutes
after its first exec), and the live rarity model starts empty at each
daemon start (the first hour is learned, not decided).
Tests: test_shadow (no model, greedy choices, budget never exceeded,
determinism, exploration rate and propensities, record codec); daemon
status; fuzz_telemetry_records covers inspection_action.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
File node ids: include/strawberry/node_ids.hpp is now the only place
they are built (file_<sha256> or file_path_<sha256 of path>). The three
modules disagreed: correlation used a 16-digit path hash and behavior
detections the raw path, so path-keyed evidence never met the rest. An
exec detection with a known hash also writes a same_path edge from the
path id to the content id. Source invariant R8 rejects hand-built file
ids in src/ and include/.
Processes carry the real uid and kernel comm: from the eBPF exec event
(which now includes the uid) or /proc. Stored on process nodes and in
process_exec telemetry (fields 14, 15).
Command lines are redacted before any storage (capture_redact_cmdline,
default on): sensitive flags and KEY=value pairs, Authorization/Bearer
credentials, URL passwords, tokens used as URL user names, well-known
token shapes (GitHub, GitLab, Slack, AWS key ids, sk-, JWTs) and mysql
-p<password>. The rules repeat to a fixed point, so redaction is
idempotent by construction. tests/test_redact.cpp covers the rules plus
idempotence, size and planted-secret properties; fuzz_cmdline_redactor
found two idempotence bugs and a leak (--password --token X) that are
fixed here.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Teacher signals and context for the learning stages, written to the
evidence stream (retention never removes it):
- label: operator actions from the control socket (quarantine, contain,
kill, suspend: positive; allowlist-add: negative; allowlist-remove and
quarantine-remove: retracted) and allowlist suppressions (negative,
once per entry, subject and rule). Never from the MCP.
- rules_generation: every rules load attempt, with sources, counts and
the error of a failed reload.
- policy_generation: each policy change, with a sha256 of the policy.
- lifecycle: daemon start and stop.
governor_sample (pressure level, CPU/IO/memory, per-class admissions) goes
to the telemetry stream once a minute.
Database::append_evidence writes records the evidence view ignores.
Tests: record round-trips; suppression labels and dedupe and rules
generations in the pipeline test; operator labels, policy generations and
lifecycle in the daemon test; the record fuzzer covers the new decoders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each rule match from a fresh scan now writes a rule_match record: the
rule, namespace, severity, rules generation and its matched strings
(identifier, offset, length, 64-bit FNV-1a digest of the captured bytes).
The matched bytes themselves are never stored. Cache hits write no
rule_match, since their scan_result already names the rules.
Hits are collected in the libyara callback (libyara 4+) and capped at
16 strings per rule, 4 hits per string and 256 per scan. The callback
also tolerates a null user_data now.
Tests: caps, offsets and digests against a real scan; record round-trip
with the encode cap; the telemetry record fuzzer covers rule_match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 2 increment 2.
- Connect, ptrace and module-load events go to telemetry raw, before the
behavior engine filters them (pid, uid, comm, address/port or target);
the join to processes happens when the data is read.
- Persistence and drop-location writes go to telemetry before any rule
decides on them.
- The exec guard reports every verdict through a new on_observation hook
(allow verdicts and cache hits included; ExecDecision still reports
only non-allow): verdict, answered, scanned, cache hit, degraded, top
rule/severity and latency, as guard_decision records.
Tests: record round-trips; the live exec-guard test checks one
observation per checked exec, including allows and cache hits;
fuzz_telemetry_records covers the new decoders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 2 increment 1. Until now only detections were stored, so the
learning stages would never see a normal example.
- TelemetrySink: bounded lock-free multi-producer queue (8192); try_emit
never blocks, allocates or throws; a full queue drops and counts.
- A guarded writer thread drains it every 2 s (or at a quarter full)
into the store's root-only telemetry stream through the pipeline,
writing a gap record whenever events were dropped.
- The pipeline emits one process_exec (pid, parent, node ids, exe,
capped cmdline, sha256, deleted/memfd, live/inventory) and one
scan_result (outcome: scanned, cache_hit, too_large, unavailable,
read_error, no_rules, yara_error, pid_scan; bytes, scan time, rules
generation, matched rules) for every handled exec, detection or not.
- Config telemetry=true (default); status gains a telemetry block
(emitted, dropped, queued, written, gaps).
Tests: C1 (never blocks; exactly once and in order under concurrency,
clean under ThreadSanitizer), record round-trips, C2 (one process_exec
and one scan_result per handled exec, stored in the telemetry stream);
fuzz_telemetry_records.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>