3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f76db466eb |
Stage 4: keep the live rarity model across daemon restarts
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / macos (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
The shadow scorer's rarity model started empty at every daemon start, so each restart cost an hour of warm-up with no decisions. The daemon now saves it to model_dir/rarity.state (checksummed, atomic write, hourly and at stop) and restores it at start. A restored state younger than the 7-day half-life skips the warm-up; decay catches up on the first event. A file made with other rarity settings, a bad checksum, or a negative or non-finite count is refused with a reason (status model.rarity_state, doctor model.rarity_state) and the model starts empty. The file must be owned by the daemon user and not group/world writable, like model files. Also fixes status model.warmup_windows, which was never filled in. Tests: R1/R2 in test_learn (round trip, decay clock, refusals), S8 in test_shadow (warm-up skipped only when recent), daemon restart restores the saved state, doctor warning; new fuzz_rarity_state target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
bbfda149a5 |
Stage 4: execute a promoted model's inspections, under guardrails
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / macos (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
A promoted model (promoted.sbm, written only by `model promote` after the gate) can now request real inspections. Only two, both additive: - memory_scan: through the daemon's memory-scan queue; - deep_rescan: the root's image re-scanned on demand; new matches are recorded as indicators of kind model_inspection (findings titled "YARA match found by a deeper rescan the inspection model requested") and positive labels. watch_subtree has no mechanism and rescan_on_next_rules is the retro-hunt's job, so both are masked out of acting decisions (kExecutableActions), and replay's policy evaluation uses the same action set. Guardrails: - InspectionExecutor (bounded queue, one low-priority worker, submit() never blocks); WorkClass::model_inspect lets the governor defer it, and a deferred decision is dropped. - Each action rechecks that the promoted model is still active at the decision's generation (demoting or replacing it revokes queued work), that the root's node id and raw image path still match (decisions carry root_image, not the feature template), and skips protected processes. A deep rescan discards its result if the process, image or rules changed during the scan. - An acting decision is durably written before it is dispatched; every attempt becomes an inspection_outcome record (queued, scanned, skipped, deferred, revoked, queue_full, failed). - Source invariant R10: no file may couple learning types to the exec-guard, OpenSnitch or policy APIs (daemon_app.cpp and main.cpp, the wiring points, are reviewed by hand). Models never block, contain, kill or create findings directly. - Config model_act and model_lambda; status model block reports acting stats; timings.model_inspection. docs/evaluation/stage4-execution-gate.md amends the gate's method (not its thresholds): executable action set, greedy promoted decisions, budgets as estimated inspection seconds by scan profile. Also: TerminalSafeBuf's destructor no longer lets an exception escape; the health message clamps huge durations; replay timestamp overflow; the Docker test build runs with -j2. Tests: test_inspection_executor (every skip and execute path, revocation, queue limits); shadow and replay acting cases; R10 fixture; fault injection and invariant updates. Validation reported by the session that wrote it: 48/48 local, 46/46 Docker plus privileged live checks, 20 fuzz targets (9.5M runs), three paired KPI runs without regressions (no acting model was loaded, so active-inspection cost is not measured yet). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0006a86ff4 |
Stage 4: shadow mode for the inspection model
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / macos (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
The daemon now scores live windows with the shadow model and logs the inspection it would request. Nothing is executed. - learn/shadow: a live replay engine (same windows and rarity as `strawberry replay`) fed every telemetry record on the writer thread. When a window closes, the model's value v gives each action a benefit v x reach - lambda x CPU cost; the best positive one is chosen if the hour's budget (10/30/120 CPU-s by scan profile) allows, else none. With probability 0.02 a uniformly random other action is taken instead, and every decision records its propensity for off-policy evaluation. Decisions are inspection_action telemetry records. - Replay gains a live mode (no ranking bookkeeping, so memory stays bounded) and a window-closed callback. - Daemon: model_shadow (default on) loads <model_dir>/shadow.sbm when it appears or changes, checked every minute; an invalid file is reported and not used. Status gains a `model` block (kernel, generation, decisions by action, exploration, would-be spend, budget refusals). - learn_sources in src/meson.build: tests that compile the daemon from sources take the learning library whole. Limits, documented: decisions come when a window closes (a few minutes after its first exec), and the live rarity model starts empty at each daemon start (the first hour is learned, not decided). Tests: test_shadow (no model, greedy choices, budget never exceeded, determinism, exploration rate and propensities, record codec); daemon status; fuzz_telemetry_records covers inspection_action. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |