2 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f4adc12a74 |
gate: judge the model against a calibrated rarity comparator
The rarity-greedy comparator acted on every evaluated window, so it spent each hour's budget on the first ~15 windows and scored 0 hits in every cell. Criteria 2 and 3 of the Stage 4 gate were void against it. rarity_calibrated keeps the same rarity value, chooser, costs and budget, and adds a per-node threshold frozen before the evaluated span: the m-th largest value over the trailing 24 h of label-free history between the warm-up and the --eval-after bound, with m set so expected spend matches the hourly budget. Without that history the comparator is uncalibrated and the gate fails by name; it never falls back to another rule. rarity_greedy (the old first-come rule) and rarity_hindsight (per-hour ordering, not causal) are reported as diagnostics and never judged. fleet gate pools counts, keeps thresholds per node, and fails a pooled cell when any node is uncalibrated or lacks the comparator. Contract change: in `fleet gate --json`, rarity_hits and rarity_cpu_seconds now describe the calibrated comparator; the old numbers are rarity_greedy_hits and rarity_greedy_cpu_seconds. Pre-registration: docs/evaluation/stage4-rarity-baseline-preregistration.md, sha256 4916c95cd2581e3749acfe46c3ccf827aa5ce98c06150b49b978fc70da013f95, indexed in YAMS at 2026-11-20T18:55:48Z, before the code. This commit is dated after the registration time in that document (2026-11-21T00:00Z); the YAMS record is what predates it. Checked: meson suite 58/58, scripts/fleet_e2e.sh, clang-tidy and the strict analyzer on the changed TUs. Not run: fold on the changed TUs, make fuzz, the live Docker tests and the perf KPI. The comparator has not been run on validation-range data. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
bbfda149a5 |
Stage 4: execute a promoted model's inspections, under guardrails
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / macos (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
A promoted model (promoted.sbm, written only by `model promote` after the gate) can now request real inspections. Only two, both additive: - memory_scan: through the daemon's memory-scan queue; - deep_rescan: the root's image re-scanned on demand; new matches are recorded as indicators of kind model_inspection (findings titled "YARA match found by a deeper rescan the inspection model requested") and positive labels. watch_subtree has no mechanism and rescan_on_next_rules is the retro-hunt's job, so both are masked out of acting decisions (kExecutableActions), and replay's policy evaluation uses the same action set. Guardrails: - InspectionExecutor (bounded queue, one low-priority worker, submit() never blocks); WorkClass::model_inspect lets the governor defer it, and a deferred decision is dropped. - Each action rechecks that the promoted model is still active at the decision's generation (demoting or replacing it revokes queued work), that the root's node id and raw image path still match (decisions carry root_image, not the feature template), and skips protected processes. A deep rescan discards its result if the process, image or rules changed during the scan. - An acting decision is durably written before it is dispatched; every attempt becomes an inspection_outcome record (queued, scanned, skipped, deferred, revoked, queue_full, failed). - Source invariant R10: no file may couple learning types to the exec-guard, OpenSnitch or policy APIs (daemon_app.cpp and main.cpp, the wiring points, are reviewed by hand). Models never block, contain, kill or create findings directly. - Config model_act and model_lambda; status model block reports acting stats; timings.model_inspection. docs/evaluation/stage4-execution-gate.md amends the gate's method (not its thresholds): executable action set, greedy promoted decisions, budgets as estimated inspection seconds by scan profile. Also: TerminalSafeBuf's destructor no longer lets an exception escape; the health message clamps huge durations; replay timestamp overflow; the Docker test build runs with -j2. Tests: test_inspection_executor (every skip and execute path, revocation, queue limits); shadow and replay acting cases; R10 fixture; fault injection and invariant updates. Validation reported by the session that wrote it: 48/48 local, 46/46 Docker plus privileged live checks, 20 fuzz targets (9.5M runs), three paired KPI runs without regressions (no acting model was loaded, so active-inspection cost is not measured yet). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |