strawberry/scripts
trvon f4adc12a74
gate: judge the model against a calibrated rarity comparator
The rarity-greedy comparator acted on every evaluated window, so it spent
each hour's budget on the first ~15 windows and scored 0 hits in every
cell. Criteria 2 and 3 of the Stage 4 gate were void against it.

rarity_calibrated keeps the same rarity value, chooser, costs and budget,
and adds a per-node threshold frozen before the evaluated span: the m-th
largest value over the trailing 24 h of label-free history between the
warm-up and the --eval-after bound, with m set so expected spend matches
the hourly budget. Without that history the comparator is uncalibrated
and the gate fails by name; it never falls back to another rule.

rarity_greedy (the old first-come rule) and rarity_hindsight (per-hour
ordering, not causal) are reported as diagnostics and never judged.
fleet gate pools counts, keeps thresholds per node, and fails a pooled
cell when any node is uncalibrated or lacks the comparator.

Contract change: in `fleet gate --json`, rarity_hits and
rarity_cpu_seconds now describe the calibrated comparator; the old
numbers are rarity_greedy_hits and rarity_greedy_cpu_seconds.

Pre-registration: docs/evaluation/stage4-rarity-baseline-preregistration.md,
sha256 4916c95cd2581e3749acfe46c3ccf827aa5ce98c06150b49b978fc70da013f95,
indexed in YAMS at 2026-11-20T18:55:48Z, before the code. This commit is
dated after the registration time in that document (2026-11-21T00:00Z);
the YAMS record is what predates it.

Checked: meson suite 58/58, scripts/fleet_e2e.sh, clang-tidy and the
strict analyzer on the changed TUs. Not run: fold on the changed TUs,
make fuzz, the live Docker tests and the perf KPI. The comparator has
not been run on validation-range data.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-20 19:20:55 -07:00
..
analyze_changed.sh Audit flow: add the clang static analyzer as a strict per-change gate 2026-10-28 09:55:03 -06:00
build_minimal.sh Implement runtime pipeline, reporting, and local UI 2026-04-26 16:31:01 -04:00
check_deps.sh Implement runtime pipeline, reporting, and local UI 2026-04-26 16:31:01 -04:00
compiler_matrix.sh ci: local compiler matrix (trixie/noble/forky x gcc/clang) 2026-11-20 10:32:11 -07:00
fleet_e2e.sh gate: judge the model against a calibrated rarity comparator 2026-11-20 19:20:55 -07:00
fold_changed.sh Harden against crashes and I/O faults 2026-10-25 18:17:17 -06:00
fold_scan_in_docker.sh Harden against crashes and I/O faults 2026-10-25 18:17:17 -06:00
install.sh Stage 7e: scheduled stop->reduce->start telemetry-reduction maintenance (opt-in) 2026-11-11 13:21:13 -07:00
install_common.sh Rules R1: ship pinned yara-forge-core source snapshot in the release (no per-node fetch) 2026-11-11 13:21:12 -07:00
macos_install.sh Stage 4: ship a training trace set; install traces 2026-10-28 04:50:04 -06:00
perf_kpi.py KPI: bound collection and reject incomplete measurements 2026-11-03 06:57:00 -07:00
perf_kpi.sh KPI: bound collection and reject incomplete measurements 2026-11-03 06:57:00 -07:00
perf_kpi_report.py kpi: budget the per-phase RSS peak, not only the last sample 2026-11-19 16:19:41 -07:00
perf_kpi_runtime.py KPI: bound collection and reject incomplete measurements 2026-11-03 06:57:00 -07:00
perf_kpi_workload.py KPI: bound collection and reject incomplete measurements 2026-11-03 06:57:00 -07:00
perf_kpi_workload.sh KPI: bound collection and reject incomplete measurements 2026-11-03 06:57:00 -07:00
profile_capture.sh profiling: pinned Tracy client, headless capture script, scan bench, tray + scan-stage zones 2026-11-14 18:29:57 -07:00
refresh-rules-snapshot.sh Rules R1: ship pinned yara-forge-core source snapshot in the release (no per-node fetch) 2026-11-11 13:21:12 -07:00
release.sh Release: one version source, CI version contract, scripts/release.sh 2026-11-09 08:55:05 -07:00
run_exec_guard_live_docker.sh Add exec prevention, behavior telemetry and persistence watch 2026-10-25 18:17:16 -06:00
run_fuzzers.sh fuzz: target the YARA rule partitioner 2026-11-17 13:45:12 -07:00
run_live_bpf_daemon_smoke_docker.sh Implement runtime pipeline, reporting, and local UI 2026-04-26 16:31:01 -04:00
run_live_bpf_pipeline_smoke_docker.sh Implement runtime pipeline, reporting, and local UI 2026-04-26 16:31:01 -04:00
run_live_bpf_smoke.sh Implement runtime pipeline, reporting, and local UI 2026-04-26 16:31:01 -04:00
run_live_bpf_smoke_docker.sh Implement runtime pipeline, reporting, and local UI 2026-04-26 16:31:01 -04:00
run_live_tests_docker.sh Stage 7e: scheduled stop->reduce->start telemetry-reduction maintenance (opt-in) 2026-11-11 13:21:13 -07:00
strawberry-reduce.service Stage 7e: scheduled stop->reduce->start telemetry-reduction maintenance (opt-in) 2026-11-11 13:21:13 -07:00
strawberry-reduce.sh Stage 7e: scheduled stop->reduce->start telemetry-reduction maintenance (opt-in) 2026-11-11 13:21:13 -07:00
strawberry-reduce.timer Stage 7e: scheduled stop->reduce->start telemetry-reduction maintenance (opt-in) 2026-11-11 13:21:13 -07:00
strawberry-trackerd.service Privacy: host-namespace tracker relay + install/units/docs (WS2b/WS2c) 2026-10-28 17:56:23 -06:00
strawberry-update.path Add policy, threat-intel feeds and OpenSnitch containment 2026-10-25 18:17:05 -06:00
strawberry-update.service CLI: yams-style commands, with doctor and doctor fix 2026-10-27 09:32:02 -06:00
strawberry-update.timer Add policy, threat-intel feeds and OpenSnitch containment 2026-10-25 18:17:05 -06:00
strawberry.service CLI: yams-style commands, with doctor and doctor fix 2026-10-27 09:32:02 -06:00
test_reduce_wrapper.sh Stage 7e: scheduled stop->reduce->start telemetry-reduction maintenance (opt-in) 2026-11-11 13:21:13 -07:00
test_rules_snapshot.sh Rules R1: ship pinned yara-forge-core source snapshot in the release (no per-node fetch) 2026-11-11 13:21:12 -07:00
test_salt_states.sh Stage 4 over fleet data: per-node 3x3 policy evaluation, pooled fleet promotion gate 2026-11-15 09:18:04 -07:00
threat_traces.py threat traces checker + fleet gate in the dockerized Salt test 2026-11-15 09:36:24 -07:00
watch_cpu_capture.sh profiling: watch_cpu_capture.sh takes one Tracy capture when a process spins 2026-11-14 18:52:09 -07:00