strawberry/docs/evaluation
trvon a4179c474d
docs: say what the gate actually enforces, and what a passing verdict needs
The gate's own documentation claimed more than the tool did, in ways that
mattered once the integrity checks landed.

- "passes only if all 9 cells pass" was enforced by scripts, not by the gate; it
  is now enforced (previous commit) and the text says which criterion the tool
  applies: complete evidence, refused false, every pooled cell passing.
- The model digest proves agreement ACROSS reports. It cannot prove that the
  file on disk now is the one that was evaluated. Corrected everywhere.
- The remedy was described inconsistently: cutting the training bundles earlier
  is NOT enough (evaluation still replays the whole store), a bounded evaluation
  range is required. Written consistently, with --eval-after NS|auto.
- scripts/fleet_e2e.sh asserted three gate checks; promotion_gate emits five. It
  now asserts >= 5, requires the provenance fields, and adds the honest-evidence
  invariant: today every cell is in-sample, so at least one cell must report
  in_sample true, none may pass, and each must carry the out-of-sample failure.
  A pass over in-sample cells is itself a failure.
- The Salt/range runbooks advertised the flow as producing promotion evidence;
  it collects gate INPUT, and a passing verdict requires out-of-sample windows.
2026-11-19 17:30:35 -07:00
..
stage3-rarity-preregistration.md Inject scripted attack traces into replay and score them 2026-10-26 22:37:44 -06:00
stage4-execution-gate.md Stage 4: execute a promoted model's inspections, under guardrails 2026-10-27 21:03:57 -06:00
stage4-fleet-gate.md docs: say what the gate actually enforces, and what a passing verdict needs 2026-11-19 17:30:35 -07:00
stage4-promotion-gate.md docs: say what the gate actually enforces, and what a passing verdict needs 2026-11-19 17:30:35 -07:00
stage4-threat-traces-preregistration.md docs: say what the gate actually enforces, and what a passing verdict needs 2026-11-19 17:30:35 -07:00