The pre-registered gate (docs/evaluation/stage4-promotion-gate.md) asks for the
criteria to hold on 3 non-overlapping periods x 3 training seeds; until now the
tool reported a single run and the operator recorded the rest. This produces
the 9 results for a fleet, with the criteria unchanged:
- replay --policy-eval --eval-model FILE (repeatable) --periods N: one pass
evaluates every model on every equal time slice of the ranked history; each
(model, period) cell simulates the model and rarity-greedy with their own
hourly budgets exactly as policy_eval does (policy_eval.cells). With one
model equal to --model and one period the cell equals the single-model
evaluation (test R11b); periods partition windows and positives.
- strawberry fleet gate DIR: pools the cells of every node report (windows,
positives, inspections, CPU-s, hits, refusals) per (model, period) and
applies learn::promotion_gate unchanged to each; passes only if all pass.
Malformed / cell-less reports are skipped and listed.
- Evaluation stays on the node (raw telemetry never leaves it): the central
cycle now also publishes all three seed models to dist/gate/; strawberry.gate
(sensors) evaluates them with the pre-registered evaluation traces; the
operator-run orch.strawberry_gate pushes the reports back and pools them;
evidence under central.work_dir/gate-<stamp>/. Nothing promotes.
- docs/evaluation/stage4-fleet-gate.md: the procedure (new dated document;
criteria and numbers unchanged).
Validated: test_fleet_gate (pooling sums, pooling reaching 10 positives
neither node has alone, one failing cell fails the fleet, malformed reports
skipped), R11b, Salt render checks (no models -> refusal; 3 models x 3
periods; orchestration pools), and scripts/fleet_e2e.sh end to end: 3 nodes ->
9 pooled cells with 28-50 positives each; the untrained synthetic models
correctly do not pass (exit code matches the verdict).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>