Commit graph

4 commits

Author SHA1 Message Date
trvon
6f5c62b31c
Stage 4 over fleet data: per-node 3x3 policy evaluation, pooled fleet promotion gate
The pre-registered gate (docs/evaluation/stage4-promotion-gate.md) asks for the
criteria to hold on 3 non-overlapping periods x 3 training seeds; until now the
tool reported a single run and the operator recorded the rest. This produces
the 9 results for a fleet, with the criteria unchanged:

- replay --policy-eval --eval-model FILE (repeatable) --periods N: one pass
  evaluates every model on every equal time slice of the ranked history; each
  (model, period) cell simulates the model and rarity-greedy with their own
  hourly budgets exactly as policy_eval does (policy_eval.cells). With one
  model equal to --model and one period the cell equals the single-model
  evaluation (test R11b); periods partition windows and positives.
- strawberry fleet gate DIR: pools the cells of every node report (windows,
  positives, inspections, CPU-s, hits, refusals) per (model, period) and
  applies learn::promotion_gate unchanged to each; passes only if all pass.
  Malformed / cell-less reports are skipped and listed.
- Evaluation stays on the node (raw telemetry never leaves it): the central
  cycle now also publishes all three seed models to dist/gate/; strawberry.gate
  (sensors) evaluates them with the pre-registered evaluation traces; the
  operator-run orch.strawberry_gate pushes the reports back and pools them;
  evidence under central.work_dir/gate-<stamp>/. Nothing promotes.
- docs/evaluation/stage4-fleet-gate.md: the procedure (new dated document;
  criteria and numbers unchanged).

Validated: test_fleet_gate (pooling sums, pooling reaching 10 positives
neither node has alone, one failing cell fails the fleet, malformed reports
skipped), R11b, Salt render checks (no models -> refusal; 3 models x 3
periods; orchestration pools), and scripts/fleet_e2e.sh end to end: 3 nodes ->
9 pooled cells with 28-50 positives each; the untrained synthetic models
correctly do not pass (exit code matches the verdict).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 09:18:04 -07:00
trvon
39ffa02ae6
Salt: dockerized multi-node fleet test (master/central + 3 sensors) + state fixes
deploy/salt/test/: builds a release tarball from HEAD in ubuntu:24.04 (CI path),
boots systemd containers (rootless, no --privileged), master==central + 3
sensors with the host's Salt onedir mounted read-only, applies the real states,
runs a workload with planted YARA detections, then two orchestrated cycles.
43 checks (install/config/idempotence, real daemons + telemetry, R2 cache
miss/hit, cp.push collect, 3-seed cycle, canary hold then fleet rollout, no
promoted model anywhere, fleet bits in round 2, --since cursor, dedup).

State fixes found by running them for real:
- init.sls no longer includes strawberry.distribute: a plain state.apply on
  sensors shipped models fleet-wide and bypassed the canary; only the
  orchestration distributes now.
- install.sls: CI tarballs carry a +g<sha> suffix that 'strawberry version'
  does not print, so install.sh re-ran on every apply; record the installed
  release and compare base version + marker.
- orch collect: mv -f (re-collected bundles must not be skipped).
- central cycle: builds the overlay first; a training refusal for too few
  teacher labels (normal on a young fleet) publishes the overlay only and
  keeps the sensors' shadow, any other error still fails.
- distribute.sls: serves only what the cycle published (cp.hash_file; the
  cp.list_master file-list cache can miss a just-published file).
Render check gains overlay-only and full publish cases (20 checks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 12:16:31 -07:00
trvon
b1c7cc41dd
fleet: e2e harness + fleet train/overlay JSON, *.sbx-only bundle listing
fleet train advertised --json but printed text, so the central cycle's
train-seedN.json evidence was not JSON; it now emits one object incl.
duplicate_windows. fleet overlay gains --json (bundles_superseded,
overlays_skipped). Both commands read only *.sbx (no dotfiles) from the
bundle dir, so a cursor or stray file is never fed to the merge.

scripts/fleet_e2e.sh drives the real binary + central cycle script over N
synthetic sensors: --since auto export chain, Salt-pull simulation, 3-seed
train + overlay, sha256 publish, determinism, overlay distribution and a
round-2 export carrying fleet rarity features. 6/6 checks pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 07:00:57 -07:00
trvon
25d0da1121
Salt: install/config/export/central/distribute states + central cycle orchestration
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 06:43:42 -07:00