The gate's own documentation claimed more than the tool did, in ways that mattered once the integrity checks landed. - "passes only if all 9 cells pass" was enforced by scripts, not by the gate; it is now enforced (previous commit) and the text says which criterion the tool applies: complete evidence, refused false, every pooled cell passing. - The model digest proves agreement ACROSS reports. It cannot prove that the file on disk now is the one that was evaluated. Corrected everywhere. - The remedy was described inconsistently: cutting the training bundles earlier is NOT enough (evaluation still replays the whole store), a bounded evaluation range is required. Written consistently, with --eval-after NS|auto. - scripts/fleet_e2e.sh asserted three gate checks; promotion_gate emits five. It now asserts >= 5, requires the provenance fields, and adds the honest-evidence invariant: today every cell is in-sample, so at least one cell must report in_sample true, none may pass, and each must carry the out-of-sample failure. A pass over in-sample cells is itself a failure. - The Salt/range runbooks advertised the flow as producing promotion evidence; it collects gate INPUT, and a passing verdict requires out-of-sample windows. |
||
|---|---|---|
| .. | ||
| range | ||
| README.md | ||
| run-fleet-loop.sh | ||
Strawberry distributed-learning validation
run-fleet-loop.sh validates the full hub-and-spoke loop end to end with the real
binaries, isolated per-node dirs/configs, and file-based transport — no mocks.
What it proves
seed N nodes (store + rarity.state)
-> round-1 export : bundles carry rarity sketches, NO fleet feature
-> central fleet train : merges N distinct nodes, trains the shadow model
-> central fleet overlay : sums per-node rarity sketches -> non-empty overlay
-> distribute overlay
-> round-2 export : feature vectors now carry the fleet-wide rarity
Hard assertions (exit non-zero on any failure):
- each round-1 bundle has rarity sketches (
rarity>0) andfleet_feat=0; - central sees exactly N distinct nodes (no
node_uidcollision); - the overlay is non-empty (sketches merged);
- the round-2 export carries the fleet feature (
fleet_feat>0).
A passing run prints: PASS: fleet loop validated (round-1 fleet_feat=0 -> round-2 fleet_feat=<N> ...).
Run it (local, no Docker)
meson compile -C builddir strawberry strawberry-fleet-seed strawberry-fleet-inspect
bash deploy/validation/run-fleet-loop.sh # NODES=2 EXECS=240 by default
Override binaries/params via env: SB, SEED, INS, NODES, EXECS, WORK.
Tools (build-only, not installed)
strawberry-fleet-seed <store> --node <n> --execs <N> [--seed S] [--model-dir <d>]— writes a deterministic synthetic telemetry store (and, with--model-dir, ararity.state, simulating a node that has been running the daemon).strawberry-fleet-inspect <bundle>— printsnode_uid windows features labels rarity fleet_featfor an export bundle (fleet_feat= feature vectors carrying a non-zero fleet slot).
Scope / honesty
This validates the distributed DATA + LEARNING path (export → merge → train → overlay → distribute → re-export) across isolated nodes. It uses a seeder instead of the live eBPF daemon, so it does NOT re-validate live telemetry capture (the single-node daemon path covers that). The Docker-swarm harness (when built) runs this same script body inside containers for host-isolation realism; the logic proven here is identical.
Integration facts this surfaced (true of real deployments)
- Nodes must export with a distinct
node_id(config) or they collide to onenode_uidon a shared identity; in containers, distinct machine-ids also suffice. - The fleet overlay only forms once nodes have a
rarity.state(accumulated by the running daemon) — a cold node's first export carries no rarity sketches, sofleet_featstays 0 until a later round. fleet train'smax_node_sharecap is for multi-node fleets; a single node is cap-degenerate.