# Strawberry distributed deployment (Salt) — contract for the deploying agent This directory holds the **Salt pillar** (data contract) for deploying Strawberry's distributed learning. The SLS **states** (install/config/egress/distribute) are to be authored separately (by the deploying agent) and consume this pillar. - `pillar/strawberry.sls` — the full pillar schema with defaults + required fields. - `pillar/top.sls` — example role assignment (sensor vs central). ## Architecture (what this release actually is) **Hub-and-spoke, NOT peer-to-peer.** ``` sensor nodes central node ------------ ------------ daemon -> store (also a sensor, optionally) strawberry export --out bundle.sbx --(Salt pull)--> bundle_dir/ strawberry fleet train bundle_dir --out shadow.sbm strawberry fleet overlay bundle_dir --out fleet-overlay.sbo model_dir/ <--(Salt push)---------- distribute: shadow.sbm + fleet-overlay.sbo (round 2+ exports carry fleet_bits) ``` - There is **no node-to-node transport**, no gossip/DHT, no decentralized aggregation. - Egress is **central-pull** (Salt orchestration / `cp.push` of a sealed, redacted bundle). The bundle artifact contract is stable, so an object store / mTLS collector can replace the pull later without daemon changes. - **Promotion is operator-only, never automatic.** States distribute `shadow.sbm`; a model only acts after `strawberry model promote` passes the pre-registered gate (`docs/evaluation/stage4-promotion-gate.md`). Keep promotion a human step. ## CLI surface the states invoke (exists today on `main`) - `strawberry export --out ` — sensor egress. Auto-includes fleet rarity when `/fleet-overlay.sbo` is present. - `strawberry fleet train --out ` — central merge + train shadow model. ## Current state and remaining gaps (honest status) Closed on `main`: - `strawberry fleet overlay --out fleet-overlay.sbo` exists (sums per-node rarity sketches; the pillar's `central.overlay_out` can run). - `node_id` is a daemon config key: export stamps `node_uid` via `read_node_uid(config.node_id)`. - Incremental export: `strawberry export --export-dir DIR --since auto` (or `--since `). `auto` reads `DIR/.export-cursor` (one decimal int64 = the previous export's `time_high_ns`); a missing or unparsable cursor means a full export (with a note). After a successful export the cursor is rewritten atomically (tmp + rename, 0640). The cursor does not match the prune glob (`export-*.sbx`). The next export starts 120 s before the cursor so a window that was still open is re-shipped. Labeled windows always re-ship (the replay exposes no label timestamp, so a delayed label cannot be dated). Rarity records are always included (the sketch is cumulative). Merge semantics on the central node (repeated exports pulled into ONE bundle dir are safe): - `fleet train` dedups per (node_uid, window_id) across bundles: first-seen order kept, label = positive in ANY bundle, features/start/rarity from the newest bundle (manifest `created_unix_ns`, tie: later path). Windows with the same id on different nodes are separate. - `fleet overlay` uses only the NEWEST bundle per node (its rarity sketch is cumulative, so summing every bundle would count a node k times). A bundle stamped `fleet-overlay` in the dir is skipped. Remaining gap: 1. **No multi-node validation.** The pipeline is unit/component + single-binary tested; it has **not** been run as a real multi-node fleet (N daemons -> central pull -> merge across hosts -> distribute -> re-export). A docker-swarm end-to-end is the recommended acceptance test before fleet rollout (the plan's deferred `docker-live` milestone). ## Paths (binary defaults) - store / `db_path`: `/var/lib/strawberry/evidence` - `model_dir`: `/var/lib/strawberry/model` (holds `shadow.sbm`, `promoted.sbm`, `rarity.state`, `fleet-overlay.sbo`) ## Security invariants the states must preserve - `capture_redact_cmdline: true` (argv redaction before storage) — do not disable on a fleet. - No new inbound listener on sensor nodes — export is an outbound read-only artifact pulled over the trusted channel; the daemon sandbox stays intact. - Reviewed, sha256-pinned binaries only; verify the pin before install.