strawberry/deploy/salt/README.md
trvon 265915ac3b
salt: README diagram uses the fleet overlay CLI
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 06:58:16 -07:00

80 lines
4.4 KiB
Markdown

# Strawberry distributed deployment (Salt) — contract for the deploying agent
This directory holds the **Salt pillar** (data contract) for deploying Strawberry's
distributed learning. The SLS **states** (install/config/egress/distribute) are to be
authored separately (by the deploying agent) and consume this pillar.
- `pillar/strawberry.sls` — the full pillar schema with defaults + required fields.
- `pillar/top.sls` — example role assignment (sensor vs central).
## Architecture (what this release actually is)
**Hub-and-spoke, NOT peer-to-peer.**
```
sensor nodes central node
------------ ------------
daemon -> store (also a sensor, optionally)
strawberry export --out bundle.sbx --(Salt pull)--> bundle_dir/
strawberry fleet train bundle_dir --out shadow.sbm
strawberry fleet overlay bundle_dir --out fleet-overlay.sbo
model_dir/ <--(Salt push)---------- distribute: shadow.sbm + fleet-overlay.sbo
(round 2+ exports carry fleet_bits)
```
- There is **no node-to-node transport**, no gossip/DHT, no decentralized aggregation.
- Egress is **central-pull** (Salt orchestration / `cp.push` of a sealed, redacted bundle).
The bundle artifact contract is stable, so an object store / mTLS collector can replace
the pull later without daemon changes.
- **Promotion is operator-only, never automatic.** States distribute `shadow.sbm`; a model
only acts after `strawberry model promote` passes the pre-registered gate
(`docs/evaluation/stage4-promotion-gate.md`). Keep promotion a human step.
## CLI surface the states invoke (exists today on `main`)
- `strawberry export <store_dir> --out <file>` — sensor egress. Auto-includes fleet rarity
when `<model_dir>/fleet-overlay.sbo` is present.
- `strawberry fleet train <bundle_dir> --out <model.sbm>` — central merge + train shadow model.
## Current state and remaining gaps (honest status)
Closed on `main`:
- `strawberry fleet overlay <bundle_dir> --out fleet-overlay.sbo` exists (sums per-node rarity
sketches; the pillar's `central.overlay_out` can run).
- `node_id` is a daemon config key: export stamps `node_uid` via `read_node_uid(config.node_id)`.
- Incremental export: `strawberry export --export-dir DIR --since auto` (or `--since <unix_ns>`).
`auto` reads `DIR/.export-cursor` (one decimal int64 = the previous export's `time_high_ns`);
a missing or unparsable cursor means a full export (with a note). After a successful export the
cursor is rewritten atomically (tmp + rename, 0640). The cursor does not match the prune glob
(`export-*.sbx`). The next export starts 120 s before the cursor so a window that was still open
is re-shipped. Labeled windows always re-ship (the replay exposes no label timestamp, so a
delayed label cannot be dated). Rarity records are always included (the sketch is cumulative).
Merge semantics on the central node (repeated exports pulled into ONE bundle dir are safe):
- `fleet train` dedups per (node_uid, window_id) across bundles: first-seen order kept, label =
positive in ANY bundle, features/start/rarity from the newest bundle (manifest
`created_unix_ns`, tie: later path). Windows with the same id on different nodes are separate.
- `fleet overlay` uses only the NEWEST bundle per node (its rarity sketch is cumulative, so summing
every bundle would count a node k times). A bundle stamped `fleet-overlay` in the dir is skipped.
Remaining gap:
1. **No multi-node validation.** The pipeline is unit/component + single-binary tested; it has
**not** been run as a real multi-node fleet (N daemons -> central pull -> merge across hosts
-> distribute -> re-export). A docker-swarm end-to-end is the recommended acceptance test
before fleet rollout (the plan's deferred `docker-live` milestone).
## Paths (binary defaults)
- store / `db_path`: `/var/lib/strawberry/evidence`
- `model_dir`: `/var/lib/strawberry/model` (holds `shadow.sbm`, `promoted.sbm`, `rarity.state`,
`fleet-overlay.sbo`)
## Security invariants the states must preserve
- `capture_redact_cmdline: true` (argv redaction before storage) — do not disable on a fleet.
- No new inbound listener on sensor nodes — export is an outbound read-only artifact pulled
over the trusted channel; the daemon sandbox stays intact.
- Reviewed, sha256-pinned binaries only; verify the pin before install.