82 lines
4.3 KiB
Markdown
82 lines
4.3 KiB
Markdown
# Scheduled telemetry reduction
|
|
|
|
`strawberry store reduce` rewrites sealed telemetry segments and drops dead,
|
|
benign process subtrees (stage 7a). Evidence (findings, detections and their
|
|
provenance chains) is untouched. On the measured corpus it cut execs by about
|
|
91% and windows by about 92% (replay and correlation roughly 11x cheaper) with
|
|
all detections preserved.
|
|
|
|
`store reduce` needs exclusive access to the store, so it refuses while the
|
|
daemon is running. The scheduled path automates **stop, reduce, start**:
|
|
|
|
- `strawberry-reduce.timer` fires weekly (randomized by up to 1 hour).
|
|
- `strawberry-reduce.service` runs `/usr/local/lib/strawberry/strawberry-reduce.sh`.
|
|
- The wrapper checks `store_reduce_enabled` in `/etc/strawberry/strawberry.conf`.
|
|
If it is not `true`/`1`, it prints `strawberry-reduce: disabled
|
|
(store_reduce_enabled)` and exits 0 without touching the daemon.
|
|
- Otherwise it stops `strawberry.service`, runs `store reduce --yes --json`
|
|
under `nice -n 19`, and starts the daemon again. An EXIT trap restarts the
|
|
daemon on every exit path (reduce failure, signal, error). Only `kill -9` of
|
|
the wrapper itself can bypass it. The wrapper's exit code is the reduce exit
|
|
code, so a failed run shows as a failed unit in `systemctl`/`journalctl -u
|
|
strawberry-reduce`.
|
|
|
|
The scheduled path is **7a only**: no `--skeleton` or `--merge`. A manual
|
|
`strawberry store reduce` is never gated and always available.
|
|
|
|
## Before you enable it
|
|
|
|
- **Monitoring gap.** The daemon is stopped for the length of each run, and no
|
|
live telemetry or detections are recorded during it.
|
|
- **Retention change.** Reduction permanently drops benign telemetry. Run the
|
|
recall gate against a copy of your store first to confirm detections survive.
|
|
- The unit is intentionally less sandboxed than `strawberry-update.service`:
|
|
it must reach systemd to stop and start the daemon and rewrite the store.
|
|
|
|
## Enable
|
|
|
|
The installer places the units and wrapper but enables nothing.
|
|
|
|
```sh
|
|
# 1. in /etc/strawberry/strawberry.conf
|
|
store_reduce_enabled=true
|
|
# 2. start the timer
|
|
sudo systemctl enable --now strawberry-reduce.timer
|
|
# one-off run: sudo systemctl start strawberry-reduce.service
|
|
```
|
|
|
|
To turn it off, `systemctl disable --now strawberry-reduce.timer` (or set
|
|
`store_reduce_enabled=false`, which makes any fired run a no-op).
|
|
|
|
## Merge horizon and CPR mode (manual `store reduce --merge`)
|
|
|
|
`--merge` (stage 7c) thins repeated `(node, type, target)` connect/file_write
|
|
accesses of old records. Two knobs, both valid only together with `--merge`
|
|
(otherwise a usage error):
|
|
|
|
- `--merge-horizon H` (whole hours, >= 1): merge only records older than `H`
|
|
hours before the newest sealed telemetry record. Default (flag absent) is the
|
|
2-day GC margin. Values below 1 hour are never used: the library clamps to
|
|
1 h (`kMinMergeHorizonNs`) so records still being written near the seal
|
|
boundary are not merged. GC (7a) and skeletonization (7b) always keep the
|
|
2-day margin; the active segment is never touched.
|
|
- `--merge-mode sample|cpr` (default `sample`). `sample` is the 7c sampler
|
|
(groups > 8 keep head 4, tail 1 and a 1-in-32 sample); with no horizon flag
|
|
it is byte-identical to before. `cpr` is the Causality-Preserving Reduction
|
|
of Xu et al. (CCS'16): a repeated access is dropped only when the node's
|
|
immediately preceding record was the same access, so every causal ordering
|
|
survives (connect X, exec child, connect X keeps both connects). Each run of
|
|
identical accesses keeps its first record, and the last occurrence of every
|
|
group is always kept.
|
|
|
|
Safety argument: merge never removes a node, a process_exec, a flagged record
|
|
or anything in a flagged node's keep closure; it only thins repeats of the same
|
|
access by non-flagged, live-or-dead benign nodes, so a shorter horizon cannot
|
|
cause a delayed retro-hunt label to lose a subtree. CPR is causality-preserving
|
|
per node: the run-length-collapsed access sequence of each node is unchanged.
|
|
Limits: interleaving is judged from same-node records and from an exec whose
|
|
parent is that node; each sealed segment start is a run boundary (the plan pass
|
|
is per segment), so a run spanning two segments keeps its first record in each.
|
|
|
|
`--json` adds `merge_mode` and `merge_horizon_hours` (48 = default margin).
|
|
The scheduled wrapper has no `--merge` knob, so it is unchanged.
|