Commit graph

221 commits

Author SHA1 Message Date
trvon
cc23b4637c
Release v0.2.10
All checks were successful
release / draft (push) Successful in 19s
release / build (debian:trixie, trixie) (push) Successful in 1h5m36s
release / build (ubuntu:24.04, noble) (push) Successful in 1h1m17s
release / publish (push) Successful in 13s
v0.2.10
2026-11-19 18:56:57 -07:00
trvon
9a2c43808e
fleet e2e: add out-of-sample evaluation check via --eval-after auto 2026-11-19 18:22:58 -07:00
trvon
d9da7e2986
yara: drop partition subsets before reload and trim glibc heap 2026-11-19 18:22:57 -07:00
trvon
a4179c474d
docs: say what the gate actually enforces, and what a passing verdict needs
The gate's own documentation claimed more than the tool did, in ways that
mattered once the integrity checks landed.

- "passes only if all 9 cells pass" was enforced by scripts, not by the gate; it
  is now enforced (previous commit) and the text says which criterion the tool
  applies: complete evidence, refused false, every pooled cell passing.
- The model digest proves agreement ACROSS reports. It cannot prove that the
  file on disk now is the one that was evaluated. Corrected everywhere.
- The remedy was described inconsistently: cutting the training bundles earlier
  is NOT enough (evaluation still replays the whole store), a bounded evaluation
  range is required. Written consistently, with --eval-after NS|auto.
- scripts/fleet_e2e.sh asserted three gate checks; promotion_gate emits five. It
  now asserts >= 5, requires the provenance fields, and adds the honest-evidence
  invariant: today every cell is in-sample, so at least one cell must report
  in_sample true, none may pass, and each must carry the out-of-sample failure.
  A pass over in-sample cells is itself a failure.
- The Salt/range runbooks advertised the flow as producing promotion evidence;
  it collects gate INPUT, and a passing verdict requires out-of-sample windows.
2026-11-19 17:30:35 -07:00
trvon
383c22cbd3
exec guard: answer a cache hit without holding the cache lock
Both hit paths (fanotify and Endpoint Security) called finish() while holding
cache_mutex_. finish() is the answer: it writes the kernel response, runs the
observation callback, bumps stats under stats_mutex_ and pushes to
report_mutex_. Holding the cache lock across that serialised every cache-hit
exec behind one mutex even though the daemon runs two workers, so on Linux a
slow answer was a slow exec - in the path whose whole job is to keep launches
fast.

detail::answer_cached_assessment copies the entry under the lock, releases it,
and only then answers. cache_mutex_ is now a leaf lock (also at the work-loop
insert), so the old edge cache_mutex_ -> {pending, stats, report} is gone and no
inversion can be introduced. Answer-exactly-once is unchanged: respond() still
checks pending->responded under the pending mutex, and the hit path still
answers once and returns.

The seam is declared in exec_guard.hpp so the tests no longer redeclare an
internal function in their own translation unit.
2026-11-19 17:29:38 -07:00
trvon
6618328059
fleet gate: refuse incomplete evidence and unprovable pooled provenance
Two ways a verdict could be built on evidence the tool did not actually have.

Pooling only ever upgraded provenance: one node that knew its training range set
the pooled flag, so a cell whose other nodes could not prove a range was still
reported as evaluated, and a cell could pass with no evaluated range at all. A
node with an unknown or unusable range now marks the pooled cell incomplete,
zeroes the displayed range (no range is shown that only some nodes could prove),
and appends "provenance incomplete: N of M contributing node report(s)". A known
range can no longer upgrade an unknown cell.

The documented "passes only if all 9 cells pass" was not enforced: passed was
just "some cells and not refused", so a one-seed, one-period report set exited 0.
A completeness gate now requires 3 seeds, 3 periods contiguous from 0, the full
seed x period cross product, and per-report consistency, and names every reason
("need 3 training seeds...", "missing: seed3 period 2", "inconsistent
--eval-model / --periods across node reports") before any verdict is computed.
2026-11-19 17:28:46 -07:00
trvon
d81ccbe7c8
gate: a bounded evaluation range, so out-of-sample evidence is possible
The in-sample check landed without any way to bound an evaluation, so every cell
was in-sample and every promotion path refused. Cutting the training bundles
earlier does not help: training spans the bundles, evaluation replays the whole
store with no lower bound, and the store keeps everything (retention 0), so
eval_first <= training_last always held.

replay --policy-eval gains --eval-after NS|auto (0 = unbounded). The bound is
applied where windows are ranked and where injected traces are planned, so the
periods partition only the out-of-sample span; auto derives it as the latest
training-range end over the loaded --eval-model files. Comparisons are written
without first + warm-up arithmetic so INT64_MAX timestamps cannot overflow. A
bound past the end of the store does not produce a silent empty cell: the cell
carries eval_error and promotion_gate refuses naming it.

Provenance also gets honest about width: training_last_ns is the last training
window's END (a window spans window_ns, so using its start left up to 60 s of
trained-on data looking disjoint), and the single-model path (model promote) now
uses the ranked span instead of the whole store, which had guaranteed a refusal.

Verified: R11c passes with 128 out-of-sample windows and a derived bound, and
unbounded returns to in-sample. Refuses an explicit bound later than all history.
2026-11-19 17:27:59 -07:00
trvon
cae67981ab
detection: mark a timed-out scan incomplete without bypassing quarantine
Review of 893127c found that reporting `scanned = false` for a timed-out
guard scan was wrong in a way that mattered: decide_exec answers `allow` on
`!scanned` BEFORE it looks at the operator quarantine, so a quarantined binary
whose scan overran its deadline would have been allowed to run. It also dropped
the only `scanned = true` on the normal assessment path, so every ordinary
assessment became "unscanned".

The assessment now keeps `scanned = true` (quarantine, allowlist and IOC
decisions still apply, and the guard still fails open on a thin result) and
carries a separate `incomplete` flag, which alone prevents caching:
guard_assessment_cacheable() and the pipeline's cache_put both exclude it.

Also fixes the same verdict being lost one caller up: scan_on_demand set
ok = false with an explanatory error, then unconditionally reset ok = true after
the match loop, so control scan-fd, MCP and retro-hunt all reported a timed-out
scan as a completed one with no error.

Tests: a timed-out assessment keeps scanned true and top_rule evidence, is
marked incomplete, and a second exec re-scans instead of hitting the cache; a
quarantined incomplete assessment still blocks in both enforce and audit, while
an incomplete clean one is allowed and not cacheable.
2026-11-19 16:48:19 -07:00
trvon
666608c4c0
detection: ctime in the scan key, retry after a failed scan, never hold live execs
Three launch-path defects from the audit.

The scan cache key was dev/ino/size/mtime, and mtime is settable via utimensat,
so a rewrite that restored size and mtime (cp -p, install -p, package restore, or
a deliberate attacker) served the old verdict with no TTL. The key now includes
ctime, and the fanotify path calls the shared detail::image_cache_key instead of
re-deriving the fields, so the guard and the pipeline cannot drift apart again.

recent_images_ was inserted before the scan and never removed, so an image that
was unavailable or whose buffer grant was refused was never retried for that
process instance - contradicting scan_image's own comment. The key is now
dropped when the outcome is unavailable, read_error or yara_error, while
completed scans stay deduped.

Live exec scans were deferred for up to ~5 s per emergency episode. The live
queue is bounded at 4096 and full means dropped exec events, so a fork storm
during an emergency could lose telemetry the guard depends on. The live lane now
takes a single non-blocking probe (kLiveExecHold is 0, pinned by a
static_assert) and scans anyway, fail-open; background inventory keeps the
bounded hold.

Tests: a same-size, mtime-restored rewrite is re-scanned rather than served from
cache; a transient scan failure is retried on the next event while a completed
scan is not; the live-lane probe never blocks.
2026-11-19 16:21:34 -07:00
trvon
7149a90893
gate: refuse in-sample and degenerate evidence, and identify the model
Four findings from a four-lane audit made the promotion gate unable to support
its own claims.

In-sample evaluation: training walks the whole post-warm-up store and the gate
evaluates the same windows, with nothing checking provenance - replay.cpp itself
listed point_in_time_splits_and_overlap_not_assessed. Models now record the
window range they learned from (file format v2; v1 still loads as unknown) and a
content fingerprint, evaluation reports its own range plus in_sample, and
promotion_gate fails with the reason: an overlapping range, no recorded range,
or no digest. Unprovable provenance never counts as out-of-sample.

Degenerate baseline: with rarity-greedy on 0 hits, hits_per_cpu_second() returns
0.0, so ">= 1.5x rarity" reduced to "model hit something" and the recall check
was vacuous. A cell whose baseline scored no hits or spent no CPU now fails,
naming the budget refusals that starved it.

Stale models: cells keyed on a bare filename, so a later cycle that refused to
train left an older cycle's dist/gate models to be judged, and nothing bound a
verdict to the file it judged. Cells and reports carry the model digest, and
fleet gate sets refused (non-zero exit) for a cell without one or for a
(model, period) whose digests disagree across nodes instead of pooling them.

3 seeds are not generalization: they vary initialisation and the per-node
shuffle over identical windows. The procedure document now says so and states
what would test generalization (disjoint periods, models trained on an earlier
cut, out-of-sample ranges).

Tests: in-sample flagged, disjoint range not flagged, unprovable provenance
fails, 0-hit and 0-CPU baselines fail, mixed digests refused, legacy report
refused, range round-trip and fingerprint, v1 compatibility.
2026-11-19 16:20:40 -07:00
trvon
c60b44fb4d
kpi: budget the per-phase RSS peak, not only the last sample
compare() budgeted daemon_rss_last_mb only, so a regression that spiked and
then settled passed: the peak was printed but never checked, leaving just the
absolute 512 MB ceiling. Both metrics are now budgeted per phase through one
helper — max(32 MB, 15% of the baseline) for the phase-end value and
max(64 MB, 15%) for the peak.

The 64 MB floor is measured, not chosen: with a shared 32 MB floor three
same-version repeat pairs (8-r1->8-r2 idle peak +59.6 MB, 9-r1->9-r2 storm
+36.3 MB, 9-r1->9-r3 +36.0 MB) would be called regressions, because a single
transient sample spreads about twice as wide as the phase-end value. 64 MB
clears all nine same-version pairs while every v0.2.8 -> v0.2.9 pair still
trips it; re-checked four same-version pairs here with zero over-budget phases.

The peak check adds real signal: on the stored runs it catches v0.2.9's idle
peak at +137.4 MB where the last-sample check saw +102.4 MB. Missing samples
still fail closed and never read as "no growth".
2026-11-19 16:19:41 -07:00
trvon
0e3d89d918
range: export the shared constants and keep shellcheck quiet
range_common.sh is a sourced library: range_up.sh, range_down.sh and
range_status.sh consume its constants, which shellcheck cannot see when it
analyses the file alone, so it reported every one as unused. Exporting them
states that interface, and the two $(cd ...) assignments are split from their
export so a failing cd is not masked.
2026-11-19 11:49:41 -07:00
trvon
ac4150f6e7
build: spell out the optional in the control-reply helper
`return cond ? parsed["data"] : nlohmann::json::object();` became ambiguous
between std::optional's converting constructor and nlohmann::json's greedy one
when that translation unit was recompiled. Name the type explicitly; no
behaviour change.
2026-11-19 11:49:41 -07:00
trvon
893127c691
detection: a timed-out or degraded scan is never a clean verdict
Two launch-path defects let a weaker-than-complete scan become the cached
answer for an executable until the next rules reload. In enforce mode that
means the exec guard silently stops consulting YARA for that binary.

1. A YARA deadline overrun was reported as success. timed_yara_scan/
   timed_yara_scan_ranges returned `ok || timed_out`, so assess_exec set
   have_matches and cached the image with an EMPTY match list; the guard cached
   that assessment too. The guard's YARA budget is exactly prevention_timeout_ms
   (1 s by default), so one slow scan was enough. They now return a
   BoundedYaraScan {ok, timed_out} and only `completed() == ok && !timed_out`
   may set have_matches or reach cache_put; scan_image returns yara_error and
   caches nothing; the assessment reports scanned == false so the guard fails
   open instead of inheriting a clean answer. Matches found before the deadline
   are still reported as evidence, and a timeout is counted in yara_timeouts.

2. The guard cached degraded and partial assessments, contradicting
   exec_guard.hpp ("not cached") and the pipeline, which already excluded them.
   A degraded result is produced by a transient condition (the scan-buffer lease
   is momentarily held), so one collision pinned the image to hash/IOC-only for
   the whole rules+policy generation. The rule now lives in one place,
   guard_assessment_cacheable(), used at both the Linux and macOS cache stores.

Tests: a timed-out assess_exec is not scanned, still reports the partial match,
and a second exec re-scans instead of hitting a cache; the cache predicate is
covered for clean, degraded, partial and unscanned assessments.
2026-11-19 11:49:41 -07:00
trvon
31e44caab0
cli: honor <group> --help, and add fleet run for a standalone central node
`strawberry fleet --help` reported "unknown subcommand 'fleet --help'" and
then dumped the group's usage, even though docs/cli-ux.md promises that
`strawberry <cmd> --help` prints the same text as `strawberry help <cmd>`.
A group word followed by -h/--help now prints the group listing.

Centralized training had no single entry point: `fleet train` and
`fleet overlay` had to be run by hand (or through Salt), so a standalone
central node had no "run" command at all. `strawberry fleet run <bundle_dir>
[--out DIR]` trains and builds the overlay by calling the same two code paths,
so the behaviour cannot diverge, and prints the model/overlay paths.

It deliberately does not distribute to nodes, apply the overlay, or promote a
model: distribution stays behind the Salt canary cycle and promotion stays
behind `model promote` plus the fleet promotion gate. The JSON output carries
distributed=false and promoted=false so a script cannot mistake it.

Tests: fleet --help and fleet -h print the group without error; fleet run maps
to --fleet-run with optional dir, --out/--seed/--epochs passthrough, rejects
unknown options; a bare `fleet` still explains itself. Verified end to end on
two seeded bundles: model + overlay written to the requested directory.
2026-11-18 12:11:34 -07:00
trvon
aa7143e005
range: read Caldera link states correctly, treat a vanished operation as failed
The verifier treated every negative link status as "Caldera never ran it", but
Caldera's states are not signed that way: HIGH_VIZ=-5, UNTRUSTED=-4,
EXECUTE=-3, DISCARD=-2, PAUSE=-1, SUCCESS=0, ERROR=1, TIMEOUT=124. EXECUTE is
the normal in-flight state, so a healthy incident was failed on its first poll
and the range reported zero verified incidents while the marker really ran.

Links are now classified explicitly: EXECUTE and PAUSE keep polling, SUCCESS
still requires a completion timestamp, and DISCARD/UNTRUSTED/HIGH_VIZ/TIMEOUT
or a nonzero exit are terminal failures.

Reconcile also treated a missing operation as pending forever, so an incident
whose operation disappeared (Caldera recreated after the OOM) was retried
indefinitely. A 404 is now terminal ("operation no longer exists"), while any
other read error still stays pending and never becomes "executed". The Caldera
client turns a malformed JSON body into a RuntimeError instead of leaking a
decode error from deep inside the client.

Verified against the live range: a queued link (-3) polled to SUCCESS with a
finish timestamp and was recorded verified with the executing agent; synthetic
cases cover every state code, SUCCESS without a finish, 404 and 500.
2026-11-17 16:16:23 -07:00
trvon
433615044e
governor: release a memory episode on the memory hold, not the current RSS
compute_level picked the release hold from the RSS *after* the drop, so an
episode raised by the daemon's own memory released in 150 ms as soon as RSS
fell below the warning ratio — ignoring the documented 1000 ms memory hold.
A warning-level memory episode could therefore clear almost immediately.

The episode is now classified when it is raised: memory_driven_episode_ is set
when memory drove the level, and that decides the release hold. Critically this
does not clear when the level steps down (critical -> warning is the same
episode); the next escalation re-classifies it.

Tests: a memory warning -> calm transition is still warning at 200 ms and
999 ms and clears at 1000 ms, while a CPU-driven episode still clears in 150 ms.
2026-11-17 13:52:37 -07:00
trvon
39c5bc4efe
range: host-safety preflight, verified teardown, per-range scheduler identity
Host safety: range_up refuses to start below 10 GiB MemAvailable or 20 GiB free
disk, range_status prints the headroom, and the README documents both floors
and the 8 GiB pause a monitor should enforce. The 2026-11-17 global OOM was
caused by fifteen concurrent compilers plus the range, not the range alone, but
starting blind is what made it easy.

Teardown: sensor ids come from the vms/ contents rather than SB_RANGE_SENSORS,
so a lowered count or missing pid file cannot leave a VM behind; vm_pid accepts
a PID only when its command line still names that sensor's QEMU and disk, so a
recycled PID is never counted as live or signalled; a /proc sweep covers QEMUs
whose pid file was lost; --destroy refuses while any of them survives and
deletes only the sensor-N disks and seeds this range created.

Scheduler: range_up writes a per-range pid file and up/down/status act only on
a PID whose command line runs this repo's range_attack.py with SB_RANGE_DIR
resolving here, replacing the host-wide pkill that could kill another range.

Status: agents are reported live/stale/unknown per sensor group instead of a
raw registration total (six registrations for three VMs after restarts), and
the scheduler line now separates verified incidents from those without
execution proof, reporting "rule-pushed" rather than implying observed labels.
2026-11-17 13:45:22 -07:00
trvon
3cf5461d72
range: verify incidents actually ran, and keep pacing across restarts
The scheduler recorded an incident as complete the moment Caldera accepted the
operation, timed labels from that creation moment, and advanced its label
counter once a rule reload was acknowledged. Restarts also reset its clock, so
incidents could land minutes apart instead of one interval, and a malformed
state file was silently replaced — reusing marker ids that the range's
"unknown at exec time" contract depends on.

Now an incident is verified only from Caldera's own record: a link for this
ability, a non-negative exit, a completion timestamp and the agent that ran it,
polled with bounded retries. Unverified outcomes keep no execution time and are
excluded from label aging, so rule pushes are no longer presented as observed
labels. The next due time is persisted (a restart waits out the interval), the
pending id and marker are saved before the Caldera call, and load() fails
closed — exit 2, file untouched — for malformed, unreadable or wrong-schema
state, defaulting only when the file is absent. The Caldera client is pinned to
the range's loopback origin and /api/v2 paths.
2026-11-17 13:45:18 -07:00
trvon
ed59a0ee26
kpi: gate on daemon RSS, not just container memory
The constrained-lane report gated idle CPU, exec-decision p95 and dropped
events, so a real memory regression could pass unnoticed — v0.2.9 grew daemon
RSS by 132-141 MiB against v0.2.8 in all three interleaved pairs.

Each phase now reports daemon RSS (max, last, sample count) from the existing
status.jsonl resources.rss_mb samples, with container memory kept as a
secondary metric, and reports fail closed when daemon RSS samples are missing
rather than silently skipping the check. --compare prints a per-phase delta
column against a baseline-relative budget of max(32 MB, 15% of the baseline)
and exits non-zero with OVER BUDGET lines, so a regression is visible instead
of buried in container statistics that also include the harness.

Measured on the stored runs: the v0.2.8 -> v0.2.9 pairs breach the budget in
all six phases (+87.2 to +141.0 MB), while a same-version pair does not (worst
spread +29.1 MB). The docstring states exactly what a non-zero exit covers.
2026-11-17 13:45:17 -07:00
trvon
169a05cd2a
fuzz: target the YARA rule partitioner
rule_partition.cpp parses rule sources (splitting rules, extracting top-level
condition identifiers and facts), but had no fuzz target, contradicting the
repository rule that every parser of attacker-influenced input gets one.
fuzz_rule_partition asserts the splitter is a total, exact partition of its
input, that rule/condition slicing stays within the rule, and that partition
selection agrees with magic bytes and the size bucket boundaries.

It is registered in fuzz/meson.build (with rule_partition.cpp), swept by
scripts/run_fuzzers.sh, and documented next to the other targets. Seeds cover
comments, strings, regexes, hex blocks, an include and a truncated source.
2026-11-17 13:45:12 -07:00
trvon
a1d77dda9a
governor: fail-safe host pressure, diagnostic diskstats, capped scan buffers
Host pressure stays fail-safe: any of CPU/I/O/memory PSI above its profile
threshold, or low MemAvailable, still defers background work and can degrade
the exec guard. That is deliberate — global I/O PSI also counts waits with no
block-device counter (FUSE/network-backed I/O, display page flips), so idle
disks never prove pressure is benign. An audit of an earlier version that
vetoed I/O PSI below 5% block utilization rejected it on exactly that basis.

/proc/diskstats is therefore diagnostic telemetry only, exposed as
host.block_io_util_pct. It is keyed by major:minor so a reused device name
cannot compare unrelated counters, and reports unknown (-1) for the first
sample, a new/replaced/reset device, or an oversized read, instead of a false
0%.

scan_buffer_budget() now derives from the same effective budget that pressure
uses — the configured/profile budget capped by cgroup memory.max — with the
existing 32 MiB floor, so a cgroup-capped daemon cannot be granted scan
buffers larger than its own limit. Lock order stays mutex_ -> buffer_mutex_.

Tests: deterministic fail-safe cases across every diskstat value, the
new-device regression, scanner budget under a 100 MiB cgroup cap, and a seeded
property asserting severe I/O PSI always holds work whatever diskstats says.
2026-11-17 13:45:07 -07:00
trvon
af5bfa78e5
validation: multi-sensor range for real promotion-gate evidence
An isolated local lab that extends the single-VM de-risk (deploy/validation/vm,
2026-11-08) to a multi-sensor fleet run over days, to produce the delayed-label
history the Stage 4 fleet gate needs. All local/offline: Caldera on loopback
(rootless Docker, capped), sensor VMs reach it only via the qemu slirp gateway,
no traffic leaves the host; payloads are inert markers (a copy of /bin/sleep
with a marker string), never real malware.

- range_up.sh: boot/resume Caldera + N Arch VMs running the real daemon
  (eBPF + fanotify), benign workload and a Caldera agent as systemd units;
  idempotent and travel-safe (VM disks + scheduler state persist under
  $SB_RANGE_DIR, the scheduler paces by its own uptime, so suspend/reboot just
  pauses it and range_up.sh resumes).
- range_workload.sh: varied benign activity so the detector learns a real
  baseline the emulated incidents must stand out from.
- range_attack.py: one distinct Caldera incident per 30-90 min per sensor
  (fresh ability + operation so each lands in its own replay window); incidents
  are unknown to the rules at exec time, then a later rule-file update + rules
  reload lets the retro-hunt label the earlier windows -- the delayed extra
  positives the gate measures, produced the production way, not injected.
- range_status.sh / range_down.sh (pause or --destroy), README, cloud-init.

Nothing here writes to the repo; the $SB_RANGE_DIR runtime tree is disposable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-11-15 09:43:22 -07:00
trvon
3fd5d23b77
threat traces checker + fleet gate in the dockerized Salt test
- scripts/threat_traces.py: enforces the mechanical rules of the 2026-10-28
  threat-trace pre-registration: required # meta keys; eval families first
  reported 2025-07-01..2026-10-28 with a new-family/new-variant bucket and
  write-up sources only; training families before the cutoff with every
  fifth family (first-report order) in dev; no family in both files;
  RFC 5737 addresses and .example domains only; size caps; and that every
  file compiles with replay's parser. --sha256 prints the freeze digest.
- tests/trace_check.cpp: strawberry-trace-check dev tool (replay's parser).
- strawberry.gate: optional gate.warmup_hours (unset = the pre-registered
  12 h; TEST pillars only).
- deploy/salt/test/fleet_run.sh: gate phase - tops sensor history past the
  10 min injection floor, runs orch.strawberry_gate, asserts 3 gate models
  were published, 9 pooled cells over 3 sensors with positives, sensors keep
  their reports, and still no promoted model anywhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 09:36:24 -07:00
trvon
6f5c62b31c
Stage 4 over fleet data: per-node 3x3 policy evaluation, pooled fleet promotion gate
The pre-registered gate (docs/evaluation/stage4-promotion-gate.md) asks for the
criteria to hold on 3 non-overlapping periods x 3 training seeds; until now the
tool reported a single run and the operator recorded the rest. This produces
the 9 results for a fleet, with the criteria unchanged:

- replay --policy-eval --eval-model FILE (repeatable) --periods N: one pass
  evaluates every model on every equal time slice of the ranked history; each
  (model, period) cell simulates the model and rarity-greedy with their own
  hourly budgets exactly as policy_eval does (policy_eval.cells). With one
  model equal to --model and one period the cell equals the single-model
  evaluation (test R11b); periods partition windows and positives.
- strawberry fleet gate DIR: pools the cells of every node report (windows,
  positives, inspections, CPU-s, hits, refusals) per (model, period) and
  applies learn::promotion_gate unchanged to each; passes only if all pass.
  Malformed / cell-less reports are skipped and listed.
- Evaluation stays on the node (raw telemetry never leaves it): the central
  cycle now also publishes all three seed models to dist/gate/; strawberry.gate
  (sensors) evaluates them with the pre-registered evaluation traces; the
  operator-run orch.strawberry_gate pushes the reports back and pools them;
  evidence under central.work_dir/gate-<stamp>/. Nothing promotes.
- docs/evaluation/stage4-fleet-gate.md: the procedure (new dated document;
  criteria and numbers unchanged).

Validated: test_fleet_gate (pooling sums, pooling reaching 10 positives
neither node has alone, one failing cell fails the fleet, malformed reports
skipped), R11b, Salt render checks (no models -> refusal; 3 models x 3
periods; orchestration pools), and scripts/fleet_e2e.sh end to end: 3 nodes ->
9 pooled cells with 28-50 positives each; the untrained synthetic models
correctly do not pass (exit code matches the verdict).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 09:18:04 -07:00
trvon
5018d81d48
Release v0.2.9
Some checks failed
release / draft (push) Successful in 19s
release / build (debian:trixie, trixie) (push) Successful in 1h4m50s
release / build (ubuntu:24.04, noble) (push) Failing after 1h0m33s
release / publish (push) Has been skipped
v0.2.9
2026-11-15 08:49:04 -07:00
trvon
1c9b005645
store: incremental read-only refresh (fixes the tray spinning a core)
Root cause, from the Tracy capture the CPU watcher took when the tray spun
(2026-11-15 08:15, 67% for 15 s): database.reload_read_only rebuilt the whole
evidence view from every segment — mean 1.4 s, max 1.7 s, 4x in 20 s — because
every 3 s poll that saw the daemon append evidence triggered a full reload.

- store::scan_store_read_only_from + ReadCursor (per segment: id, sealed,
  bytes, records delivered). When the previously seen segments are an
  unchanged prefix (same ids, sealed ones byte-identical), only records
  appended since are delivered: the rest of the previously last segment
  (skipping what was applied) and new segments. Any other change —
  compaction, retention, reduction, a rewritten or shrunk segment — delivers
  everything and the caller rebuilds.
- Database read-only refresh uses it; records apply in the same order a full
  rebuild uses, so the view is identical. read_only_reload_counts() reports
  full vs incremental refreshes.

Live, on the real evidence store: the change-driven reload went from ~1.4 s
to 2.8 ms; the tray did ~97 ms of work in a 60 s window.

Test (test_incremental_view, seeded property): a long-lived reader
refreshing across random appends, upserts, seals and a compaction always
equals a freshly opened reader; appends refresh incrementally (42-49 per run),
the compaction forces a rebuild, and no change reports no change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 08:46:00 -07:00
trvon
8411c70ba8
macOS: build and test fixes found re-validating on the Mac (57/57 green)
- fleet.cpp: HOST_NAME_MAX is Linux-only; use a 256-byte buffer (the
  POSIX/RFC 1035 limit gethostname() truncates to).
- test_replay: two dangling pointers - victim(run(...)) returned a pointer
  into a temporary report destroyed at the end of the statement. Passed on
  Linux by allocator luck; failed deterministically on macOS for every seed.
- test_yara_scanner: a brace initializer with a comma inside assert() (the
  macro splits it on macOS), and ::close needs <unistd.h> outside the
  Linux-only block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 08:35:34 -07:00
trvon
db4c49155f
yara: rule partitions — scan each file only with the rules that can match it
Why YARA-X scanned ~1.6x faster than libyara (perf + source audit, 2026-11-14):
not the language — both spend ~5 s in the atom search loop — but verification
(libyara 2.0 s vs 0.3 s). YARA-X derives filesize bounds and header constraints
per rule at compile time and disables those patterns before the search, so their
atom hits are dropped with one bit test. In the yara-forge core pack 1787 rules
require an MZ header and 2498 bound filesize, so for big Linux ELFs most of
libyara's verification is wasted.

Rule partitioning gets the same win with libyara, and drops the rules' atoms from
the automaton entirely:
- src/rule_partition.cpp (pure): lexer-aware splitter (comments, text, regex
  after = or matches, hex), facts from TOP-LEVEL `and` terms only (MZ /
  pe.is_pe, ELF, Mach-O magic; filesize < / <=), exclusion per (MZ / non-MZ) x
  size bucket (<1, 1-4, 4-16, >=16 MiB). Global rules and rules referenced by
  kept rules always stay.
- YaraScanner builds subsets in a background thread after each load, in
  priority order (big non-PE first) under rule_partition_budget_pct (default
  100 ~ one extra full set: 3 subsets, +59 MiB; all 16 class x size subsets
  would cost +428 MiB). Cached under rules_cache_dir/partitions/. File, buffer
  and fd scans pick their subset; process-memory scans use the full set.
- Fails closed to the full rules: include directives, a split count that
  differs from libyara's, or a subset compile / count error.
- Status rules.partitions (state, reason, subsets, kept_rules/scans per
  partition, full_scans); config rule_partitions (default true) and
  rule_partition_budget_pct.

Measured on the 5132-rule pack: big ASan ELFs 1.64x, 1200 system binaries
1.59x, 0 match-set differences, including 348 positive files across
MZ/ELF/raw x small/5 MiB. Tests: splitter traps, facts, exclusion soundness
property, real pack = 5132 rules; differential test (24 file shapes x memory /
ranges / fd scans, every subset built) and fail-closed; bench --partitions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 06:58:39 -07:00
trvon
d5df2f489a
docs: libyara profile and engine A/B results
perf on a symbol build of libyara 4.5.8: Aho-Corasick walk 59%, atom-hit
verification 24%, rule compile ~5%. -O3 -march=native LTO libyara scans ~2.5%
slower than the distro -O2 build (memory-bound walk); fast mode 1.01x; loaded
segments 1.11x (2x on debug builds, opt-in).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 19:11:21 -07:00
trvon
7a7fa81c74
profiling: watch_cpu_capture.sh takes one Tracy capture when a process spins
Polls a profiled process's CPU every 5 s and, after sustained use above a
threshold, attaches scripts/profile_capture.sh for a fixed window. For bugs
that only appear sometimes (the tray spinning a core during the 2026-11-14
hang). Verified attach on the live profiled tray: zones tray.on_reply /
update_status / update_tray / poll arrive (~4 ms per 3 s poll when healthy).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 18:52:09 -07:00
trvon
4586eed50d
yara engine: range/fast-mode scanning API, ELF load ranges, engine A/B bench, opt-in guard partial scan
Strawberry drives libyara (system 4.5.8), no custom engine. Profiling the scan path
(Tracy + user-space perf) shows 86% of scan time inside libyara's string-search loop;
our callback runs only on matches. Engine-usage variants measured with the new
`strawberry-scan-bench --engine` A/B over 3434 ELF files / 2.9 GB (5132 rules):
  full 50.1 MiB/s | fast mode 1.01x | loaded segments only 1.11x (2x on debug/ASan
  builds; 1.08x in the 4-32 MiB band the exec guard full-scans). 0 match-set
  differences, but the corpus has no rule matches, so parity is not proven.

- YaraScanner::scan_memory_ranges (yr_scanner + yr_scanner_scan_mem_blocks over
  zero-copy blocks; file offsets and filesize keep file semantics) with
  EngineOptions{fast_mode}; elf_load_ranges() (PT_LOAD + first page, merged,
  bounds-checked against hostile headers; empty -> scan everything).
- Exec guard: opt-in `guard_loaded_segments_only` (default off) scans only the
  loaded segments of ELF images >= 4 MiB at launch; such results are marked
  `partial`, never cached, and the pipeline still scans the whole file. Status
  pipeline.guard_partial_scans. Off by default: an 8% faster launch decision is
  not worth missing unloaded-section strings until the background scan.
- Fast mode is not adopted (1%, and it drops #count / @offset semantics).

Tests: ELF ranges (synthetic, BE, truncated, absurd phnum, phoff overflow, clamped
filesz, /bin/true), range scans (unmapped string unseen, at/uint32/filesize keep
file offsets, multi-block), guard partial scan catches mapped markers / misses
unmapped ones / pipeline full scan detects them / default off scans everything,
config round trip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 18:51:10 -07:00
trvon
11d3614348
profiling: pinned Tracy client, headless capture script, scan bench, tray + scan-stage zones
- third_party/tracy.wrap pins the Tracy client to v0.15.0 (sha256-checked), matching
  the Arch tracy 0.15.0 capture/export tools (client and tools must match exactly);
  built with tracy_enable + on_demand, so nothing is recorded or paid for unless a
  capture is connected. -Denable_profiling=true picks it up; default builds are
  unchanged (macros are no-ops).
- scripts/profile_capture.sh: headless (no GUI, no root) `run OUT -- CMD` or
  `attach OUT SECONDS` -> OUT.tracy + OUT.zones.csv + a top-zones table.
- strawberry-scan-bench (dev tool, not installed): the real scan_on_demand path over
  files with the real governor and rule set, so the scan path can be profiled
  without root or the daemon.
- Zones: scan_on_demand, image.read, image.sha256 (next to yara_scan/scan_image/
  assess_exec), tray.* (poll, on_reply, refresh_evidence, update_status/_policy,
  refresh_findings, notify) and evidence/database reloads.

Baseline (6 yams ASan binaries, 668 MiB, idle host): YARA 86%, read 9%, sha256 5%,
~61 MiB/s; PT_LOAD segments are 35-48% of those files. DEVELOPER.md: profiling
section, and the balanced row corrected (64 MiB image cap, 768 MiB budget).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 18:29:57 -07:00
trvon
109ec03649
Release v0.2.8
All checks were successful
release / draft (push) Successful in 19s
release / build (debian:trixie, trixie) (push) Successful in 57m18s
release / build (ubuntu:24.04, noble) (push) Successful in 52m22s
release / publish (push) Successful in 13s
v0.2.8
2026-11-14 15:36:26 -07:00
trvon
85239c85d5
tray: one outstanding status/policy poll; back off while the daemon is not answering
A slow daemon (17-22 s per request during the 2026-11-14 hang) made every
3 s poll open new status + policy requests on top of the unanswered ones.
Now each verb has at most one request in flight, and failed status polls
back off 2x per failure up to 30 s, restoring the normal rate on success.
(The tray's ~1 core of CPU at the time is not yet root-caused: profiling it
needs root here; see the Tracy setup follow-up.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 15:36:15 -07:00
trvon
5685600e57
Daemon memory: budget scan buffers, stream large images on the launch path, report throttling
2026-11-14 host hang: a yams ASan test run (~110 MiB binaries) pushed
strawberry.service past MemoryHigh=768M; the kernel throttled the daemon
(own memory.pressure full ~72%), so the exec guard stalled launches (589
1-second prevention timeouts in 80 min, even start-swayidle.sh), scans hit
p95 67 s and the control socket stopped answering.

- Exec guard (assess_exec) no longer copies an image before deciding: a cached
  scan answers with no read; an image over guard_full_scan_max_bytes, or when
  the scan-buffer budget is exhausted, is hashed by streaming (1 MiB chunks,
  new HashUtils::sha256_md5_fd) and decided on hash / IOC / quarantine
  (degraded; the pipeline still scans it). Only small budgeted images are copied.
- Scan-buffer budget (ResourceGovernor::acquire_scan_buffer, RAII
  ScanBufferLease): in-memory image copies bounded to memory budget / 4
  (balanced 192 MiB); the pipeline waits up to 30 s, on-demand scans 60 s;
  an oversized request is granted only alone, so no deadlock or starvation.
- Balanced pipeline_max_image_bytes 128 -> 64 MiB: larger images get hash/IOC
  checks only (as on the efficient profile).
- Response hashing streams instead of copying the whole executable.
- Status: resources.cgroup memory_high_mb / memory_high_events /
  memory_full_pct / throttled, and resources.scan_buffers; doctor check
  resources.memory (problem when throttled at MemoryHigh, with the fix).

Tests: budget grant/refuse/exclusive/wait/timeout/move, PSI full + memory.events
parsers, streaming hash = buffer hash across chunks, guard streams a 33 MiB
image with no YARA pass and no buffer grant, streams when the budget is
exhausted, full-scans + caches when budget is free, doctor throttle check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 15:36:14 -07:00
trvon
7cce78e270
Release v0.2.7
All checks were successful
release / draft (push) Successful in 19s
release / build (debian:trixie, trixie) (push) Successful in 59m6s
release / build (ubuntu:24.04, noble) (push) Successful in 53m17s
release / publish (push) Successful in 13s
v0.2.7
2026-11-14 13:27:22 -07:00
trvon
ccad00c431
CI: release build timeout 60 -> 120 min
The v0.2.6 trixie leg was cancelled by the job timeout at 60:04 (v0.2.5 took
~56 min on the single-CPU runner); the cold build + full suite outgrew it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 13:27:19 -07:00
trvon
0f1a1edf7c
Release v0.2.6
Some checks failed
release / draft (push) Successful in 20s
release / build (debian:trixie, trixie) (push) Has been cancelled
release / build (ubuntu:24.04, noble) (push) Successful in 54m16s
release / publish (push) Has been skipped
v0.2.6
2026-11-14 12:25:30 -07:00
trvon
80070a6497
fleet test: judge --since auto against a same-moment full export
Round-1 bundles are not a fair baseline: the 120 s cursor overlap is a large
share of a few-minute test history. Verified on the live fleet: incremental
192/206/194 windows vs full 230/244/232.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 12:16:32 -07:00
trvon
0410eac61c
fleet-inspect: print manifest time range and window start min/max
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 12:16:32 -07:00
trvon
39ffa02ae6
Salt: dockerized multi-node fleet test (master/central + 3 sensors) + state fixes
deploy/salt/test/: builds a release tarball from HEAD in ubuntu:24.04 (CI path),
boots systemd containers (rootless, no --privileged), master==central + 3
sensors with the host's Salt onedir mounted read-only, applies the real states,
runs a workload with planted YARA detections, then two orchestrated cycles.
43 checks (install/config/idempotence, real daemons + telemetry, R2 cache
miss/hit, cp.push collect, 3-seed cycle, canary hold then fleet rollout, no
promoted model anywhere, fleet bits in round 2, --since cursor, dedup).

State fixes found by running them for real:
- init.sls no longer includes strawberry.distribute: a plain state.apply on
  sensors shipped models fleet-wide and bypassed the canary; only the
  orchestration distributes now.
- install.sls: CI tarballs carry a +g<sha> suffix that 'strawberry version'
  does not print, so install.sh re-ran on every apply; record the installed
  release and compare base version + marker.
- orch collect: mv -f (re-collected bundles must not be skipped).
- central cycle: builds the overlay first; a training refusal for too few
  teacher labels (normal on a young fleet) publishes the overlay only and
  keeps the sensors' shadow, any other error still fails.
- distribute.sls: serves only what the cycle published (cp.hash_file; the
  cp.list_master file-list cache can miss a just-published file).
Render check gains overlay-only and full publish cases (20 checks).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 12:16:31 -07:00
trvon
758ef9db07
fleet: node-share cap never below the fair share 1/N
Seen in the dockerized 3-sensor run: max_node_share=0.25 (pillar default) with
3 similar-sized nodes cut every node to 25% of the total, dropping 158/468
windows (34%) while correcting no imbalance. The cap is now
max(max_node_share, 1/N) * total, so a balanced fleet is never trimmed and
only nodes above an equal share are cut. Test: balanced 3-node fleet keeps
all windows; a dominant 4th node is still trimmed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 11:29:06 -07:00
trvon
b1c7cc41dd
fleet: e2e harness + fleet train/overlay JSON, *.sbx-only bundle listing
fleet train advertised --json but printed text, so the central cycle's
train-seedN.json evidence was not JSON; it now emits one object incl.
duplicate_windows. fleet overlay gains --json (bundles_superseded,
overlays_skipped). Both commands read only *.sbx (no dotfiles) from the
bundle dir, so a cursor or stray file is never fed to the merge.

scripts/fleet_e2e.sh drives the real binary + central cycle script over N
synthetic sensors: --since auto export chain, Salt-pull simulation, 3-seed
train + overlay, sha256 publish, determinism, overlay distribution and a
round-2 export carrying fleet rarity features. 6/6 checks pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 07:00:57 -07:00
trvon
265915ac3b
salt: README diagram uses the fleet overlay CLI
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 06:58:16 -07:00
trvon
8f4cb586b8
Rules R2: node-local compiled-rules cache keyed on libyara version + sources
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 06:54:28 -07:00
trvon
ebade1ba80
Store 7c/7d: tunable merge horizon + CPR interleave-aware merge mode
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 06:50:09 -07:00
trvon
4436bee457
Fleet F1: merge dedup per node+window, overlay newest-per-node, export --since
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 06:50:03 -07:00
trvon
25d0da1121
Salt: install/config/export/central/distribute states + central cycle orchestration
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 06:43:42 -07:00
trvon
300cadf33e
rules: first pinned snapshot (yara-forge-core 20261108, 5132 rules)
Generated by scripts/refresh-rules-snapshot.sh (monthly cadence). Bundled as-is
per the R1 license decision; IOC lists stay fetch-only. build-release.sh now
verifies MANIFEST.sha256 and install.sh installs from it with rule_auto_fetch=false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 06:39:49 -07:00