Bump project + daemon version 0.1.0 -> 0.2.0. Correct meson `license` from the
stale `MIT` to `GPL-3.0-or-later` to match the GPLv3 LICENSE file and README.
First tagged release carrying Stage 6 fleet learning and Stage 3.5 provenance
work (3.5a fleet-train flags, 3.5b2 write->exec data-flow feature).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3.5b's ordered exec-lineage proved inert in clean A/Bs; replace it with a general
write->exec data-flow feature: WindowSummary.self_written_execs (exec of a file
written earlier in the same window) at x[42], tracked via OpenWindow.written_paths.
Honest status: this is the conceptually-correct provenance feature (it separates
download->exec-the-same-file from download->exec-unrelated), but it ALSO shows no
AUC gain on detector-labeled data -- for the same structural reason as 3.5b: the
teacher labels (DownloadThenExecute etc.) correlate with existing count/location
features or re-derive the detector. Validating provenance LEARNING requires the
novel-chain generalization benchmark (train on detector-covered, hold out
detector-evading attacks); committed as that benchmark's feature foundation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Feed ordered parent->child exe-template bigrams into the window feature vector
(WindowSummary.lineage_edges, note_lineage_edge, a salted hash loop in featurize,
wired at replay on_exec). The model already got per-kind lineage surprisal
(x[25..29]); this adds path ORDER, which the unordered token bag discarded.
Clean A/B on a fixed corpus (same 697 windows, same flags, only the binary
differs): model AUC 0.734 -> 0.737 (rarity 0.520 unchanged). Null on this
shallow-lineage data (marker execs are 1-process, synthetic data flat) but the
feature is correct and order-sensitive (test: A>B != B>A). Kept as the path
representation that Stage 3.5c's retro-hunt attribution head needs; its payoff is
expected on real multi-stage attack telemetry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expose the merge cap and train/test split on `strawberry fleet train` so a
single-node real-attack bundle can pre-seed a model. Default stays temporal +
cap 0.25 (behavior-preserving); `--split random` + `--max-node-share 1` lets the
Caldera-attack export (15 positive windows) train. Verified: real-only train AUC
1.0 vs rarity 0.563; default still refuses identically.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- README: front door states the full system (core + experimental shadow-only
inspection model + distributed fleet learning); new "Fleet learning" usage
section; learn/ + fleet_export/merge + governor in Architecture; a proper
"Documentation" map; license corrected MIT -> GPLv3 (matches LICENSE).
- AGENTS.md: crisp YAMS work-loop (retrieve/inspect/persist/report), replaced
the stale ~64-line dated "Current Project Reality" with an accurate concise
state + pointers (PLAN/DEVELOPER/git/YAMS), trimmed stale work-order/milestone,
added "Keep This File Lean".
- CLAUDE.md: work-loop + evidence rule up top.
- gitignore the local validation/attack-range harness (deploy/validation/{docker,vm}).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Forgejo now reads .forgejo/workflows and ignores .github/workflows, whose
ubuntu-24.04/macos-15 labels no runner here provides. The test lane runs on
the ghost-ci runner (label docker).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run-fleet-loop.sh + README: proves the distributed loop end to end with real binaries
(seed N nodes -> round-1 export -> central fleet train + fleet overlay -> distribute ->
round-2 export), asserting the fleet signal propagates (round-1 fleet_feat=0 ->
round-2 fleet_feat>0, N distinct nodes, non-empty overlay). Verified: 2 nodes PASS.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pillar data (deploy/salt/pillar/) + README for the deploying agent: hub-and-spoke
(sensor export -> central fleet train + overlay -> distribute), role assignment,
CLI surface, security invariants, and the honest gaps (no `fleet overlay` CLI yet,
node_id not wired, no multi-node/docker-swarm validation).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
release.yml builds, tests and packs a Linux install tree per distro
(debian:trixie, ubuntu:24.04), one at a time with JOBS=2 and nice.
Branches (main, ci/**) and PRs run the test lane and print each sha256;
a v* tag uploads the tarballs and .sha256 files to a draft release and
publishes it only when every build passed.
runner-probe.yml checks that a runner with the docker label picks up
jobs, can clone this repo with the job token and reach the API.
packaging/ci/build-release.sh is the shared build step. It refuses to
run outside CI because it installs packages.
Forgejo now reads .forgejo/workflows and ignores .github/workflows,
whose ubuntu-24.04/macos-15 labels no runner here provides.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brick.html is a ~9.5 MB brick (structure audit) report; .pi/ and .clangd are
local tool state. None belong in the tree.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Do not turn an empty cached digest into an unbounded descriptor hash. Distinguish size refusals from YARA results so the exec guard applies its own bounded scan/degrade policy. Cover cached read budgets across size boundaries, known hashes, inventory/live events and real guard-cache reuse. Fold/live/KPI and post-rollout CPU acceptance remain pending; OpenSnitch checksum work is separate.
Use suspend-aware BOOTTIME waits and owned-container cleanup without changing the workload. Reject missing coverage and preserve producer/report failures. Include runtime/helper regressions, constrained build-job support and live exec-guard crash safety. Document the failed live/idle gates without relaxing thresholds or claiming production acceptance.
Preserve independent support, typed rule namespaces and stable process lifetimes. Share authoritative managed-set reconciliation across daemon and offline correlation, load offline policy without seeding/migration, and release resources after shutdown-flush exceptions. Keep retirement history, transaction ownership and fault/allocation regressions; no whole-set crash-atomicity or live acceptance claim.
Reuse the existing resolver/sampler for opt-in aggregate training summaries. Reject incomplete or undecodable snapshots without usable partial reports, retain best-effort readers elsewhere, and document cooperative extent checks rather than atomic snapshot guarantees. Include command-table/help, JSON, fixture and corruption regressions; no model training or promotion.
Represent full-range ordered clock differences without signed overflow and allow complete decay. Add seeded restore-clock regressions and a dedicated fuzz seed.
Allowlisting a binary suppressed only *new* detections: the findings
already stored for it stayed in the evidence store, and every reload
re-materialised them. That is why the ~9k iproute2 `ss` false positives
kept "coming back" after the hash was allowlisted a day earlier -- nothing
retracted them, and the only cleanup was `doctor fix
retire-allowlisted-findings`, which operators do not know to run.
allowlist-add now retires the now-allowlisted findings at the moment of
the action (alongside the existing containment release) and reports the
count as `retired_findings` in its reply. retire_allowlisted_findings is
hash-based and self-filtering, so a path/rule entry that matches no
fully-allowlisted finding retires nothing.
Regression test (test_daemon_runtime): a detected binary's finding is
materialised (forced correlation), allowlist-add reports
retired_findings >= 1, and after correlation re-runs the finding does not
come back (A already makes correlation skip allowlisted entities).
Also refresh the doctor findings.allowlisted message, which claimed
allowlisting does not delete stored findings -- it now does.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
/tmp is tmpfs (RAM) on the dev host; a large GCC LTO relink writes its
ltrans files to $TMPDIR and, when that is /tmp, can exhaust memory and
freeze the machine (observed this session). Point the build's TMPDIR at a
disk-backed build/tmp, created as an order-only prerequisite of the
compiling targets (build, conan-build, fuzz). Override with BUILD_TMPDIR=.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On a host that sits at critical/emergency for a long time (its own apps holding
most of RAM, or the daemon slightly over its soft budget), the starvation guard
was disabled (`!hard`), so correlation, retro-hunt and inventory deferred
forever — observed deferred 153 min with forced=0, which also starves the
retro-hunt delayed labels that Stage 4 training needs.
- Tiered starvation guard: it now fires under critical/emergency too, but after a
level-scaled window (1x warning, 4x critical, 12x emergency) so background work
is rare under load, not zero. A true host-memory crunch (host_severe_,
MemAvailable below the severe floor) still holds everything, since forcing work
then could tip the host toward OOM. exec guard/scan paths unchanged.
- Raise the balanced own-memory soft budget 512 -> 768 MiB: the daemon's steady
footprint grew (telemetry buffers, the ingest server, the shadow model), so 512
left it permanently over budget -> pinned critical/emergency. 768 fits the real
baseline, still well under the 1.5 GiB cgroup cap.
Tests updated for the 768 budget and the tiered guard (critical forces after the
4x window; severe holds it). Suite 51/51; analyzer + clang-tidy clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The teacher-label join made a window a positive if it was detected
(replay.cpp), but it never applied the allowlist, so a window running a cleared
false positive (e.g. /usr/bin/ss) was labelled a teacher positive — poisoning
Stage 4 training. Negative labels were written to the stream but replay dropped
them.
- replay_store now collects negative labels into ReplayOptions.negative_labels
(it used to ignore any non-positive label).
- A window whose exec hash is negatively labelled is marked `suppressed`; the
example/policy-eval positive is gated on `!suppressed`, so such a window is
kept as a *negative* training example (useful signal) and never a positive,
even if a cleared rule "detected" it.
- DetectionPipeline seeds a negative label for every allowlisted sha256 at init,
so evidence stored before the allowlist (the historical ss storm) is covered
without waiting for a re-exec.
Test: a detected window with a stored negative label for its hash is a positive
before the label and a kept-negative after. Suite 51/51; analyzer + clang-tidy
clean on replay.cpp and detection_pipeline.cpp.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Correlation runs over the whole store and never consulted the allowlist, so an
allowlisted hash (e.g. the /usr/bin/ss feed false positive the operator cleared)
regenerated the same findings on every pass from already-stored indicators —
`doctor fix retire-allowlisted-findings` cleared them but they came straight back
(~9,400 findings observed again). This also meant the findings were a poisoned
teacher-label source for training.
- classify() now skips an indicator whose (hash, path, rule) is allowlisted, the
single chokepoint shared by CorrelationEngine::build_findings (the oracle) and
IncrementalCorrelator::update, so both agree. New AllowlistFn; build_findings /
correlate take an optional predicate (default = no filtering, existing tests and
the incremental==full oracle unchanged); IncrementalCorrelator::set_allowlist.
- DetectionPipeline wires the predicate to PolicyStore::match, so the live
incremental path skips allowlisted evidence. Retiring the findings now sticks.
Test: build_findings with an allowlist predicate emits nothing for the matched
hash and is unaffected for others. Suite 51/51; analyzer + clang-tidy clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
strawberry-tray did a one-shot QSystemTrayIcon::isSystemTrayAvailable() check at
startup and, if no StatusNotifier host was up yet, permanently fell back to
window-only mode (never creating the tray icon). Under sway the app is usually
`exec`ed before waybar registers its tray module, so the indicator never showed
even after waybar came up.
- main.cpp no longer disables the tray on that one-shot check; it keeps the tray
and logs a hint.
- AppController::start(): if the tray host isn't available yet, retry every 1.5 s;
when it appears, hide()+show() the icon to (re)register with the host. Only if
no host shows up within ~15 s does it fall back to opening the window.
Verified live on sway: with waybar up, the fixed binary registers a
StatusNotifierItem (icon visible); the retry covers the start-before-waybar case.
Suite 51/51; clang static analyzer clean on the changed app TUs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`extern char** environ;` inside namespace strawberry declared strawberry::environ
(undefined at link on arm64). Use `*_NSGetEnviron()` (<crt_externs.h>), the
macOS-blessed way to get the environment for posix_spawn. macOS validated on the
Mac via ssh: meson compile + 46/46 tests green in a throwaway worktree at the
previous commit with this file patched in. Linux unaffected (macOS-only code).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the tracker-attribution transport on macOS. There is no network
sandbox to work around there, so the daemon (root) reads Little Snitch's traffic
stream directly and feeds the same on_opensnitch_connection consumer — no relay
or ingest socket (those are the Linux sandbox answer).
- LittleSnitchParser (cross-platform, unit-tested): parses `littlesnitch
log-traffic` CSV via a header-name->index map (tolerates column reordering),
yielding the shared redacted OpenSnitchConnection for outbound rows only
(program, uid, host/ip/port, protocol 6->tcp/17->udp). Little Snitch's CSV has
no argv/env, so redaction is structural. split_csv_line handles quoted fields.
- LittleSnitchSource (macOS only): posix_spawns `littlesnitch log-traffic
--stream`, reads the pipe on a guarded thread, parses and calls back; stop()
SIGTERMs the CLI (EOF -> loop ends) then reaps it.
- Daemon: start_tracker_attribution gains an __APPLE__ branch that starts the
source; shutdown/status handle it. New config littlesnitch_cli (default CLI
path). status privacy.listener source "little-snitch".
- docs: macOS section rewritten (needs Little Snitch "Allow access via Terminal").
Test E10 covers the parser (header, outbound/inbound, deny, quoted exe, no-dest).
Linux suite 51/51; analyzer + clang-tidy clean (Linux view). The __APPLE__ code
compiles/validates on the Mac next (ssh); live spawn needs root + LS Terminal access.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Second half of the sandbox-safe tracker transport. A small relay in the host
network namespace bridges OpenSnitch to the sandboxed daemon, so the daemon keeps
PrivateNetwork=true + RestrictAddressFamilies=AF_UNIX and never sees argv/env.
- `strawberry opensnitch relay` (new CLI verb, Map::flag -> --opensnitch-relay):
binds the OpenSnitch loopback port via the existing OpenSnitchEventListener
(root-peer check, argv/env dropped at parse), and forwards each redacted tuple
(encode_connection) to the daemon's AF_UNIX ingest socket, reconnecting on
failure. Foreground loop until SIGTERM/SIGINT; drops (not blocks) when the
socket is down. Linux-only. The listener callback runs on one thread, so the
sink needs no lock.
- scripts/strawberry-trackerd.service: runs the relay in the host namespace with
only CAP_NET_BIND_SERVICE, no PrivateNetwork, and /proc/net kept readable for
the peer check (no ProcSubset=pid); otherwise the same tight sandbox as
strawberry-update.service. install.sh installs+manifests+starts it when
OpenSnitch is present and privacy_trackers is not false.
- docs/tracker-attribution.md: replace the "known limitation" with the relay
architecture (rejected alternatives noted). API_CONTRACT: privacy.listener now
describes the ingest socket + relay; also documents feed-add max_severity (B1).
Suite 51/51; clang static analyzer and clang-tidy clean on main.cpp and
cli_commands.cpp. Full relay->OpenSnitch end-to-end needs host validation
(sudo + a real OpenSnitch); the socket path is covered by E9.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First half of the sandbox-safe tracker transport (architecture deep dive). The
daemon's old loopback TCP listener could not bind under its own sandbox
(PrivateNetwork=true + RestrictAddressFamilies=AF_UNIX). Replace it with an
AF_UNIX ingest socket the daemon *can* bind under the sandbox; a host-namespace
relay (next change) will bind the OpenSnitch port, strip argv/env, and forward
redacted tuples here.
- opensnitch_events.hpp/cpp: encode_connection / decode_connection serialize only
the redacted fields (never argv/env, which the parser already drops) as one
JSON object — the relay->daemon wire form. New TrackerIngestServer: an AF_UNIX
stream listener (Linux-only, like the TCP listener) that accepts only a root
peer (SO_PEERCRED), frames with take_json_objects, decodes and calls back;
bounded clients/object/buffer as before.
- daemon: start_tracker_attribution now opens the ingest socket
(opensnitch_ingest_socket, default /run/strawberry/trackers.sock) instead of
the in-sandbox TCP listener, so it works with the sandbox unchanged and is
simply idle until a relay connects. status privacy.listener reports the ingest
socket + relay_events; shutdown stops it. New config key opensnitch_ingest_socket.
- Tests: E8 encode/decode round-trip + redaction (no args/env on the wire);
E9 TrackerIngestServer end to end (connect, two tuples, callback, stats);
daemon-runtime configs set privacy_trackers=false (not under test).
Suite 51/51; clang static analyzer and clang-tidy clean on the changed TUs. The
relay run-mode + systemd unit + install wiring + docs follow in WS2b/WS2c.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The scan path (handle_process) and the exec-guard recording path
(record_exec_decision) had grown two near-identical repeat-aggregation maps
(repeats_ / exec_repeats_) plus three copies of the IOC-feed severity idiom.
Refactor to one implementation; no behavior change (the existing aggregation and
IOC-severity tests are the oracle and pass unchanged).
- New include/strawberry/repeat_aggregator.hpp: RepeatAggregator owns the
per-key window map, the 20-full-then-summary cadence and the aggregated count;
observe(key, now, seen_ns, pid) returns {full, at_threshold, aggregated,
emit_summary, notify, state}. Each caller keeps its own key, summary-id prefix,
summary shape and notify behaviour. Both pipeline sinks now hold one instance
(scan notifies at most hourly; exec passes notify_ns=0). status
repeats_aggregated sums both totals (atomic, lock-free read).
- ioc_hit_severity(feed): one source for the three IOC match sites (scan_image,
assess_exec, handle_process) that was the policy_->ioc_feed_severity ternary.
- Leaves the latency-critical assess_exec/decide path untouched; both sinks still
run under mutex_, off the FAN response.
Unit test tests/test_repeat_aggregator.cpp (window reset, threshold, summary and
notify cadence, per-key independence). Suite 51/51; clang static analyzer and
clang-tidy clean on detection_pipeline.cpp; fold's 7 int-overflow are the accepted
monotonic-timestamp FP class (unchanged by the move); perf-kpi PASS, no regression
(exec_decision p50 unchanged, exec_assess/db_commit within noise or faster).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Allowlisting a hash suppresses new detections but does not delete findings
already stored for it (e.g. the /usr/bin/ss false-positive storm). That cleanup
now lives in doctor instead of a one-off compaction.
- DetectionPipeline::retire_allowlisted_findings(dry_run): a finding is stale
when it references at least one hash and every hash it references (from
metadata "hashes" and file_<sha256> nodes) is allowlisted, so a finding that
also implicates a non-allowlisted binary is never dropped. Mirrors
retire_superseded_findings (retire_finding + transaction).
- node_ids.hpp gains content_hash_of_file_node_id() (the inverse of
file_node_id for the content case) so the pipeline parses file ids through the
canonical helper, not a "file_" literal (source invariant R8).
- Doctor check findings.allowlisted (warn) with maintenance action
retire-allowlisted-findings; DoctorInputs.allowlisted_findings gathered by the
daemon; run_maintenance handles the action; CLI `doctor fix` help + completion
and API_CONTRACT updated.
Tests: test_retire_allowlisted_findings (two execs of a flagged binary -> a
finding; allowlist the hash -> dry-run lists it, apply retires it, idempotent);
doctor check ok/warn cases; CLI completion string. Suite 50/50; clang static
analyzer, clang-tidy and source invariants clean on the changed TUs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The packaged strawberry.service runs with PrivateNetwork=true and
RestrictAddressFamilies=AF_UNIX, so the OpenSnitch-events loopback listener
cannot bind (EAFNOSUPPORT/EPERM) and could not reach OpenSnitch's host namespace
anyway. Rather than surface a bare errno:
- start_tracker_attribution() translates the sandbox bind failure into a plain
explanation in status privacy.error (names PrivateNetwork/RestrictAddressFamilies
and points at docs/tracker-attribution.md). Nothing blocks, no finding.
- install.sh no longer implies tracker events are working: it says OpenSnitch was
*configured* to send, and to check `strawberry status` (privacy.listener), since
the daemon can only receive if its sandbox permits a loopback listener.
- docs/tracker-attribution.md documents the limitation and that the planned fix is
a host-namespace relay over the AF_UNIX control socket (sandbox preserved);
relaxing the sandbox is an operator choice, not a default.
Transport redesign deferred to the architecture deep dive. Suite 50/50; clang
static analyzer and clang-tidy clean on daemon_app.cpp.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The exec-guard would-block recording path stored one indicator (and, via
correlation's per-process grouping, one finding) per exec, with no aggregation,
unlike the scan path. A critical feed or rule matching a frequently-run binary
would flood evidence the way /usr/bin/ss did.
- record_exec_decision now collapses repeated identical decisions (keyed by
verdict|hash|rule) with the same 10-min window / 20-full-then-summary scheme
as handle_process (new exec_repeats_ map; counts into repeats_aggregated_).
Beyond the window it refreshes one "exec-repeat" summary indicator and returns
before the per-exec graph write.
- The actor's /proc identity reads are now done before taking mutex_, not under
it. The clang static analyzer (new strict gate) flagged ::read in a critical
section (unix.BlockInCriticalSection) here; this also honors the AGENTS rule
that scans/reads stay off DetectionPipeline::mutex_. Analyzer now clean.
Test: test_repeated_would_block_is_summarized drives 30 identical would-block
decisions and asserts exactly 20 full exec-audit indicators + 1 exec-repeat
summary and repeats_aggregated == 10. Suite 50/50; analyzer and clang-tidy clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A benign system binary (/usr/bin/ss) whose SHA-256 was on a noisy community hash
feed produced thousands of critical "would have been blocked" findings: IOC hash
matches were hardcoded severity "critical" at every site, and in exec_guard an
ioc_match forces a prevention hit regardless of the severity threshold.
- FeedSpec gains `max_severity` (append-only; serialize/parse/validate, control
socket `feed-add` accepts it). Empty = default for the kind: hash feeds are
advisory "high", not "critical". PolicyStore::ioc_feed_severity() resolves it.
- All three IOC severity sites now use the feed's severity
(detection_pipeline.cpp scan_image, assess_exec, handle_process), replacing the
hardcoded "critical".
- Trust gating: ExecAssessment::ioc_match is set only when the matched feed
resolves to "critical" (a curated feed that should force prevention). A
community feed stays advisory — its "high" is subject to prevention_min_severity
(default "critical"), so it no longer would-blocks system tools per exec.
Tests: test_ioc_hash_detection uses an explicit critical feed (trusted path) and
asserts assess_exec forces the hit; new test_ioc_community_feed_is_advisory
asserts a default feed is "high", ioc_match is not forced, and decide_exec allows
it in audit and enforce; test_policy covers max_severity validation, the advisory
default, and a critical round-trip. Suite 50/50; clang static analyzer and
clang-tidy clean on the changed TUs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>