Commit graph

228 commits

Author SHA1 Message Date
trvon
fdd66e8bab
Stage 4 groundwork: window features, inspection MLP, model files
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Nothing runs in the daemon yet; these are the pieces the next increments
train and run in shadow.

- learn/features: a WindowSummary accumulated from a window's events
  (exec locations, memfd/deleted/uid0 execs, depth, connection classes
  and ports, persistence and drop writes, ptrace, module loads, scan
  outcomes, capped surprisal per tuple kind, bounded tokens) and
  featurize(): 48 explicit features scaled to about [0, 1] plus signed
  hashing of tokens into 208 dims, with the governor level and budget
  left as context. A window's own detections are never features: they
  are the teacher labels.
- learn/mlp: 256 -> 128 -> 64 ReLU with a policy head over the
  inspection actions (none, memory_scan, deep_rescan, watch_subtree,
  rescan_on_next_rules) and a value head. Scalar reference, AVX2+FMA
  chosen at runtime, NEON on aarch64; safe default (no inspection) on a
  wrong input size or an inconsistent model. Deterministic init per seed.
- learn/model_file: "SBMD" format with bounded dimensions, exact length,
  a sha256 trailer and finite weights checked on load, each failure
  named; atomic save into a 0700 directory.

Tests: test_model (kernels against scalar for arbitrary sizes, forward
pass invariants, deterministic init, round-trip, every flipped byte and
truncation rejected, non-finite weights rejected, features bounded and
deterministic). fuzz_model_file. NEON is built and tested only on aarch64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 09:53:25 -06:00
trvon
cedd7e7ad9
test_cli: completions list --yes for doctor fix
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Fixes the test left failing by 47c3af7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 09:45:12 -06:00
trvon
47c3af735c
CLI: exit code 3 for "could not run here", confirm destructive actions
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Decided 2026-10-27 (docs/cli-ux.md):
- Exit codes: 0 done/healthy, 1 negative result (unhealthy, not found,
  daemon refused), 2 usage, 3 could not run here (daemon unreachable,
  permission denied, store not readable, feature not built). Daemon
  unreachable used to exit 2, like a usage error.
- Process kill/suspend, quarantine, allowlist-add, protection-set to
  enforce, store compact and doctor fix (without --dry-run) say what they
  will do and ask "Proceed? [y/N]" on a terminal; Enter or EOF is no.
  Without a terminal they need --yes, else exit 2 with nothing done.
  This applies to the flag form too (--policy-set process-kill ...).
  The desktop app talks to the control socket directly and is unaffected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 09:44:52 -06:00
trvon
bf1e779706
CLI: safe terminal output, completions, no accidental daemons; UX guide
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Evidence strings are attacker-controlled: every CLI command's stdout
  and stderr now pass through a filter that shows control characters
  (C0, DEL, C1) and invalid UTF-8 as \xNN (src/term_safe.hpp), so a
  crafted process name or path cannot inject terminal escape sequences.
  The daemon's journal output is unchanged.
- `strawberry` alone prints help; an unknown word (`strawberry fix`) is an
  error with a "did you mean" hint. Both used to start a daemon as the
  calling user with the word as its config path. Config paths (contain
  '/' or end in .conf) still run the daemon, for older units.
- Shell completions generated from the command table: `strawberry
  completion bash|zsh|fish`; install.sh installs them under
  /usr/local/share.
- One correlation pass at daemon start (stored findings catch up with the
  current rules, e.g. stable ids), and retire-superseded-findings
  correlates first, so replacements exist before old copies are retired.
  The user's first run reported "no duplicates" for that reason.
- docs/cli-ux.md: CLI UX rules adapted from yams (grammar, flags, human
  and JSON output, status words, security rules for output, help and
  completions, adding a command, known gaps). Exit code 3 and
  confirmation prompts are proposed there, pending a decision. AGENTS.md
  and CLAUDE.md point to it.

tests/test_cli.cpp: bare and unknown words, terminal_safe and the stream
filter, completion scripts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 09:41:19 -06:00
trvon
fb6b905860
CLI: yams-style commands, with doctor and doctor fix
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
strawberry [--config PATH] [--json] <command> [<subcommand>] [args] [--flags]

status, doctor, doctor fix <action>, daemon run, findings list/show/
correlate, process search/trace/trace-node/suspend/resume/kill, scan
file/processes/memory, rules reload/update/check/fetch, policy show/set,
opensnitch enable-checksums, store verify/compact, replay, demo seed,
help [command], version.

One table in src/cli_commands.cpp defines each command (words,
positionals, flags, help) and how it maps onto the existing flag form,
which main() still parses; help is generated from the same table.
Unknown options and wrong arity print the command's usage. Process
actions validate node ids before building their JSON payload.

The older flag form (strawberry --status, strawberry <config>) still
works. The systemd units now run `strawberry daemon run --config ...` and
`strawberry rules fetch --config ...`. README, AGENTS.md and the
false-positive guide use the command form.

doctor renders the daemon's checks with their fixes; doctor fix runs a
maintenance action (with --dry-run). Against an older daemon the CLI says
to reinstall rather than "unknown command". replay explains that
telemetry is root-only instead of reporting zeros, and keeps config
output out of --json.

tests/test_cli.cpp: translations, flag-form passthrough, errors, help.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 09:32:02 -06:00
trvon
b55723547d
Add doctor checks and maintenance actions to the daemon
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- doctor (control verb, read-only): store health and disk floor,
  telemetry growth per day and days until the floor when retention is
  unlimited, leftover store files, superseded findings, retro-hunt state,
  thread/write errors, logs (journald rotates them), model (none yet).
  Each check carries a maintenance action or advice. Pure checks over
  gathered inputs (src/doctor.cpp), so they test without a daemon.
- maintain (control payload verb, privileged, rate limited, audited):
  retire-superseded-findings and clean-store, each with dry_run.
- finding_retired evidence record: removes a finding from view; it stays
  in the stream's history. CorrelationEngine::superseded_findings finds
  old hash/rule findings by recomputing the stable id, and old lineage
  findings only when the current one's parent process is among their
  nodes (a reused pid on another boot is not a match). Lineage findings
  now record parent_process_node_id.
- store::inspect_store_files / clean_store_leftovers: orphan/ files older
  than N days and *.compact / MANIFEST.tmp temp files; never a listed
  segment.

Tests: test_doctor (check statuses, superseded detection including pid
reuse and no-replacement cases, retirement surviving a reopen, cleanup
keeping recent orphans and listed segments); daemon test exercises both
verbs over the socket. fuzz_evidence_records covers finding_retired.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 09:27:23 -06:00
trvon
d54d987b3a
Retro-hunt: re-scan binaries seen before when the rules change
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
The main source of teacher labels over time: rules improve daily, while
most malware runs once.

- The pipeline remembers every binary that ran (image_seen evidence
  record: sha256, path, first/last seen, size; at most one write per hash
  a day). Deleted and memfd images are skipped: nothing to find again.
- Each rules load computes a digest of the rule files and IOC hash lists
  (paths after resolving symlinks, sizes, mtimes).
- RetroHunter (src/retro_hunt.cpp, own thread, idle priority, paced at
  retro_hunt_pace_ms, deferred by the governor as WorkClass::retro_hunt)
  re-scans when the digest differs from the last completed pass, most
  recently seen first. Only paths that still hold the same content are
  scanned (O_NOFOLLOW, regular files, hash compared); others count as
  changed or missing. A completed pass is recorded (retro_hunt record) so
  a restart does not repeat it; an interrupted one starts over.
- New matches (rules not recorded before for that hash, not allowlisted)
  become indicators with scan_kind retro_hunt, a file/indicator graph
  link, a positive label and a notification. Correlation turns them into
  findings titled "YARA match on a binary that ran here (found after a
  rules update)". Nothing is blocked or contained from here.
- Replay reads positive labels from the evidence stream and marks windows
  that ran a binary labelled bad later (LATE in the CLI).
- Status: a retro_hunt block (API_CONTRACT.md); config: retro_hunt,
  retro_hunt_pace_ms.

Tests: test_retro_hunt (remembered once; passes only on a digest change
and survive a restart; a clean-at-exec binary matching a new rule becomes
an indicator, label, finding and notification; a replaced binary is
counted as changed, never scanned as the old one; a stop abandons a pass
unrecorded); test_replay R9 (late labels, from a real store).
fuzz_evidence_records covers the new records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 08:59:38 -06:00
trvon
da41b04512
Replay: say how much history there is when traces cannot be injected
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 22:40:52 -06:00
trvon
4e25666a70
Inject scripted attack traces into replay and score them
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- src/learn/trace.cpp: a small trace format (exec, connect, write,
  ptrace, module with +offsets and process aliases) compiled to
  telemetry records with pids above any pid_max and node ids that name
  the trace and copy.
- --replay --inject FILE [--inject-copies N]: copies are spread evenly
  over the ranked history (a first pass reads the time range), scored
  but never learned from, and checked against alert budgets of 1, 3 and
  10 per day: is the copy's best window above the (budget x days)-th best
  real window? Ties count against. Injection leaves real windows and
  their ranking unchanged.
- config/traces/standard.trace: the six plan traces (browser to curl|sh,
  systemd user persistence, memfd loader, ptrace injection, download
  then exec, python from a PDF viewer).
- docs/evaluation/stage3-rarity-preregistration.md: method and pass
  thresholds, written before any run on host data.

Tests: test_trace (standard traces, error messages with line numbers,
records); test_replay R8 (every copy found, real ranking unchanged, a
novel chain caught at 10/day and a bare shell not). fuzz_trace_dsl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 22:37:44 -06:00
trvon
5fa2031c62
Replay: reorder over 2 minutes, not 30 s
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
On the host, 586 of 303k records arrived more than 30 s late: exec records
of large images are written after their paced, deferrable scan. Replay is
offline, so holding 2 minutes of records costs nothing that matters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 22:27:00 -06:00
trvon
6fa5e68405
Make replay rankings fair over time and readable
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
From the first replay of real host data (3.5 h, 266k records):

- A never-seen event scored higher the more the model had seen, so the
  ranking favoured whatever happened last (the reinstall's systemctl
  and generator runs). Each event is now capped at 16 bits (never
  seen), and a window scores the sum of its three most surprising
  distinct events.
- Warm-up: on a fresh history nearly everything is new. Windows in the
  first 12 h are learned from but not ranked (--warmup H); detections
  are always listed. The CLI says so when nothing is ranked yet.
- Windows with the same root program and top pattern are one line with
  a count ("x40 similar").
- Counts: file writes split by persistence; events from processes
  started before recording are reported as such, not as an error;
  records more than 30 s out of order are counted as late.

tests/test_replay.cpp R7: warm-up, equal scores for a novel event early
and late, grouping, late-record counting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 22:10:51 -06:00
trvon
92cb4e9ba8
Add --replay, and keep non-UTF-8 host strings from breaking JSON
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Replay (src/learn/replay.cpp, strawberry --replay [dir] [--top N] [--json]):
reads the telemetry stream read-only, orders events by time (a 30 s
reorder buffer absorbs the writer's batching), joins behavior events to
the latest exec of their pid at that time (keeps PID reuse apart), builds
60 s windows (a root exec, its descendants and their behavior, at most 64
events), scores each exec, connection and persistence write with the
rarity model before observing it, and marks windows whose processes had
detections. Prints the most unusual windows with the tuples that made
them unusual. Deterministic. tests/test_replay.cpp: chain windows, PID
reuse, order insensitivity within the horizon, determinism, ranking,
detection marking, a real store round trip. fuzz_replay feeds arbitrary
record streams.

UTF-8: file names are bytes, and the fuzzer showed nlohmann's dump()
throwing on a path that is not UTF-8. The same applied to the daemon's
control replies (kill/suspend/quarantine name the exe path) and to the
OpenSnitch containment rule, so a binary with such a name could not be
contained, and json_escape wrote the raw bytes into status JSON. Now:
- every .dump() in src/ uses error_handler_t::replace (U+FFFD), enforced
  by source invariant R9;
- json_escape writes invalid sequences as � (valid_utf8 and
  utf8_sequence_length in json_util.hpp);
- containment refuses such a path with a clear error (a rule could never
  match the real bytes); quarantine still blocks execution by hash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 21:49:04 -06:00
trvon
134aebf7a9
Add the tokenizer and rarity model for the learning stages
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
src/learn/ (pure libraries; nothing in the daemon calls them yet, Stage 3
uses them in replay):

- tokenize: path templates (user homes, per-uid dirs, pids, digit and hex
  runs, UUIDs, mktemp-style names abstracted) and command-line shapes
  (flags kept, values, paths, URLs, IPs, numbers and encoded blobs
  classed; the first plain word kept as a subcommand; long argument lists
  summarized). Bounded and noexcept.
- rarity: decayed count-min sketches over parent>child, grandparent>
  parent>child, exe+command-line shape, exe+connection class and
  exe+persistence location, scored as surprisal in bits; a bounded
  first-seen table; deterministic hashing for replay. About 1.3 MiB.

tests/test_learn.cpp: tokenizer cases and idempotence; count-min never
underestimates (also after decay); determinism; rare scores above common;
half-life decay; bounded first-seen. fuzz_tokenizer found two
idempotence bugs (truncation before templating, and a cut that left a
" (deleted)" suffix), fixed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 21:21:28 -06:00
trvon
72d239dee9
Give hash, rule and lineage findings one stable id each
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Their ids included the member pids (and rules or hashes), so every new
process joining a group created a new finding and left the previous one
behind as a duplicate. The id is now the group key alone: the hash, the
rule name, or the parent process. The finding is updated in place as the
group grows; the pids stay in its metadata and summary.

Findings stored under the old ids stay as they are (one last duplicate
per group after upgrading). The incremental property test now also checks
that add-only runs leave exactly one finding per group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 21:08:17 -06:00
trvon
e746eaa70d
Correlate incrementally: rebuild only the groups that changed
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Every finding is a function of one group of indicators (a process, a
parent, a hash or a rule). IncrementalCorrelator keeps group membership
(ids and keys, not indicator contents) and rebuilds only the groups that
new or changed indicators touch. Each pass used to copy every indicator
and rebuild every finding; it now costs about the size of what changed.

- correlation_engine.cpp: the four group loops become per-group
  functions (re-indented; logic unchanged), shared by the full pass and
  the incremental one.
- EvidenceView keeps a bounded change log of written indicator ids
  (resets after clear(), at first use, or past 65536 entries);
  Database::take_indicator_changes / find_indicators expose it, with
  rolled-back writes reported as removed.
- DetectionPipeline holds its lock only to read changes and members and
  to store findings, as before.

Fix found by the oracle test: an indicator naming a process node but no
pid formed a second group for the same process with the same finding id,
and the two findings overwrote each other. The group pid now comes from
the node id when the pid is missing (finding ids are unchanged).

tests/test_correlation_incremental.cpp: random batches of new and updated
indicators give exactly the findings the full pass after every batch
stored (I1, I2); a batch touching one process rebuilds only its groups
(I3); the store's change log, including a rolled-back write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 19:51:57 -06:00
trvon
4366a368bb
Document the telemetry and command-line redaction settings
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 17:59:07 -06:00
trvon
c452be93d4
Unify file ids, record uid/comm, redact command lines
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
File node ids: include/strawberry/node_ids.hpp is now the only place
they are built (file_<sha256> or file_path_<sha256 of path>). The three
modules disagreed: correlation used a 16-digit path hash and behavior
detections the raw path, so path-keyed evidence never met the rest. An
exec detection with a known hash also writes a same_path edge from the
path id to the content id. Source invariant R8 rejects hand-built file
ids in src/ and include/.

Processes carry the real uid and kernel comm: from the eBPF exec event
(which now includes the uid) or /proc. Stored on process nodes and in
process_exec telemetry (fields 14, 15).

Command lines are redacted before any storage (capture_redact_cmdline,
default on): sensitive flags and KEY=value pairs, Authorization/Bearer
credentials, URL passwords, tokens used as URL user names, well-known
token shapes (GitHub, GitLab, Slack, AWS key ids, sk-, JWTs) and mysql
-p<password>. The rules repeat to a fixed point, so redaction is
idempotent by construction. tests/test_redact.cpp covers the rules plus
idempotence, size and planted-secret properties; fuzz_cmdline_redactor
found two idempotence bugs and a leak (--password --token X) that are
fixed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 17:58:48 -06:00
trvon
2044873b95
Record labels, generations, lifecycle and governor samples
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Teacher signals and context for the learning stages, written to the
evidence stream (retention never removes it):

- label: operator actions from the control socket (quarantine, contain,
  kill, suspend: positive; allowlist-add: negative; allowlist-remove and
  quarantine-remove: retracted) and allowlist suppressions (negative,
  once per entry, subject and rule). Never from the MCP.
- rules_generation: every rules load attempt, with sources, counts and
  the error of a failed reload.
- policy_generation: each policy change, with a sha256 of the policy.
- lifecycle: daemon start and stop.

governor_sample (pressure level, CPU/IO/memory, per-class admissions) goes
to the telemetry stream once a minute.

Database::append_evidence writes records the evidence view ignores.

Tests: record round-trips; suppression labels and dedupe and rules
generations in the pipeline test; operator labels, policy generations and
lifecycle in the daemon test; the record fuzzer covers the new decoders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 17:23:09 -06:00
trvon
b46e367e6d
Record YARA string hits as rule_match telemetry
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Each rule match from a fresh scan now writes a rule_match record: the
rule, namespace, severity, rules generation and its matched strings
(identifier, offset, length, 64-bit FNV-1a digest of the captured bytes).
The matched bytes themselves are never stored. Cache hits write no
rule_match, since their scan_result already names the rules.

Hits are collected in the libyara callback (libyara 4+) and capped at
16 strings per rule, 4 hits per string and 256 per scan. The callback
also tolerates a null user_data now.

Tests: caps, offsets and digests against a real scan; record round-trip
with the encode cap; the telemetry record fuzzer covers rule_match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 11:08:58 -06:00
trvon
8ab8c9b353
Record raw behavior events and every exec-guard verdict
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Stage 2 increment 2.
- Connect, ptrace and module-load events go to telemetry raw, before the
  behavior engine filters them (pid, uid, comm, address/port or target);
  the join to processes happens when the data is read.
- Persistence and drop-location writes go to telemetry before any rule
  decides on them.
- The exec guard reports every verdict through a new on_observation hook
  (allow verdicts and cache hits included; ExecDecision still reports
  only non-allow): verdict, answered, scanned, cache hit, degraded, top
  rule/severity and latency, as guard_decision records.

Tests: record round-trips; the live exec-guard test checks one
observation per checked exec, including allows and cache hits;
fuzz_telemetry_records covers the new decoders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 09:57:17 -06:00
trvon
a32e045412
Record every exec and scan result as telemetry
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Stage 2 increment 1. Until now only detections were stored, so the
learning stages would never see a normal example.

- TelemetrySink: bounded lock-free multi-producer queue (8192); try_emit
  never blocks, allocates or throws; a full queue drops and counts.
- A guarded writer thread drains it every 2 s (or at a quarter full)
  into the store's root-only telemetry stream through the pipeline,
  writing a gap record whenever events were dropped.
- The pipeline emits one process_exec (pid, parent, node ids, exe,
  capped cmdline, sha256, deleted/memfd, live/inventory) and one
  scan_result (outcome: scanned, cache_hit, too_large, unavailable,
  read_error, no_rules, yara_error, pid_scan; bytes, scan time, rules
  generation, matched rules) for every handled exec, detection or not.
- Config telemetry=true (default); status gains a telemetry block
  (emitted, dropped, queued, written, gaps).

Tests: C1 (never blocks; exactly once and in order under concurrency,
clean under ThreadSanitizer), record round-trips, C2 (one process_exec
and one scan_result per handled exec, stored in the telemetry stream);
fuzz_telemetry_records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 09:16:26 -06:00
trvon
f1b948df24
Keep exec scanning alive under memory pressure; summarize repeats
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
The KPI detect phase still dropped events after the correlation fix.
Cause: every detection kept ~12 KB of evidence in memory, 20k detections
pushed RSS past the 512 MB budget, the governor went to emergency, and in
emergency every live exec scan waited 5 s, so the queue overflowed and
stayed full (memory that is evidence never comes back by waiting).

- The governor holds live exec scans for at most 5 s per emergency
  episode, then lets them run (counted as forced); background work still
  waits.
- Repeated detections of the same image and rules: the first 20 per
  10-minute window are stored in full, then one summary indicator per
  key keeps the count and first/last pid and time (rewritten at most
  every 30 s). All detections are counted; alerts repeat at most once a
  minute per key. Status reports pipeline.repeats_aggregated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 07:32:30 -06:00
trvon
c50d415066
Correlate outside the pipeline lock; per-exec logs opt-in
The KPI detect phase dropped 1-2.6k events in 2 of 3 runs: each
correlation pass computed findings over every indicator (300+ ms with
~20k indicators) while holding the pipeline lock, so the exec worker
stalled until its queue (4096) overflowed. Correlation now snapshots the
indicators under the lock, computes findings without it
(CorrelationEngine::build_findings), and takes the lock again only to
store them in one transaction.

"Scanning process" and "New process detected" were logged for every exec
on the host; they now need STRAWBERRY_TRACE_EXECS=1. Detections, errors
and state changes are still logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 07:06:05 -06:00
trvon
7c1a7fc1c1
Bound finding size and say why a store write is refused
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Found by the KPI detect phase (2-3 persist failures per run):
- a finding for one rule across thousands of processes listed every
  related id and outgrew the 256 KiB block limit, so the correlation
  commit was refused; related lists are now capped at 200 (the most
  recent indicators) with the totals in metadata
- a single record up to the 1 MiB record limit may now form a block on
  its own (the decoder bound only loosens)
- a refused append records its reason ("rejected: ...") instead of
  logging an empty error
- the Database no longer logs every indicator, node, edge and finding it
  stores (175k lines in a 4-minute run); failures are still logged

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 06:43:46 -06:00
trvon
4ca4b5c8a8
Measure the evidence write path in the KPI
A detect phase runs 40 distinct binaries carrying a KPI-only YARA marker,
then repeats them, so db_commit and correlation are timed (the earlier
workload never produced a detection).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 06:43:45 -06:00
trvon
833ae7c423
Add the store's disk floor, retention, verify and compact
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Stage 1 increment 7.
- Disk floor: below max(store_min_free_mb, store_min_free_pct) free,
  telemetry is refused and counted (a gap record with the count and
  time range is written once space returns) or, with
  store_floor_action=evict_oldest_telemetry, the oldest sealed
  telemetry is deleted; evidence continues until store_hard_min_free_mb.
- Retention (telemetry only, off by default): store_retention_days and
  store_max_mb, applied when a segment is sealed. Evidence is never
  deleted by retention.
- Status gains a store block (floor, free space, per-stream sizes,
  compression ratio, drops, deletions); health reports a low disk,
  a failed evidence store and unsaved detections.
- strawberry --store-verify checks the manifest and every segment
  (read-only, safe while the daemon runs; root-only telemetry is noted,
  not flagged). strawberry --store-compact rewrites sealed segments at
  zstd level 19 with the daemon stopped, never touching a damaged one.
- The manifest records each sealed segment's uncompressed size.

Tests L7-L11 cover the floor, gaps, retention by age and size,
eviction, compaction and verify.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 06:19:29 -06:00
trvon
337e67d5ad
Read the daemon's evidence store from the CLI
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
--list-findings, --show-finding, --search-process and the trace report
opened config.db_path, which is empty by default, so as a user they read
~/.local/share/strawberry/database.db and showed nothing. They now use
an explicit db_path, else the path the running daemon reports over the
control socket (as the app and MCP do), else /var/lib/strawberry if it
exists, else the per-user default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 05:48:19 -06:00
trvon
9b937119c8
Record routine indicators as evidence only
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Per the triage decisions:
- DeletedExecutable when a new file exists at the executable's path (an
  update or reinstall) is stored as evidence only. A binary with nothing
  left at its path still raises a low finding.
- scripts/install.sh writes /var/lib/strawberry/installed-files.sha256
  (hashes of the units it installs, root-owned, written first). A
  PersistenceWrite whose file matches its listed hash is evidence only;
  other content at those paths, or a suspicious writer, is reported as
  before. The daemon ignores a manifest that is not root-owned or is
  group/world-writable.

Evidence-only indicators make no finding, alert or containment on their
own; correlation counts them when the same process has another signal.
Status reports pipeline.evidence_only. Tests cover the manifest, the
behavior hook, correlation and the pipeline (replaced vs deleted).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 05:37:58 -06:00
trvon
f69fcb0e19
Attach the fork tracepoint on current kernels
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
sched_process_fork's comm fields became 4-byte __data_loc offsets in
newer kernels, moving child_pid from offset 44 to 20. The hand-written
tracepoint struct read past the event, so the kernel refused to attach
the program and parent-PID enrichment was always off ("Fork tracepoint
✘" in the app). The struct is now CO-RE (preserve_access_index), so
parent_pid/child_pid relocate against the running kernel's BTF. Checked
in a privileged container on kernel 7.2: attaches now, did not before.

The exec smoke test reports whether the tracepoint attached and, when it
did, requires a parent pid on the exec event. The eBPF smokes still
miss the test's own exec inside the container (an older problem, noted
in the live-test runner and tracked separately).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 21:33:39 -06:00
trvon
1c5ca2b3cd
Read the autostart test file without istreambuf_iterator
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
The autostart test used std::istreambuf_iterator, which source
invariant R1 forbids; use read_small_file_or_empty. The previous commit
was pushed with source_invariants failing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 21:00:19 -06:00
trvon
2e5637fe6c
Add "Start at login" to the tray app
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
A checkable tray menu item and a Diagnostics-tab checkbox write or
remove ~/.config/autostart/strawberry-tray.desktop (XDG autostart,
atomic write), pointing at the installed strawberry-tray when there is
one. Sessions that do not run autostart entries on their own (sway, i3,
Hyprland, niri without uwsm/dex) get a note with the exact line to add
to the compositor config instead of a silent no-op; GNOME, KDE, XFCE and
the like, or an active xdg-desktop-autostart.target, need nothing more.

The logic is Qt-free (app/autostart.cpp) with a unit test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 20:59:52 -06:00
trvon
97e21ee81b
Triage findings and cut routine false positives
Triage of all 26 findings on a developer workstation (none malicious)
is in docs/false-positives.md: binaries replaced while running
(auto-updates, reinstalls), Strawberry's own installer writing its
systemd units, sanitizer/self-debugging ptrace, deliberate tests, and
the test marker matching process memory.

Fixes:
- behavior detections now honor path and rule allowlist entries (they
  only checked sha256, which is computed for high severity only)
- finding summaries include what happened (file written, ptrace
  target) and metadata.details
- Strawberry_Test_Marker only matches files that start with the marker
  (EICAR-style); it no longer fires on processes holding the text
- ptrace between a parent and child running the same binary
  (LeakSanitizer StopTheWorld) is low instead of medium; ProcessRef
  carries the parent pid

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 20:56:35 -06:00
trvon
648457802a
Forbid empty catch blocks
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Every catch now handles the exception or records it with
STRAWBERRY_NOTE_SWALLOWED() (strawberry/swallowed.hpp): an atomic count,
exported in status as resources.swallowed_exceptions, plus a file:line
log line written with write(2), so it is safe out of memory and in
destructors. The 16 empty catches (store error fallbacks, destructors,
YARA compiler callback, thread entry logging) now use it. Status also
reports thread_errors, which was counted but never exported.

Enforced by source invariant R7 (with a self-test fixture) and by
clang-tidy bugprone-empty-catch in a new .clang-tidy (errors; inherits
parent config). AGENTS.md and CLAUDE.md state the rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:52:45 -06:00
trvon
dc50f06f8d
Drop unqlite-cpp and import legacy databases
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Nothing shipped used the library after the event store landed. A small
reader/writer for its file format (src/store/legacy_kv) replaces it for
the legacy code, and the submodule, subproject and library test are
removed.

A writer open now imports a legacy database.db file into the event
store: indicators are replayed in an order consistent with the old
hash/rule indexes, the result is verified against the old file, the
old file is kept as database.db.migrated-<ts>, and an import
interrupted between its final renames is finished on the next open.
Read-only handles read a legacy file directly and switch to the store
once it is imported. Ids re-stored under another hash are reported
(their old index entries cannot be carried over).

Tests: legacy import properties I1-I6 (locked and corrupt files are
left untouched), the differential test now shares its comparisons, and
fuzz_legacy_kv covers the file parser. A copy of a real 137 KB legacy
database imported cleanly into 7 KB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:46:21 -06:00
trvon
43635b8a47
Install all build dependencies in CI
nlohmann-json3-dev and libzstd-dev are required; add the optional
updater libraries too. No submodules to check out any more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:46:20 -06:00
trvon
ad1676a594
Integrate features into the daemon, build and docs
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Wire the policy, feeds, containment, exec guard, behavior, governor,
crash safety and event store into the daemon, detection pipeline, CLI
and meson build (zstd now required); Dockerfile, conan, systemd unit
(LimitCORE=0, resource limits) and installer updates; existing tests
adapted (store paths are directories now). Detections are counted and
notified even when evidence cannot be persisted (persist_failures in
status). prevention_timeout_ms is clamped and behavior dedupe handles
out-of-order events. Docs: API contract, developer guide, plan, UX,
agent rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:17:30 -06:00
trvon
adb40e5b62
Add compressed event store
zstd block codec (CRC32C, varint tagged records), segment files with
sealed footers and block index, crash-atomic MANIFEST, LogStore
(lock, recovery, rolls, orphaning, heal with backoff), the evidence
view, and the legacy unqlite store kept for import. Property tests
under injected faults and crashes, a differential test against the old
store, and fuzz targets for every decoder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:17:17 -06:00
trvon
6628ea9aea
Harden against crashes and I/O faults
Fix for the 2026-10-25 host freeze (an uncaught iostream exception on a
/proc read aborted the daemon; its core dump deadlocked with the exec
guard): non-throwing /proc reads, crash handlers that release fanotify
permission fds, guarded thread entries, from_chars parsing.

Adds the io fault-injection seam with property tests, source invariant
checks (R1-R6) with self-test fixtures, libFuzzer targets and seeds
(including I/O-fault fuzzing), the fold changed-files scan scripts and
the agent workflow docs (CLAUDE.md, prompts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:17:17 -06:00
trvon
2d5e05e8db
Add resource governor, scan profiles and KPI harness
PSI/cgroup-aware governor with efficient/balanced/aggressive profiles,
background pacing and thread priorities; timing histograms and optional
Tracy zones; a capped Docker KPI harness with report/compare tooling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:17:16 -06:00
trvon
1d1ab9f57e
Add exec prevention, behavior telemetry and persistence watch
fanotify exec-permission guard (audit/enforce, fail-open deadline),
eBPF behavior engine (IOC connections, ptrace, module loads,
download-then-execute) and fanotify persistence watch, with unit tests
and privileged Docker live tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:17:16 -06:00
trvon
fadc6363f9
Add policy, threat-intel feeds and OpenSnitch containment
PolicyStore (allowlist, protection settings, operator actions), the
rule/feed updater with HTTPS-only fetches and a systemd timer, IOC
hash/domain/network sets, OpenSnitch blocklist containment, the root
control socket (SCM_RIGHTS credentials, verb allowlist) and scanner
health snapshots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:17:05 -06:00
trvon
47bb058fdc
Add MCP server for agent access
A stdio JSON-RPC MCP server exposing status, findings, process search
and YARA path scans. Evidence fields are marked as untrusted data;
protection actions stay read-only from MCP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:17:04 -06:00
trvon
76ec40e4f5
Replace web UI with a Qt tray app
Remove the browser UI, its Python server and the web API contract test.
Add the Qt6 tray app (status, findings, process search, policy
controls, notifications) and EvidenceStore, the read-only view of the
daemon's evidence that the app and MCP server share. The GUI demo
config becomes strawberry.demo.conf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:17:03 -06:00
trvon
577404a39b Prioritize process inventory collection 2026-04-27 19:48:20 -04:00
trvon
158050f3ea Document web deployment modes 2026-04-27 19:30:34 -04:00
trvon
886fa6ff82 Add web API contract test 2026-04-27 19:26:05 -04:00
trvon
1527787888 Wire process detail panel 2026-04-27 18:46:41 -04:00
trvon
6b337ccdd6 Add process detail API 2026-04-27 17:27:36 -04:00
trvon
4e055d771f Document web API contract 2026-04-27 16:58:11 -04:00
trvon
09ccf940db Plan C++ web API service 2026-04-27 16:51:31 -04:00