Second piece of tracker attribution (docs/tracker-attribution.md): which
programs reach domains on a trackers list. Advisory only: never a block,
never a finding (source invariant R11).
- OpenSnitch event listener: OpenSnitch's JSON "remote" logger is the
hostname source (Strawberry's eBPF connects have none). It streams over
TCP and cannot leave fields out, so events carry command lines and
environments. The listener binds 127.0.0.1 at a port below 1024 (only
root can), accepts only root-owned peers (/proc/net/tcp), bounds
clients/object size/buffering, and reads only pid, uid, path,
destination and action; arguments and environment are never copied.
- TrackerContacts: per program x domain, bounded at 20000 keys. A root-only
tracker_contact telemetry record (type 25) on the first contact, then at
most hourly with the count since.
- Status `privacy` block; control verb privacy-trackers; CLI
`strawberry privacy trackers [--json]`.
- `strawberry opensnitch enable-events` adds the logger to OpenSnitch's
config (with a .bak); the installer runs it when OpenSnitch is present,
unless privacy_trackers=false. Config: privacy_trackers (default true),
opensnitch_events (default 127.0.0.1:979).
- EasyPrivacy is a default trackers feed (GPL-3.0+ or CC-BY-SA-3.0,
downloaded at run time).
Tests: E1-E7 (test_opensnitch_events: event fields, stream splitting,
peer uid, address rules, listener end to end, contacts and record, config
editor), the control verb, CLI translation and help, daemon status. New
fuzz_opensnitch_events (90 s clean).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
First piece of tracker attribution (docs/tracker-attribution.md): which
programs reach advertising and tracking domains, built with the
measurement rules of the CarolinaCon 2026 privacy talk (bridgescope):
a listed domain is a circumstantial label, never proof.
- Feed kind "trackers": hosts lines, plain domains and adblock
whole-domain rules (||host^, ||host^$third-party). Cosmetic rules,
exceptions, paths and other options are skipped; the comment cut used
to turn "site.example##.ad" into a domain.
- TrackerSet: suffix matching (a.b.t.example matches t.example, never
nott.example, an IP literal or a bare TLD); first feed reported.
- List parsing moves from rule_updater.cpp to ioc_list.cpp (no network
code) so the query-side core can use it.
- Source invariant R11: tracker lists never meet exec-guard or OpenSnitch
code. They are not in the OpenSnitch blocklist sync.
Tests: T1-T3 (test_tracker_set), trackers in the list-parsing property
and the list fuzzer (60 s clean), R11 self-test fixture.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shadow scorer's rarity model started empty at every daemon start, so
each restart cost an hour of warm-up with no decisions. The daemon now
saves it to model_dir/rarity.state (checksummed, atomic write, hourly and
at stop) and restores it at start. A restored state younger than the
7-day half-life skips the warm-up; decay catches up on the first event.
A file made with other rarity settings, a bad checksum, or a negative or
non-finite count is refused with a reason (status model.rarity_state,
doctor model.rarity_state) and the model starts empty. The file must be
owned by the daemon user and not group/world writable, like model files.
Also fixes status model.warmup_windows, which was never filled in.
Tests: R1/R2 in test_learn (round trip, decay clock, refusals), S8 in
test_shadow (warm-up skipped only when recent), daemon restart restores
the saved state, doctor warning; new fuzz_rarity_state target.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
include/strawberry/platform_process.hpp answers what /proc answers on
Linux: running PIDs (proc_listallpids), executable path
(proc_pidpath), parent and start time (sysctl KERN_PROC_PID, readable
for every process), argv (KERN_PROCARGS2) and a boot id
(kern.bootsessionuuid). A process is identified by its start time in
microseconds since the epoch plus the boot id, so node ids survive PID
reuse as on Linux. It is header-only, so the per-test source lists stay
as they are.
Users on macOS: resolve_process_ref (default root), the pipeline's
image hashing and exec-decision actor, and YaraScanner::scan_process
(scans the image by path). A synthetic procfs root still takes the
procfs code, so the existing procfs tests run on macOS too.
The resource governor samples Mach/sysctl on macOS: own CPU from
getrusage, RSS from task_info, load from getloadavg, available memory
from host_statistics64 (free + inactive + purgeable + speculative).
The behavior engine knows macOS debuggers (debugserver, dtrace) and
kext loaders (kernelmanagerd, kmutil), and macOS temp and Downloads
paths count as drop locations. The Linux lists are unchanged.
The KERN_PROCARGS2 parser is portable: a property test (exact argv
recovery, truncation never invents data, total on junk) and the
fuzz_procargs target run on every platform. test_platform_process
checks the live helpers against this process; the daemon test reads
the stopped state through sysctl on macOS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing runs in the daemon yet; these are the pieces the next increments
train and run in shadow.
- learn/features: a WindowSummary accumulated from a window's events
(exec locations, memfd/deleted/uid0 execs, depth, connection classes
and ports, persistence and drop writes, ptrace, module loads, scan
outcomes, capped surprisal per tuple kind, bounded tokens) and
featurize(): 48 explicit features scaled to about [0, 1] plus signed
hashing of tokens into 208 dims, with the governor level and budget
left as context. A window's own detections are never features: they
are the teacher labels.
- learn/mlp: 256 -> 128 -> 64 ReLU with a policy head over the
inspection actions (none, memory_scan, deep_rescan, watch_subtree,
rescan_on_next_rules) and a value head. Scalar reference, AVX2+FMA
chosen at runtime, NEON on aarch64; safe default (no inspection) on a
wrong input size or an inconsistent model. Deterministic init per seed.
- learn/model_file: "SBMD" format with bounded dimensions, exact length,
a sha256 trailer and finite weights checked on load, each failure
named; atomic save into a 0700 directory.
Tests: test_model (kernels against scalar for arbitrary sizes, forward
pass invariants, deterministic init, round-trip, every flipped byte and
truncation rejected, non-finite weights rejected, features bounded and
deterministic). fuzz_model_file. NEON is built and tested only on aarch64.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- src/learn/trace.cpp: a small trace format (exec, connect, write,
ptrace, module with +offsets and process aliases) compiled to
telemetry records with pids above any pid_max and node ids that name
the trace and copy.
- --replay --inject FILE [--inject-copies N]: copies are spread evenly
over the ranked history (a first pass reads the time range), scored
but never learned from, and checked against alert budgets of 1, 3 and
10 per day: is the copy's best window above the (budget x days)-th best
real window? Ties count against. Injection leaves real windows and
their ranking unchanged.
- config/traces/standard.trace: the six plan traces (browser to curl|sh,
systemd user persistence, memfd loader, ptrace injection, download
then exec, python from a PDF viewer).
- docs/evaluation/stage3-rarity-preregistration.md: method and pass
thresholds, written before any run on host data.
Tests: test_trace (standard traces, error messages with line numbers,
records); test_replay R8 (every copy found, real ranking unchanged, a
novel chain caught at 10/day and a bare shell not). fuzz_trace_dsl.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replay (src/learn/replay.cpp, strawberry --replay [dir] [--top N] [--json]):
reads the telemetry stream read-only, orders events by time (a 30 s
reorder buffer absorbs the writer's batching), joins behavior events to
the latest exec of their pid at that time (keeps PID reuse apart), builds
60 s windows (a root exec, its descendants and their behavior, at most 64
events), scores each exec, connection and persistence write with the
rarity model before observing it, and marks windows whose processes had
detections. Prints the most unusual windows with the tuples that made
them unusual. Deterministic. tests/test_replay.cpp: chain windows, PID
reuse, order insensitivity within the horizon, determinism, ranking,
detection marking, a real store round trip. fuzz_replay feeds arbitrary
record streams.
UTF-8: file names are bytes, and the fuzzer showed nlohmann's dump()
throwing on a path that is not UTF-8. The same applied to the daemon's
control replies (kill/suspend/quarantine name the exe path) and to the
OpenSnitch containment rule, so a binary with such a name could not be
contained, and json_escape wrote the raw bytes into status JSON. Now:
- every .dump() in src/ uses error_handler_t::replace (U+FFFD), enforced
by source invariant R9;
- json_escape writes invalid sequences as � (valid_utf8 and
utf8_sequence_length in json_util.hpp);
- containment refuses such a path with a clear error (a rule could never
match the real bytes); quarantine still blocks execution by hash.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
src/learn/ (pure libraries; nothing in the daemon calls them yet, Stage 3
uses them in replay):
- tokenize: path templates (user homes, per-uid dirs, pids, digit and hex
runs, UUIDs, mktemp-style names abstracted) and command-line shapes
(flags kept, values, paths, URLs, IPs, numbers and encoded blobs
classed; the first plain word kept as a subcommand; long argument lists
summarized). Bounded and noexcept.
- rarity: decayed count-min sketches over parent>child, grandparent>
parent>child, exe+command-line shape, exe+connection class and
exe+persistence location, scored as surprisal in bits; a bounded
first-seen table; deterministic hashing for replay. About 1.3 MiB.
tests/test_learn.cpp: tokenizer cases and idempotence; count-min never
underestimates (also after decay); determinism; rare scores above common;
half-life decay; bounded first-seen. fuzz_tokenizer found two
idempotence bugs (truncation before templating, and a cut that left a
" (deleted)" suffix), fixed here.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
File node ids: include/strawberry/node_ids.hpp is now the only place
they are built (file_<sha256> or file_path_<sha256 of path>). The three
modules disagreed: correlation used a 16-digit path hash and behavior
detections the raw path, so path-keyed evidence never met the rest. An
exec detection with a known hash also writes a same_path edge from the
path id to the content id. Source invariant R8 rejects hand-built file
ids in src/ and include/.
Processes carry the real uid and kernel comm: from the eBPF exec event
(which now includes the uid) or /proc. Stored on process nodes and in
process_exec telemetry (fields 14, 15).
Command lines are redacted before any storage (capture_redact_cmdline,
default on): sensitive flags and KEY=value pairs, Authorization/Bearer
credentials, URL passwords, tokens used as URL user names, well-known
token shapes (GitHub, GitLab, Slack, AWS key ids, sk-, JWTs) and mysql
-p<password>. The rules repeat to a fixed point, so redaction is
idempotent by construction. tests/test_redact.cpp covers the rules plus
idempotence, size and planted-secret properties; fuzz_cmdline_redactor
found two idempotence bugs and a leak (--password --token X) that are
fixed here.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 2 increment 1. Until now only detections were stored, so the
learning stages would never see a normal example.
- TelemetrySink: bounded lock-free multi-producer queue (8192); try_emit
never blocks, allocates or throws; a full queue drops and counts.
- A guarded writer thread drains it every 2 s (or at a quarter full)
into the store's root-only telemetry stream through the pipeline,
writing a gap record whenever events were dropped.
- The pipeline emits one process_exec (pid, parent, node ids, exe,
capped cmdline, sha256, deleted/memfd, live/inventory) and one
scan_result (outcome: scanned, cache_hit, too_large, unavailable,
read_error, no_rules, yara_error, pid_scan; bytes, scan time, rules
generation, matched rules) for every handled exec, detection or not.
- Config telemetry=true (default); status gains a telemetry block
(emitted, dropped, queued, written, gaps).
Tests: C1 (never blocks; exactly once and in order under concurrency,
clean under ThreadSanitizer), record round-trips, C2 (one process_exec
and one scan_result per handled exec, stored in the telemetry stream);
fuzz_telemetry_records.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing shipped used the library after the event store landed. A small
reader/writer for its file format (src/store/legacy_kv) replaces it for
the legacy code, and the submodule, subproject and library test are
removed.
A writer open now imports a legacy database.db file into the event
store: indicators are replayed in an order consistent with the old
hash/rule indexes, the result is verified against the old file, the
old file is kept as database.db.migrated-<ts>, and an import
interrupted between its final renames is finished on the next open.
Read-only handles read a legacy file directly and switch to the store
once it is imported. Ids re-stored under another hash are reported
(their old index entries cannot be carried over).
Tests: legacy import properties I1-I6 (locked and corrupt files are
left untouched), the differential test now shares its comparisons, and
fuzz_legacy_kv covers the file parser. A copy of a real 137 KB legacy
database imported cleanly into 7 KB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fix for the 2026-10-25 host freeze (an uncaught iostream exception on a
/proc read aborted the daemon; its core dump deadlocked with the exec
guard): non-throwing /proc reads, crash handlers that release fanotify
permission fds, guarded thread entries, from_chars parsing.
Adds the io fault-injection seam with property tests, source invariant
checks (R1-R6) with self-test fixtures, libFuzzer targets and seeds
(including I/O-fault fuzzing), the fold changed-files scan scripts and
the agent workflow docs (CLAUDE.md, prompts).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>