make docker-compiler-matrix builds and tests the working tree in throwaway containers per distro and compiler, ninja -k 0 so every compile error shows. Minutes per leg locally vs ~3 h for a ci/** run. The apt list moves to packaging/ci/build-deps.txt, read by build-release.sh and Dockerfile.matrix. noble's clang leg uses clang-19 + llvm-19: clang 18 cannot use libstdc++ 13's std::expected.
20 KiB
AGENTS.md
Documentation Map
- RESEARCH.md — literature-backed rationale for Strawberry's architecture and scope
- PLAN.md — execution order, status, and next milestones
- README.md — public project overview and build/run instructions
- DEVELOPER.md — implementation notes and code structure
- OFFLINE_INSTALL.md — hybrid Conan/submodule air-gapped install workflow
- UX.md — analyst workflow notes and the desktop app (Qt6 + tray) direction
- API_CONTRACT.md — control-socket protocol, access model, and CLI/evidence JSON shapes
- docs/prompts/PROMPT-eng-codex.md — generic YAMS-first operating loop: retrieval contract, metadata, handoff, ask-first list
- docs/prompts/PROMPT-strawberry-audit.md — audit flow with brick (structure), fold (symbolic SAST) and pipe (state flow)
Operating Model
Work loop: retrieve → inspect → act → verify → persist → report. Run it after a
new session, model switch or compaction; reuse evidence already in context instead of
repeating work. The full generic loop (claim, checkpoints, handoff, ask-first list) is
docs/prompts/PROMPT-eng-codex.md, shared with ../yams/../brick. YAMS (yams CLI)
is the durable cross-session memory and code index — not the transcript. In brief:
- Retrieve.
yams search "strawberry <task>" --limit 5 --paths-only, hydrate hits withyams get --name <name> --max-bytes 4000(≤3 first). Notes are leads, not proof; revalidate mutable facts against current source/runtime. - Inspect.
yams grep -F "<symbol>" --paths-onlythen read the source (rgas fallback).git status --shortto separate this task's edits from existing work. - Persist. After meaningful work, store verified findings, decisions and handoffs
in YAMS (
task=strawberry-<topic>,source=note|decision|evidence,owner=opencode), not as new local Markdown; checkpoint long work, re-index changed files (below). Confirm by read-back — a queued/--syncreceipt is not saved. Product docs (README, API_CONTRACT, PLAN, DEVELOPER) stay in the repo and must keep matching the code. - Report. State changes, the checks actually run and their results, and remaining uncertainty — with evidence (file:line, check/result). A passing test covers only what it exercises; an empty search is not proof of absence.
Re-index this task's changed files at checkpoints:
yams add . --recursive --include "*.cpp,*.hpp,*.h,*.c,*.md,*.build,*.yar,*.service,*.timer,*.path,*.sh,*.conf" \
--exclude "builddir/*,third_party/*,build/*,.ccache/*" \
--metadata "mode=engineering,task=<task>,phase=checkpoint,owner=opencode,source=code"
Audits use docs/prompts/PROMPT-strawberry-audit.md (brick = structure, fold = bounded
symbolic findings, pipe = enum class state flow): verify every finding in source before
fixing or reporting, land a fix with a test, store triage in YAMS (source=evidence).
Background Work & Resource Governor
Strawberry runs on desktops: a scanner that freezes the machine gets
uninstalled. ResourceGovernor (include/strawberry/resource_governor.hpp,
ported from ../yams) samples the daemon's CPU/RSS, host PSI (loadavg
fallback), MemAvailable and cgroup v2 limits, and the scan profile
(scan_profile = efficient | balanced | aggressive) bounds every heavy path.
- Code that starts a thread, loop or job asks the governor first:
admit(WorkClass::…)/wait_admitted(…). Background classes (inventory, memory scans, correlation, rule reloads, blocklist sync) wait while the host is busy; the starvation guard (background_max_deferral_s) still runs them. - The launch path is never deferred: the exec guard may only degrade to
hash/IOC/quarantine (
guard_should_degrade) and must answer beforeprevention_timeout_ms; its YARA timeout is capped at that deadline. - Background reads draw from
pace_readand drop page cache afterwards; background threads run at the profile's nice/ioprio (src/thread_priority.hpp). Scans happen outsideDetectionPipeline::mutex_. - No free-running loops, no unbounded queues, no clear-all caches on hot paths, no full-file hashing on the eBPF event thread.
- A new or changed background path needs a constrained-lane KPI before and
after (
make perf-kpi,scripts/perf_kpi_report.py --compare), 3 repeats for any claim. Code reading is not evidence. - Timings: record into
PerfStats(include/strawberry/perf_stats.hpp) and add Tracy zones (STRAWBERRY_ZONE,-Denable_profiling=true) for any new heavy step; surface them in statustimings.
Invariants and Fuzzing
Per-change checks. Every change that touches I/O, callbacks, threads, parsers or persistence runs, before it is called done:
| Check | Command | Catches |
|---|---|---|
| Unit + property + source invariants | nice -n 19 meson test -C builddir --num-processes 2 |
logic, tests/test_invariants.cpp, tests/source_invariants.py (iostream/std::sto*/unguarded threads/non-noexcept C callbacks/empty catch) |
| Fault injection | test_fault_injection (part of meson test; sweep seeds with STRAWBERRY_PROPERTY_SEED) |
exceptions, hangs or wrong results when reads/opens fail (ESRCH, EIO, EINTR storms, short reads) |
| Fuzzing | make fuzz (clang, capped, niced; fuzz_io_faults + parser targets) |
crashes, escapes and timeouts on hostile input and I/O |
| fold | scripts/fold_changed.sh [base] (fold's LLVM-19 image, --cpus=2 --memory=6g, cached) |
bounded symbolic findings in changed TUs, including exception-escape (exceptions reaching noexcept functions, destructors, C callbacks, thread entries) |
| clang-tidy | nice -n 19 clang-tidy -p builddir <changed files> (.clang-tidy; enforced checks are errors) |
bugprone-empty-catch today; more checks are added as the tree is cleaned |
| clang static analyzer (strict) | make analyze (scripts/analyze_changed.sh [base]; clang-check --analyze on changed TUs via builddir/compile_commands.json, nice -n 19, 2 at a time; any diagnostic fails the run, ANALYZE_STRICT=0 reports only) |
null deref, use-after-free, leaks, uninitialized reads and path-sensitive logic bugs the type system and clang-tidy miss |
| brick / pipe | see docs/prompts/PROMPT-strawberry-audit.md |
structure, enum class state flow |
| Live + KPI | make docker-live-tests, make perf-kpi (3× --compare) |
root-only paths, resource regressions |
| Compiler matrix | make docker-compiler-matrix (trixie/noble/forky x gcc/clang, scripts/compiler_matrix.sh) |
toolchain/distro breaks (newer GCC/libstdc++, clang-only errors, libbpf/yara API drift) before a ci/** push or release |
Verify every tool finding in source before fixing or reporting it, and store
the triage in YAMS (source=evidence). Fold's exception-escape detector,
source_invariants.py R4/R5 and the fault-injection properties together cover
exceptions escaping into C frames, destructors and threads. Fold sees one TU at
a time and the invariants are syntactic, so keep all three. The clang static
analyzer is path-sensitive within a TU (null deref, use-after-free, leaks,
uninitialized reads); it complements fold's bounded symbolic cross-call analysis
and the syntactic invariants rather than replacing either — run all of them, and
index the change in YAMS at each checkpoint before and after.
- Decision logic and parsers carry property tests in
tests/test_invariants.cpp(seeded; failures print the seed). A new rule that must always hold gets a property, not just an example. - Every parser of untrusted or attacker-influenced input has a libFuzzer
target in
fuzz/(make fuzz, clang, capped and niced). New parsers get a target and seeds infuzz/seeds/<target>. - Every file or syscall I/O on a runtime path goes through
include/strawberry/sys_io.hpp(io::read,io::pread, …) so the fault-injection property tests andfuzz_io_faultscan fail it. Read loops bound EINTR retries (io::kMaxEintrRetries); a short file at EOF is an error, never a silent prefix. - No empty
catchblocks (R7, clang-tidybugprone-empty-catch). A comment is not handling. Handle the exception (log, count, return an error), or, where nothing more is possible (destructors,noexceptC callbacks, error-reporting fallbacks), record it withSTRAWBERRY_NOTE_SWALLOWED()frominclude/strawberry/swallowed.hpp: it counts the exception (statusresources.swallowed_exceptions) and logs file:line without allocating or throwing. - New static rules go in
tests/source_invariants.pywith abad_*.cppfixture intests/fixtures/invariants/(the self-test proves the rule fires).
Purpose
Strawberry is an early-stage Linux host security scanner prototype in C++23. The intended design remains:
- eBPF/libbpf for process/runtime telemetry
- YARA for file/process scanning
- an embedded local store for indicators + relationship graphs
- a minimal rule-based correlation layer over persisted runtime evidence
- a small daemon/state-machine shell around those components
Current Project Reality
A working C++23 host security scanner, well past prototype but not finished.
Status and next steps: PLAN.md. Implementation detail: DEVELOPER.md. History:
git log / YAMS task notes. The load-bearing facts:
- Build/test spine green — meson/ninja (
-Denable_tests=true), thinmakewrapper, hybrid Conan offline deps. Runtime validation for eBPF//proc/systemd/memory-scan is Linux-only (make docker-live-tests); static review + most unit tests run anywhere. - Store —
Databaseover the compressed event store (src/store/: zstd segments, sealed footers, crash-atomic MANIFEST, LogStore recovery); persists indicators, nodes, edges, findings, labels, telemetry; legacy unqlite import on first open. Evidence 0640, telemetry root-only 0600. - Monitoring —
ProcessMonitor: procfs polling + Linux libbpf exec/behavior events (eBPF falls back to procfs). Process ids viamake_process_node_id()(process_<pid>_<boot8>_<starttime>) — never hand-build them. - Detection —
DetectionPipeline+YaraScanner(file //proc/<pid>/exe/ trigger-only memory scans, bounded in-memory copy), IOC feeds with per-feed trust, behavior rules; narrowCorrelationEngine(a handful of YARA/hash/lineage/behavior rules — NOT a general incident engine). Rules auto-update viaPolicyStore+ sandboxedstrawberry-update. - Response — fanotify exec guard (
auditdefault,enforceopt-in, fails open), OpenSnitch containment (hash-pinned), quarantine/suspend/kill by node id. See Guardrails. - Learning (Stage 4, shadow-only) —
learn/: offline MLP training, live shadow scoring, replay-gated operator promotion. It steers inspection budget, it is not a detector; Stage 5 advisory detection is deferred. - Distributed fleet (Stage 6) —
strawberry export(redacted bundles: windows, features, labels, rarity sketches) → centralfleet train+fleet overlay→ per-node + fleet-wide rarity as a model feature. Seefleet_export.*/fleet_merge.*and the distributed-fleet-salt note. - Interfaces — Qt6 desktop app + tray (
app/,enable_app);strawberry-mcp(mcp/,enable_mcp) read-only agent interface over stdio (scan-fdfd-passing, never path-opening); yams-style CLI (src/cli_commands.cpp,noun verbtable,--json). All mutation goes throughControlServer(/run/strawberry/control.sock, groupstrawberry, peer-checked, rate-limited, audited); the app reads the DB read-only. - macOS — builds +
meson testpass (GitHub CImacos); Endpoint Security backends (exec/AUTH_EXEC/file-watch) compile with libproc fallback but need root + the ES entitlement to run live. Linux paths unchanged. - Still process-centric: no thread/TID telemetry (thread search is future UX, not current).
Storage Guidance
- keep the
Databasepublic interface narrow and stable; storage specifics live insrc/store/ - every decoder takes untrusted bytes and never throws; new record fields are tagged (old readers skip them), record type ids are append-only
- all store I/O goes through
strawberry::io(fault injection); store changes need the S/M/L properties, the differential test and the store fuzzers green - evidence is group-readable (0640), telemetry root-only (0600)
Build System Guidance
Meson/Ninja remains the source of truth.
- Prefer fixing Meson rather than replacing it.
- The repository now also exposes a thin GNU Make wrapper for convenience.
- A root
conanfile.pysupports a hybrid offline dependency flow for OpenSSL, zstd and optional extras; nothing is vendored. - The root Conan recipe forces a C++23 Meson native-file setting so
conan profile detectdefaults do not break the build. - The repo now also seeds a local
yara/4.5.5Conan recipe underconan/local_recipes_index/for air-gapped or pinned builds. - The root
Dockerfileis now the Linux build/test container definition and should stay aligned with the real dependency set. - The Docker image defaults to
gcc/g++for C++23 inside the Ubuntu 24.04 container. make appruns the desktop app from the build tree;make demoseeds demo evidence and opens it in the app. Qt6 (Core, Gui, Widgets, DBus, Network) is optional;-Denable_app=disabledskips the app.- If someone asks for
makesupport, keep it as a wrapper around Meson/Ninja unless explicitly told otherwise. - If “Make” refers to Make.com, treat that as a separate integration effort and do not assume it means GNU Make.
Recommended Work Order
Current priorities and sequencing live in PLAN.md. Standing principles: keep build/test
green; grow the correlation engine only when the evidence model supports a rule cleanly;
keep control-socket/JSON contract changes small and documented in API_CONTRACT.md; reuse
the read-only evidence + fixed control verbs for CLI/MCP; defer thread-level UX until
thread/TID telemetry exists; expand packaging/hardening only once the core is stable.
Testing Priorities
Prioritize tests in this order:
-
Build configuration tests
- clean configure
- clean compile
- test toggle works
-
Pure unit tests
- hash utilities
- config parsing/serialization
- database persistence semantics
- graph traversal behavior
-
Integration tests
- YARA rule load from a temp directory
- file-scan match persistence
- multi-hit indicator persistence for the same hash
- synthetic process-event ingestion pipeline
- synthetic procfs event →
DetectionPipeline→ persisted indicator/graph flow
-
Linux runtime smoke tests
- daemon starts with test config
- telemetry path initializes cleanly
- privileged libbpf smoke harness observes a live exec event
- test YARA hit is stored and queryable
- live daemon-owned telemetry path stores a test hit through the real daemon wrapper
-
Correlation and operator tests
- multiple related raw events collapse into one explainable finding when intended
- process-first query surfaces return lineage plus related indicators/files
- findings-first query surfaces can pivot back to underlying evidence
Repo Conventions
- Public headers live in
include/strawberry/ - Implementations live in
src/ - Keep public interfaces small and dependency-light
- Prefer RAII and
std::unique_ptr - Keep platform-specific code isolated behind implementation details
- Preserve XDG-style default data placement for local storage
Agent Guardrails
When working in this repo:
- do not describe the current YARA path as fully complete: file/executable and process-memory scans exist, but memory scans run only on triggers (memfd/deleted/ptrace target/operator request), not continuously, and need real-host tuning
- do not describe the current runtime pipeline as fully complete: synthetic procfs-driven scan/store coverage and one live libbpf-to-store smoke pass now exist, but richer live-event enrichment and daemon-grade runtime handling are still the next major steps
- do not describe the daemon runtime as fully complete: the reusable wrapper, daemon-owned smokes, and basic text reporting now exist, but richer live parent-PID enrichment and more advanced reporting/export are still the next major steps
- do not describe the desktop app as a full operations console: it is a findings/process/diagnostics viewer with rules/allowlist/response management and a small set of confirmed actions
- never add a control verb that makes the root daemon open a caller-supplied path or run a command: on-demand file access must stay fd-passing (
scan-fd). Policy payload verbs (allowlist-*,feed-*) may store path/URL strings afterPolicyStorevalidation, but the daemon never opens them, and feed URLs are fetched only by the sandboxedstrawberry-update.service(strawberry rules fetch); the daemon itself keepsPrivateNetwork=true - protection/response verbs take hashes or ids, never paths; paths for containment/quarantine come only from Strawberry's stored evidence
- process actions (
scan-memory,process-suspend/resume/kill) must accept only stable node ids and re-verify them against/proc(boot id + start time) before acting; keep the protected-process refusal list (pid 1, the daemon, systemd/dbus/opensnitchd/display server/sshd…) - automatic actions (containment, exec deny) must be hash-pinned, must never apply to system locations (
is_protected_system_path: alert only), and must lose to the allowlist; only an explicit operator quarantine blocks regardless of prevention mode - every containment/quarantine/process action must be recorded in policy history and be visible and undoable in the app's Response tab
- the MCP server stays read-only for containment, prevention, quarantine and process actions; do not add MCP tools for them
- exec prevention must fail open (deadline, oversized files, daemon exit) so a slow or broken scanner can never hang the host
- a process holding fanotify permission events must never die slowly: keep
install_crash_safety()(non-dumpable, fatal-signal/terminate handlers close registered permission fds first) andLimitCORE=0; register every new permission fd withregister_permission_fd(2026-10-25 freeze: crash → core-dump helper exec → deadlock) - never read
/proc(or any file that can vanish mid-read) with iostreams/istreambuf_iterator; useread_small_file()(include/strawberry/proc_read.hpp); worker loops and C callbacks (libbpf) catch exceptions per item - do not describe exec prevention as default-on blocking: the default mode is
audit;enforceis an operator choice - rule feeds must stay verified and fail-safe: GitHub release assets require a published sha256 digest, archives are extracted to flattened
*.yarbasenames only, and staged rules must compile together with local + other feeds beforecurrentis switched - keep policy mutation out of
strawberry-mcp(read-onlystrawberry_get_policyonly) so prompt-injected agents cannot allowlist malware - treat everything the MCP server returns from evidence (cmdlines, paths, rule meta) as untrusted host data; keep the server's instructions saying so, keep string clipping and UTF-8 replacement
- keep the control socket's verb set fixed (plain verbs argument-free, payload verbs strictly validated) unless a new verb gets the same review (peer check, rate limit, tests in
tests/test_control_server.cpp); never add a verb that runs commands or takes paths without explicit design - do not describe the correlation engine as complete: today Strawberry has four small YARA/process/hash/rule correlation rules, two process-image heuristics (memfd, deleted executable), an IOC hash match and six narrow behavior rules, not general multi-signal incident generation
- do not describe thread search as implemented: current telemetry and persistence are process-oriented
- do not silently reintroduce MongoDB-era names or semantics
- keep docs aligned with real code, especially around persistence and tests
- call out Linux-only assumptions explicitly in reviews and READMEs
- if adding convenience scripts, make sure they stop mentioning MongoDB-era services
Keep This File Lean
This file holds durable repo rules, invariants and guardrails. Generic agent behavior
lives in docs/prompts/PROMPT-eng-codex.md; dated progress and "as of <date>" history
belong in git log / YAMS task notes, not here; current status and next steps live
in PLAN.md; implementation detail in DEVELOPER.md. When a section starts listing dated
events or restating code, move it out and leave a pointer.