Nothing shipped used the library after the event store landed. A small reader/writer for its file format (src/store/legacy_kv) replaces it for the legacy code, and the submodule, subproject and library test are removed. A writer open now imports a legacy database.db file into the event store: indicators are replayed in an order consistent with the old hash/rule indexes, the result is verified against the old file, the old file is kept as database.db.migrated-<ts>, and an import interrupted between its final renames is finished on the next open. Read-only handles read a legacy file directly and switch to the store once it is imported. Ids re-stored under another hash are reported (their old index entries cannot be carried over). Tests: legacy import properties I1-I6 (locked and corrupt files are left untouched), the differential test now shares its comparisons, and fuzz_legacy_kv covers the file parser. A copy of a real 137 KB legacy database imported cleanly into 7 KB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
25 KiB
Implementation Plan
Execution plan for turning Strawberry from scaffold into a minimally real, research-backed prototype.
Linked Documents
Planning Principle
This plan follows the literature in RESEARCH.md:
- reliable runtime telemetry before advanced analytics
- signatures as enrichment, not the only detector
- graph context as a force multiplier
- compact, inspectable local workflows before bigger platform ambitions
Current Status Snapshot
As of April 27, 2026:
- Phase 0 is complete: the repo configures and compiles again
- Phase 1 is complete: the current unit suite passes
- Phase 2 is complete in the current checkout: Strawberry persists indicators and graph data across reopen through vendored
third_party/unqlite-cpp(replaced by the compressed event store on 2026-10-25; the submodule was removed) - direct persistence tests now cover reopen, commit, rollback, and read-only behavior
- Phase 3 has started: optional libyara-backed rule loading and file scanning are implemented
- YARA hit metadata now persists through the database path, including multi-hit support for the same hash and process→indicator graph linkage
scan_processnow performs a best-effort/proc/<pid>/exescan when libyara is available- a Conan-backed libyara build and test pass succeeded on April 25, 2026 on the current macOS host, including real rule compilation and file-scan assertions
- Linux-specific
/proc/<pid>/exebehavior is now covered by a Docker-validated unit test on April 26, 2026 - a minimal procfs-polling process-event path now exists on April 26, 2026
- a reusable
DetectionPipelinenow exists, and a Docker-validated synthetic runtime test proves procfs event ingestion can drive executable scanning plus persisted indicator/graph storage on April 26, 2026 - a minimal libbpf exec-event backend now compiles in the Linux Docker build on April 26, 2026
- a privileged Linux Docker smoke run now validates live libbpf tracepoint attach and exec-event delivery on April 26, 2026
- a second privileged Linux Docker smoke run now validates live libbpf exec-event persistence through
DetectionPipelineon April 26, 2026 - a reusable
StrawberryDaemonwrapper now owns the daemon shell around config loading, process-monitor startup, andDetectionPipeline - a daemon-level synthetic test now validates procfs event ingestion through
StrawberryDaemoninto persisted indicator/graph storage on April 26, 2026 - a third privileged Linux Docker smoke run now validates live libbpf exec-event persistence through
StrawberryDaemonitself on April 26, 2026 - a new
IncidentReporternow renders concise human-readable summaries over persisted graph state with first/last seen timing, and the main binary can print them with--trace-process/--trace-node - the operator UI is now the Qt6 desktop app + tray indicator
strawberry-tray(October 24, 2026); the temporary Python web evaluator was removed - a synthetic demo-trace seed path exists (
make demo), so the app/reporting flow can be evaluated without a running daemon - the active direction remains system improvements first so real Linux process collection and correlation evidence quality lead the work
- a first minimal correlation engine now exists: Strawberry can group process-executable YARA indicators by process/hash and persist an explainable finding
- a second correlation rule now flags multiple matched descendants under one parent process lineage
- a third correlation rule now flags the same matched executable hash across multiple processes
- a fourth correlation rule now flags the same YARA rule across multiple processes
- findings now have JSON output through the CLI for UI/automation consumers
- a process search query surface now exists for PID, path, hash, YARA rule, and finding pivots, with JSON output for UI/automation consumers
- the correlation engine is intentionally small and rule-based; it is not yet a general subgraph matcher, anomaly detector, or full incident engine
- the current data model is still process-centric, not thread-centric; there is no first-class TID/thread telemetry or thread search UX yet
- procfs polling now has an optional startup inventory mode that emits already-running processes before continuing to watch for newly observed PIDs; this is the current path for collecting a fuller Linux process corpus for correlation refinement
- the libbpf path now has a best-effort fixed-width live argv preview (
argv0/argv1); both the/bin/echosmoke and the staged pipeline smoke validated it on April 26, 2026 - the libbpf path now opportunistically attempts
sched_process_forkfor parent-PID enrichment, but the privileged container smoke confirms that this additional tracepoint may be denied while the main exec-event path still succeeds - procfs-driven runtime events now also persist
parent_pidandevent_timestamp_ns, and create minimalspawnedlineage edges on April 26, 2026 - matched runtime events now also persist executable-image file nodes with
executedandcontent_matchedprovenance edges on April 26, 2026 - a thin GNU Make wrapper now exists for local
make setup/build/test/run - a root Conan 2 recipe and air-gap workflow doc now exist for offline dependency installs
- a seeded local
yara/4.5.5Conan recipe now exists underconan/local_recipes_index/ - a Linux-oriented root
Dockerfilenow exists for build/test validation with libyara enabled - the Docker image now defaults to
gcc/g++because the Ubuntu 24.04 container path was the cleanest working route for C++23 + vendoredunqlite-cpp
Host-readiness hardening (October 24, 2026):
- the BPF object now builds on Arch (UAPI
__u32/__u64types instead of glibc<stdint.h>); previously it silently failed there and the daemon fell back to procfs polling execveattracepoints are attached, sofexecveand memfd (fileless) executions are observed- events resolve through
/proc/<pid>/exeand scan/hash that magic link, so relative paths, replaced files, deleted binaries, memfd images and other mount namespaces are handled MemfdExecution/DeletedExecutableheuristic indicators feed per-process correlation- process nodes use a stable identity (
process_<pid>_<boot8>_<starttime>), so PID reuse no longer merges processes - the BPF backend now also emits the startup inventory, and its timestamps are converted to wall clock
- the daemon fails closed when no rules compile (
require_rules=true) or the DB cannot be opened for writing - database: single-writer
flock, read-only CLI/UI queries (no lost updates), transactional per-event writes, batched correlation, and a scan-result cache - rule severity comes from
meta.severity(defaultmedium); starter rules were replaced with a small, high-signal set - systemd unit: capability-bounded,
NoNewPrivileges, read-only/homeand shared/tmp(so binaries there can still be scanned), private state dir - (the web UI server hardened in this pass was later replaced by the desktop app; see Phase 8)
Next for host use: worker-pool scanning off the ring-buffer thread with drop counters, fuller argv capture, on-demand memory scans for flagged processes, file-write/network/module telemetry, structured alert output, and an allowlist.
Phase 0 — Make the repo buildable ✅
Goals
- repair build metadata
- eliminate obvious compile blockers
- get the project configuring cleanly
Delivered
- Replaced invalid
meson.optionsusage with valid Meson option handling. - Fixed the
DaemonStateMachinetype definition problem ininclude/strawberry/daemon_states.hpp. - Removed compile-time migration fallout and stale MongoDB-era references from the active build path.
- Re-ran configure and compile until the build graph was stable.
Exit Criteria
meson setup builddir -Denable_tests=truesucceedsmeson compile -C builddirsucceeds
Phase 1 — Repair and modernize tests ✅
Goals
- align tests with current public interfaces
- establish a trustworthy regression baseline
Delivered
- Updated
tests/test_config.cppto usedb_pathand current config defaults. - Kept
HashUtilstests compiling and passing. - Added database-focused tests for:
- indicator round-trip
- graph round-trip
- persistence across reopen
Exit Criteria
meson test -C builddir --verbosepasses for the current unit suite
Phase 2 — Real embedded persistence ✅
Goals
- turn the previous in-memory
Databasestub into real local persistence - keep the public interface stable
Delivered
- Integrated
git@github.com:trvon/unqlite-cpp.gitbehindsrc/database.cpp.- vendored location:
third_party/unqlite-cpp
- vendored location:
- Persisted:
IndicatorEntryGraphNodeGraphEdge
- Preserved XDG-friendly defaults.
- Added direct persistence tests under
/tmpfor:- reopen
- commit
- rollback
- read-only enforcement
Exit Criteria
- indicators survive process restart
- graph nodes and edges survive reopen
- query helpers work against persisted data
Phase 3 — Real YARA integration 🟡
Goals
- replace placeholder load/scan logic with actual libyara-backed behavior
Delivered so far
- Implemented rule compilation/loading from a file or directory when Strawberry is built with libyara.
- Implemented file scanning with real match extraction when libyara is available.
- Implemented a best-effort
scan_processbridge that scans/proc/<pid>/exerather than process memory. - Added YARA-focused tests that:
- exercise graceful fallback when libyara is absent
- contain real scan assertions for libyara-enabled builds
- now include a Linux-only
/proc/<pid>/exesmoke test forscan_process
- Persisted YARA hit metadata through the database path and added graph linkage tests for process→indicator relationships.
- Validated the real libyara file-scan path through the Conan-backed dependency flow and fixed a Conan/Meson C++ standard mismatch by forcing a C++23 consumer toolchain.
- Validated the Linux
scan_processpath in the Docker Ubuntu 24.04 test image on April 26, 2026. - Added a reusable
DetectionPipelineand a synthetic runtime integration test that proves procfs event ingestion can trigger YARA matching and persisted indicator/graph storage on April 26, 2026.
Remaining Tasks
- Thread the persisted YARA hit path through more realistic end-to-end daemon/runtime flows.
- If feasible, add true process-memory scanning as a guarded Linux-only feature.
- Record at least one libyara-enabled synthetic end-to-end scan/store run on Linux.
- Start the minimal
libbpf/eBPF exec-event ingestion path.
Exit Criteria
- a temp rule set can be loaded on a libyara-enabled host
- a synthetic malicious sample triggers a stored match
- YARA hit metadata is persisted alongside related process/graph context
- at least one libyara-enabled automated test pass is recorded
Phase 4 — Real eBPF process telemetry
Goals
- replace placeholder monitoring logic with actual runtime event capture
Delivered so far
- Replaced the empty
ProcessMonitorloop with a minimal procfs-polling backend that emits new-process callbacks. - Captured per-process basics from procfs:
- pid
- executable path
- SHA-256 hash of the executable
- command line
- Added a synthetic procfs unit test for new-process detection and callback delivery.
- Added a minimal libbpf-backed exec-event build path:
- BPF object source for
sys_enter_execve/sys_exit_execve - user-space libbpf loader and ring-buffer handling
- automatic fallback to procfs polling when libbpf is unavailable or attach/load fails
- BPF object source for
- Validated in the Linux Docker build that the eBPF object compiles cleanly on Ubuntu 24.04.
- Added a privileged Docker live smoke harness and validated one real
/bin/echoexec event reaches user space.
This is now a useful Linux-oriented bridge with both a real privileged Docker smoke pass and a real privileged Docker libbpf-to-store smoke pass, but it is not yet a richer daemon-grade runtime ingestion pipeline.
Tasks
- Fold the live libbpf-to-store harness behavior into a daemon/runtime harness, not just standalone smoke executables. ✅
- Capture at least:
- pid
- parent pid if available
- executable path
- timestamp
- Improve event context beyond the current executable path +
commbridge in the live libbpf path (for example, parent PID and richer argv handling if feasible). - Keep a Linux-only smoke path in Docker/CI for the live telemetry path.
Exit Criteria
- a real Linux exec event reaches user space
- Strawberry creates at least one persisted process node and relationship edge from live telemetry
Phase 5 — Basic provenance graph usefulness
Goals
- make the graph layer operationally useful, not just present
Tasks
- Record process lineage edges.
- Optionally record executable file edges.
- Add simple queries:
- related nodes from a process
- all matches connected to a lineage
- Add a concise CLI or log output for incident trace summaries. ✅
Exit Criteria
- one suspicious event can be traced through stored relationships in a human-readable way
Phase 6 — Packaging and automation surfaces
Goals
- add convenience surfaces only after the core path is real
Tasks
- Keep the GNU Make wrapper aligned with Meson.
- Keep the Conan/offline workflow aligned with real dependency usage, including the local YARA recipe.
- Keep the Linux Docker image aligned with the real dependency and test path.
- Clean up install/service scripts as runtime features become real.
- If the target is Make.com, define a small stable CLI contract first, then automate it.
- Document Linux-only runtime expectations clearly.
Exit Criteria
- operators can build, test, and run the scanner through one stable interface
- external automation does not depend on undocumented internal behavior
Phase 7 — Detection correlation engine
Goals
- move from stored evidence to ranked, explainable findings
- correlate multiple related events into one analyst-facing detection object
Delivered so far
- Added
FindingRecordpersistence/query support toDatabase. - Added
CorrelationEnginewith a deterministicprocess_executable_yara_matchrule. - Added
lineage_multiple_yara_matchesfor parent processes with multiple matched descendants. - Added
same_hash_multiple_process_matchesfor repeated matched executable hashes across processes. - Added
same_yara_rule_multiple_processesfor repeated YARA rule hits across processes. - Added CLI surfaces:
--correlate--list-findings--show-finding <id>--jsonfor structured finding output
- Updated the demo seed path to emit a correlated finding.
- Extended the minimal local web GUI with finding list/show endpoints and controls.
- Added unit coverage proving multiple related process YARA hits, lineage hits, and same-hash process reuse collapse into deterministic findings.
Tasks
- Extend the minimal correlation input model over existing persisted data:
- process nodes
- file nodes
- indicator nodes
spawned,executed,matched, andcontent_matchededges- timestamps, hashes, paths, command lines, and parent PID where available
- Add more small rule/heuristic detections before ML:
- suspicious executable matched by YARA and executed by a short-lived process
- repeated matched descendants in the same lineage window
- one file image tied to multiple suspicious process executions
- process lineage containing multiple matched artifacts in a bounded time range
- Enrich durable finding records with:
- finding ID
- severity or score
- reason/explanation
- related node IDs
- related indicator IDs
- first/last seen timestamps
- Expand CLI and programmatic queries for:
- list findings
- show one finding
- pivot from finding → process lineage → indicators
- Add tests proving that additional raw event families collapse into one correlated finding when appropriate.
Exit Criteria
- Strawberry can generate at least one explainable finding from multiple persisted graph artifacts ✅
- the finding can be queried independently of the raw node/edge dump ✅
- at least two distinct correlation rules exist ✅
Phase 8 — Analyst UX and search
Goals
- move from a debug-style trace viewer to a real operator workflow
- make process-centric investigation fast and explainable
- a native desktop app + tray indicator as the operator surface, with no network listener
- agent-facing access (CLI today, MCP next) over the same read-only evidence and fixed control verbs
Delivered so far
- Added
ProcessQueryfor process-centric pivots over graph nodes, indicators, and findings. - Added CLI support:
--search-process <query>--search-process <query> --json
- Added local web UI process search backed by structured JSON.
- Added parent/child lineage summaries to process search results.
- Added first/last seen evidence timestamps to process search results.
- Added local web UI deep links from process search results to trace and finding views.
- Added local web UI finding cards that pivot from findings back to process traces and process search.
- Added related file/indicator summaries to process search cards, making the local evaluator a lightweight process detail panel.
- Added client-side sort/filter controls for process search and finding cards.
- Added operator-friendly ns/us/ms/s duration formatting for trace, finding, and process-search timestamps.
- Added local-web copy/download affordances for rendered traces, process-search results, and findings.
- Split process-search cards into structured identity, timing, lineage, related-file, related-indicator, and finding sections.
- Added explicit UI server host/port controls, with safe loopback-only default and
--allow-remoterequired for non-loopback binds. - Documented the Python UI server as a temporary evaluator and the target C++ Boost.Beast/Asio API service direction.
- Added
API_CONTRACT.mdto pin the current web JSON contract before porting endpoints into C++. - Added
/api/processfor exact process-detail JSON by PID or process node ID. - Wired the local web UI to
/api/processwith a focused process-detail panel and detail pivots from search/finding results. - Added a loopback web API contract test for health/defaults/trace/findings/process search/process detail against seeded demo data.
- Added
WEB_DEPLOYMENT.mdcovering local, SSH-tunnel, container/lab-bind, and reverse-proxy prototype modes. - Added unit coverage for search by PID, path, hash, YARA rule, finding metadata, lineage summaries, and process seen timestamps.
- October 24, 2026: replaced the web evaluator (items 3, 6–9, 11–19) with a Qt6 desktop app + tray indicator (
app/,strawberry-tray):- Findings, Processes and Diagnostics tabs; tray colour by scanner health and unseen findings; desktop notifications for new high/critical findings.
- Daemon
ControlServerUnix socket (status,reload-rules,rescan; groupstrawberry; peer-credential checked; rescan rate-limited), plusstrawberry --status/--reload-rules/--rescan. - The service runs with
Group=strawberry, so evidence is0640group-readable; the app reads it viaDatabase::open_read_only(). - Tests:
tests/test_control_server.cpp,app/tests/test_tray_state.cpp, pipeline stats/rescan coverage. - Removed
scripts/strawberry_ui_server.py,ui/,scripts/test_web_api_contract.py,WEB_DEPLOYMENT.md;API_CONTRACT.mdnow documents the socket protocol and CLI JSON.
Tasks
- ✅ Agent-facing interface (October 24, 2026):
strawberry-mcp(MCP over stdio) with status, findings, process search/detail, fd-passingscan_paths, reload and rescan;--read-onlymode;strawberry --scan-file. Next: optionally record on-demand scan hits as evidence, and an MCP resource/notification for new findings. - ✅ (October 24, 2026) Auto-updating YARA rule feeds (YARA Forge core by default, digest-verified, compile-checked, atomic switch, sandboxed updater with daily timer + "Update now"), allowlist (hash/path/rule) with suppression in the pipeline, Rules/Allowlist tabs in the app, read-only
strawberry_get_policyMCP tool; process scans from bounded memory copies; exec events via a bounded worker queue with drop counters. - Expand the process-first views as evidence grows (argv, file writes, network).
- Defer thread-level pivots until Strawberry actually captures TID/thread telemetry.
- Package the app (desktop entry/autostart) for more desktops; keep it read-only plus fixed verbs.
Exit Criteria
- an operator can start from a process or finding and pivot across related evidence without using raw DB internals ✅
- scanner health and new high-severity findings are visible without opening a terminal ✅
- operator controls never require root or a network listener ✅
Phase 9 — Protection and response (October 25, 2026)
Delivered (all unit/integration tested; root-only paths validated in a privileged container):
- M1 threat-intel feeds: list feed kinds (
hashes,domains,ips,nets) with validation, caps and a shrink guard; default CERT-PL, Spamhaus DROP, Feodo C2 and malware-hash feeds;IOC_KnownMalwareHash; YARAmeta.score→ severity - M2 OpenSnitch: blocklist rules from list feeds; automatic per-binary containment at ≥
contain_min_severity(hash-pinned withEnableChecksums); Response tab with undo - M3 exec prevention: fanotify
FAN_OPEN_EXEC_PERMguard,off/audit/enforce, fail-open deadline, verdict cache, system paths never denied - M4 behavior telemetry: eBPF connect/ptrace/module events, fanotify persistence/drop watcher, six behavior rules
- M5 memory scan + response:
yr_rules_scan_procmemory scans (automatic for memfd/deleted/ptrace targets, or on request);quarantine(blocks execution in every mode + network);process-suspend/resume/killwith PID/start-time re-verification and protected-process refusals
Open items:
- host validation with OpenSnitch hash pinning on, prevention in
audit, thenenforce - behavior rules need real-host tuning (false-positive review of persistence writes and connects)
- network events are kept in memory only and persisted when relevant; no long-term network history
- no thread/TID telemetry yet
Phase 10 — Resource governor and profiling (October 25, 2026)
Delivered: always-on timing histograms (status timings), optional Tracy
build, constrained-lane KPI (make perf-kpi), ResourceGovernor (PSI /
loadavg / MemAvailable / cgroup v2, levels with hysteresis, deferral with a
starvation guard, read pacing), scan profiles, shared guard/pipeline scan
cache, scans outside the pipeline lock, no hashing on the eBPF thread, paced
inventory queue, governed memory scans, rule-reload backoff, unchanged
OpenSnitch lists not rewritten, per-thread nice/ioprio, systemd limits.
Open: database whole-file rewrite per commit and O(F²) finding index (measure
with timings.db_commit / correlation on a long-running host first); fuzz
and invariant tests for parsers and decision logic.
Immediate Next Actions
These are the best next steps now:
- validate startup process inventory on Linux with
STRAWBERRY_PROCESS_MONITOR_BACKEND=procfsandscan_existing_processes_on_start=true - feed that fuller process corpus through
DetectionPipelineso correlation work is based on real host-wide evidence rather than GUI demo data - improve runtime enrichment that matters for correlation: stable executable identity, parent/child lineage, timestamps, command-line capture, and exit/short-lived-process handling
- keep improving libbpf enrichment in parallel, especially parent-PID coverage and a hybrid BPF-events-plus-procfs-inventory mode when feasible
- add or refine correlation rules only when the collected evidence model supports them cleanly
- keep desktop-app work limited to what new evidence needs; next interface work is agent access (MCP/CLI)
Note: event timestamps are wall-clock nanoseconds (eBPF boot-time stamps are converted at ingestion); synthetic demo data may still use small raw values.
Suggested Next Deliverable
The next meaningful deliverable should be:
- buildable repository
- passing unit tests
- persisted local indicators and graph data
- real YARA file scanning on a libyara-enabled host
- privileged Docker-validated live libbpf exec-event delivery
- one synthetic end-to-end daemon-owned scan stored in the database
- one privileged Docker-validated live daemon-owned scan stored in the database
- one Linux procfs startup inventory pass that collects already-running processes into the same pipeline
- one minimal explainable correlated finding generated from the stored graph/evidence path
- one process-first CLI/query surface that can pivot from a finding into lineage and related artifacts
That gives Strawberry a real detection and investigation spine before deeper eBPF, provenance, and desktop UX work.