strawberry/PLAN.md
trvon 74aa9664ad
Pipeline: keep oversized cache refusals unscanned
Do not turn an empty cached digest into an unbounded descriptor hash. Distinguish size refusals from YARA results so the exec guard applies its own bounded scan/degrade policy. Cover cached read budgets across size boundaries, known hashes, inventory/live events and real guard-cache reuse. Fold/live/KPI and post-rollout CPU acceptance remain pending; OpenSnitch checksum work is separate.
2026-11-03 06:57:35 -07:00

546 lines
33 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Implementation Plan
Execution plan for turning Strawberry from scaffold into a minimally real, research-backed prototype.
## Linked Documents
- [RESEARCH.md](RESEARCH.md)
- [AGENTS.md](AGENTS.md)
- [DEVELOPER.md](DEVELOPER.md)
- [UX.md](UX.md)
- [API_CONTRACT.md](API_CONTRACT.md)
## Planning Principle
This plan follows the literature in [RESEARCH.md](RESEARCH.md):
- reliable runtime telemetry before advanced analytics
- signatures as enrichment, not the only detector
- graph context as a force multiplier
- compact, inspectable local workflows before bigger platform ambitions
## Active work — data engine and staged learning (October 27, 2026)
The newer data-engine **stages** are separate from the original scaffold
**phases** retained below. Safety/event-store work and richer telemetry have
landed, followed by incremental correlation, rarity/replay, injected trace
evaluation and retro-hunt. Stage 4 has offline model training, live shadow
scoring and operator promotion with replay gates.
The current Stage 4 increment wires promoted decisions to bounded additional
memory scans and uncached image rescans, with governor admission, stable process
identity checks, outcome telemetry and a shared executable-action replay policy.
See [the execution-policy gate](docs/evaluation/stage4-execution-gate.md).
Old promotion files are not automatically activated; they require re-evaluation.
The increment has passed local and clean-object Docker tests, privileged Linux
smokes, sanitizer/property checks and three paired default-lane KPI comparisons.
Those KPI runs had no acting model; they do not validate active inspection cost
or efficacy. Detailed coverage and limitations are in the execution-policy gate.
The live rarity model is kept across daemon restarts (`model_dir/rarity.state`,
checksummed, saved hourly and at stop), so a restart no longer repeats the
one-hour shadow warm-up.
This does **not** complete Stage 4: measured-cost budget enforcement, online
training, full multi-period/multi-seed promotion evidence and live model-triggered
inspection validation remain. Stage 5 advisory detections have not been enabled.
Offline training-data follow-up: replay now retains a seeded reservoir of teacher
positives even after negative examples fill the sample, instead of retaining only
first-arriving positives. Sampling bookkeeping is bounded by retained examples.
Restored rarity clocks also handle full-range elapsed times and complete decay
without signed overflow. The per-program cap now bounds the combined positive +
negative count per entity, and replay label joins resolve operator retractions
in stored order (allowlist-remove by entry id, quarantine-remove by hash),
preserving independent positives and other entries. Parent audit repairs cover
multi-hash entry removal, delayed suppression observations, quarantine provenance,
malformed-label failures and capped-entity refresh with bounded slot back-links
and allocated capacity (geometric compaction, not just logical entry limits).
Fresh host training readiness is **unavailable**: this session cannot read the
root-only telemetry stream; historical scarcity is not current evidence. The
readiness proposal requires total vs retained labels, temporal split, provenance,
concentration, age and full replay/training RSS measurement. Measured-cost
calibration and the unchanged three-period × three-seed promotion gate remain;
these fixes do not train, promote or activate an operational model.
Reuse-first diagnostics follow-up: existing replay now has an opt-in
`--training-summary` mode, not a parallel readiness engine. It reuses the label
resolver/sampler, reports actual retained classes and anonymous concentration,
keeps ordinary collection off, and says `assessment: not_assessed`. The reader
exposes skipped/invalid input observations; replay rejects incomplete evidence
or telemetry without exporting a usable partial report (including to trainer
loader callers). Decode availability is observed per segment: a block that needs
a zstd dictionary this build does not have is *unavailable*, and a complete block
header that fails validation is *invalid*; only a short tail an active writer
still owns stays tolerated. Snapshot summaries additionally require a stable current
manifest and no torn tail; ordinary active-tail/fragment/fallback replay remains
supported. Verifier JSON config chatter is captured, but verifier `ok` with
partial-coverage notes is intentionally not redefined as a full-read verdict.
CLI-local wall histograms/Tracy zones are not RSS or actual CPU evidence. Local
synthetic regressions cover these contracts; independent full validation remains
required before acceptance. No operational model is trained/promoted/activated,
no daemon/control/persistence format changes, and no host-readiness or promotion
claim follows from these descriptive counts.
Findings-engine repair follow-up (in progress): regressions reproduce scoped
allowlist cleanup hiding independent evidence, stale incremental membership after
policy changes, namespace/source rule-name conflation, network attribution across
PID reuse, and misleading instance counts. The repair keeps real indicator
support and immutable policy views, explicitly rebuilds/withdraws managed findings
on policy changes, preserves typed rule identities and history, and joins network
history only to an observed stable process lifetime without per-connect procfs
reads. Review found and reproduced an offline full-rebuild gap; daemon and
offline persistence now share managed-set reconciliation, and the legacy CLI
reads policy without seeding or migrating it. Fold then exposed an unguarded
shutdown-flush exception path. A one-shot allocation-failure RED regression now
passes with best-effort flush handling and a destructor guard, including raw-data
recovery and writer release. Post-rework local functional validation passes 52/52 Meson tests, 16 seeded
checks, 24 ASan/UBSan runs and 22 fuzz targets (6,390,642 executions, no crash
artifacts or sanitizer diagnostics). Native/LSP checks retain baseline warnings;
Fold's six selected daemon TUs retain 49 source-triaged candidates but no longer
report the shutdown escape. This is not diagnostic-clean or production sign-off.
The runner recorded 21m02s monotonic elapsed time, but UTC timestamps span 34m14s.
Subsequent journal review found a suspend during the batch (02:18:20–02:31:34 UTC);
`CLOCK_MONOTONIC` excludes suspended time. The historical wall cap is not
retroactively certified. An audit-only replacement uses `CLOCK_BOOTTIME` timerfd /
pidfd waits and records all three clocks; 15 focused regression/integration tests
pass. At that follow-up its future heavy-run copy remained unarmed; no controlled
physical-suspend test or new heavy validation was run. All 10 deferred YAMS artifacts
were later recovered byte-exactly without a service restart.
The separately authorized November 2 privileged/KPI follow-up hardened the harness
and built immutable actual-dirty-baseline/candidate images (14 harness tests passed
inside each). Five live fixtures passed; three dedicated BPF smokes failed and
remain unresolved, not attributed to the FE changes. Three alternating constrained
KPI pairs loaded 5,140 rules from the same three files. All six workloads completed
within 300 BOOTTIME seconds, but **all six failed the 2% idle-CPU gate**: baseline
3.16/5.42/2.12%, candidate 2.54/2.99/4.15%. Every run recorded zero dropped events
and a 0.007 ms exec-decision p95; maxima were approximately 1.01–1.04 s with
1–3 guard timeouts, so p95 is not a universal launch-latency bound. Maximum sampled
daemon RSS was 444.27–462.20 MiB baseline and 449.49–463.28 MiB candidate; these
are whole-daemon measurements without an acting model, not model inspection cost.
Sampler errors and the failed BPF predicates are retained. No thresholds were
relaxed or failed BPF tests rerun. The full container unit matrix was not rebuilt;
the earlier native functional/sanitizer/fuzz evidence remains separate. These
validation runs did not deploy the changes, establish the host-crash cause, or clear
the privileged/resource/real-data/model-promotion gates.
The subsequent CPU-cache repair distinguishes an oversized-image refusal from a
completed scan. Repeated cache hits no longer hash an unknown oversized image to
EOF, and a refusal cannot satisfy the exec guard as a clean YARA result. Local
regressions cover known/unknown hashes, inventory/live events and size boundaries;
52/52 native tests, six ASan/UBSan suite runs and 22 fuzz targets passed (10,253,990
executions, no crash artifacts or sanitizer diagnostics; leak detection disabled).
This does not clear Fold, live/KPI acceptance or post-rollout CPU measurements.
OpenSnitch's separately observed MD5 cost remains a distinct follow-up; disabling
checksums or weakening firewall rules is not part of this repair.
## Historical Status Snapshot
As of **April 27, 2026** (historical; later sections and the active-work summary
above supersede remaining-task lists from this snapshot):
- Phase 0 is complete: the repo configures and compiles again
- Phase 1 is complete: the current unit suite passes
- Phase 2 is complete in the current checkout: Strawberry persists indicators and graph data across reopen through vendored `third_party/unqlite-cpp` (replaced by the compressed event store on 2026-10-25; the submodule was removed)
- direct persistence tests now cover reopen, commit, rollback, and read-only behavior
- Phase 3 has started: optional libyara-backed rule loading and file scanning are implemented
- YARA hit metadata now persists through the database path, including multi-hit support for the same hash and process→indicator graph linkage
- `scan_process` now performs a best-effort `/proc/<pid>/exe` scan when libyara is available
- a Conan-backed libyara build and test pass succeeded on **April 25, 2026** on the current macOS host, including real rule compilation and file-scan assertions
- Linux-specific `/proc/<pid>/exe` behavior is now covered by a Docker-validated unit test on **April 26, 2026**
- a minimal procfs-polling process-event path now exists on **April 26, 2026**
- a reusable `DetectionPipeline` now exists, and a Docker-validated synthetic runtime test proves procfs event ingestion can drive executable scanning plus persisted indicator/graph storage on **April 26, 2026**
- a minimal libbpf exec-event backend now compiles in the Linux Docker build on **April 26, 2026**
- a privileged Linux Docker smoke run now validates live libbpf tracepoint attach and exec-event delivery on **April 26, 2026**
- a second privileged Linux Docker smoke run now validates live libbpf exec-event persistence through `DetectionPipeline` on **April 26, 2026**
- a reusable `StrawberryDaemon` wrapper now owns the daemon shell around config loading, process-monitor startup, and `DetectionPipeline`
- a daemon-level synthetic test now validates procfs event ingestion through `StrawberryDaemon` into persisted indicator/graph storage on **April 26, 2026**
- a third privileged Linux Docker smoke run now validates live libbpf exec-event persistence through `StrawberryDaemon` itself on **April 26, 2026**
- a new `IncidentReporter` now renders concise human-readable summaries over persisted graph state with first/last seen timing, and the main binary can print them with `--trace-process` / `--trace-node`
- the operator UI is now the Qt6 desktop app + tray indicator `strawberry-tray` (October 24, 2026); the temporary Python web evaluator was removed
- a synthetic demo-trace seed path exists (`make demo`), so the app/reporting flow can be evaluated without a running daemon
- the active direction remains **system improvements first** so real Linux process collection and correlation evidence quality lead the work
- a first minimal correlation engine now exists: Strawberry can group process-executable YARA indicators by process/hash and persist an explainable finding
- a second correlation rule now flags multiple matched descendants under one parent process lineage
- a third correlation rule now flags the same matched executable hash across multiple processes
- a fourth correlation rule now flags the same YARA rule across multiple processes
- findings now have JSON output through the CLI for UI/automation consumers
- a process search query surface now exists for PID, path, hash, YARA rule, and finding pivots, with JSON output for UI/automation consumers
- the correlation engine is intentionally small and rule-based; it is not yet a general subgraph matcher, anomaly detector, or full incident engine
- the current data model is still **process-centric**, not **thread-centric**; there is no first-class TID/thread telemetry or thread search UX yet
- procfs polling now has an optional startup inventory mode that emits already-running processes before continuing to watch for newly observed PIDs; this is the current path for collecting a fuller Linux process corpus for correlation refinement
- the libbpf path now has a best-effort fixed-width live argv preview (`argv0`/`argv1`); both the `/bin/echo` smoke and the staged pipeline smoke validated it on **April 26, 2026**
- the libbpf path now opportunistically attempts `sched_process_fork` for parent-PID enrichment, but the privileged container smoke confirms that this additional tracepoint may be denied while the main exec-event path still succeeds
- procfs-driven runtime events now also persist `parent_pid` and `event_timestamp_ns`, and create minimal `spawned` lineage edges on **April 26, 2026**
- matched runtime events now also persist executable-image file nodes with `executed` and `content_matched` provenance edges on **April 26, 2026**
- a thin GNU Make wrapper now exists for local `make setup/build/test/run`
- a root Conan 2 recipe and air-gap workflow doc now exist for offline dependency installs
- a seeded local `yara/4.5.5` Conan recipe now exists under `conan/local_recipes_index/`
- a Linux-oriented root `Dockerfile` now exists for build/test validation with libyara enabled
- the Docker image now defaults to `gcc`/`g++` because the Ubuntu 24.04 container path was the cleanest working route for C++23 + vendored `unqlite-cpp`
Host-readiness hardening (**October 24, 2026**):
- the BPF object now builds on Arch (UAPI `__u32/__u64` types instead of glibc `<stdint.h>`); previously it silently failed there and the daemon fell back to procfs polling
- `execveat` tracepoints are attached, so `fexecve` and memfd (fileless) executions are observed
- events resolve through `/proc/<pid>/exe` and scan/hash that magic link, so relative paths, replaced files, deleted binaries, memfd images and other mount namespaces are handled
- `MemfdExecution` / `DeletedExecutable` heuristic indicators feed per-process correlation
- process nodes use a stable identity (`process_<pid>_<boot8>_<starttime>`), so PID reuse no longer merges processes
- the BPF backend now also emits the startup inventory, and its timestamps are converted to wall clock
- the daemon fails closed when no rules compile (`require_rules=true`) or the DB cannot be opened for writing
- database: single-writer `flock`, read-only CLI/UI queries (no lost updates), transactional per-event writes, batched correlation, and a scan-result cache
- rule severity comes from `meta.severity` (default `medium`); starter rules were replaced with a small, high-signal set
- systemd unit: capability-bounded, `NoNewPrivileges`, read-only `/home` and shared `/tmp` (so binaries there can still be scanned), private state dir
- (the web UI server hardened in this pass was later replaced by the desktop app; see Phase 8)
Next for host use: worker-pool scanning off the ring-buffer thread with drop counters, fuller argv capture, on-demand memory scans for flagged processes, file-write/network/module telemetry, structured alert output, and an allowlist.
## Phase 0 — Make the repo buildable ✅
### Goals
- repair build metadata
- eliminate obvious compile blockers
- get the project configuring cleanly
### Delivered
1. Replaced invalid `meson.options` usage with valid Meson option handling.
2. Fixed the `DaemonStateMachine` type definition problem in `include/strawberry/daemon_states.hpp`.
3. Removed compile-time migration fallout and stale MongoDB-era references from the active build path.
4. Re-ran configure and compile until the build graph was stable.
### Exit Criteria
- `meson setup builddir -Denable_tests=true` succeeds
- `meson compile -C builddir` succeeds
## Phase 1 — Repair and modernize tests ✅
### Goals
- align tests with current public interfaces
- establish a trustworthy regression baseline
### Delivered
1. Updated `tests/test_config.cpp` to use `db_path` and current config defaults.
2. Kept `HashUtils` tests compiling and passing.
3. Added database-focused tests for:
- indicator round-trip
- graph round-trip
- persistence across reopen
### Exit Criteria
- `meson test -C builddir --verbose` passes for the current unit suite
## Phase 2 — Real embedded persistence ✅
### Goals
- turn the previous in-memory `Database` stub into real local persistence
- keep the public interface stable
### Delivered
1. Integrated `git@github.com:trvon/unqlite-cpp.git` behind `src/database.cpp`.
- vendored location: `third_party/unqlite-cpp`
2. Persisted:
- `IndicatorEntry`
- `GraphNode`
- `GraphEdge`
3. Preserved XDG-friendly defaults.
4. Added direct persistence tests under `/tmp` for:
- reopen
- commit
- rollback
- read-only enforcement
### Exit Criteria
- indicators survive process restart
- graph nodes and edges survive reopen
- query helpers work against persisted data
## Phase 3 — Real YARA integration 🟡
### Goals
- replace placeholder load/scan logic with actual libyara-backed behavior
### Delivered so far
1. Implemented rule compilation/loading from a file or directory when Strawberry is built with libyara.
2. Implemented file scanning with real match extraction when libyara is available.
3. Implemented a best-effort `scan_process` bridge that scans `/proc/<pid>/exe` rather than process memory.
4. Added YARA-focused tests that:
- exercise graceful fallback when libyara is absent
- contain real scan assertions for libyara-enabled builds
- now include a Linux-only `/proc/<pid>/exe` smoke test for `scan_process`
5. Persisted YARA hit metadata through the database path and added graph linkage tests for process→indicator relationships.
6. Validated the real libyara file-scan path through the Conan-backed dependency flow and fixed a Conan/Meson C++ standard mismatch by forcing a C++23 consumer toolchain.
7. Validated the Linux `scan_process` path in the Docker Ubuntu 24.04 test image on **April 26, 2026**.
8. Added a reusable `DetectionPipeline` and a synthetic runtime integration test that proves procfs event ingestion can trigger YARA matching and persisted indicator/graph storage on **April 26, 2026**.
### Remaining Tasks
1. Thread the persisted YARA hit path through more realistic end-to-end daemon/runtime flows.
2. If feasible, add true process-memory scanning as a guarded Linux-only feature.
3. Record at least one libyara-enabled synthetic end-to-end scan/store run on Linux.
4. Start the minimal `libbpf`/eBPF exec-event ingestion path.
### Exit Criteria
- a temp rule set can be loaded on a libyara-enabled host
- a synthetic malicious sample triggers a stored match
- YARA hit metadata is persisted alongside related process/graph context
- at least one libyara-enabled automated test pass is recorded
## Phase 4 — Real eBPF process telemetry
### Goals
- replace placeholder monitoring logic with actual runtime event capture
### Delivered so far
1. Replaced the empty `ProcessMonitor` loop with a minimal procfs-polling backend that emits new-process callbacks.
2. Captured per-process basics from procfs:
- pid
- executable path
- SHA-256 hash of the executable
- command line
3. Added a synthetic procfs unit test for new-process detection and callback delivery.
4. Added a minimal libbpf-backed exec-event build path:
- BPF object source for `sys_enter_execve` / `sys_exit_execve`
- user-space libbpf loader and ring-buffer handling
- automatic fallback to procfs polling when libbpf is unavailable or attach/load fails
5. Validated in the Linux Docker build that the eBPF object compiles cleanly on Ubuntu 24.04.
6. Added a privileged Docker live smoke harness and validated one real `/bin/echo` exec event reaches user space.
This is now a useful Linux-oriented bridge with both a real privileged Docker smoke pass and a real privileged Docker libbpf-to-store smoke pass, but it is **not yet a richer daemon-grade runtime ingestion pipeline**.
### Tasks
1. Fold the live libbpf-to-store harness behavior into a daemon/runtime harness, not just standalone smoke executables. ✅
2. Capture at least:
- pid
- parent pid if available
- executable path
- timestamp
3. Improve event context beyond the current executable path + `comm` bridge in the live libbpf path (for example, parent PID and richer argv handling if feasible).
4. Keep a Linux-only smoke path in Docker/CI for the live telemetry path.
### Exit Criteria
- a real Linux exec event reaches user space
- Strawberry creates at least one persisted process node and relationship edge from live telemetry
## Phase 5 — Basic provenance graph usefulness
### Goals
- make the graph layer operationally useful, not just present
### Tasks
1. Record process lineage edges.
2. Optionally record executable file edges.
3. Add simple queries:
- related nodes from a process
- all matches connected to a lineage
4. Add a concise CLI or log output for incident trace summaries. ✅
### Exit Criteria
- one suspicious event can be traced through stored relationships in a human-readable way
## Phase 6 — Packaging and automation surfaces
### Goals
- add convenience surfaces only after the core path is real
### Tasks
1. Keep the GNU Make wrapper aligned with Meson.
2. Keep the Conan/offline workflow aligned with real dependency usage, including the local YARA recipe.
3. Keep the Linux Docker image aligned with the real dependency and test path.
4. Clean up install/service scripts as runtime features become real.
5. If the target is **Make.com**, define a small stable CLI contract first, then automate it.
6. Document Linux-only runtime expectations clearly.
### Exit Criteria
- operators can build, test, and run the scanner through one stable interface
- external automation does not depend on undocumented internal behavior
## Phase 7 — Detection correlation engine
### Goals
- move from stored evidence to ranked, explainable findings
- correlate multiple related events into one analyst-facing detection object
### Delivered so far
1. Added `FindingRecord` persistence/query support to `Database`.
2. Added `CorrelationEngine` with a deterministic `process_executable_yara_match` rule.
3. Added `lineage_multiple_yara_matches` for parent processes with multiple matched descendants.
4. Added `same_hash_multiple_process_matches` for repeated matched executable hashes across processes.
5. Added `same_yara_rule_multiple_processes` for repeated YARA rule hits across processes.
6. Added CLI surfaces:
- `--correlate`
- `--list-findings`
- `--show-finding <id>`
- `--json` for structured finding output
6. Updated the demo seed path to emit a correlated finding.
7. Extended the minimal local web GUI with finding list/show endpoints and controls.
8. Added unit coverage proving multiple related process YARA hits, lineage hits, and same-hash process reuse collapse into deterministic findings.
### Tasks
1. Extend the minimal correlation input model over existing persisted data:
- process nodes
- file nodes
- indicator nodes
- `spawned`, `executed`, `matched`, and `content_matched` edges
- timestamps, hashes, paths, command lines, and parent PID where available
2. Add more small rule/heuristic detections before ML:
- suspicious executable matched by YARA and executed by a short-lived process
- repeated matched descendants in the same lineage window
- one file image tied to multiple suspicious process executions
- process lineage containing multiple matched artifacts in a bounded time range
3. Enrich durable finding records with:
- finding ID
- severity or score
- reason/explanation
- related node IDs
- related indicator IDs
- first/last seen timestamps
4. Expand CLI and programmatic queries for:
- list findings
- show one finding
- pivot from finding → process lineage → indicators
5. Add tests proving that additional raw event families collapse into one correlated finding when appropriate.
### Exit Criteria
- Strawberry can generate at least one explainable finding from multiple persisted graph artifacts ✅
- the finding can be queried independently of the raw node/edge dump ✅
- at least two distinct correlation rules exist ✅
## Phase 8 — Analyst UX and search
### Goals
- move from a debug-style trace viewer to a real operator workflow
- make process-centric investigation fast and explainable
- a native desktop app + tray indicator as the operator surface, with no network listener
- agent-facing access (CLI today, MCP next) over the same read-only evidence and fixed control verbs
### Delivered so far
1. Added `ProcessQuery` for process-centric pivots over graph nodes, indicators, and findings.
2. Added CLI support:
- `--search-process <query>`
- `--search-process <query> --json`
3. Added local web UI process search backed by structured JSON.
4. Added parent/child lineage summaries to process search results.
5. Added first/last seen evidence timestamps to process search results.
6. Added local web UI deep links from process search results to trace and finding views.
7. Added local web UI finding cards that pivot from findings back to process traces and process search.
8. Added related file/indicator summaries to process search cards, making the local evaluator a lightweight process detail panel.
9. Added client-side sort/filter controls for process search and finding cards.
10. Added operator-friendly ns/us/ms/s duration formatting for trace, finding, and process-search timestamps.
11. Added local-web copy/download affordances for rendered traces, process-search results, and findings.
12. Split process-search cards into structured identity, timing, lineage, related-file, related-indicator, and finding sections.
13. Added explicit UI server host/port controls, with safe loopback-only default and `--allow-remote` required for non-loopback binds.
14. Documented the Python UI server as a temporary evaluator and the target C++ Boost.Beast/Asio API service direction.
15. Added `API_CONTRACT.md` to pin the current web JSON contract before porting endpoints into C++.
16. Added `/api/process` for exact process-detail JSON by PID or process node ID.
17. Wired the local web UI to `/api/process` with a focused process-detail panel and detail pivots from search/finding results.
18. Added a loopback web API contract test for health/defaults/trace/findings/process search/process detail against seeded demo data.
19. Added `WEB_DEPLOYMENT.md` covering local, SSH-tunnel, container/lab-bind, and reverse-proxy prototype modes.
20. Added unit coverage for search by PID, path, hash, YARA rule, finding metadata, lineage summaries, and process seen timestamps.
21. **October 24, 2026: replaced the web evaluator (items 3, 6–9, 11–19) with a Qt6 desktop app + tray indicator** (`app/`, `strawberry-tray`):
- Findings, Processes and Diagnostics tabs; tray colour by scanner health and unseen findings; desktop notifications for new high/critical findings.
- Daemon `ControlServer` Unix socket (`status`, `reload-rules`, `rescan`; group `strawberry`; peer-credential checked; rescan rate-limited), plus `strawberry --status/--reload-rules/--rescan`.
- The service runs with `Group=strawberry`, so evidence is `0640` group-readable; the app reads it via `Database::open_read_only()`.
- Tests: `tests/test_control_server.cpp`, `app/tests/test_tray_state.cpp`, pipeline stats/rescan coverage.
- Removed `scripts/strawberry_ui_server.py`, `ui/`, `scripts/test_web_api_contract.py`, `WEB_DEPLOYMENT.md`; `API_CONTRACT.md` now documents the socket protocol and CLI JSON.
### Tasks
1. ✅ Agent-facing interface (October 24, 2026): `strawberry-mcp` (MCP over stdio) with status, findings, process search/detail, fd-passing `scan_paths`, reload and rescan; `--read-only` mode; `strawberry --scan-file`. Next: optionally record on-demand scan hits as evidence, and an MCP resource/notification for new findings.
2. ✅ (October 24, 2026) Auto-updating YARA rule feeds (YARA Forge core by default, digest-verified, compile-checked, atomic switch, sandboxed updater with daily timer + "Update now"), allowlist (hash/path/rule) with suppression in the pipeline, Rules/Allowlist tabs in the app, read-only `strawberry_get_policy` MCP tool; process scans from bounded memory copies; exec events via a bounded worker queue with drop counters.
3. Expand the process-first views as evidence grows (argv, file writes, network).
3. Defer thread-level pivots until Strawberry actually captures TID/thread telemetry.
4. Package the app (desktop entry/autostart) for more desktops; keep it read-only plus fixed verbs.
### Exit Criteria
- an operator can start from a process or finding and pivot across related evidence without using raw DB internals ✅
- scanner health and new high-severity findings are visible without opening a terminal ✅
- operator controls never require root or a network listener ✅
## Phase 9 — Protection and response (October 25, 2026)
Delivered (all unit/integration tested; root-only paths validated in a privileged container):
- **M1 threat-intel feeds**: list feed kinds (`hashes`, `domains`, `ips`, `nets`) with validation, caps and a shrink guard; default CERT-PL, Spamhaus DROP, Feodo C2 and malware-hash feeds; `IOC_KnownMalwareHash`; YARA `meta.score` → severity
- **M2 OpenSnitch**: blocklist rules from list feeds; automatic per-binary containment at ≥ `contain_min_severity` (hash-pinned with `EnableChecksums`); Response tab with undo
- **M3 exec prevention**: fanotify `FAN_OPEN_EXEC_PERM` guard, `off`/`audit`/`enforce`, fail-open deadline, verdict cache, system paths never denied
- **M4 behavior telemetry**: eBPF connect/ptrace/module events, fanotify persistence/drop watcher, six behavior rules
- **M5 memory scan + response**: `yr_rules_scan_proc` memory scans (automatic for memfd/deleted/ptrace targets, or on request); `quarantine` (blocks execution in every mode + network); `process-suspend/resume/kill` with PID/start-time re-verification and protected-process refusals
Open items:
- host validation with OpenSnitch hash pinning on, prevention in `audit`, then `enforce`
- behavior rules need real-host tuning (false-positive review of persistence writes and connects)
- network events are kept in memory only and persisted when relevant; no long-term network history
- no thread/TID telemetry yet
## Phase 10 — Resource governor and profiling (October 25, 2026)
Delivered: always-on timing histograms (status `timings`), optional Tracy
build, constrained-lane KPI (`make perf-kpi`), `ResourceGovernor` (PSI /
loadavg / MemAvailable / cgroup v2, levels with hysteresis, deferral with a
starvation guard, read pacing), scan profiles, shared guard/pipeline scan
cache, scans outside the pipeline lock, no hashing on the eBPF thread, paced
inventory queue, governed memory scans, rule-reload backoff, unchanged
OpenSnitch lists not rewritten, per-thread nice/ioprio, systemd limits.
Open: database whole-file rewrite per commit and O(F²) finding index (measure
with `timings.db_commit` / `correlation` on a long-running host first); fuzz
and invariant tests for parsers and decision logic.
## Immediate Next Actions
These are the best next steps now:
1. validate startup process inventory on Linux with `STRAWBERRY_PROCESS_MONITOR_BACKEND=procfs` and `scan_existing_processes_on_start=true`
2. feed that fuller process corpus through `DetectionPipeline` so correlation work is based on real host-wide evidence rather than GUI demo data
3. improve runtime enrichment that matters for correlation: stable executable identity, parent/child lineage, timestamps, command-line capture, and exit/short-lived-process handling
4. keep improving libbpf enrichment in parallel, especially parent-PID coverage and a hybrid BPF-events-plus-procfs-inventory mode when feasible
5. add or refine **correlation rules** only when the collected evidence model supports them cleanly
6. keep desktop-app work limited to what new evidence needs; next interface work is agent access (MCP/CLI)
Note: event timestamps are wall-clock nanoseconds (eBPF boot-time stamps are converted at ingestion); synthetic demo data may still use small raw values.
## Suggested Next Deliverable
The next meaningful deliverable should be:
- buildable repository
- passing unit tests
- persisted local indicators and graph data
- real YARA file scanning on a libyara-enabled host
- privileged Docker-validated live libbpf exec-event delivery
- one synthetic end-to-end daemon-owned scan stored in the database
- one privileged Docker-validated live daemon-owned scan stored in the database
- one Linux procfs startup inventory pass that collects already-running processes into the same pipeline
- one minimal explainable correlated finding generated from the stored graph/evidence path
- one process-first CLI/query surface that can pivot from a finding into lineage and related artifacts
That gives Strawberry a real detection and investigation spine before deeper eBPF, provenance, and desktop UX work.