Commit graph

4 commits

Author SHA1 Message Date
trvon
728b2462e7
Governor: background work no longer starves under sustained pressure
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / macos (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
On a host that sits at critical/emergency for a long time (its own apps holding
most of RAM, or the daemon slightly over its soft budget), the starvation guard
was disabled (`!hard`), so correlation, retro-hunt and inventory deferred
forever — observed deferred 153 min with forced=0, which also starves the
retro-hunt delayed labels that Stage 4 training needs.

- Tiered starvation guard: it now fires under critical/emergency too, but after a
  level-scaled window (1x warning, 4x critical, 12x emergency) so background work
  is rare under load, not zero. A true host-memory crunch (host_severe_,
  MemAvailable below the severe floor) still holds everything, since forcing work
  then could tip the host toward OOM. exec guard/scan paths unchanged.
- Raise the balanced own-memory soft budget 512 -> 768 MiB: the daemon's steady
  footprint grew (telemetry buffers, the ingest server, the shadow model), so 512
  left it permanently over budget -> pinned critical/emergency. 768 fits the real
  baseline, still well under the 1.5 GiB cgroup cap.

Tests updated for the 768 budget and the tiered guard (critical forces after the
4x window; severe holds it). Suite 51/51; analyzer + clang-tidy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-28 22:33:25 -06:00
trvon
713122f845
Read process facts on macOS without procfs
include/strawberry/platform_process.hpp answers what /proc answers on
Linux: running PIDs (proc_listallpids), executable path
(proc_pidpath), parent and start time (sysctl KERN_PROC_PID, readable
for every process), argv (KERN_PROCARGS2) and a boot id
(kern.bootsessionuuid). A process is identified by its start time in
microseconds since the epoch plus the boot id, so node ids survive PID
reuse as on Linux. It is header-only, so the per-test source lists stay
as they are.

Users on macOS: resolve_process_ref (default root), the pipeline's
image hashing and exec-decision actor, and YaraScanner::scan_process
(scans the image by path). A synthetic procfs root still takes the
procfs code, so the existing procfs tests run on macOS too.

The resource governor samples Mach/sysctl on macOS: own CPU from
getrusage, RSS from task_info, load from getloadavg, available memory
from host_statistics64 (free + inactive + purgeable + speculative).

The behavior engine knows macOS debuggers (debugserver, dtrace) and
kext loaders (kernelmanagerd, kmutil), and macOS temp and Downloads
paths count as drop locations. The Linux lists are unchanged.

The KERN_PROCARGS2 parser is portable: a property test (exact argv
recovery, truncation never invents data, total on junk) and the
fuzz_procargs target run on every platform. test_platform_process
checks the live helpers against this process; the daemon test reads
the stopped state through sysctl on macOS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 09:55:09 -06:00
trvon
f1b948df24
Keep exec scanning alive under memory pressure; summarize repeats
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
The KPI detect phase still dropped events after the correlation fix.
Cause: every detection kept ~12 KB of evidence in memory, 20k detections
pushed RSS past the 512 MB budget, the governor went to emergency, and in
emergency every live exec scan waited 5 s, so the queue overflowed and
stayed full (memory that is evidence never comes back by waiting).

- The governor holds live exec scans for at most 5 s per emergency
  episode, then lets them run (counted as forced); background work still
  waits.
- Repeated detections of the same image and rules: the first 20 per
  10-minute window are stored in full, then one summary indicator per
  key keeps the count and first/last pid and time (rewritten at most
  every 30 s). All detections are counted; alerts repeat at most once a
  minute per key. Status reports pipeline.repeats_aggregated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 07:32:30 -06:00
trvon
2d5e05e8db
Add resource governor, scan profiles and KPI harness
PSI/cgroup-aware governor with efficient/balanced/aggressive profiles,
background pacing and thread priorities; timing histograms and optional
Tracy zones; a capped Docker KPI harness with report/compare tooling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:17:16 -06:00