- doctor (control verb, read-only): store health and disk floor,
telemetry growth per day and days until the floor when retention is
unlimited, leftover store files, superseded findings, retro-hunt state,
thread/write errors, logs (journald rotates them), model (none yet).
Each check carries a maintenance action or advice. Pure checks over
gathered inputs (src/doctor.cpp), so they test without a daemon.
- maintain (control payload verb, privileged, rate limited, audited):
retire-superseded-findings and clean-store, each with dry_run.
- finding_retired evidence record: removes a finding from view; it stays
in the stream's history. CorrelationEngine::superseded_findings finds
old hash/rule findings by recomputing the stable id, and old lineage
findings only when the current one's parent process is among their
nodes (a reused pid on another boot is not a match). Lineage findings
now record parent_process_node_id.
- store::inspect_store_files / clean_store_leftovers: orphan/ files older
than N days and *.compact / MANIFEST.tmp temp files; never a listed
segment.
Tests: test_doctor (check statuses, superseded detection including pid
reuse and no-replacement cases, retirement surviving a reopen, cleanup
keeping recent orphans and listed segments); daemon test exercises both
verbs over the socket. fuzz_evidence_records covers finding_retired.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The main source of teacher labels over time: rules improve daily, while
most malware runs once.
- The pipeline remembers every binary that ran (image_seen evidence
record: sha256, path, first/last seen, size; at most one write per hash
a day). Deleted and memfd images are skipped: nothing to find again.
- Each rules load computes a digest of the rule files and IOC hash lists
(paths after resolving symlinks, sizes, mtimes).
- RetroHunter (src/retro_hunt.cpp, own thread, idle priority, paced at
retro_hunt_pace_ms, deferred by the governor as WorkClass::retro_hunt)
re-scans when the digest differs from the last completed pass, most
recently seen first. Only paths that still hold the same content are
scanned (O_NOFOLLOW, regular files, hash compared); others count as
changed or missing. A completed pass is recorded (retro_hunt record) so
a restart does not repeat it; an interrupted one starts over.
- New matches (rules not recorded before for that hash, not allowlisted)
become indicators with scan_kind retro_hunt, a file/indicator graph
link, a positive label and a notification. Correlation turns them into
findings titled "YARA match on a binary that ran here (found after a
rules update)". Nothing is blocked or contained from here.
- Replay reads positive labels from the evidence stream and marks windows
that ran a binary labelled bad later (LATE in the CLI).
- Status: a retro_hunt block (API_CONTRACT.md); config: retro_hunt,
retro_hunt_pace_ms.
Tests: test_retro_hunt (remembered once; passes only on a digest change
and survive a restart; a clean-at-exec binary matching a new rule becomes
an indicator, label, finding and notification; a replaced binary is
counted as changed, never scanned as the old one; a stop abandons a pass
unrecorded); test_replay R9 (late labels, from a real store).
fuzz_evidence_records covers the new records.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zstd block codec (CRC32C, varint tagged records), segment files with
sealed footers and block index, crash-atomic MANIFEST, LogStore
(lock, recovery, rolls, orphaning, heal with backoff), the evidence
view, and the legacy unqlite store kept for import. Property tests
under injected faults and crashes, a differential test against the old
store, and fuzz targets for every decoder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>