Four findings from a four-lane audit made the promotion gate unable to support
its own claims.
In-sample evaluation: training walks the whole post-warm-up store and the gate
evaluates the same windows, with nothing checking provenance - replay.cpp itself
listed point_in_time_splits_and_overlap_not_assessed. Models now record the
window range they learned from (file format v2; v1 still loads as unknown) and a
content fingerprint, evaluation reports its own range plus in_sample, and
promotion_gate fails with the reason: an overlapping range, no recorded range,
or no digest. Unprovable provenance never counts as out-of-sample.
Degenerate baseline: with rarity-greedy on 0 hits, hits_per_cpu_second() returns
0.0, so ">= 1.5x rarity" reduced to "model hit something" and the recall check
was vacuous. A cell whose baseline scored no hits or spent no CPU now fails,
naming the budget refusals that starved it.
Stale models: cells keyed on a bare filename, so a later cycle that refused to
train left an older cycle's dist/gate models to be judged, and nothing bound a
verdict to the file it judged. Cells and reports carry the model digest, and
fleet gate sets refused (non-zero exit) for a cell without one or for a
(model, period) whose digests disagree across nodes instead of pooling them.
3 seeds are not generalization: they vary initialisation and the per-node
shuffle over identical windows. The procedure document now says so and states
what would test generalization (disjoint periods, models trained on an earlier
cut, out-of-sample ranges).
Tests: in-sample flagged, disjoint range not flagged, unprovable provenance
fails, 0-hit and 0-CPU baselines fail, mixed digests refused, legacy report
refused, range round-trip and fingerprint, v1 compatibility.
Nothing runs in the daemon yet; these are the pieces the next increments
train and run in shadow.
- learn/features: a WindowSummary accumulated from a window's events
(exec locations, memfd/deleted/uid0 execs, depth, connection classes
and ports, persistence and drop writes, ptrace, module loads, scan
outcomes, capped surprisal per tuple kind, bounded tokens) and
featurize(): 48 explicit features scaled to about [0, 1] plus signed
hashing of tokens into 208 dims, with the governor level and budget
left as context. A window's own detections are never features: they
are the teacher labels.
- learn/mlp: 256 -> 128 -> 64 ReLU with a policy head over the
inspection actions (none, memory_scan, deep_rescan, watch_subtree,
rescan_on_next_rules) and a value head. Scalar reference, AVX2+FMA
chosen at runtime, NEON on aarch64; safe default (no inspection) on a
wrong input size or an inconsistent model. Deterministic init per seed.
- learn/model_file: "SBMD" format with bounded dimensions, exact length,
a sha256 trailer and finite weights checked on load, each failure
named; atomic save into a 0700 directory.
Tests: test_model (kernels against scalar for arbitrary sizes, forward
pass invariants, deterministic init, round-trip, every flipped byte and
truncation rejected, non-finite weights rejected, features bounded and
deterministic). fuzz_model_file. NEON is built and tested only on aarch64.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>