Commit graph

1 commit

Author SHA1 Message Date
trvon
a72016c546
bench: wire gold precision/recall gates into fold/yams manifests
Round 3, iteration 1 of the fold optimization loop
(docs/research/optimization-loop-2026-07-30.md).

The foundation audit found gold scoring machinery (26 oracle files in
bench/gold/, fold_eval/scoring.py, gold_threshold_failures gates)
fully implemented but wired into zero project manifests — the eval
measured finding drift, never precision/recall.

New gold standards anchored on the reviewed dispositions:
- bench/gold/fold_self_gold.json: 9 cases (5 SafetyObserver stub
  clones, 1 detector-internal clone, 1 loop_inventory clone, 2
  sql-injection sites reviewed false-positive-but-intentionally-
  reported)
- bench/gold/yams_self_gold.json: 7 cases (3 clone, 4 high-complexity)

Both manifests now enforce gold_thresholds precision 1.0 / recall 1.0:
any detector drift that loses a reviewed finding (recall drop) or
emits a new unmatched finding (precision drop) fails the evaluation.

Proof runs: fold TP=9/FP=0/FN=0, yams TP=7/FP=0/FN=0, precision 1.0,
recall 1.0, fingerprints exactly preserved (0fec78e5.../247523c5...),
full suite 137/137. Gold files live in bench/gold/ alongside the
existing oracles (the tests/benchmarks schema-test glob correctly does
not sweep that directory).
2026-08-31 17:49:43 -06:00