Backlog #7. Repro showed chromium finding LOCATIONS (rule/uri/line/
col/id) were byte-identical across runs while the summary
finding_fingerprint drifted (064e60c9 -> 4573bb71): summarize_sarif_findings
hashed the full canonical finding record, which carries Z3-timing-
volatile fields (quality metrics, uncertainty reasons, confidence,
score). The stability gate already used identity-only keys, so the
fingerprint contradicted it on Z3-heavy corpora.
finding_identity_key now = {finding_id, rule_id, uri, line, backend}
(measurement artifacts excluded), and the fingerprint is the sha256
of the sorted identity keys — consistent with how change-time drift
and dispositions already define 'a finding'.
Verified: chromium fingerprint STABLE across identical runs
(f5629f0f). Repinned the four baselines to the identity-based
fingerprint (fold 6fe221a0, yams c9ad7d60, sage 3da6ecfa, chromium
f5629f0f).
Backlog #5 (operator-confirmed). chromium_source.json and
sage_rust.json were validated but unwired. This adds:
- reviewed dispositions (5 parallel review subagents): chromium 11
(10 FP/1 accepted-risk — deliberate modular arithmetic in
base/file_path.cc); sage 2714 (9 actionable uncontained path-join
reads in skill install/daemon library/model-downloader; 890 FP;
1815 accepted-risk quality bulk)
- wall/RSS baselines for both corpora
Measured both corpora produce finding drift run-to-run (chromium Z3
fingerprint drifts; sage +5 findings) and chromium's wall swings up
to ~2x under load — so they are wired as STRESS/DIVERSITY corpora:
a new additive evaluation.max_new_findings override lets a manifest
declare its finding-drift tolerance (CLI default unchanged for
fold/yams), allow_finding_drift is on, and chromium's wall/RSS
thresholds are relaxed. Final: both evals pass.
Also fixes a pre-existing /tmp-literal default (now platform tempdir)
in run_project_eval.py.
Round 4, iteration 2 of the fold optimization loop
(docs/research/optimization-loop-2026-07-30.md).
The yams wall-regression gate at 20% fired on a 113% concurrent-load
swing (8.1s -> 17.3s on identical code) and passed quiet at -11% —
with measured identical-code walls of 7.2-11.4s (quiet) and 17.3s
(loaded), a 20% threshold on a ~9s workload is noise-dominated and
flaky. A flaky gate is worthless as a regression oracle.
Calibrated to 50%: fires above 12.15s sustained (50% over the 8.10s
baseline) — catches true 2x regressions while tolerating the measured
40-113% machine swings. Validated: yams eval ok (7.37s vs 8.104s
baseline, -9%), gold precision/recall 1.0, full suite 137/137.
Fingerprints exactly preserved.
Round 3, iteration 2 of the fold optimization loop
(docs/research/optimization-loop-2026-07-30.md).
The foundation audit found both manifests declaring
wall_regression_pct/rss_regression_pct with no baseline artifacts —
the regression gates never engaged in the entire campaign.
New: tests/benchmarks/projects/baselines/{fold_source,yams_source}.json
(full prior-eval summaries with matching tu-set/config fingerprints
and exact finding fingerprints). Both manifests now reference their
baselines, activating the eval's baseline-compare path
(compare_performance_lanes + rss_regression).
Engage proof:
- fold: baseline_comparison engaged (paired-by-repetition, baseline
103.65s vs candidate 82.7s, -20.2%), status ok
- yams: gate FIRED on a 113% concurrent-load swing (proving it
discriminates), then passed quiet (7.181s vs 8.104s)
Also extends the schema-test prune to {runs, baselines} — third
instance of the same glob-sweep fragility the foundation audit
predicted. Full suite 137/137; fingerprints exactly preserved
(0fec78e5.../247523c5...); gold precision/recall 1.0 both.
Round 3, iteration 1 of the fold optimization loop
(docs/research/optimization-loop-2026-07-30.md).
The foundation audit found gold scoring machinery (26 oracle files in
bench/gold/, fold_eval/scoring.py, gold_threshold_failures gates)
fully implemented but wired into zero project manifests — the eval
measured finding drift, never precision/recall.
New gold standards anchored on the reviewed dispositions:
- bench/gold/fold_self_gold.json: 9 cases (5 SafetyObserver stub
clones, 1 detector-internal clone, 1 loop_inventory clone, 2
sql-injection sites reviewed false-positive-but-intentionally-
reported)
- bench/gold/yams_self_gold.json: 7 cases (3 clone, 4 high-complexity)
Both manifests now enforce gold_thresholds precision 1.0 / recall 1.0:
any detector drift that loses a reviewed finding (recall drop) or
emits a new unmatched finding (precision drop) fails the evaluation.
Proof runs: fold TP=9/FP=0/FN=0, yams TP=7/FP=0/FN=0, precision 1.0,
recall 1.0, fingerprints exactly preserved (0fec78e5.../247523c5...),
full suite 137/137. Gold files live in bench/gold/ alongside the
existing oracles (the tests/benchmarks schema-test glob correctly does
not sweep that directory).
Iteration 9 of the fold optimization loop
(docs/research/optimization-loop-2026-07-30.md).
Iteration 9 targeted the control-walk condition-fact cost (35.1s of
the sql-injection profile). Measurement-first: a scratch patch (never
committed) counted duplicate feasibility queries in a live scan —
1314 total, 1268 distinct (3.5% duplicate) — so a memo cache caps out
at ~1.2s of 35.1s, far below the >=20% KPI. The checks are genuine,
distinct SMT solving; memoization is a measured dead end. Scratch
patch fully reverted; production byte-identical.
Viable future directions recorded for the next campaign: fast-path
widening beyond affine parameter comparisons, incremental solving over
the pushed path stack, dual-walk (CFG/AST fallback) elimination.
This commit folds the formatting normalization applied to two
campaign-authored files after their initial commit.
Iteration 5 of the fold optimization loop
(docs/research/optimization-loop-2026-07-30.md).
The load-sensitivity flake in symbolic summary recomputation
(observed once: empty return_post in a 7/10-run burst under CPU
saturation) did not reproduce in three bounded rig attempts:
- 8 subshell busy-loops + 10 filtered runs: 10/10 PASS
- 8 verified 'yes' workers + 10 full runs: 10/10 PASS
- concurrent real fold sql-injection+uninit-read scan (-j 4)
+ 10 full runs: 10/10 PASS
No defect reproduced, so no fix ships; the rig is committed as the
standing recipe with the suspect documented (isValidPredicate's 100ms
Z3 solver timeout in summary_encoder.cpp — check() returning unknown
under CPU starvation flips predicate-record decisions). Production
code byte-identical.
Iteration 4 of the fold optimization loop
(docs/research/optimization-loop-2026-07-30.md).
BenchmarkSchemaTest claimed to validate checked-in manifests but
globbed tests/benchmarks/**/*.json, sweeping the designated runtime
output directory tests/benchmarks/runs/ (optimization-loop harness
artifacts) and failing 32 assertions + 5 errors on every full-suite
run once run artifacts existed.
Fix: prune runs/ from the glob; add tests/benchmarks/runs/ to
.gitignore so run artifacts are never accidentally committed.
fold:benchmark_evaluation green; full suite 137/137 with zero
failures. No production code change.
Iteration 2 of the fold optimization loop
(docs/research/optimization-loop-2026-07-30.md).
The disposition baselines predated source edits; finding IDs encode
location, so line moves surfaced as new+resolved pairs and both
project evaluations gated red:
- fold: 8 new findings exceeded limit 0 (review coverage 0.111)
- yams: 3 new findings exceeded limit 0
Each finding was verified against source before re-baselining:
- fold: 5 SafetyObserver virtual-stub clones (line-moved, same
methods) accepted-risk; 2 detector-internal clone sites re-hashed,
accepted-risk; 2 sql-injection sites re-verified — all SQL text is
compile-time constants or fixed internal table lists (store.cpp:22,
:251) — false-positive
- yams: 3 clone IDs refreshed after line moves (document_metadata.h
600/623, connection_pool.cpp 42); 4 high-complexity locations
updated
Also fixes the disposition target fingerprint to the exact
runner-reproduced configuration fingerprint (manifest_configuration +
SOURCE_SETTING_FLAGS defaults), which the prior fold baseline had
stale.
Green proof (tests/benchmarks/runs/optimization-2026-07-31-dispositions/):
fold eval status failed -> ok, yams failed -> ok; finding fingerprints
unchanged (fold 0fec78e5..., yams 247523c5...); churn 0, pairwise
Jaccard 1.0, review_coverage 1.0. No gate weakened and no production
code change.
Unit + integration coverage for the session's work: incremental project
graph/index, pcode encoder, unreachable/dead-branch, equiv/cross-TU,
div-by-zero/sign-conversion/int-truncation, use-after-free, quality-precision,
scan-router profile, source TU runner, detector plan, and the eval fixtures.
- int-overflow: unsigned wrap is defined, not UB — split to
fold.unsigned-wrap so signed-UB findings aren't drowned; on by
default (--report-unsigned-wrap=false opts out)
- use-after-free: model return/throw as path-terminating so
`delete p; return;` in an error branch can't poison the fall-through
- oob-access: real sizeof for complete structs (was 1); recognize
std::min/clamp/modulo index bounds
- memory-leak: static-storage singletons + return-to-caller are
caller-owned, not leaks
YAMS scan: UAF 4→0, struct-OOB ~10→0, singleton-leak ~15→0,
min/modulo-bound OOB →0. 137/137 tests pass.
