080bf9655f
Extend ir::Loop classification: a monotone single-induction-variable for-loop with a symbolic bound (e.g. `for (i=0; i<n; ++i)`) is now Affine (was General). The walker summarizes Affine loops like Counted ones — bind the induction variable to a fresh symbolic, assert its start bound plus the loop condition (encoded to a Z3 bool) as path constraints, walk the body once — exposing high-index OOB that bounded unroll misses. Findings are over-approximations (LoopHavoc/heuristic), never proved. Net effect on the gate: oob recall 0.531 -> 0.559, F1 0.637 -> 0.652 (precision dips 0.796 -> 0.782) — a contract-sanctioned over-approximation tradeoff. Also regenerate bench/fp-baseline.json deterministically from a clean run: the prior committed oob numbers (0.574) were not reproducible (transient warm-state artifact); fresh runs are now bit-identical, so the gate is reliable. Tests: test_ir_loop (affine classification), test_loop_summary (symbolic-bound OOB exposed, not proved). Suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
78 lines
2.9 KiB
Markdown
78 lines
2.9 KiB
Markdown
# fold FP baseline
|
|
|
|
Generated: 2026-05-31T12-37-44Z
|
|
Fold version: 0.1.0
|
|
|
|
## Per-detector precision / recall / F1
|
|
|
|
| Detector | TP | FP | FN | Gold | Findings | Precision | Recall | F1 |
|
|
|----------|----|----|----|------|----------|-----------|--------|----|
|
|
| dead | 1 | 1 | 0 | 1 | 2 | 50.0% | 100.0% | 66.7% |
|
|
| div-by-zero | 53 | 26 | 36 | 89 | 79 | 67.1% | 59.6% | 63.1% |
|
|
| int-overflow | 89 | 34 | 76 | 165 | 123 | 72.4% | 53.9% | 61.8% |
|
|
| int-truncation | 27 | 3 | 60 | 87 | 30 | 90.0% | 31.0% | 46.2% |
|
|
| null-deref | 42 | 11 | 43 | 85 | 53 | 79.2% | 49.4% | 60.9% |
|
|
| oob | 222 | 62 | 175 | 397 | 284 | 78.2% | 55.9% | 65.2% |
|
|
| sign-conversion | 132 | 50 | 48 | 180 | 182 | 72.5% | 73.3% | 72.9% |
|
|
|
|
## Literature context
|
|
|
|
- CodeQL FP rate: ~5%
|
|
- Semgrep FP rate: ~12%
|
|
- Source: Bennett et al. 2024 (EASE), Lipp et al. 2022 (ISSTA)
|
|
|
|
## Per-detector per-corpus breakdown
|
|
|
|
### dead
|
|
|
|
| Corpus | TP | FP | FN | Gold | Findings | Prec | Rec | F1 |
|
|
|--------|----|----|----|------|----------|------|-----|----|
|
|
| security-bugs | 1 | 1 | 0 | 1 | 2 | 50.0% | 100.0% | 66.7% |
|
|
|
|
### div-by-zero
|
|
|
|
| Corpus | TP | FP | FN | Gold | Findings | Prec | Rec | F1 |
|
|
|--------|----|----|----|------|----------|------|-----|----|
|
|
| juliet-369 | 51 | 26 | 36 | 87 | 77 | 66.2% | 58.6% | 62.2% |
|
|
| security-bugs | 2 | 0 | 0 | 2 | 2 | 100.0% | 100.0% | 100.0% |
|
|
|
|
### int-overflow
|
|
|
|
| Corpus | TP | FP | FN | Gold | Findings | Prec | Rec | F1 |
|
|
|--------|----|----|----|------|----------|------|-----|----|
|
|
| juliet-190 | 45 | 16 | 31 | 76 | 61 | 73.8% | 59.2% | 65.7% |
|
|
| juliet-191 | 42 | 17 | 43 | 85 | 59 | 71.2% | 49.4% | 58.3% |
|
|
| security-bugs | 2 | 1 | 2 | 4 | 3 | 66.7% | 50.0% | 57.1% |
|
|
|
|
### int-truncation
|
|
|
|
| Corpus | TP | FP | FN | Gold | Findings | Prec | Rec | F1 |
|
|
|--------|----|----|----|------|----------|------|-----|----|
|
|
| juliet-197 | 27 | 3 | 60 | 87 | 30 | 90.0% | 31.0% | 46.2% |
|
|
|
|
### null-deref
|
|
|
|
| Corpus | TP | FP | FN | Gold | Findings | Prec | Rec | F1 |
|
|
|--------|----|----|----|------|----------|------|-----|----|
|
|
| juliet-476 | 40 | 11 | 43 | 83 | 51 | 78.4% | 48.2% | 59.7% |
|
|
| security-bugs | 2 | 0 | 0 | 2 | 2 | 100.0% | 100.0% | 100.0% |
|
|
|
|
### oob
|
|
|
|
| Corpus | TP | FP | FN | Gold | Findings | Prec | Rec | F1 |
|
|
|--------|----|----|----|------|----------|------|-----|----|
|
|
| juliet-121 | 35 | 9 | 41 | 76 | 44 | 79.5% | 46.1% | 58.3% |
|
|
| juliet-122 | 37 | 6 | 53 | 90 | 43 | 86.1% | 41.1% | 55.6% |
|
|
| juliet-124 | 40 | 5 | 35 | 75 | 45 | 88.9% | 53.3% | 66.7% |
|
|
| juliet-126 | 56 | 20 | 23 | 79 | 76 | 73.7% | 70.9% | 72.3% |
|
|
| juliet-127 | 52 | 22 | 23 | 75 | 74 | 70.3% | 69.3% | 69.8% |
|
|
| security-bugs | 2 | 0 | 0 | 2 | 2 | 100.0% | 100.0% | 100.0% |
|
|
|
|
### sign-conversion
|
|
|
|
| Corpus | TP | FP | FN | Gold | Findings | Prec | Rec | F1 |
|
|
|--------|----|----|----|------|----------|------|-----|----|
|
|
| juliet-194 | 52 | 24 | 28 | 80 | 76 | 68.4% | 65.0% | 66.7% |
|
|
| juliet-195 | 62 | 26 | 18 | 80 | 88 | 70.5% | 77.5% | 73.8% |
|
|
| juliet-196 | 18 | 0 | 2 | 20 | 18 | 100.0% | 90.0% | 94.7% |
|
|
|