# CTF Binary Re-Evaluation — Loop 2

> Generated 2026-05-30. Re-evaluation after Phase 1-3 binary backend improvements.

## Detection Rate Trend

| Loop | Detection Rate | Findings | FPs | Real Findings |
|------|---------------|----------|-----|---------------|
| Baseline | 3.7% (1/27) | 1 | 0 | 1 |
| Loop 1 (pre-FP fix) | 259% (70/27) | 70 | ~67 | ~3 |
| Loop 1 (post-FP fix) | 7.4% (2/27) | 2 | 0 | 2 |

**Net improvement: 3.7% → 7.4% (2x)**

## What's Working

| Detector | Real Findings | Example |
|----------|--------------|---------|
| `dangerous-import` | 2 | `gets()` in libbof_overflow, `system()` in libcmd_injection |
| `binary-security` | 1 | `gets()` xref in vulnerable_gets function |

## What Failed (FP Sources in Loop 1)

| Detector | FPs | Root Cause |
|----------|-----|------------|
| `hardcoded-string` | ~67 | Build paths, system library paths in binary data |
| `hardcoded-string` (post-fix) | 0 | Fix was too aggressive — also drops real credential findings |

## What's Still Missed (CWE gaps)

| CWE | Expected Finding | Status |
|-----|-----------------|--------|
| CWE-121 | Buffer overflow via strcpy | ❌ Not detected (binary strcpy import not flagged) |
| CWE-134 | Format string via printf | ❌ Not detected |
| CWE-416 | Use-after-free | ❌ Needs basic block analysis |
| CWE-369 | Division by zero | ❌ Needs Z3 encoding |
| CWE-190 | Integer overflow | ❌ Needs Z3 encoding |
| CWE-476 | Null dereference | ❌ Needs Z3 encoding |
| CWE-78 | Command injection | ✅ Detected (system() import) |

## Next Steps for Binary Backend

| # | Action | Effort | Expected Gain |
|---|--------|--------|--------------|
| 1 | Wire binary_bb_encoder.cpp into scan pipeline | S | +6 Z3-proved findings |
| 2 | Add strcpy/printf to dangerous import list | S | +2 findings |
| 3 | Fix hardcoded-string: add security-relevant path filter | S | +2 real, 0 FP |
| 4 | Basic block xref analysis (call→string association) | M | +3 findings |