# CTF Detector Gap Analysis — Binary Mode

> Date: 2026-05-29 | Source: Task #158 | Baseline: 7.4% detection (2/27 ground-truth)

## Summary

| Metric | Value |
|--------|-------|
| CTF binaries evaluated | 10 |
| Ground-truth vulnerabilities | 27 across 7 CWE categories |
| Current binary findings (real) | 2 (`gets()`, `system()`) |
| Current false positives | ~0 (post-FP fix, over-suppression) |
| Source-mode baseline | 66.7% (18/27) |
| **Total gap** | **25 undetected vulns across 6 CWEs** |

---

## Per-CWE Breakdown

### Gap #1: CWE-121 — Fortified Import Normalization (S effort, +4 vulns)

**Root cause:** Mach-O binaries built with `-D_FORTIFY_SOURCE` import
`___strcpy_chk` instead of `strcpy`. `normalizedSymbolName()` strips `_` prefix
but not `_chk` suffix. Fix: strip `_chk` suffix (~5 lines in `r2_backend.cpp:315`).

### Gap #2: CWE-134 — Binary Format String (S effort, +3 vulns)

**Root cause:** Binary mode cannot distinguish `printf("Hello")` (safe) from
`printf(user_input)` (vulnerable). Source mode has full AST-based detection.
Fix: use r2 `axtj` xref to find call sites, then disassemble to check if
format arg is constant `.rodata` string or variable register (~50 lines).

### Gap #3: Hardcoded-String Re-Enable (S effort, +2 vulns)

**Root cause:** Post-Loop-1 FP fix was over-aggressive — killed all credentials.
Fix: add bare-secret patterns (`supersecret`, `adminpass`), add `sk-` prefix
API key pattern, add build-artifact path filter (~30 lines).

### Gap #4: Binary BB Encoder Granularity (M effort, +6 vulns)

**Root cause:** `binary_bb_encoder` uses `abstract_blocks = true` which produces
unconstrained Z3 variables. Needs per-instruction `pdbj` parsing to fire
per-instruction hooks (sdiv→div-by-zero, ldr/str→null-deref, add/sub/mul→int-overflow).
Pipeline IS wired in `scan_runner.cpp:3279`, observer logic IS correct — encoder
needs per-instruction granularity (~100 lines).

### Gap #5: Binary UAF Observer (M effort, +2 vulns)

**Root cause:** `BinaryBBObserver` has `onPointerDerefIr` but doesn't track
whether ptr was freed. Fix: add `std::unordered_set<unsigned> freed_regs_`,
track on `free()` calls, check on derefs (~80 lines).

---

## Detection Ranked by Gain-per-Effort

| Rank | Gap | Effort | CWEs Closed | +Vulns | Cumulative |
|------|-----|--------|-------------|--------|------------|
| **#1** | Fortified import `_chk` strip | S | CWE-121 | +4 | 6/27 (22%) |
| **#2** | Binary format-string xref | S | CWE-134 | +3 | 9/27 (33%) |
| **#3** | Hardcoded-string re-enable | S | CWE-798 | +2 | 11/27 (41%) |
| **#4** | BB encoder per-instruction | M | CWE-369,190,476 | +6 | 17/27 (63%) |
| **#5** | Binary UAF observer | M | CWE-416 | +2 | 19/27 (70%) |

3 remaining FPs require interprocedural binary analysis or runtime fuzzing.