fold/docs/internal/ctf-detector-gaps.md

CTF Detector Gap Analysis — Binary Mode

Date: 2026-05-29 | Source: Task #158 | Baseline: 7.4% detection (2/27 ground-truth)

Summary

Metric	Value
CTF binaries evaluated	10
Ground-truth vulnerabilities	27 across 7 CWE categories
Current binary findings (real)	2 (gets(), system())
Current false positives	~0 (post-FP fix, over-suppression)
Source-mode baseline	66.7% (18/27)
Total gap	25 undetected vulns across 6 CWEs

---

Per-CWE Breakdown

Gap #1: CWE-121 — Fortified Import Normalization (S effort, +4 vulns)

Root cause: Mach-O binaries built with -D_FORTIFY_SOURCE import ___strcpy_chk instead of strcpy. normalizedSymbolName() strips _ prefix but not _chk suffix. Fix: strip _chk suffix (~5 lines in r2_backend.cpp:315).

Gap #2: CWE-134 — Binary Format String (S effort, +3 vulns)

Root cause: Binary mode cannot distinguish printf("Hello") (safe) from printf(user_input) (vulnerable). Source mode has full AST-based detection. Fix: use r2 axtj xref to find call sites, then disassemble to check if format arg is constant .rodata string or variable register (~50 lines).

Gap #3: Hardcoded-String Re-Enable (S effort, +2 vulns)

Root cause: Post-Loop-1 FP fix was over-aggressive — killed all credentials. Fix: add bare-secret patterns (supersecret, adminpass), add sk- prefix API key pattern, add build-artifact path filter (~30 lines).

Gap #4: Binary BB Encoder Granularity (M effort, +6 vulns)

Root cause: binary_bb_encoder uses abstract_blocks = true which produces unconstrained Z3 variables. Needs per-instruction pdbj parsing to fire per-instruction hooks (sdiv→div-by-zero, ldr/str→null-deref, add/sub/mul→int-overflow). Pipeline IS wired in scan_runner.cpp:3279, observer logic IS correct — encoder needs per-instruction granularity (~100 lines).

Gap #5: Binary UAF Observer (M effort, +2 vulns)

Root cause: BinaryBBObserver has onPointerDerefIr but doesn't track whether ptr was freed. Fix: add std::unordered_set<unsigned> freed_regs_, track on free() calls, check on derefs (~80 lines).

---

Detection Ranked by Gain-per-Effort

Rank	Gap	Effort	CWEs Closed	+Vulns	Cumulative
#1	Fortified import _chk strip	S	CWE-121	+4	6/27 (22%)
#2	Binary format-string xref	S	CWE-134	+3	9/27 (33%)
#3	Hardcoded-string re-enable	S	CWE-798	+2	11/27 (41%)
#4	BB encoder per-instruction	M	CWE-369,190,476	+6	17/27 (63%)
#5	Binary UAF observer	M	CWE-416	+2	19/27 (70%)

3 remaining FPs require interprocedural binary analysis or runtime fuzzing.