Foundation for replacing fold's dead/abstract binary encoder with a real
SSA P-Code frontend. Verified root-cause gap: serializeBinaryIR() emits only
data-discovery nodes (sections/strings/imports/exports); the control-flow
BinaryExprKind values (BasicBlock/CallSite/...) are never constructed, so the
Z3-BB encoder's encodable filter is always empty and encodeBinaryBBs never
runs — hence 0 binary Z3 findings. The P-Code pipeline supersedes that path.
Phase 0 — docs/internal/ghidra-pcode-integration-research.md: P-Code opcode
core, the decompiler C++ embedding API (LoadImage/SleighArchitecture/Funcdata),
the CORE+DECCORE+SLEIGH source sets, no-bison finding, and pypcode as build
reference.
Phase 1 — vendored Ghidra decompiler at pinned commit (Ghidra_12.1_build,
7e89d94e34...) under third_party/ghidra-decompiler/ (cpp + AArch64 specs +
Apache-2.0 LICENSE + VENDOR.md + .gitignore for build artifacts). Validated by
hand on this clang-21/macOS toolchain:
- full SSA engine (CORE+DECCORE+SLEIGH, 80 files incl. heritage/funcdata/
action) compiles clean, -std=c++11 -O2, NO BFD, NO bison (generated
parsers are committed upstream; ruleparse isn't in the lib set);
- sleigh compiler builds; AArch64 .slaspec -> .sla compiles;
- tools/pcode_dump.cpp (custom LoadImage, no BFD) lifts the real arm64
sym._div_by_zero_direct to correct P-Code: `mov w9,#0 -> COPY(const 0)`,
`sdiv -> INT_SDIV(w8,w9)` — divisor zero-ness explicit in the data flow,
so the Phase-3 encoder proves div-by-zero trivially, multi-arch.
Not yet meson-wired (next step) and no fold-side IR/encoder yet (Phase 2-3);
this commit is the validated vendor + smoke tool.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Record that rust_ir_encoder and binary_bb_encoder do not route through
SecurityWalker/unrollLoop and have no loop handling, so "cross-frontend
ir::Loop reuse" is gated on a shared clang-free loop-walk core rather than
being a standalone slice. Lay out the realistic sequencing (clang-free CFG
foundation -> rust/binary lowering -> recall-preserving while(1) acceleration)
and note no safe small gate-validatable slice exists short of starting the
foundation. Clang counted+affine summarization is complete.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The old fp-baseline.json lived under gitignored bench/results/, so it had
no history and nothing compared against it — precision drift went unnoticed.
- Move the committed baseline to a tracked path (bench/fp-baseline.json)
regenerated from the current build.
- Add `score_fp.py --gate`: diff per-detector P/R/F1 vs the baseline and
exit non-zero on any drop beyond --tolerance (default 0.02).
- Add `--max-files-per-cwe` deterministic stride sampling so huge CWEs are
tractable; restrict gold to the scanned subset so recall stays meaningful.
- Document the gate (local, not CI) in bench/README.md.
- docs/internal/loop-summarization-research.md: Workstream B research + the
null-deref drift root-cause appendix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Each R2Session construction runs full auto-analysis
- 4-6 aaa runs on a single binary → 60-90s timeout
- Fix: share one R2Session across all extraction calls
- Progressive analysis levels: aa (quick), aaa (full), aaaa (experimental)
- SafetyObserver::seedInterproceduralContext() called before function walks
- OOBObserver override scans all call sites for ConstantArrayType args
- Seeds param_buffer_sizes_ from any caller, regardless of decl order
- Research doc covers Inferbo/ARCHER/FunFrog approaches and fold's infrastructure
- Only 5/15 functions tested; extractCalls/extractXrefs/pdbj untested
- CTF failures mapped to specific r2_backend.cpp code paths
- Top fix: flag dangerous imports + hardcoded strings (S effort, +12 findings)
- Full basic block analysis needed for div-by-zero/null-deref (L effort, +6)
- Z3 solver <0.01% of wall time (74 queries, ~0ms)
- Real bottleneck: AST walkers + CFG traversal (~85%)
- Top optimization: TU pre-filter (skip files with no security ops)
- Bitwuzla multi-solver would not help — SMT queries are already fast
- Bitwuzla BV solver is 5.1x faster than Z3 (SMT-COMP 2023)
- ESBMC pattern: dispatch to 6 solvers, race for first result
- Proposal: add --solver bitwuzla | z3 | portfolio-all flag
- Expected 2-5x speedup on fold's BV-64 queries
