Problem: normalizeClangTuPath, normalizeQuarantinePath and fan-out's
normalizePath were three copies of the same absolute + weakly_canonical
routine. Coverage, worker records and the quarantine file must agree on a
TU's spelling, and three copies can drift apart.
Fix: move the routine to fold/cli/tu_path.hpp as normalizeTuPath and use it
at every site. No behaviour change.
Test: existing test_scan_coverage (path spellings), test_tu_quarantine and
test_subprocess_fanout, updated to the new name. Full suite: 140/140.
Completes the binary_bb_encoder deletion (3fa6d0f56) whose scan_runner include
and src/symbolic/meson.build entry were not captured in that commit — HEAD
referenced the deleted source. Also records build-config changes already in use:
- third_party/ghidra-decompiler/meson.build: link-only zlib (cpp.find_library)
to avoid the macOS SDK include path shadowing libc++ <cstdio>/<cstdlib>.
- meson.build/meson_options.txt: reserve optional analysis-lane feature names
(retdec/remill/souffle/phasar/crab) with vendor-manifest-only stubs; only
ghidra is wired.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drives the dual-stream router over a real binary end-to-end.
- r2_backend: detectBinaryArch() (iIj) + extractFunctionBytes() (aflj + p8,
skipping import thunks) — r2 supplies arch + per-function bytes; the Ghidra
lifter consumes them.
- binary_pcode_scan.cpp (fold-ghidra lib): scanBinaryWithPcode() maps the
r2 arch to a vendored SLEIGH spec (arm64-LE -> AARCH64:LE:64:v8A today;
unknown arch -> {} so the caller falls back to the r2-disasm passes),
enumerates functions, routes each through both lift streams, and merges +
dedups across functions. max_functions caps the sweep (robustness).
- meson: -DFOLD_GHIDRA_SPECS_DIR baked to the vendored specs path so the lifter
locates .ldefs/.sla at runtime; binary_pcode_scan added to fold-ghidra; the
pcode_scan driver gains a whole-binary `pcode_scan <binary>` mode.
Validated on the CTF corpus: libdiv_by_zero.dylib -> exactly 2 div-by-zero
(div_by_zero_direct + mod_by_zero, correctly attributed; div_by_zero_variable
abstains pending path-forking), and 0 on the other 9 dylibs. This reaches
parity with the shipped slice-4 div-by-zero TPs through the proper Z3/P-Code
pipeline. Next (Phase 4 part 2): wire into scan_runner additively (dedup the
div-by-zero overlap with the structural passes), then the encoder increments
(path-forking, null-deref, overflow) for the remaining corpus gains.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wires the whole pipeline together: binary bytes -> raw+SSA lift -> P-Code
encoder -> Z3 -> BinaryBBObserver -> merged finding.
- include/fold/binary/pcode_router.{hpp} + src/binary/pcode_router.cpp:
routeFunctionToFindings() lifts a function both ways (liftRaw + liftSSA),
encodes each into a fresh Z3 context/observer, tags findings with the lift
level (evidence.backend = binary-pcode-raw|ssa), and merges + dedups by
(kind, line, detector) — raw wins on collision (more direct hazard evidence).
- meson: gated `fold-ghidra` static lib (ghidra_lifter + pcode_router) linking
fold_core_dep + ghidra_decompiler_dep (first fold_core <-> Ghidra link), and
the `pcode_scan` smoke driver. Defined after fold_core so both deps are in
scope; built only when -Dghidra=enabled.
- pcode_encoder fix: value map now keys on (space, offset) without size and
coerces on read, so a sub-register divisor read (w9, size 4) sees the
full-register zero write (x9, size 8) the lifter emits for `mov w9,#0`
(ARM64 zero-extend). New unit test locks this aliasing.
Validated end-to-end: pcode_scan on arm64 sym._div_by_zero_direct emits exactly
one finding — `[binary-pcode-raw] division by zero proven by Z3` — with the SSA
stream correctly contributing nothing (it folds x/0 -> 0) and the router
merging. Full suite 71 Ok / 0 Fail / 1 pre-existing binary_backend timeout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a -Dghidra feature option (default disabled while the encoder is under
construction; enable with -Dghidra=enabled) and third_party/ghidra-decompiler/
meson.build that:
- builds libghidra_decompiler.a from the CORE+DECCORE+SLEIGH object sets
(80 files; cpp_std=c++11, warnings/werror disabled — vendored third-party);
- builds the ghidra_sleigh spec compiler and runs it per-arch to emit .sla
into the build dir (single-file form; source tree stays clean, .sla are
build artifacts);
- builds tools/pcode_dump (custom LoadImage, no BFD).
fold_core does not link it yet (the lifter/encoder glue is Phase 2-3); enabling
ghidra only adds these targets, leaving the normal build untouched.
Validated end-to-end through meson: ninja builds the lib + .sla + pcode_dump,
and the meson-built pcode_dump lifts arm64 sym._div_by_zero_direct to
INT_SDIV(w8,w9) with w9=COPY(const 0) — identical to the hand-build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enumeration: replace the hand-rolled top-level `tu->decls()` walks (TU
security prefilter + every interprocedural pre-pass/cross-TU primer in
div-by-zero/int-overflow/null-deref/oob) with a shared RAV-based
`fold::detect::forEachFunctionWithBody`, so functions nested in namespaces
and classes are analyzed. Fixes 0-findings on namespaced C++ (dbench CTF).
TU prefilter: extracted to fold/cli/tu_security_prefilter; flags unsigned
-/* wraps so int-overflow candidates in size arithmetic are not skipped.
Timeout robustness: cooperative per-TU walk deadline (WalkOptions) checked
in the walk/unrollLoop/CFG-block checkpoints and the BatchSafetyObserver
flush loop, plus a non-progress unroll fixpoint — slow TUs degrade with
partial findings instead of being SIGALRM-quarantined.
OOB precheck: treat `*(p+i)` and indirect callees as candidates.
null-deref drift fix: remove the unsound `priorFunctionTextProvesNonNull`
source-text heuristic (added in 331a68467) that treated a textual `p!=0`
before a deref as proof of non-null, silently suppressing real null-derefs
(CWE-476 F1 0.485 -> 0.583, recall 0.379 -> 0.517).
Tests: test_function_enumeration, test_tu_security_prefilter, namespaced
cross-TU int-overflow regression. Full suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- setup.sh creates build/llvm_ar.ini with ar=llvm-ar path
- meson configure passes llvm_ar.ini as additional native file
- Fixes 'llvm-ar: command not found' when meson test rebuilds
- Also find llvm-ar in meson.build for direct ninja invocations
- Replaced run_target with custom_target (build_by_default: true)
- Single ninja builds all 3: fold + fold-rust + fold-swift
- meson install copies all 3 to prefix/bin automatically
- No more separate ninja fold-rust step needed
Phase-1 (C3) of the CLI design philosophy refactor. Addresses
audit item #5 from docs/cli-philosophy.md.
Today scan_router throws std::invalid_argument /
std::runtime_error on bad inputs (missing --binary, unknown mode,
no compile DB, etc.). runScan catches them and logs "scan failed:
<msg>" — the user sees what failed but not how to recover.
After this change, those throw sites use the new
fold::cli::UserError (subclass of std::runtime_error carrying a
vector of recovery hints). runScan's catch dispatches on
UserError first and formats via emitUserError, which prints:
<message>
Try:
<hint 1>
<hint 2>
All hints are exact, copy-pasteable command lines — no
placeholders. Applied at six throw sites: unsupported mode,
no compile DB, empty compile DB, rust mode without --paths,
binary mode without --binary, unsupported hybrid strategy.
New files:
- include/fold/cli/error_format.hpp: UserError exception type +
emitErrorWithRecovery / emitUserError helpers.
- src/cli/error_format.cpp: implementation.
Wired:
- src/cli/scan_router.cpp: throws now carry hints.
- src/cli/main.cpp::runScan: catches UserError first.
- meson.build: error_format.cpp in fold_core_sources.
44/44 tests still green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase-1 (C2) of the CLI design philosophy refactor. Addresses
audit item #4 from docs/cli-philosophy.md.
Today --detectors / --source-detectors / --binary-detectors all
accept any string and silently drop unknown names downstream.
After this change CLI11 rejects unknown names at parse time with
exit code 105 (ValidationError) and a full list of valid names.
New files:
- include/fold/cli/detector_registry.hpp: canonical names +
validator + family helpers (sourceLeafDetectors,
binaryLeafDetectors).
- src/cli/detector_registry.cpp: the table (single source of
truth across CLI11, scan_router family expansion, and any
future MCP-tool routing per principle 9).
Wiring:
- main.cpp: new appendValidatedDetectors() helper validates
every name via the registry before pushing into ScanOptions.
Throws CLI::ValidationError with the registry's "did you
mean" message on rejection so the user sees the full valid
set inline. Applied to all three detector flags.
- meson.build: registry compiled into fold-core (used by both
the CLI executable and downstream tests).
44/44 tests still green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
R3-oob: oob_access now hoists z3::model out of both solver branches
and passes it to recordSinkWitnessWithModel. Predicted seeds for
`int buf[N]; return buf[idx];` now contain Z3's chosen boundary-
violating idx (e.g. `04 00 00 00` for buf[4]) instead of zeros.
P1.7: tests/integration/test_fuzz_prep.cpp materializes a
null-deref-behind-`flag==0xAB`-guard fixture, drives runFuzzPrep
in-process, and asserts the witness seed contains the exact magic
byte (0xAB at offset 0) plus a well-formed harness (extern "C",
size-gate at total_bytes, leaky(arg0, arg1) call).
Moves fuzz_prep_runner.cpp from the fold executable target into
fold_core_sources so the integration test can link runFuzzPrep
directly.
Tests: 37 → 38 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Build the static-analysis-driven fuzzing pipeline alongside the radare2
+ z3 work. fold's symbolic engine already proves sink feasibility; this
adds the path that retains the Z3 model, lowers it to libFuzzer
harnesses + seed corpora, and writes a findings.predicted.json bridge
so a future fold fuzz-run can promote crashes to ConcreteWitness
confidence.
* fold::fuzz value types (PathWitness, ParamWitness) + WitnessRecorder
collector — process-local, lock-free hot-path check
* model_to_bytes — z3::model → little-endian std::byte at source width;
pointer null-vs-nullable disambiguation
* harness_emitter — generates .harness.cc + .schema.json; honors
is_c_linkage via `extern "C"` wrapper for C targets; nested-namespace
emission for C++
* corpus_writer — packs witness bytes per harness layout, FNV-1a hash
for filename, 3 canonical seeds (zeros / 0xFF / single byte)
* param_schema — schemaForFunction + recordSinkWitness +
recordSinkWitnessWithModel; latter looks up each parameter's z3::expr
via env->tryGet() and mines the model for byte payloads
* SafetyObserver: const SymbolicEnv* env added to onPointerDeref,
onDivision, onTruncation, onSignedUnsignedConversion,
onArraySubscript; encoder.cpp passes &env at every call site;
CompositeSafetyObserver in scan_runner fans through
* null_deref, div_by_zero, int_overflow, int_truncation,
sign_conversion now record witnesses with full model bytes;
oob_access records schema-only (sat sites don't capture a model yet)
* fold fuzz-prep subcommand — runs scan in-process (workers can't
ship recorder state), drains the recorder, emits one
{harness,schema,corpus/} bundle per function
Smoke validation: 4 detectors on a 5-line C fixture produce
non-partial witnesses whose bytes are the exact magic values that
trigger each sink (flag=0xAB for null-deref; x=0 for div-by-zero;
a+b that overflows INT_MIN for int-overflow).
Tests: 34 → 37 (add test_model_to_bytes, test_harness_emitter,
test_corpus_writer). All green.
Also folds in a small worker-toolchain optimization that had been
sitting in the working tree: preloadHostToolchainEnv() avoids
repeated popen calls during worker init.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
