trevon/fold
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
fold/docs/architecture/component-and-cli-map.md
trvon 8d85829481
 docs: document convergence contracts
2026-07-29 00:58:19 -04:00

14 KiB
Raw Permalink Blame History

Fold — Component & CLI Architecture Map

A reference map of fold's layered components, analysis backends, CLI surface, and detector taxonomy, plus an audit of the profiling and testing infrastructure.

Fold is a bounded-symbolic SAST + binary-analysis tool. It analyzes C/C++ (clang), Rust and Swift (external frontends), and compiled binaries/firmware (radare2 + Ghidra P-Code), encodes program semantics into Z3, and emits findings classified by how strongly they were established (a confidence model, not a severity scale).

1. Layered components

Fold enforces a strict no-upward-includes layering (see CLAUDE.md). Lower layers never include higher ones.

flowchart TD
    ir["fold::ir<br/>frontend-agnostic vocabulary<br/>Expr · Symbol · SourceLoc · Call · CFG · summaries"]
    report["fold::report<br/>Finding · Confidence · triage · SARIF/MD/JSON"]
    symbolic["fold::symbolic<br/>encoder family (Z3) · SymbolicEnv · SafetyObserver<br/>heap/interval/octagon · stdlib models · loop invariants"]
    detect["fold::detect<br/>detectors subclassing the observer surface<br/>+ security_walker composite engine"]
    cli["fold::cli<br/>scan router/runner · detector plan · presets · worker fanout"]

    ir --> report --> symbolic --> detect --> cli

    subgraph backends [side modules]
        binary["fold::binary<br/>r2_backend · ghidra_lifter · pcode_router<br/>rop_gadgets · gadget_semantics · rop_chain"]
        frontend["fold::frontend<br/>clang_backend · rust_backend · swift_backend<br/>ast_visitor · cfg_builder · comp_db"]
        confirm["fold::confirm<br/>FindingConfirmationTask (deeper bounded re-proof)"]
        fuzz["fold::fuzz<br/>harness_emitter · corpus · asan_parser · runner"]
        mcp["fold::mcp<br/>JSON-RPC stdio server · ToolRegistry · ScanCache"]
        lsp["fold::lsp<br/>Content-Length framed JSON-RPC diagnostics"]
    end

    frontend --> symbolic
    binary --> symbolic
    detect --> confirm
    detect --> fuzz
    cli --> mcp
    cli --> lsp

All of src/binary, src/frontend, src/ir, src/symbolic, src/detect, src/index, src/report, src/confirm, src/witness, src/fuzz, src/mcp, src/lsp compile into the fold_core static library (top-level meson.build:217-228); the fold executable (src/cli, meson.build:292) links it. The optional fold_ghidra lib is gated behind ghidra_enabled.

Frontend-agnostic invariant (M30): fold::ir headers name no clang::* types. Clang bridges live in *_clang.cpp (src/symbolic/ir_expr_clang.cpp, clang_location_mapper.cpp). A Rust/Swift frontend builds ir::Expr directly. Detectors and observer hooks must not break this.

2. Analysis flow

flowchart LR
    subgraph inputs [Inputs]
        cdb[compile_commands.json]
        bin[binary / firmware]
        rs[".rs (HIR JSON)"]
        sil["Swift SIL (SWAN)"]
    end

    cdb --> clang[clang_backend]
    rs --> rustfe[rust_backend → fold-rust]
    sil --> swiftfe[swift_backend → fold-swift]
    bin --> r2[r2_backend + ghidra_lifter]

    clang --> irc[fold::ir]
    rustfe --> irc
    swiftfe --> irc
    r2 --> pcode[P-Code]

    irc --> enc[encoder.cpp — Z3 encoding]
    pcode --> penc[pcode_encoder.cpp — Z3]

    enc --> det[detectors via SafetyObserver +<br/>security_walker engine modules]
    penc --> det

    det --> triage[report::triage<br/>score · suppress · baseline]
    triage --> out{--format}
    out --> sarif[SARIF 2.1.0]
    out --> native[Versioned findings JSON]
    out --> md[Markdown]
    out -. deprecated json alias .-> sarif

    det -. confirm/fuzz .-> promote[Proved → ConcreteWitness]
    promote --> triage

Mode resolution (src/cli/scan_router.cpp resolveMode): auto infers source/binary/hybrid/rust/swift from inputs; explicit --mode overrides. Hybrid runs source + binary lanes and merges/corroborates. Source mode fans out across fold worker subprocesses when --processes > 1 (src/cli/subprocess_fanout.cpp).

3. CLI command tree

Entry: src/cli/main.cpp (CLI11). Exactly one subcommand required. Global --quiet/-q. Exit codes 0 (no actionable findings) / 1 (findings) / 2 (error).

flowchart TD
    fold[fold]
    fold --> scan
    fold --> grep
    fold --> confirm
    fold --> fuzz
    fold --> mcp
    fold --> lsp
    fold --> skill
    fold -. hidden .-> worker

    fuzz --> fprep[prep]
    fuzz --> frun[run]

    scan --> sg["flag groups:<br/>Input · Output · Detectors ·<br/>Analysis · Hybrid ·<br/>Performance/Misc · Suppression"]
Subcommand Purpose
scan Source/binary/hybrid/Rust/Swift analysis → SARIF/findings-json/Markdown
grep Query existing Fold SARIF/findings-json reports with shared typed filters and pagination
confirm Re-run a finding with deeper bounded symbolic analysis; optional GraphML witnesses
fuzz prep Emit libFuzzer harnesses + Z3-derived seed corpora
fuzz run Compile + run harnesses; ingest crashes as ConcreteWitness findings
mcp JSON-RPC 2.0 stdio MCP server (canonical tools include fold.scan*, fold.query_findings, and fold.explain)
lsp LSP diagnostics server
skill Print the embedded SKILL.md for agents (-o to file)
worker Internal — process a shard (stdin file list → NDJSON), for subprocess fanout

Key scan flags (selection): --mode, --compile-db, --binary, paths (Input); --output/-o, --format/-f sarif\|findings-json\|markdown\|json, --fix-output (Output; json is a deprecated SARIF alias); --detectors / --scope (Detectors); --budget-seconds, --loop-unroll, --profile-solver (Analysis); --processes/-j (Performance); --suppressions, --baseline (Suppression).

Compartmentalizing scans: --scope security|quality|redundancy|binary|all is the friendly front-end that expands to a detector category; it composes with --detectors. --profile is metrics-only; modeling presets live on --preset <name> (currently freestanding). A value passed to --profile is a deprecated alias for --preset.

4. Detector taxonomy

Categories live in src/cli/detector_plan.cpp (kSourceDetectorDescriptors) as SourceDetectorCategory{Redundancy, Security, Quality}. Binary detectors are named separately (src/cli/detector_registry.cpp kBinaryLeaves).

flowchart TD
    root[detectors]
    root --> sec[Security]
    root --> qual[Quality]
    root --> red[Redundancy]
    root --> binr[Binary]

    sec --> slist["div-by-zero · null-deref · int-overflow<br/>int-truncation · sign-conversion · oob · taint-track<br/>sql-injection · uninit-read · format-string<br/>use-after-free · double-free · memory-leak<br/>hardcoded-credential · weak-crypto · deprecated-banned"]
    qual --> qlist["dead-statement · unreachable · unused-function<br/>complexity · deprecated-api · use-after-move<br/>static-ptr-asymmetry · iterator-invalidation"]
    red --> rlist["clone · equiv · class-dup · dead<br/>always-condition · redundant-store"]
    binr --> blist["binary-security · binary-null-deref<br/>dangerous-import · hardcoded-string<br/>+ ROP/gadget exploit synthesis"]

Confidence model (include/fold/report/finding.hpp), strongest → weakest: ConcreteWitness (fuzzer-reproduced crash) > Proved (Z3 UNSAT of negation) > Latent (proved modulo caller precondition) > Heuristic (over-approximated) > Syntactic (AST/CFG-shape match). There is no Severity enum — report::triage::scoreFinding derives a 0–1 score from confidence + summary backing + cross-TU corroboration.

Supported frontends: C/C++ (clang, full suite), Rust (fold-rust HIR→IR; kinds rust-unsafe-api, rust-unchecked-index, rust-path-traversal, rust-command-injection, rust-crypto-misuse), Swift (fold-swift SWAN SIL, reuses the Rust IR/kinds), and binaries (radare2 + Ghidra P-Code, heuristic hazards + ROP primitives).

5. Audit — testing infrastructure

Framework: Catch2, gated behind -Dbuild-tests.

Suite Wiring Count
Unit tests/unit/meson.build 89 base targets plus feature-gated tests
Integration tests/integration/meson.build 13 base targets plus feature-gated tests
Benchmark / architecture tests/benchmarks/meson.build evaluator, source-structure, source-metamorphic, UBSan/ASan/Clang-SA differential, and organic-pair gates
Confirmation / KLEE smoke tests/confirmation/ —
Fixtures tests/fixtures/ by category

Doc correction: fixed test-count claims become stale as feature-gated suites grow. The validated release configuration on 2026-07-13 registered 113 Meson tests; use meson test -C build/release --list as the source of truth.

Product-level test features:

  • fold fuzz prep/run — libFuzzer + ASan/UBSan harness generation and crash ingestion (promotes Proved → ConcreteWitness).
  • Precision/recall harness in bench/: run_fold_on_juliet.py, run_svcomp.py, run_curl_ab.py, run_security_targets.py, run_quality_gate.py, compare_literature.py, plus baselines fp-baseline*.json and literature_baselines.json. This is the real precision guard the unit suite cannot provide.

What "green" means (per CLAUDE.md): build clean, all tests pass, and finding-count parity on a known OSS target (curl is canonical) for any encoder/observer change.

6. Audit — profiling infrastructure

Fold is self-instrumented; no external profiler (Tracy/perf/Instruments) is wired.

  • ScanProfile / DetectorProfile (src/cli/profile.cpp, include/fold/cli/profile.hpp): ~45 phase-level counters + peak_rss_kb, serialized via ScanProfile::to_json. Emitted by bare --profile (consolidated metrics JSON), written to --profile-output.
  • SolverStats — per-detector Z3 query counts/time, surfaced by --profile-solver; encoder reachability/bail metrics via --profile-reach.
  • bench/perf_audit_run.py — multi-run perf harness; dated baseline snapshots under docs/internal/perf-audit/ (e.g. baseline-2026-05-23.json, baseline-phases-2026-05-23.json).

Gap / opportunity: profiling measures fold's own runtime. It does not measure target-code complexity. Complexity/hotspot analysis of the analyzed program is a separate capability (a Quality-category detector — see complexity), complementing the existing dead-code detectors (dead, dead-statement, unreachable, unused-function).

For the structural-quality detector map, Yams rollout tiers, and the benchmark contract that joins target-code findings to Fold runtime/reliability metrics, see Code quality and Yams benchmark.

7. Key files index

Concern Path
CLI entry / flag wiring src/cli/main.cpp
Scan routing / mode resolve src/cli/scan_router.cpp
Source scan coordination src/cli/scan_runner.cpp
Backend shard enrichment / dispatch src/cli/backend_shard_runner.cpp
Incremental function fingerprints src/cli/incremental_fingerprint.cpp
Source call graph and conservative dispatch resolution src/cli/source_call_graph.cpp
Symbolic-summary SCC/fixpoint pipeline src/cli/source_summary_pipeline.cpp
Per-TU detector execution and solver-stat ownership src/cli/source_tu_runner.cpp
Finding proof/evidence annotation src/cli/finding_evidence.cpp
TU quarantine persistence src/cli/tu_quarantine.cpp
Detector taxonomy & plan src/cli/detector_plan.{hpp,cpp}
Detector name registry src/cli/detector_registry.cpp
Modeling presets src/cli/scan_preset.cpp
Scan options struct include/fold/cli/scan_options.hpp
Profiling src/cli/profile.cpp, include/fold/cli/profile.hpp
Finding / confidence model include/fold/report/finding.hpp
Triage / scoring src/report/triage.cpp
Output formats src/report/{sarif,markdown,finding_json,findings_json_report}.cpp
Z3 encoder dispatch / foundations / operators / expression arms src/symbolic/encoder{,_foundation,_operators,_expr,_expr_arms}.cpp
Composite security-walker facade / analysis / contracts / refinement / control src/detect/security_walker{,_analysis,_contracts,_refinement,_engine_core,_engine_control}.cpp
Div-by-zero guard facts and guard context src/detect/div_by_zero{,_guard_facts,_guard_context}.cpp
Null, overflow, and OOB candidate analysis src/detect/{null_deref,int_overflow,oob_access}_analysis.cpp
radare2 orchestration / session / serialization / disassembly state / extraction src/binary/r2_{backend,session,serialize,disasm_state,extract}.cpp
Benchmark case/run schemas and artifact hashes scripts/benchmarks/fold_eval/schema.py
Classification, sliced micro/macro, and paired statistics scripts/benchmarks/fold_eval/metrics.py, performance.py
Canonical SARIF and metamorphic finding comparison scripts/benchmarks/fold_eval/sarif.py, metamorphic.py
Runtime confirmation state model scripts/benchmarks/fold_eval/confirmation.py
Source metamorphic benchmark runner scripts/benchmarks/run_source_metamorphic.py
UBSan/ASan vulnerable/fixed differential runner scripts/benchmarks/run_sanitizer_differential.py
Clang Static Analyzer differential runner scripts/benchmarks/run_clang_sa_differential.py
Pinned organic vulnerable/fixed runner scripts/benchmarks/run_organic_pairs.py
Shared Fold source comparator lane scripts/benchmarks/fold_eval/source_lane.py
Reviewed disposition and change-time evaluator scripts/benchmarks/fold_eval/dispositions.py
Yams performance, reliability, and change-time runner scripts/benchmarks/run_code_quality.py
Production source-size ratchet scripts/check_source_structure.py, tests/architecture/source_budgets.json