fold/docs/research/fold-production-complexity-audit-2026-06-03.md
2026-06-04 20:20:29 -04:00

7.5 KiB

Fold Production Source Complexity Audit

Date: 2026-06-03 Scope: production C++ in src/ and include/fold/, not benchmark harness code.

Summary

The highest complexity is concentrated in a few production orchestration and encoder files:

  • src/cli/scan_runner.cpp
  • src/symbolic/encoder.cpp
  • src/symbolic/rust_ir_encoder.cpp
  • src/symbolic/pcode_encoder.cpp
  • src/detect/security_walker.cpp
  • large enum/string mapping helpers in ir / report / ghidra_lifter

The benchmark harness refactor was useful cleanup, but it was not the primary maturity risk. Fold's production complexity risk is orchestration coupling: detector registration, frontend-specific observers, reporting/profile stamping, and backend-specific passes are currently concentrated in large functions.

Methods

  • Rough lexical complexity pass over src/ and include/fold/.
  • clang-tidy cognitive complexity check on current production diff files: readability-function-cognitive-complexity, threshold 30.
  • Focused Fold source scan over touched production files:
./build/release/src/cli/fold scan --mode source \
  --compile-db build/release/compile_commands.json \
  src/binary/binary_pcode_scan.cpp src/binary/ghidra_lifter.cpp \
  src/binary/pcode_router.cpp src/binary/r2_backend.cpp \
  src/cli/scan_runner.cpp src/symbolic/pcode_encoder.cpp \
  src/symbolic/rust_ir_encoder.cpp \
  --detectors security --format json

Top rough lexical complexity results

Approx CC LOC Location Function
81 402 src/symbolic/encoder.cpp:1339 encodeExpr
57 133 src/ir/expr_json.cpp:390 toJson
55 128 src/ir/expr_json.cpp:260 fromJson
53 138 src/symbolic/pcode_encoder.cpp:229 Encoder::step
50 100 include/fold/ir/pcode.hpp:163 opName
49 99 src/binary/ghidra_lifter.cpp:90 mapOp
42 203 src/detect/security_walker.cpp:1711 walkCFG
39 466 src/cli/scan_runner.cpp:2032 handleTranslationUnitImpl
39 165 src/symbolic/summary_encoder.cpp:403 walk
32 218 src/cli/fuzz_run_runner.cpp:244 runFuzzRun

clang-tidy cognitive complexity highlights

Changed production files flagged by clang-tidy:

Cognitive complexity Location Function
335 src/cli/scan_runner.cpp:1651 runClangDetectorsOnShard
154 src/cli/scan_runner.cpp:1852 nested CreateASTConsumer
76 src/symbolic/rust_ir_encoder.cpp:126 encodeRustNode
75 src/cli/scan_runner.cpp:290 buildProjectCallGraph
61 src/cli/scan_runner.cpp:298 nested CreateASTConsumer
50 src/cli/scan_runner.cpp:480 computeSummariesForFunctions
46 src/cli/scan_runner.cpp:2032 handleTranslationUnitImpl
35 src/cli/scan_runner.cpp:3139 runDetectorsOnShard
35 src/binary/r2_backend.cpp:174 runCommand
34 src/binary/r2_backend.cpp:874 decodeBase64Candidate

Focused Fold scan findings

Focused source scan returned 26 user-facing findings across the current production diff files:

  • fold.int-overflow: 14
  • fold.null-deref: 4
  • fold.sign-conversion: 4
  • fold.div-by-zero: 2
  • fold.memory-leak: 1
  • fold.uninit-read: 1

Many are bounded-model false positives on sizes/counters or nullable Clang API objects, but one finding revealed a real C++ undefined behavior risk:

Fixed during audit

src/cli/scan_runner.cpp:3355 and src/cli/scan_runner.cpp:3526 used:

1LL << (w - 1)

When w == 64, this computes 1LL << 63, which is undefined behavior for a signed long long. These expressions built signed min/max bit-vector bounds in Rust/Swift overflow observers.

Fix applied:

const std::uint64_t sign_bit = 1ULL << (w - 1);
z3::expr min_val = ctx.bv_val(sign_bit, w);
z3::expr max_val = ctx.bv_val(sign_bit - 1, w);

Validation passed:

ninja -C build/release
meson test -C build/release fold:rust_ir_encoder fold:rust_frontend fold:scan_e2e --print-errorlogs
git diff --check

Findings and recommendations

1. scan_runner.cpp is the primary complexity hotspot

runClangDetectorsOnShard owns too many responsibilities:

  • quarantine filtering;
  • incremental cache/fingerprint handling;
  • detector selection booleans;
  • detector profile construction;
  • nested AST action and consumer definitions;
  • per-TU watchdog/signal isolation;
  • composite security detector orchestration;
  • individual detector execution;
  • Rust and Swift backend-specific Z3 observer definitions;
  • BinaryIR/P-Code additive passes.

This makes detector additions risky because one file/function accumulates every new backend lane and every profile/provenance convention.

Recommended refactor:

  • Extract detector selection/profile construction into a DetectorPlan helper.
  • Extract Clang TU execution into a ClangShardRunner or TranslationUnitRunner.
  • Move Rust/Swift IR observer definitions out of scan_runner.cpp into frontend or symbolic helper files.
  • Keep runDetectorsOnShard as a thin backend dispatcher.

2. Rust/Swift observer code should not live inside scan orchestration

The inline RustSafetyObserver and SwiftSafetyObserver duplicate emission logic and symbolic overflow bound construction. This caused the 64-bit signed shift UB in two places.

Recommended refactor:

  • Create a shared language-IR observer helper, e.g. src/symbolic/language_ir_observer.cpp with:
    • backend tag (rust / swift);
    • detector enable flags;
    • shared emitFinding;
    • shared signed-bound helpers.
  • Keep Rust/Swift frontend-specific differences as small option structs.

3. P-Code encoder step is dense but bounded

Encoder::step is a large switch over P-Code operations. This is natural for an interpreter, but hazard-specific branches are now mixed with basic value-modeling branches.

Recommended refactor:

  • Extract helpers:
    • stepMemory(op);
    • stepBoolean(op);
    • stepCompare(op);
    • stepCall(op) already exists;
    • stepDivision(op).
  • Keep switch cases as dispatch only.
  • Add unit tests around division-address gating and CBranch fallthrough behavior before refactoring.

4. Opcode/string mapping functions are large but low-risk

mapOp, opName, kindToString, fromJson, and SARIF/finding rule maps are large switch/if mapping functions. They inflate lexical complexity but are less risky than orchestration code.

Recommended refactor:

  • Prefer table-driven maps generated from one canonical enum/key source where wire-format stability matters.
  • Do not prioritize over scan_runner and observers unless they keep producing drift bugs.

5. Fold source scan should become a production-code review gate

The focused scan found the signed-shift UB. It also surfaced likely model false positives, but those are useful pressure cases for Fold itself.

Recommended benchmark extension:

  • Add a production self-scan golden that runs a small subset such as src/symbolic/rust_ir_encoder.cpp and src/cli/scan_runner.cpp with a loose must_not_report list for known false positives and a warning-only summary.
  • Do not make the full self-scan a hard gate until false positives are triaged.

Suggested sequencing

  1. Extract Rust/Swift language IR observers and signed-bound helpers.
  2. Introduce DetectorPlan for source detector selection/profile setup.
  3. Extract ClangShardRunner / TU execution from runClangDetectorsOnShard.
  4. Split PcodeEncoder::step into operation-family helpers.
  5. Table-drive enum/wire-format mapping only after behavior-critical refactors.