fold/docs/internal/library-expansion-followups.md
2026-05-27 10:36:57 -04:00

10 KiB

Library-expansion matrix — outstanding follow-ups

After the sequential sweep documented in docs/internal/library-expansion-matrix.md, three follow-up workstreams remain. None block any shipped row; they cover the loose-thread cleanup the audit pattern surfaced.

Ordering reflects priority. (1) is needed for the C5 detector to be runnable end-to-end; (2) addresses a real capability gap catalogued during the C4 audit; (3) closes the methodology loop on rows that shipped before the per-row baseline convention existed.

1. scan_runner.cpp dispatch hook for uninit-local-read

Status: verified locally on 2026-05-26. The scan-runner diff contains both C2's selective clang::CallGraph merge and C5's uninit-local-read dispatch hook. Focused tests and a CLI smoke passed; the changes are still uncommitted in the worktree.

Background. The C5 wiring commit (48dac13d5) landed:

  • include/fold/detect/uninit_local_read.hpp
  • src/detect/uninit_local_read.cpp
  • FindingKind::UninitLocalRead in include/fold/report/finding.hpp
  • SARIF rule id + name in src/report/sarif.cpp
  • finding-name parser in src/report/finding_json.cpp
  • registry entry in src/cli/detector_registry.cpp
  • 5 unit tests + meson wiring

…but did not commit the four scan_runner.cpp hunks that:

  1. include fold/detect/uninit_local_read.hpp
  2. set want_uninit_local = detectorEnabled(...) and the addDetectorProfile("uninit-local-read") entry
  3. invoke detectUninitLocalReadInContext(ctx) inside the per-TU detector dispatch, annotating findings + accumulating profile
  4. emit the spdlog::info("uninit-local-read: emitted …") line in the summary section

The hunks were withheld because scan_runner.cpp was simultaneously being edited by a parallel session implementing C2 (clang::CallGraph selective adapter — mergeClangCallGraph, addLocalCallGraphCallee, etc.). Both changes are correct individually; the bundled diff just needed a coordinated commit moment that never arrived.

What to do. Audit the current src/cli/scan_runner.cpp diff — it should contain BOTH C2 and C5 hunks. Run meson test -C build/release (expected 57/57). Commit as a single bundled commit crediting both changes, e.g.:

feat(cli): wire C2 clang::CallGraph + C5 uninit-local-read dispatch

- C2: mergeClangCallGraph + addLocalCallGraphCallee augment
  fold's existing ir::ProjectCallGraph with clang TU-local edges
- C5: scan-runner dispatch for the uninit-local-read detector
  registered in 48dac13d5

Pre-commit clang-format hook will catch two long-line violations (at the include block + a const_cast line); run clang-format -i src/cli/scan_runner.cpp first.

Verification. After commit, the canonical smoke is:

build/release/src/cli/fold scan \
  --compile-db /tmp/c5wiring_smoke/compile_commands.json \
  --detectors uninit-local-read \
  -o /tmp/out.json -f json
# Expected: "uninit-local-read: emitted 3 findings"

(/tmp/c5wiring_smoke/uninit.c was the fixture used during C5-wiring development; recreate from the test cases in tests/unit/test_uninit_local_read.cpp if it's gone.)

2. Native UAF detector

Status: implemented locally on 2026-05-26 as an intra-procedural, heuristic use-after-free source detector with unit tests and CLI dispatch. Curl-scale baseline captured at docs/internal/perf-audit/runs/matrix/UAF-curl-2026-05-26.json (18 SARIF findings after report filtering; 11 ms detector wall).

Carved out from C4 (docs/internal/library-expansion-matrix.md). C4's original framing was clang::consumed::ConsumedAnalysis for RAII / use-after-free. Audit (commit 1dc20ca) confirmed ConsumedAnalysis only activates on classes marked [[clang::consumable]] + member functions marked [[clang::callable_when(state)]] + [[clang::return_typestate(state)]]. Curl/yams/kernel-style C code has none of these attributes.

The capability gap is real: fold has no UAF detector. The right implementation is not ConsumedAnalysis but a fold-native CFG-walker shape:

Sketch.

  • New detector: include/fold/detect/use_after_free.hpp, src/detect/use_after_free.cpp
  • FindingKind::UseAfterFree in include/fold/report/finding.hpp
  • SARIF rule id fold.use-after-free + finding-name parser
  • Registry entry "use-after-free" in src/cli/detector_registry.cpp
  • scan_runner.cpp dispatch (same template as uninit-local-read)

Detector core (per-function walk):

  • Track local set freed_ptrs: VarDecl* -> first_free_loc
  • On free(p), delete p, delete[] p: add p (and any aliases via q = p;) to freed_ptrs
  • On any subsequent read of p (DeclRefExpr usage, MemberExpr base, deref *p, subscript p[…]): emit a UAF finding citing both the free site and the use site
  • Reset on re-assignment (p = … after the free clears it from the set)

Soundness scope.

  • Intra-procedural only (interprocedural via summaries is a follow-up — the existing CrossTUCache pattern fits)
  • Trivially correct on flat function bodies; needs CFG awareness for branches (use the same BranchLiveness / walkIf scaffolding the security_walker uses)
  • False-positive sources to handle explicitly: free(NULL) doesn't poison anything; p = realloc(p, n) is not a UAF even though p is conceptually "freed" — model realloc as a refresh

Test plan.

  1. tests/unit/test_use_after_free.cpp with: simple free(p); *p; flagged; freed-but-reassigned not flagged; free(NULL) not flagged; p = realloc(p, n) not flagged; alias poisoning; delete p; p->x; and conditional free/use inside the then-branch.
  2. End-to-end SARIF smoke on a synthetic fixture (mirroring the c5wiring_smoke pattern).
  3. Curl baseline finding count after the detector lands — capture under docs/internal/perf-audit/runs/matrix/uaf-curl-<DATE>.json.

Out of scope for the first PR.

  • C++ smart-pointer typestate (covered by C4's deferred form)
  • Interprocedural UAF (free in callee, use in caller) — relies on a freed_ptr_params slot in detect::CalleeProperties
  • Heap-region precision via symbolic::HeapModel (isUseAfterFree already exists but the connector to the new detector is the C++-aware bit)

Naming. "use-after-free" matches the SARIF taxonomy and CWE-416 ("Use After Free"). The existing HeapAccessInfo::is_use_after_free field already uses this spelling.

3. Per-row precision baselines for shipped rows

Status: Z4, C5, C7, and the native UAF follow-up now have local baselines under docs/internal/perf-audit/runs/matrix/:

  • Z4-curl-2026-05-26.json
  • C5-curl-2026-05-26.json
  • C7-curl-2026-05-26.json
  • UAF-curl-2026-05-26.json
  • C9-synthetic-2026-05-26.json
  • Z5-synthetic-2026-05-26.json
  • Z10-serialize-corpus-2026-05-26.json
  • C8-contracts-synthetic-2026-05-26.json

All listed shipped-row baseline artifacts now have at least one local artifact. Curl-scale artifacts for Z5/C8/C9 remain optional follow-up work where the existing blockers or cost justify a dedicated session.

What to capture per row.

Row Target Tool Output
Z5 synthetic minimal-witness fixture one-off C++ invoking refineModelForMinimalWitness done: docs/internal/perf-audit/runs/matrix/Z5-synthetic-2026-05-26.json; curl fuzz-prep remains blocked by pre-existing crash
Z10 encoder-shaped n=100 corpus one-off C++ invoking z3 to_string() with alias thresholds done: docs/internal/perf-audit/runs/matrix/Z10-serialize-corpus-2026-05-26.json
C5 curl baseline fold scan --detectors uninit-local-read --profile done: docs/internal/perf-audit/runs/matrix/C5-curl-2026-05-26.json
C7 curl baseline fold scan --detectors deprecated-api --profile done: docs/internal/perf-audit/runs/matrix/C7-curl-2026-05-26.json
C8 synthetic twin baseline fold scan --detectors null-deref --profile with /*@ requires */ annotations vs without done: docs/internal/perf-audit/runs/matrix/C8-contracts-synthetic-2026-05-26.json
C9 synthetic + curl fold scan --fix-output … synthetic done: docs/internal/perf-audit/runs/matrix/C9-synthetic-2026-05-26.json; curl patch-count still optional

Naming convention. docs/internal/perf-audit/runs/matrix/<row>-<target>-<YYYY-MM-DD>.json, matching Z4's path.

Effort. Each curl baseline is ≈9 minutes wall (one --runs 1 of perf_audit_run.py). Total ≈54 min for the four curl-shaped rows + a few one-off Python invocations for Z10/Z5. Best run in a single dedicated session with the laptop idle.

Update step. Each row's Status column in docs/internal/library-expansion-matrix.md gets a link added to the new baseline file, mirroring Z4's:

**shipped** — `docs/internal/perf-audit/runs/matrix/<row>-<target>-<DATE>.json`,
              `tests/unit/test_<row>.cpp`

Cross-cutting cleanup

These showed up repeatedly across the matrix sweep and don't fit any single row, but should be cleaned up at some point:

  • Untracked workspace junk. profile.json, reach.txt, reach_stderr.txt, scan_output.log, taint_trace.log, trace.log, yams_full_scan.log, yams_security_bench.json, scratch/, tools/fold-rust/-, tools/fold-rust/target/ are all in git status -u as untracked. They're scratch artifacts. Either gitignore them (preferred — these are reproducible outputs) or delete them.

  • Parallel-session uncommitted edits in docs/USAGE.md, docs/internal/rust-frontend.md, src/frontend/rust_backend.cpp, src/mcp/tool_registry.cpp, tests/integration/meson.build, tests/integration/test_cross_tu_security.cpp, tests/integration/test_mcp_stdio.cpp, tools/fold-rust/src/main.rs. These belong to a Rust-frontend workstream that's orthogonal to the library-expansion matrix. Audit and commit separately.

  • Tests with chained-comparison warnings flagged by clang-tidy throughout the unit-test sweep (e.g. REQUIRE(stats.queries > 0 || !findings.empty()) parsing as v0 <= v1 != v2). Catch2 macros tickle this lint pattern even though the semantics are fine. Either suppress at the file level or wrap conditions in parens.

Suggested next-session opener

# Resume library-expansion follow-ups (see docs/internal/library-expansion-followups.md)
# Order: (1) scan_runner C2+C5 commit, (2) UAF detector, (3) baselines

(1) is mechanical (~10 minutes including verification). (2) is a real new detector (~half a session including tests + curl smoke). (3) is batch capture (~1 hour curl scans).