fold/docs/internal/library-expansion-followups.md
2026-05-27 10:36:57 -04:00

240 lines
10 KiB
Markdown

# Library-expansion matrix — outstanding follow-ups
After the sequential sweep documented in
`docs/internal/library-expansion-matrix.md`, three follow-up workstreams
remain. None block any shipped row; they cover the loose-thread
cleanup the audit pattern surfaced.
Ordering reflects priority. (1) is needed for the C5 detector to
be runnable end-to-end; (2) addresses a real capability gap
catalogued during the C4 audit; (3) closes the methodology loop
on rows that shipped before the per-row baseline convention
existed.
## 1. scan_runner.cpp dispatch hook for `uninit-local-read`
**Status:** verified locally on 2026-05-26. The scan-runner diff contains
both C2's selective `clang::CallGraph` merge and C5's
`uninit-local-read` dispatch hook. Focused tests and a CLI smoke passed; the
changes are still uncommitted in the worktree.
**Background.** The C5 wiring commit (48dac13d5) landed:
- `include/fold/detect/uninit_local_read.hpp`
- `src/detect/uninit_local_read.cpp`
- `FindingKind::UninitLocalRead` in `include/fold/report/finding.hpp`
- SARIF rule id + name in `src/report/sarif.cpp`
- finding-name parser in `src/report/finding_json.cpp`
- registry entry in `src/cli/detector_registry.cpp`
- 5 unit tests + meson wiring
…but **did not** commit the four `scan_runner.cpp` hunks that:
1. include `fold/detect/uninit_local_read.hpp`
2. set `want_uninit_local = detectorEnabled(...)` and the
`addDetectorProfile("uninit-local-read")` entry
3. invoke `detectUninitLocalReadInContext(ctx)` inside the per-TU
detector dispatch, annotating findings + accumulating profile
4. emit the `spdlog::info("uninit-local-read: emitted …")` line
in the summary section
The hunks were withheld because `scan_runner.cpp` was simultaneously
being edited by a parallel session implementing C2 (clang::CallGraph
selective adapter — `mergeClangCallGraph`, `addLocalCallGraphCallee`,
etc.). Both changes are correct individually; the bundled diff just
needed a coordinated commit moment that never arrived.
**What to do.** Audit the current `src/cli/scan_runner.cpp` diff —
it should contain BOTH C2 and C5 hunks. Run `meson test -C
build/release` (expected 57/57). Commit as a single bundled commit
crediting both changes, e.g.:
```
feat(cli): wire C2 clang::CallGraph + C5 uninit-local-read dispatch
- C2: mergeClangCallGraph + addLocalCallGraphCallee augment
fold's existing ir::ProjectCallGraph with clang TU-local edges
- C5: scan-runner dispatch for the uninit-local-read detector
registered in 48dac13d5
```
Pre-commit clang-format hook will catch two long-line violations
(at the include block + a `const_cast` line); run `clang-format
-i src/cli/scan_runner.cpp` first.
**Verification.** After commit, the canonical smoke is:
```bash
build/release/src/cli/fold scan \
--compile-db /tmp/c5wiring_smoke/compile_commands.json \
--detectors uninit-local-read \
-o /tmp/out.json -f json
# Expected: "uninit-local-read: emitted 3 findings"
```
(`/tmp/c5wiring_smoke/uninit.c` was the fixture used during
C5-wiring development; recreate from the test cases in
`tests/unit/test_uninit_local_read.cpp` if it's gone.)
## 2. Native UAF detector
**Status:** implemented locally on 2026-05-26 as an intra-procedural,
heuristic `use-after-free` source detector with unit tests and CLI dispatch.
Curl-scale baseline captured at
`docs/internal/perf-audit/runs/matrix/UAF-curl-2026-05-26.json` (18 SARIF
findings after report filtering; 11 ms detector wall).
**Carved out from C4 (`docs/internal/library-expansion-matrix.md`).** C4's
original framing was `clang::consumed::ConsumedAnalysis` for
RAII / use-after-free. Audit (commit `1dc20ca`) confirmed
ConsumedAnalysis only activates on classes marked
`[[clang::consumable]]` + member functions marked
`[[clang::callable_when(state)]]` + `[[clang::return_typestate(state)]]`.
Curl/yams/kernel-style C code has none of these attributes.
The capability gap is real: fold has no UAF detector. The right
implementation is **not** ConsumedAnalysis but a fold-native
CFG-walker shape:
**Sketch.**
- New detector: `include/fold/detect/use_after_free.hpp`,
`src/detect/use_after_free.cpp`
- `FindingKind::UseAfterFree` in `include/fold/report/finding.hpp`
- SARIF rule id `fold.use-after-free` + finding-name parser
- Registry entry `"use-after-free"` in `src/cli/detector_registry.cpp`
- scan_runner.cpp dispatch (same template as `uninit-local-read`)
**Detector core** (per-function walk):
- Track local set `freed_ptrs: VarDecl* -> first_free_loc`
- On `free(p)`, `delete p`, `delete[] p`: add `p` (and any aliases
via `q = p;`) to `freed_ptrs`
- On any subsequent read of `p` (DeclRefExpr usage, MemberExpr
base, deref `*p`, subscript `p[…]`): emit a UAF finding citing
both the free site and the use site
- Reset on re-assignment (`p = …` after the free clears it from
the set)
**Soundness scope.**
- Intra-procedural only (interprocedural via summaries is a
follow-up — the existing CrossTUCache pattern fits)
- Trivially correct on flat function bodies; needs CFG awareness
for branches (use the same `BranchLiveness` / `walkIf`
scaffolding the security_walker uses)
- False-positive sources to handle explicitly: `free(NULL)`
doesn't poison anything; `p = realloc(p, n)` is not a UAF
even though `p` is conceptually "freed" — model `realloc` as
a refresh
**Test plan.**
1. `tests/unit/test_use_after_free.cpp` with: simple `free(p); *p;`
flagged; freed-but-reassigned not flagged; `free(NULL)` not flagged;
`p = realloc(p, n)` not flagged; alias poisoning; `delete p; p->x`;
and conditional free/use inside the then-branch.
2. End-to-end SARIF smoke on a synthetic fixture (mirroring the
`c5wiring_smoke` pattern).
3. Curl baseline finding count after the detector lands —
capture under `docs/internal/perf-audit/runs/matrix/uaf-curl-<DATE>.json`.
**Out of scope for the first PR.**
- C++ smart-pointer typestate (covered by C4's deferred form)
- Interprocedural UAF (free in callee, use in caller) — relies
on a `freed_ptr_params` slot in `detect::CalleeProperties`
- Heap-region precision via `symbolic::HeapModel`
(`isUseAfterFree` already exists but the connector to the new
detector is the C++-aware bit)
**Naming.** "use-after-free" matches the SARIF taxonomy and CWE-416
("Use After Free"). The existing `HeapAccessInfo::is_use_after_free`
field already uses this spelling.
## 3. Per-row precision baselines for shipped rows
**Status:** Z4, C5, C7, and the native UAF follow-up now have local
baselines under `docs/internal/perf-audit/runs/matrix/`:
- `Z4-curl-2026-05-26.json`
- `C5-curl-2026-05-26.json`
- `C7-curl-2026-05-26.json`
- `UAF-curl-2026-05-26.json`
- `C9-synthetic-2026-05-26.json`
- `Z5-synthetic-2026-05-26.json`
- `Z10-serialize-corpus-2026-05-26.json`
- `C8-contracts-synthetic-2026-05-26.json`
All listed shipped-row baseline artifacts now have at least one local artifact.
Curl-scale artifacts for Z5/C8/C9 remain optional follow-up work where the
existing blockers or cost justify a dedicated session.
**What to capture per row.**
| Row | Target | Tool | Output |
|---|---|---|---|
| Z5 | synthetic minimal-witness fixture | one-off C++ invoking `refineModelForMinimalWitness` | **done:** `docs/internal/perf-audit/runs/matrix/Z5-synthetic-2026-05-26.json`; curl fuzz-prep remains blocked by pre-existing crash |
| Z10 | encoder-shaped n=100 corpus | one-off C++ invoking z3 `to_string()` with alias thresholds | **done:** `docs/internal/perf-audit/runs/matrix/Z10-serialize-corpus-2026-05-26.json` |
| C5 | curl baseline | `fold scan --detectors uninit-local-read --profile` | **done:** `docs/internal/perf-audit/runs/matrix/C5-curl-2026-05-26.json` |
| C7 | curl baseline | `fold scan --detectors deprecated-api --profile` | **done:** `docs/internal/perf-audit/runs/matrix/C7-curl-2026-05-26.json` |
| C8 | synthetic twin baseline | `fold scan --detectors null-deref --profile` with `/*@ requires */` annotations vs without | **done:** `docs/internal/perf-audit/runs/matrix/C8-contracts-synthetic-2026-05-26.json` |
| C9 | synthetic + curl | `fold scan --fix-output …` | **synthetic done:** `docs/internal/perf-audit/runs/matrix/C9-synthetic-2026-05-26.json`; curl patch-count still optional |
**Naming convention.**
`docs/internal/perf-audit/runs/matrix/<row>-<target>-<YYYY-MM-DD>.json`,
matching Z4's path.
**Effort.** Each curl baseline is ≈9 minutes wall (one `--runs 1`
of `perf_audit_run.py`). Total ≈54 min for the four curl-shaped
rows + a few one-off Python invocations for Z10/Z5. Best run in a
single dedicated session with the laptop idle.
**Update step.** Each row's `Status` column in
`docs/internal/library-expansion-matrix.md` gets a link added to the new
baseline file, mirroring Z4's:
```
**shipped** — `docs/internal/perf-audit/runs/matrix/<row>-<target>-<DATE>.json`,
`tests/unit/test_<row>.cpp`
```
## Cross-cutting cleanup
These showed up repeatedly across the matrix sweep and don't fit
any single row, but should be cleaned up at some point:
- **Untracked workspace junk.** `profile.json`, `reach.txt`,
`reach_stderr.txt`, `scan_output.log`, `taint_trace.log`,
`trace.log`, `yams_full_scan.log`, `yams_security_bench.json`,
`scratch/`, `tools/fold-rust/-`, `tools/fold-rust/target/` are
all in `git status -u` as untracked. They're scratch artifacts.
Either gitignore them (preferred — these are reproducible
outputs) or delete them.
- **Parallel-session uncommitted edits** in `docs/USAGE.md`,
`docs/internal/rust-frontend.md`, `src/frontend/rust_backend.cpp`,
`src/mcp/tool_registry.cpp`, `tests/integration/meson.build`,
`tests/integration/test_cross_tu_security.cpp`,
`tests/integration/test_mcp_stdio.cpp`,
`tools/fold-rust/src/main.rs`. These belong to a Rust-frontend
workstream that's orthogonal to the library-expansion matrix.
Audit and commit separately.
- **Tests with chained-comparison warnings** flagged by
clang-tidy throughout the unit-test sweep (e.g.
`REQUIRE(stats.queries > 0 || !findings.empty())` parsing as
`v0 <= v1 != v2`). Catch2 macros tickle this lint pattern even
though the semantics are fine. Either suppress at the file
level or wrap conditions in parens.
## Suggested next-session opener
```
# Resume library-expansion follow-ups (see docs/internal/library-expansion-followups.md)
# Order: (1) scan_runner C2+C5 commit, (2) UAF detector, (3) baselines
```
(1) is mechanical (~10 minutes including verification).
(2) is a real new detector (~half a session including tests + curl smoke).
(3) is batch capture (~1 hour curl scans).