fold/docs/research/assertion-invariant-maturity-audit-2026-06-03.md

# Fold Assertion and Invariant Maturity Audit

> Date: 2026-06-03
> Scope: assertion policy and invariant hardening for Fold's source, binary,
> frontend, report, and confirmation paths.

## Current assertion model

Fold does not currently have a project-local assertion layer such as
`FOLD_ASSERT`, `FOLD_DCHECK`, or `FOLD_UNREACHABLE`.

Current correctness controls are mostly:

- ordinary error returns (`std::nullopt`, `{}`, error strings);
- warning/error logging via `spdlog`;
- tests and golden fixtures;
- report-level filtering/deduplication;
- graceful optional-tool fallback for r2 CLI, KLEE, Swift/Rust frontends, and Ghidra specs.

This is appropriate for environmental/recoverable failures, but several internal
analysis invariants should be made explicit over time so bugs fail near the
cause instead of leaking misleading findings.

## Policy recommendation

Add a small project-local assertion layer only after one or two high-value call
sites prove the need. Do not migrate broadly in one sweep.

Suggested API:

```cpp
FOLD_ASSERT(cond, "message")       // always-on invariant; terminates
FOLD_DCHECK(cond, "message")       // debug/canary-only sanity check
FOLD_PRECONDITION(cond, "message") // programmer contract at API boundary
FOLD_UNREACHABLE("message")        // impossible control-flow path
```

Initial implementation should include file/line/function diagnostics, preferably
through `std::source_location` where the project C++ standard permits it, or a
small macro wrapper if not.

Do not use assertions for:

- missing optional tools;
- missing compile DB in a mode where the user can fix input;
- unsupported binary architecture;
- malformed external frontend output;
- missing KLEE/SymCC/angr adapters.

Those remain recoverable errors or skips.

## Highest-value invariant targets

| Priority | Site/flow | Recommended tier | Invariant |
|---:|---|---|---|
| 1 | Report emission / benchmark comparator | Always-on or test gate | Emitted user-facing findings have physical locations and backend provenance. |
| 1 | `scan_router` mode resolution | Always-on internal check | `source`, `binary`, `hybrid`, `auto` resolve to the intended backend set; hybrid does not silently erase source findings. |
| 1 | P-Code SSA call annotation | Debug + unit-gated invariant | Raw call-site names are not overwritten by synthesized SSA target fallback. |
| 1 | P-Code raw div/rem gating | Debug + unit-gated invariant | If machine division addresses are known, div/rem findings only originate at those addresses. |
| 2 | Rust/Swift frontend NDJSON parsing | Recoverable error + counter metric | Malformed nodes degrade per-node or per-function; they must not abort the whole scan. |
| 2 | Confirmation task/result JSON | Always-on parser validation | Required schema keys are present; unknown optional keys are tolerated; canonical names remain centralized. |
| 2 | Finding construction helpers | Always-on invariant in helper | New findings set detector name, backend, confidence, and at least one location unless explicitly internal. |
| 3 | Benchmark harness baselines | Always-on script failure | Malformed spec/baseline files fail closed, not as a passing empty suite. |
| 3 | Cache/fact identity | Debug check | Fact IDs and summary cache keys are stable and non-empty for stored entries. |

## Gap list

### Missing always-on invariant checks

- No central finding builder enforces location/backend/detector completeness.
- The report filter drops locationless findings, but the producing site may not
  fail or count the invariant violation.
- Router mode/backend resolution is tested, but not asserted at the transition
  where findings are merged.
- Confirmation task/result schemas round-trip in tests, but parser failures are
  not yet framed as schema invariants with clear diagnostics.

### Debug-only checks worth adding first

- P-Code call annotation should assert in debug/canary that an existing
  call-site-resolved `call_target` is not replaced by a different synthesized
  target.
- Raw P-Code div/rem findings should log/debug-count cases suppressed because the
  originating machine instruction is not a real division.
- Rust/Swift IR encoders should count per-node degradation events so a frontend
  regression is visible even when the scan succeeds.

### Avoid assertion misuse

- Do not assert when optional tools are absent. `r_core`, Capstone, KLEE, Swift,
  and Rust frontends all have legitimate unavailable states.
- Do not assert on user/project input errors such as missing paths or invalid
  compile databases. Return exit code 2 with actionable diagnostics.
- Do not assert inside benchmark comparators for expected benchmark failures;
  return structured JSON with failures.

## Rollout plan

### Step 1 — codify invariants in benchmark gates

Already started:

- `compare_findings.py` fails on missing physical locations;
- `compare_findings.py` fails on missing backend provenance;
- binary/source golden specs express `must_report` and `must_not_report` rules.

Next:

- add a benchmark summary field for suppressed/degraded encoder events;
- fail benchmark specs with zero tests unless explicitly allowed.

### Step 2 — add central finding validation helper

Create a small helper in report or scan code, for example:

```cpp
bool isUserFacingFindingWellFormed(const Finding& f);
```

Use it in:

- report filtering tests;
- benchmark comparator equivalent in C++ tests;
- optional debug logging before report emission.

Only after that consider always-on `FOLD_ASSERT` in narrow internal helpers.

### Step 3 — add assertion header if repeated call sites justify it

Potential file:

```text
include/fold/core/assert.hpp
```

Initial macros/functions:

- `FOLD_ASSERT`
- `FOLD_DCHECK`
- `FOLD_UNREACHABLE`

Keep it independent of detector code and avoid side-effectful expressions.

### Step 4 — migrate high-risk sites only

First candidates:

1. P-Code call-target annotation conflict check.
2. Finding-builder completeness checks in shared helper paths.
3. Router merge provenance checks.
4. Confirmation schema parse/serialize invariants.

Do not migrate every `return {}` or `std::nullopt`; many are intentional
recoverable degradation paths.

## Concrete candidate sites

| File | Candidate | Why |
|---|---|---|
| `src/binary/pcode_router.cpp` | `annotateBySite` / SSA annotation | Prevent call target confusion that caused `memchr`/`memcpy` FPs. |
| `src/symbolic/pcode_encoder.cpp` | div/rem hazard gate | Ensure known machine-address gating remains active for raw scans. |
| `src/report/filter.cpp` | post-filter sanity | Detect whether locationless findings are being produced frequently. |
| `src/cli/scan_router.cpp` | backend merge | Ensure router-stamped provenance survives merge. |
| `src/confirm/confirmation_task.cpp` | JSON parsing | Enforce canonical keys and required fields. |
| `src/frontend/rust_backend.cpp` / `src/frontend/swift_backend.cpp` | NDJSON parse degradation counters | Make frontend artifact regressions visible without crashing scans. |
| `scripts/benchmarks/compare_findings.py` | spec shape validation | Fail closed on malformed benchmark specs. |

## Suggested validation for assertion rollout

Before adding an assertion macro:

```sh
meson test -C build/release fold:pcode_encoder fold:pcode_summary fold:scan_e2e --print-errorlogs
scripts/benchmarks/summarize_maturity.py --build build/release --all
```

After adding always-on assertions, also run:

```sh
meson test -C build/release --print-errorlogs
```

## Current recommendation

Do not add a broad assertion layer yet. The immediate maturity win is to keep
encoding invariants as benchmark gates and focused tests. Add `FOLD_ASSERT` only
when a second or third high-value C++ call site needs the same invariant behavior.