# Optimization micro-benchmarks

`bench/run_micro_opt.py` is the synthetic A/B harness for the current speed and
false-positive reduction backlog. It generates tiny C++ projects, runs `fold scan
--profile`, and writes per-case SARIF/profile artifacts plus `summary.json` and
`summary.md`.

Use it before and after each optimization patch:

```sh
python3 bench/run_micro_opt.py --case all --repeats 3
# compare medians in bench/results/<timestamp>_micro-opt/summary.json
```

For quick smoke while iterating:

```sh
python3 bench/run_micro_opt.py --case all --size 4 --repeats 1 --out-dir /tmp/fold-micro-smoke
```

## Cases

| Case | Optimization validated | Main metrics to compare | Invariant |
|---|---|---|---|
| `branch-merge` | `SecurityWalker::walkIf` / `SymbolicEnv` dirty-binding or cheaper branch join logic | `null-deref.wall_ms`, `states_merged`, `states_kept_separate`, `serial_dispatch_ms` | finding count and confidence distribution stay stable |
| `defuse-null` | direct `DefUseChains` lookup by `(var, use_loc)` in `NullDerefObserver::shouldQuery` | `null-deref.wall_ms`, `z3_queries`, external elapsed | finding count stays stable |
| `cross-tu-prime` | skipping/deduplicating redundant per-TU cross-TU priming after global priming | `serial_dispatch_ms`, `total_wall_ms`, findings by rule | cross-TU callsite findings stay stable |
| `precision-fp` | shared precision evidence / confidence demotion for imprecise security findings | `by_confidence`, detector finding counts | exact findings stay; imprecise findings move from `proved` to `heuristic` where appropriate |
| `expr-cache` | query-hash/cache changes, especially avoiding expensive canonical string hashing | detector `wall_ms`, `z3_queries`, `z3_timeouts` | findings and confidence distribution stay stable |
| `curl-str-bound` | Curl-style bounded parser output (`curlx_str_*`) feeding `curlx_strlen(&out)` into a stack buffer terminator index | `oob.findings_emitted`, `oob.wall_ms`, `z3_queries` | max+1 buffer terminator writes produce 0 findings while undersized-buffer unit tests still flag |

## Suggested acceptance gates

Do not use absolute milliseconds as hard gates across machines. Prefer comparing
a before/after run on the same machine/build:

- Performance patches: median target metric should improve by at least 10% on the
  relevant case, with no new findings and no lost findings unless the patch is
  explicitly an FP reduction.
- FP/confidence patches: total findings may stay the same, but confidence
  distribution should move in the intended direction; exact/proved positives in
  the smoke cases must not disappear.
- Profile/instrumentation patches: new metrics should appear in profile JSON, and
  existing metrics should remain populated.

Each repeat writes raw artifacts:

```text
<case>-<size>-r<N>.sarif
<case>-<size>-r<N>.profile.json
<case>-<size>-r<N>.stdout.log
<case>-<size>-r<N>.stderr.log
```

Use `--keep-workdir` when you want to inspect generated source and compile DBs.

## Real-code presets

`bench/run_security_targets.py` runs the current source-security presets:

```sh
python3 bench/run_security_targets.py --preset yams-wide20-security --repeats 3
python3 bench/run_security_targets.py --preset all --repeats 1
```

Presets skip themselves when the local compile database is missing. Use
`yams-wide20-security` as the quick real-code gate, then `yams-full-security` and
`curl-security` for slower validation.

## Added case: `control-nesting`

`control-nesting` stresses nested branches/loops and should be used for changes
that target `control_walk_ms`, sink prechecks, or branch pruning counters.