27e1b23e7d
Iteration 9 of the fold optimization loop (docs/research/optimization-loop-2026-07-30.md). Iteration 9 targeted the control-walk condition-fact cost (35.1s of the sql-injection profile). Measurement-first: a scratch patch (never committed) counted duplicate feasibility queries in a live scan — 1314 total, 1268 distinct (3.5% duplicate) — so a memo cache caps out at ~1.2s of 35.1s, far below the >=20% KPI. The checks are genuine, distinct SMT solving; memoization is a measured dead end. Scratch patch fully reverted; production byte-identical. Viable future directions recorded for the next campaign: fast-path widening beyond affine parameter comparisons, incremental solving over the pushed path stack, dual-walk (CFG/AST fallback) elimination. This commit folds the formatting normalization applied to two campaign-authored files after their initial commit.
51 lines
2 KiB
Bash
Executable file
51 lines
2 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Reproduction rig for the load-sensitivity flake in symbolic summary
|
|
# recomputation (fold optimization loop iteration 5; see
|
|
# docs/research/optimization-loop-2026-07-30.md).
|
|
#
|
|
# Observed once (2026-07-31): the characterization test
|
|
# 'repeated nested summary computation stays stable across a reused
|
|
# scratch context' produced an empty return_post ('' vs 'arg0') in a
|
|
# 7/10-run burst while background fold scans saturated the machine,
|
|
# then passed 16/16 on a quiet machine with byte-identical production
|
|
# code. Three bounded rig attempts (2026-07-31) failed to reproduce:
|
|
# 1. 8 subshell busy-loops + 10 filtered test runs -> 10/10 PASS
|
|
# 2. 8 `yes` workers (verified via uptime) + 10 full runs -> 10/10 PASS
|
|
# 3. concurrent real fold sql-injection+uninit-read scan
|
|
# (-j 4, Z3 contexts churning) + 10 full runs -> 10/10 PASS
|
|
# Prime suspect remains isValidPredicate()'s 100ms Z3 solver timeout
|
|
# (summary_encoder.cpp): under CPU starvation check() returns unknown
|
|
# and predicate-record decisions flip. The flake window may also
|
|
# involve memory pressure or specific solver states not covered by the
|
|
# small fixture. Re-run this rig after scanner changes that touch the
|
|
# summary path; escalate to deterministic predicate decisions if it
|
|
# ever reproduces.
|
|
#
|
|
# Usage: tests/benchmarks/run_summary_flake_rig.sh [build-dir]
|
|
set -euo pipefail
|
|
BUILD_DIR="${1:-build/release}"
|
|
TEST_BIN="$BUILD_DIR/tests/unit/test_symbolic_summary"
|
|
cd "$(dirname "$0")/../.."
|
|
|
|
[[ -x "$TEST_BIN" ]] || {
|
|
echo "test binary missing: $TEST_BIN" >&2
|
|
exit 2
|
|
}
|
|
|
|
for i in 1 2 3 4 5 6 7 8; do yes >/dev/null & done
|
|
trap 'kill $(jobs -p) 2>/dev/null || true' EXIT
|
|
sleep 2
|
|
echo "load during run: $(uptime)"
|
|
|
|
PASS=0
|
|
FAIL=0
|
|
for i in $(seq 1 10); do
|
|
if "$TEST_BIN" >"/tmp/summary-flake-rig-$i.log" 2>&1; then
|
|
PASS=$((PASS + 1))
|
|
else
|
|
FAIL=$((FAIL + 1))
|
|
echo "FAILED run $i: /tmp/summary-flake-rig-$i.log" >&2
|
|
fi
|
|
done
|
|
echo "summary-flake rig: PASS=$PASS FAIL=$FAIL"
|
|
[[ "$FAIL" -eq 0 ]]
|