Fix audit findings: soundness, coverage reporting, docs, and review follow-ups #1
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "fix/audit-findings"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Fixes the confirmed findings from an audit of fold's soundness, reporting and docs, then the findings from a code review of those fixes. There are 18 commits on
master(b3da5a4): the 10 audit fixes, a README/docs commit, and 7 review fixes. Fixes come with a failing test written first unless noted.Commits
76bc00ftest(fanout): build the failure worker before the recovery testssubprocess_fanoutfailed on a clean build because its helper binary was not a test dependency.depends: [fanout_failure_worker].6adcc24fix(detect): never prove post-loop conditions from a bounded unrollloop_unrolliterations. Early exits (a symbolic bound,break,goto) were dropped, so a condition afterfor (i < n)was "Proved" dead or constant from the loop having run exactly N times.LoopUnrollExits(newloop_havoc.{hpp,cpp}, which also absorbs the three duplicated havoc helpers) marks the post-loop state as exact only when the unroll left through the loop's only exit. Otherwise the assigned variables are havocked and the entry path is restored.test_loop_exit_soundness.cppcovers symbolic-bound loops, abreakloop, an exact constant loop as a control (still Proved), and the helpers.67483eefix(model): apply libc models only to C library calleesmine::free(p); free(p);was a Proved double-free, andShell::system(...)was a command-injection sink.isCLibraryFunction/isCLibraryCalleerequire the resolved declaration to be global, instd, orextern "C", and never a member. Heuristic web-framework taint entries keep name matching (c_api = false).test_library_callee_identity.cpp(a namespaced free, a member free, a membersystem) with libc controls.0556cc6fix(report): keep Proved for exact bounded models onlydocs/research/soundness.mddefinesprovedas an exact bounded model, but double-free (heap flag), static-ptr-asymmetry (pattern), unused-function and dead-statement (AST facts), the stdlib copy check, and heap OOB/UAF (SAT without a precision check) all assigned Proved.sizeof/alignofon complete non-VLA types is now an exact constant (typeTraitConstant), somalloc(sizeof(T))paths can stay exact.test_proved_requires_exact_model.cpp.c9811b7fix(scan): report unanalyzed translation units and exit 2compile-errors,frontend-failed,analysis-failed,timeout,crashorworker-exit. Workers carry it onfile_finished.invocations[0].executionSuccessfulplus onetoolExecutionNotificationsentry per gap. Findings JSON gets an additivecoverageobject.2after writing the report. CLI diagnostics go to stderr.docs/USAGE.mdandAGENTS.mddocument the fields and the exit code.test_scan_coverage.cpp, plustest_cli_paths[coverage](clean TU exits 0; broken TU exits 2 and SARIF on stdout still parses).46aaac4fix(summary): key function summaries by USR, not qualified nametake(int*, long)(which null-checks) inheritedtake(int*, int)'s nonnull precondition and was reported as a null-deref. Lookups also spelled static functions differently from stores, so their summaries were never found.summaryStoreKey()(USR viasemanticFunctionId) is used for every clang-side store, lookup and dependency edge. Now that static callees' summaries are found, null-deref's summary path applies the same DEBUGASSERT and static-parameter-contract filters as its callee-deref path.test_summary_key_identity.cpp: the overload call was red before the fix; a control call to the dereferencing overload still reports.5c352b2fix(solver): fail closed when the portfolio re-check disagrees--portfolio, a SAT verdict stood even when the reporting solver's re-check timed out or disagreed, so findings were emitted with no model.reconcilePortfolioRecheck()returns the verdict only when both solvers agree, and unknown otherwise. It is applied at all six sites (null-deref, div-by-zero, int-overflow).test_model_validation.8a32f08fix(int-overflow): render witnesses in the source type's width and signint awitness printed the raw 64-bit symbol, e.g.a=2147483648.sourceIntTypesOf,formatSourceIntValue).test_int_overflow[witness].f0fe006fix(docs,tests): correct the pre-push hook claim, a dead link, MCP cleanupprior-art.mdlinked toprior-art-full.md, which never existed. The MCP testTempDirdestructor could throw and leave directories behind.TempDiruses the non-throwingremove_all. It stays under the working directory, because the MCP server rejects paths outside its workspace (moving it to the system temp directory broke three MCP tests).c785a6adocs(positioning): drop claims the code and evidence do not support2945e59docs: trim the README and move the performance roadmap to docs/docs/USAGE.md. The performance section moves todocs/architecture/performance.md, which now says the gold set is fold's own reviewed output matched by kind and file, so it catches drift rather than measuring precision/recall. The lint hook note moves toBUILD.md.Review fixes
A code review of the commits above found 10 issues. Eight were fixed across the 7 commits below, most with a red test first. #6 was confirmed by reading the code, because a portfolio disagreement cannot be forced end to end; its test covers the new helper. #10 is a refactor. Two did not reproduce, and one half of #10 is deferred (see Deferred).
9e37fd0fix(detect): model continue exits and memory stores in truncated loops (review #1, #7)LoopUnrollExitstreatedcontinueas a non-exit, so a loop with only acontinueescape was still "exact" and post-loop conditions were Proved. When the unroll was truncated, only assigned scalars were havocked; array elements, members and stores through pointers kept their pre-loop values, soif (a[0] == 0)after such a loop was Proved.continueinside the analysed loop (not inside a nested loop) counts as an escape.LoopUnrollExits::havocalso havocs the base arrays/aggregates written by subscript or member stores; a store through a pointer, arrow or unknown base havocs all heap arrays and marks the path over-approximated. The dead-branch, always-condition and unreachable sites callexits.havoc.test_loop_exit_soundness[continue]and[memory](array store and calloc'd buffer), plus an opaque-call loop guard for review #9.964769cfix(oob): require an exact encoding before a constant-index OOB is Proved (review #3)emittakesexactEncoding(env)and assigns Proved only with it.test_proved_requires_exact_model[oob]: an exact control stays Proved; an opaque call on the path is now Heuristic (was Proved).29a292cfix(oob,uaf): re-check portfolio verdicts on the reporting solver (review #2)--portfolio, OOB and UAF took the portfolio's verdict as final. The reporting solver was never checked, so OOB read its counterexample from an unchecked solver (i=0underi > 5), and a SAT the reporting solver would not reproduce was still emitted.symbolic::recheckPortfolioVerdictre-checks a decided verdict on the reporting solver and reconciles the two (disagreement or timeout -> unknown). It is applied at the four OOB and two UAF sites.test_proved_requires_exact_model[portfolio]: the counterexample must satisfy the path (wasi=0, nowi=6).0008244fix(encoder): a global C++ lookalike is not a C library function (review #8)isCLibraryFunctionaccepted any global non-member function, so a project'svoid free(Obj*)overload was libc free: calling it twice was a double free, and reading after it a use-after-free.extern "C", instd, or declared in a system header.extern "C"also covers C translation units.test_library_callee_identity[global](double-free and UAF 1 -> 0);test_encoder_callee_namesflips the global C++ helper.b8c997cfix(scan): report quarantined TUs as coverage gaps (review #5)frontend-failed.quarantined; a worker emits aquarantinedfile_finishedrecord so the parent classifies it the same way.docs/USAGE.mdlists the reason.test_subprocess_fanout[quarantine]covers serial and fan-out.ed48d44fix(solver): never memoize a core from a portfolio disagreement (review #6)recheckPortfolioVerdictand act only on the reconciled verdict.SolverStats::portfolio_disagreementscounts disagreements and is merged, printed and exported in the detector profile; the sites traceUNKNOWN (portfolio re-check disagreed).test_model_validation[portfolio](agreement, unknown, disagreement, merge).fcf474erefactor(cli): one TU path normalizer for coverage, fan-out and quarantine (review #10, part)normalizeClangTuPath,normalizeQuarantinePathand fan-out'snormalizePathwere three copies of one routine that must agree.fold/cli/tu_path.hppnormalizeTuPath, used everywhere. No behaviour change.Tests
meson test -C build/release: 140/140 pass after the review fixes (macOS arm64, LLVM 21, Apple clang for Conan deps).scripts/lint.sh --ci: exit 0 (format clean; cppcheck reports only pre-existing advisory warnings).rust_frontend's 500 ms fake-frontend timeout test flaked once at load average 22 during the audit fixes and passed on four reruns. It is timing-sensitive and unrelated to this PR.Before / after
Before is a build of
b3da5a4; after is the audit fixes (c785a6a). Findings are listed askind:confidence.mine::freethenstd::free), heap detectorscoverage0/1, gapcompile-errorscompile-errorsfrontend-failed, exit 2 (fan-out and serial)tests/fixtures/clones/exact.cppint-overflow witnessa=2147483648, b=0a=-2147483648, b=0yams dogfood (
src/cli/time_parser.cpp,src/topology/protected_relation_cover.cppviabuild/debug/compile_commands.json):--scope all: identical before and after (2 clone, 2 high-complexity, 1 int-truncation, 1 sign-conversion, 2 unsigned-wrap, all Heuristic or Syntactic). After also reports coverage 2/2 with no gaps.After the review fixes (
--scope security,redundancy --detectors always-condition,unreachable):x / d), int-overflow:latent, unsigned-wrap:heuristiccompile-errorscontinueloop (hit == 0after it)a[0] == 0after it)void free(Obj*)called twicequarantined(was exit 0 complete /frontend-failed)yams dogfood after the review fixes: identical to the table above (default: 2 clone:syntactic;
--scope all: 2 clone, 2 high-complexity, 1 int-truncation, 1 sign-conversion, 2 unsigned-wrap), coverage 2/2, exit 1.Deferred / follow-ups
fileand a scan run from another directory reports 1/1 analysed in serial and fan-out, because the front end reports the main file absolutely.advance(&i)already havocsi. The probe stays as a guard test.attachPrecisionEvidence): the null-deref and div-by-zero copies encode different downgrade rules (null-deref's sink-local exception, div-by-zero's latent carve-out). Sharing it would add precision evidence to OOB/UAF findings, a report change that belongs in its own PR.recheckPortfolioVerdict.oobandnull-derefboth enabled,Obj* o = (Obj*)malloc(sizeof(Obj)); o->v = 1;is reported as oob-access:proved with "region size is 1 bytes". The/usr/local/bin/foldbuild reports it too.per_query_ms=30000,budget_seconds=604800andper_tu_timeout=86400.SolverBudget::totalis not enforced by the main detectors, and there is no Z3memory_max_size. The long defaults are a documented product choice ("findings are valued over wall time"), and enforcingtotaltouches every detector's query loop. This needs a maintainer decision on the defaults and its own PR. TU-level timeouts and worker kills now surface as coverage gaps.unknownresults are still counted only in stats and logs. Surfacing them per function needs worker→parent stats plumbing.bench/, untouched here):_generate_juliet.py).fp-iter*.jsonfiles, andscore_fp.pyreturns[]when gold is missing.bench/score.pyalways returns 0.scripts/benchmarks/fold_eval/scoring.pymatches by kind+file only. Tightening it changes the gate's baselines, so it belongs with the gold rework./usr/local/bin/foldreports0.1.0+259ae527d4a9.dirty, 23 commits behind and built from a dirty tree. Reinstall from a clean HEAD before dogfooding.if (!p)aftermallocis reported always-false (Proved). This is still visible on p2_shadow and needs a nullable-allocation default decision.WARN+returnwhen a binary or backend is missing pass without testing anything. Several detectors (format-string, memory-leak, source double-free, iterator-invalidation) lack dedicated tests.These changes were written by Claude (Anthropic's AI assistant, via Claude Code) at the maintainer's direction, from an audit the maintainer requested. The audit fixes were each reproduced on a HEAD build, test-first. The review fixes followed the same process, except for the #6 and #10 notes above.
malloc/free heap modeling (encoder), use-after-free's free/realloc, and the taint source/sink tables selected a model by the callee's bare identifier. A project function that merely shares a libc name was modeled as libc: namespace mine { void free(void*); } mine::free(p); ... std::free(p); // double-free "proved", plus // oob "double free" and a UAF and `Pool::free(p)` behaved the same way, as did a member function named `system` as a command-execution sink. isCLibraryFunction / isCLibraryCallee (encoder_callee_names) accept a direct callee that is not a class member and is declared at global scope, in namespace std, or extern "C". The encoder's malloc-family and free models, use-after-free's free/realloc, and every libc/POSIX taint entry now require it. Entries that are deliberately method-name heuristics (get, Query, execute, query, fromJson, readString*, the request-parameter getters) are marked c_api = false and keep matching by name as before. Tests: test_library_callee_identity (namespaced and member `free` are not double-free/UAF; member `system` is not a sink; libc free twice and libc system(getenv()) still are), and isCLibraryCallee cases in test_encoder_callee_names (::free, std::free, mine::free, Pool::free, a global helper, an indirect call). Full suite 137/137.