Commit graph

32 commits

Author SHA1 Message Date
trvon
b03c8615ca
fix(sarif): point informationUri at git.trevon.dev
Some checks failed
build / build-linux (pull_request) Failing after 8m12s
build / build-macos (pull_request) Has been cancelled
build / release (pull_request) Has been cancelled
fold lives only on git.trevon.dev, but every SARIF report advertised
https://github.com/trvon/fold as the tool's information URI, so code
scanning UIs and SARIF viewers linked readers to a repository fold no
longer uses. The driver now points at https://git.trevon.dev/trevon/fold.

test_scan_e2e asserts the URI; it failed before this change.
2026-09-25 08:07:37 -06:00
trvon
fcf474e4f0
refactor(cli): one TU path normalizer for coverage, fan-out and quarantine
Some checks failed
build / build-linux (pull_request) Failing after 7m32s
build / build-macos (pull_request) Has been cancelled
build / release (pull_request) Has been cancelled
Problem: normalizeClangTuPath, normalizeQuarantinePath and fan-out's
normalizePath were three copies of the same absolute + weakly_canonical
routine. Coverage, worker records and the quarantine file must agree on a
TU's spelling, and three copies can drift apart.

Fix: move the routine to fold/cli/tu_path.hpp as normalizeTuPath and use it
at every site. No behaviour change.

Test: existing test_scan_coverage (path spellings), test_tu_quarantine and
test_subprocess_fanout, updated to the new name. Full suite: 140/140.
2026-09-24 21:46:12 -06:00
trvon
b8c997c732
fix(scan): report quarantined TUs as coverage gaps
Problem: a TU skipped because a prior run quarantined it was not analysed,
but coverage did not say so. The serial path dropped it from the attempted
set, so a scan whose only TU was quarantined was complete and exited 0; the
fan-out path saw no file_finished record and called it frontend-failed.

Fix: the shard runner counts every requested TU and records skipped ones
with gap reason `quarantined`; a worker emits a file_finished record with
that reason so the parent classifies it the same way. USAGE lists the reason.

Test: test_subprocess_fanout "[quarantine]" covers serial (was 2 attempted,
no gap) and fan-out (was frontend-failed). CLI probe: serial and fan-out
both exit 2 with a `quarantined` gap.
2026-09-24 21:41:42 -06:00
trvon
f0fe006d6b
fix(docs,tests): correct the pre-push hook claim, a dead link, MCP cleanup
Problem: the README said setup enables "tracked ... OpenGrep pre-push
hooks", but only .githooks/pre-commit is tracked, so fresh clones get no
pre-push hook. docs/research/prior-art.md pointed to prior-art-full.md,
which has never been in the repository. test_mcp_stdio's TempDir
destructor called the throwing remove_all overload; a failure during
unwinding terminates the test binary and leaves .fold-mcp-test-*
directories behind.

Fix: the README names only the tracked pre-commit hook. The pre-push
hook is not committed: it runs `opengrep --config auto`, which needs the
network. The dead reference is removed. TempDir uses the error_code
remove_all overload. It stays under the working directory, because the
MCP server rejects paths outside its workspace root (moving it to
temp_directory_path broke three MCP tests).

Test: mcp_stdio passes.
2026-09-24 20:57:32 -06:00
trvon
46aaac427a
fix(summary): key function summaries by USR, not qualified name
Problem: SummaryStore keyed summaries by getQualifiedNameAsString (plus
"@file" for internal linkage), which overloads share. When a TU calls a
function whose body lives in another TU, the lookup returns whichever
overload was summarized last. Example: `take(int*, int)` dereferences p,
`take(int*, long)` checks it, and a call `take(q, 1L)` with a null q was
reported as a null-deref because it inherited the other overload's nonnull
precondition. Lookups also used a different spelling (no "@file") than
stores for static functions, so their summaries were never found at call
sites.

Fix: summaryStoreKey()/summaryStoreKeyForCall() return the declaration's
USR (semanticFunctionId: USR, TU-qualified for internal linkage). Every
clang-side store, lookup, dependency edge and staleness check uses it:
summary_encoder, null-deref, div-by-zero, OOB and the summary priming
pipeline, which still selects targets by the call graph's qualified names.
Rust summaries keep their own keys. Now that static callees' summaries
are found, null-deref's summary-precondition path applies the same
DEBUGASSERT and static-parameter-contract filters as its callee-deref
path.

Test: tests/unit/test_summary_key_identity.cpp covers per-overload
summaries, key identity across redeclarations and distinct keys across
namespaces, and the cross-TU overload call. That call reported a false
null-deref before the fix; a control call to the dereferencing overload
still reports one. Tests that hand-built the old keys
(test_symbolic_summary, test_cross_tu_security) now derive them from the
declaration. Full suite: 140/140 (rust_frontend's 500 ms timeout test
flaked once at load average 22 and passed on four reruns).
2026-09-24 20:49:38 -06:00
trvon
c9811b7d41
fix(scan): report unanalyzed translation units and exit 2
Problem: a TU the front end could not parse left no trace. ClangTool::run's
status was discarded, TUs with compile errors were analyzed on a partial
AST and counted as done, and the isolated-worker path dropped TUs that
never produced an AST. A scan of only broken files printed no findings and
exited 0, the same as a clean scan. Separately, the scan logger wrote to
stdout, so any warning corrupted a SARIF/JSON report written to stdout.

Fix: every TU now yields an outcome (clean, compile-errors,
analysis-failed); workers carry it on their file_finished record, and the
fan-out parent adds timeout/crash/worker-exit for the TU a worker died on.
tuCoverage() turns the requested file list plus outcomes into a
ScanCoverage (requested TUs with no outcome are frontend-failed). SARIF
gets a run `invocations` entry (executionSuccessful, one
toolExecutionNotification per gap); findings JSON gets an additive
`coverage` object. A scan with any gap, or with nothing analyzed, exits 2
after writing its report. CLI diagnostics now go to stderr, as
docs/internal/cli-philosophy.md principle 12 requires. USAGE.md and
AGENTS.md document the coverage fields and the exit code.

Test: tests/unit/test_scan_coverage.cpp (classification, path spelling,
complete/untracked controls, merge, both writers, and unchanged output
without coverage); test_cli_paths "[coverage]" covers a clean TU (exit 0,
complete) and a TU with a missing include (exit 2, gap in JSON, SARIF on
stdout parseable with executionSuccessful=false). Before: both probe files
exited 0 with no coverage information. Full suite: 139/139.
2026-09-24 20:37:51 -06:00
trvon
76bc00f635
test(fanout): build the failure worker before the recovery tests
fold:subprocess_fanout reads FOLD_TEST_FANOUT_FAILURE_WORKER, but the
worker target is build_by_default: false and the test declared
`depends: []`, so a plain `meson test -C build/release` ran the two
recovery cases against a binary that was never built and failed both
(`REQUIRE(std::filesystem::exists(helper))`). The test now depends on
the worker, and the suite is green from a fresh build.
2026-09-24 20:00:17 -06:00
trvon
fd4a68dfff
test: add coverage for project graph, pcode encoder, and detector behaviors
Unit + integration coverage for the session's work: incremental project
graph/index, pcode encoder, unreachable/dead-branch, equiv/cross-TU,
div-by-zero/sign-conversion/int-truncation, use-after-free, quality-precision,
scan-router profile, source TU runner, detector plan, and the eval fixtures.
2026-08-16 16:46:07 -06:00
trvon
86b2f14a43
feat(analysis): harden project-wide scans
- persist TU-owned graphs and guarded configuration evidence
- add closed-world production reachability classification
- calibrate detector precision and reviewed self-scan policy
- terminate poisoned workers instead of longjmp recovery
2026-07-30 18:18:04 -04:00
trvon
bcdcb3036b
feat(analysis): harden quality and solver pipelines 2026-07-29 20:46:51 -04:00
trvon
b0b51f1e71
feat: converge quality and scan interfaces
- fail closed on MCP input and compile database policy
- add shared scan catalogs, finding queries, and tiered gates
2026-07-29 00:57:33 -04:00
trvon
f37222d4fb
fix(scan): harden frontend analysis 2026-07-28 11:57:38 -04:00
trvon
75e3e09cbf
fix(rust): dedupe cargo wrapper output 2026-07-28 00:38:56 -04:00
trvon
eedaa16c4f
fix(swift): clean frontend test warning 2026-07-20 14:55:52 -04:00
trvon
45b31c6b5a
feat(swift): harden SIL scanner 2026-07-20 14:52:50 -04:00
trvon
2dcbdbc75d
feat: harden Fold quality guard
Preserve source, binary, and runtime evidence boundaries while making precision and performance regressions reproducible.

- decompose analysis orchestration into bounded modules
- add differential, metamorphic, organic, and Yams change-time gates
- fix detector, P-Code, fuzz sanitizer, and report identity precision
2026-07-13 15:05:35 -04:00
trvon
8d053f4727
feat: checkpoint. Bug fixes and good improvements 2026-07-10 09:59:49 -04:00
trvon
b08410a4dd
feat: extend Rust/Swift IR encoders, add locationless filter, embedded skill
- Rust IR encoder: null-deref, div-by-zero, int-overflow, oob (was null-deref only)
- Swift SIL encoder: reuses RustExprNode-compatible IR from fold-swift
- Locationless finding filter in filterAndDedup before SARIF/JSON emit
- Encoder failures degrade to heuristic findings instead of silent skip
- Z3 exception guard in encodeRustIr (per-function fault isolation)
- Embedded fold skill via `fold skill` subcommand and --help
- Tests: fuzz-prep Rust lane, dedup locationless filter, IR encoder

Rust and Swift proved findings now flow into confirmation.tasks.json.
Confidence: concrete-witness > proved > heuristic > syntactic.
2026-06-02 16:00:42 -04:00
trvon
374aa7475d
many good updates for binary and SAST / DAST scanning 2026-06-02 12:39:03 -04:00
trvon
6f73c75d5d
feat: extend binary dangerous import list — 22 functions with CWE annotations
- Added: strncpy, strncat, snprintf, printf, fprintf, fscanf, sscanf
- Added: free, malloc, calloc, realloc (heap operation tracking)
- CWE annotations: 121 (buffer), 134 (format), 242 (banned), 78 (injection)
- Import list expanded from 11 to 22 entries
2026-05-29 21:36:45 -04:00
trvon
b4fa3c834d
feat: binary hardcoded string detection — URLs, passwords, keys, paths
- extractHardcodedStrings(): scans izzj output against 6 pattern categories
- URLs, credentials, file paths, emails, IPv4, base64 >20 chars
- +5 findings on CTF corpus, 130 build targets, 63/63 tests
2026-05-29 20:41:51 -04:00
trvon
9bdd989b7e
fix: binary harness_file, -I path resolution, caller-context schema
- Store harness_file in binary fuzz predictions to fix prep↔run mismatch
- Resolve relative -I flags against compile DB directory
- Add caller_unsafe/is_param_sourced fields to fold-rust NDJSON
- Allow empty target_sources in fuzz-run for dlopen harnesses
- Recursive IR encoder descent for nested hazards
2026-05-27 17:34:52 -04:00
trvon
9bdcd9f7a0
feat: audit fixes — complexity, CLI UX, MCP parity, binary fuzz
- CompositeSafetyObserver replaced with macros (composite_observer.hpp)
- tool_registry split per category
- Renamed --null-deref-nullable-allocations to --nullable-allocations
- --input-type hidden from help, fold-swift --help added
- MCP: fold.scan_rust, fold.scan_swift, backend filter on get_findings
- Binary fuzz: fold fuzz prep --binary <mach-o>, --compile-db optional
2026-05-27 16:29:39 -04:00
trvon
ffc7917957
feat: symbolic IR, swift backend, binary-to-fuzz pipeline, SARIF fixes
- Rust IR encoder (encodeRustIr): walks HIR expr trees, fires SafetyObserver hooks
- Swift SIL frontend (fold-swift): swiftc -emit-sil parser, demangling, NDJSON output
- SwiftBackend: --mode swift CLI routing, shells out to fold-swift
- Binary-to-fuzz: Mach-O export discovery, ARM64 ABI param-count inference, dlopen harness generation
- Language-agnostic IR: json_keys canonical constants, Call/Member/ArraySubscript/Cast/HeapNew nodes
- Dogfood fixes: SARIF kind field, cross-detector dedup, harness_file in predictions, Swift stoi safety
- Corpuses: swift-sil-z3-corpus.md, binary-fuzz-corpus.md
- AGENTS.md: cross-language wire format rule
2026-05-27 15:29:14 -04:00
trvon
95aac8209c
feat: expand rust security scanner coverage 2026-05-27 11:32:05 -04:00
trvon
ce1c353847
feat: harden security scanner routing 2026-05-27 10:36:57 -04:00
trvon
93eb1f09b2
feat(fuzz): M31 Phase 2 — closed-loop fuzz-run promotes to ConcreteWitness
End-to-end: fold scan → fuzz-prep emits Z3-derived seed → fuzz-run
compiles + libFuzzer + ASan → crash ingested → predicted Finding
promoted from Proved to ConcreteWitness in ~1s wall-clock.

P2.1 — Confidence::ConcreteWitness tier
* Strictly stronger than Proved: SMT-feasibility + observed runtime
  failure. confidenceRank() gives explicit numeric ordering so the
  enum integer values stay incidental.
* FindingKind::FuzzCrash for surprise crashes the SymEx side didn't
  predict.
* FuzzWitness evidence sub-struct (seed_path, asan_kind, asan_file,
  asan_line) + JSON round-trip.

P2.2 — ASan stderr parser
* include/fold/fuzz/asan_parser.{hpp,cpp} (~150 LOC)
* Parses the canonical SUMMARY tail line — stable across LLVM 15–22
  for both ASan (heap-buffer-overflow, SEGV, …) and UBSan
  (signed-integer-overflow, …)
* test_asan_parser.cpp: 6 cases / 27 assertions

P2.3 — FuzzRunner
* include/fold/fuzz/runner.{hpp,cpp}
* Synchronous: compile + posix_spawn + waitpid-with-deadline +
  artifact-scan. SIGTERM + 1s grace + SIGKILL on parent timeout.
* Per-source `-x c` / `-x c++` pinning so C targets keep their
  C linkage and the harness's extern "C" decl resolves.
* Drops -std= entirely — generated harnesses are pre-C++17.

P2.4 + P2.5 — fuzz-run CLI + crash ingestion
* src/cli/fuzz_run_runner.cpp: load findings.predicted.json, group
  by function, runFuzzCampaign per harness, replay each crash under
  ASan, parse SUMMARY, match (file, line) to predicted Finding.id.
* Matched crashes → augmented Finding with Confidence::ConcreteWitness
  + evidence.fuzz_witness. Unmatched → FindingKind::FuzzCrash.
* Subcommand: --prep-dir, --target-source (rep), --time-budget,
  --compiler, --report, --clang-flag.

P2.6 — integration test
* tests/integration/test_fuzz_run.cpp: probes for a clang++ that can
  link -fsanitize=fuzzer, builds the null-deref-behind-guard fixture,
  chains runFuzzPrep + runFuzzRun, asserts at least one
  concrete-witness Finding whose id matches a predicted Finding.id.
* WARN+SUCCEED when no libFuzzer-capable compiler is found so CI
  on stripped clang installs stays green.

Tests: 39 → 40 green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 15:21:05 -04:00
trvon
075dd85669
feat(fuzz): R3-oob + P1.7 — full witness coverage + e2e test
R3-oob: oob_access now hoists z3::model out of both solver branches
and passes it to recordSinkWitnessWithModel. Predicted seeds for
`int buf[N]; return buf[idx];` now contain Z3's chosen boundary-
violating idx (e.g. `04 00 00 00` for buf[4]) instead of zeros.

P1.7: tests/integration/test_fuzz_prep.cpp materializes a
null-deref-behind-`flag==0xAB`-guard fixture, drives runFuzzPrep
in-process, and asserts the witness seed contains the exact magic
byte (0xAB at offset 0) plus a well-formed harness (extern "C",
size-gate at total_bytes, leaky(arg0, arg1) call).

Moves fuzz_prep_runner.cpp from the fold executable target into
fold_core_sources so the integration test can link runFuzzPrep
directly.

Tests: 37 → 38 green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 14:39:17 -04:00
trvon
a51cdf5748
M19–M30 + scout-rule encoder refactor
Multi-phase work merged into one commit:

* M19: flow-sensitive guard tracking (stmtAlwaysExits, walkIf early-exit)
* M20–M27: non-null marking across deref, addr-of, callsite preconds,
  if/while/switch guards, AddrOf-of-subexpr, CFG-block-scoped maps
* M30 Phase A: dialect-agnostic SourceLoc + LocationMapper + frontend/
  boundary marker; ClangLocationMapper wired through SafetyObserver
* M30 Phase B v2: QueryTracer + 13 SafetyObserver hooks migrated to
  ir::Expr / ir::Call / ir::MemberRef / ir::Symbol / ir::SourceLoc
* M30 Phase C/D/D2/D3: ir::Symbol, ir::Expr (with Kind), ir::Call,
  ir::MemberRef opaque types; ir_expr_clang.cpp houses the clang
  adapters; encodeExpr migrated to switch(kind) dispatch

Scout-rule extractions from encoder.cpp (2800 → 2371 lines):
* encoder_callee_predicates.{hpp,cpp} — isMallocLikeCall, isFreeCall,
  isAssertCall as pure std::string_view predicates
* encoder_type_queries.{hpp,cpp} — typeSizeBytes, sourceWidth,
  isUnsignedLike, byteWidthExpr
* encoder_callee_names.{hpp,cpp} — calleeName, qualifiedFunctionName,
  qualifiedCalleeName (with internal-linkage @-suffix)
* encoder_coerce.{hpp,cpp} — coerceToBv, coerceToExactBv, coerceToBool,
  coerceToType (BV-64 storage model documented at the boundary)
* encoder_reachability.{hpp,cpp} — ReachabilityCounter impl, markEncoded,
  recordEncoded, recordBailedNode, public profiling glue
* encoder_summary_instantiate.cpp — SMT-LIB summary substitution
  (instantiateSummaryExpr / Predicate / PredicateInContext +
  applySummaryParamPost; kills the redundant *Impl pass-through layer)
2026-05-21 11:17:50 -04:00
trvon
c4e42cec0a
Add scan router and MCP job queue
Split source and binary scan routing with merge-only hybrid support, binary provenance, and router-backed MCP scan contexts.

Add async MCP scan jobs for long-running scans, detector precision fixes, benchmark scripts, docs, and integration coverage.
2026-05-19 00:24:12 -04:00
trvon
f218af3d8b
Add githooks, clang-format all code, fix CI Z3 paths 2026-05-15 00:08:47 -04:00
trvon
3f1234f7bb
fold: bounded symbolic SAST for C++
Clang LibTooling + Z3 BV-64 encoding.
11 detectors: clone, equiv, class-dup, dead, null-deref,
div-by-zero, int-overflow, oob, int-truncation,
sign-conversion, taint-track.

Interfaces: CLI, MCP (JSON-RPC 2.0), LSP.
Output: SARIF 2.1.0, JSON, Markdown.
2026-05-14 23:14:54 -04:00