86b2f14a43
- persist TU-owned graphs and guarded configuration evidence - add closed-world production reachability classification - calibrate detector precision and reviewed self-scan policy - terminate poisoned workers instead of longjmp recovery
138 lines
5.3 KiB
Markdown
138 lines
5.3 KiB
Markdown
# Clang API Quirks Encountered by fold
|
|
|
|
Catalog of Clang LibTooling APIs that assert or recurse on inputs fold has
|
|
encountered in real-world C++ codebases (primarily yams, fmt, spdlog,
|
|
libc++ headers). Each entry: assertion text or symptom, the fold call site
|
|
that trips it, and the precondition or guard we install.
|
|
|
|
This list grows when the process-isolated TU boundary observes a new
|
|
SIGABRT/SIGSEGV/SIGTRAP. Fatal signals terminate the worker; the parent uses
|
|
flushed `file_finished` records to identify the unfinished TU and writes it to
|
|
`quarantine.txt`. Fold never jumps across interrupted C++ frames.
|
|
|
|
## 1. `StringLiteral::getString()` — wide-char assertion
|
|
|
|
**Symptom:**
|
|
|
|
```
|
|
Assertion failed: ((isUnevaluated() || getCharByteWidth() == 1)
|
|
&& "This function is used in places that assume strings use char"),
|
|
function getString, file Expr.h, line 1869.
|
|
```
|
|
|
|
**Why:** `StringLiteral::getString()` returns a `StringRef` over the raw byte
|
|
array but assumes single-byte characters. For `wchar_t`/`char16_t`/`char32_t`
|
|
literals (e.g. `L"..."`), the bytes are 2 or 4 wide and the function
|
|
deliberately asserts.
|
|
|
|
**Fold call sites (all guarded):**
|
|
|
|
- `src/ir/hash.cpp:53` — clone summary hashing.
|
|
- `src/ir/hash.cpp:182` — clone debug dump.
|
|
- `src/detect/oob_access.cpp:81` — sprintf format-string introspection.
|
|
- `src/symbolic/encoder.cpp:431` — sprintf format-string size analysis.
|
|
|
|
**Guard:**
|
|
|
|
```cpp
|
|
if (sl->getCharByteWidth() == 1) {
|
|
// safe to call getString()
|
|
} else {
|
|
// fall back to byte-length-only hashing / bail entirely
|
|
}
|
|
```
|
|
|
|
## 2. `ASTContext::getTypeInfoImpl()` — infinite recursion on templates
|
|
|
|
**Symptom:** Stack overflow with 7,000+ frames of
|
|
`clang::ASTContext::getTypeInfoImpl + 1204`, SIGSEGV when the OS stack runs
|
|
out. Observed on `fmt::basic_format_parse_context` instantiations.
|
|
|
|
**Why:** Some heavy templated record decls have dependent-base chains that
|
|
Clang's type-info computation walks recursively without a cycle guard.
|
|
|
|
**Fold call sites:**
|
|
|
|
- `src/symbolic/encoder.cpp:typeSizeBytes` (`getTypeSizeInChars`)
|
|
- `src/symbolic/encoder.cpp:sourceWidth` (`getTypeSize`)
|
|
- `src/symbolic/encoder.cpp:1170` (`getASTRecordLayout`) — biggest offender.
|
|
|
|
**Guards** (combined):
|
|
|
|
- `qt.isNull() || qt->isDependentType()`
|
|
- `qt->isIncompleteType()`
|
|
- For `getASTRecordLayout`: also skip when
|
|
`parent->isDependentContext()`,
|
|
`parent->isLambda()`,
|
|
`cxx_parent->getDescribedClassTemplate() != nullptr`, or
|
|
`isa<ClassTemplateSpecializationDecl>(parent)`.
|
|
|
|
## 3. `Z3_solver_set_params("sat.threads")` — Z3 4.17 parameter removal
|
|
|
|
**Symptom:**
|
|
|
|
```
|
|
libc++abi: terminating due to uncaught exception of type z3::exception:
|
|
unknown parameter 'sat.threads'
|
|
```
|
|
|
|
**Why:** Z3 4.17 dropped `sat.threads` from the SAT solver's param
|
|
descriptor. Some internal paths still raise the exception when the param is
|
|
addressed indirectly (e.g. through `tactics.cpp`'s `par_or` portfolio or
|
|
`std::async` future-bound solvers).
|
|
|
|
**Guard:** every direct `s.set("sat.threads", ...)` is wrapped in
|
|
`if (supportsParam(s.get_param_descrs(), "sat.threads")) { s.set(...); }`.
|
|
Ordinary Z3 exceptions are handled at C++ boundaries. A fatal Z3 signal
|
|
terminates the isolated worker and is attributed by the parent; the process is
|
|
never reused.
|
|
|
|
## 4. SIGTRAP from libc++ debug assertions
|
|
|
|
**Symptom:** Worker dies with signal 5 (SIGTRAP); no informative output.
|
|
Observed on yams `equiv` runs.
|
|
|
|
**Why:** macOS libc++ in debug builds raises SIGTRAP via `__builtin_trap()`
|
|
instead of `abort()` on certain hardened-mode contract violations
|
|
(out-of-range iterator, etc.). Source location is lost because Clang's
|
|
template instantiation hides the offending callsite.
|
|
|
|
**Guard:** the isolated worker terminates. The parent records the unfinished TU
|
|
in `quarantine.txt` and continues remaining work in a fresh process.
|
|
|
|
## 5. ImplicitCastExpr at call args — observer gap (not a crash)
|
|
|
|
This is not a crash but a recall gap fixed in the same session as the
|
|
crash work: `observeImplicitConversion` was only wired into VarDecl init,
|
|
plain `BO_Assign`, and compound assignment. Lossy casts at call
|
|
arguments (e.g. `malloc(signed_int)` widening + signedness flip to
|
|
`size_t`) skipped the observer entirely, zero-ing recall on Juliet
|
|
CWE-194/195. Fixed at `src/symbolic/encoder.cpp` CallExpr arg-encoding
|
|
loop (Phase-2 of the session).
|
|
|
|
## Adding a new entry
|
|
|
|
1. Run `tools/audit-yams.sh /tmp/yams-audit-<date>` with process isolation
|
|
active. Crashes get logged to `quarantine.txt` with signal provenance.
|
|
2. Reproduce one of the crashed TUs alone:
|
|
|
|
```
|
|
./build/release/src/cli/fold scan --compile-db /tmp/one.json \
|
|
--detectors <det> --processes 1 \
|
|
--format sarif --output /tmp/x.sarif
|
|
```
|
|
|
|
3. Run under lldb (`lldb --batch -k 'bt 50' ...`) to get the offending
|
|
Clang API.
|
|
4. Add a guard at the Fold call site. Process isolation and quarantine contain
|
|
the failure but do not make unsupported library behavior safe.
|
|
5. Append a new section to this doc with the symptom + fix.
|
|
|
|
## Related files
|
|
|
|
- `src/cli/signal_isolation.cpp` — sigaltstack plus process-terminating signal policy.
|
|
- `src/cli/subprocess_fanout.cpp` — parent watchdog, progress reconciliation,
|
|
quarantine attribution, and fresh-worker recovery.
|
|
- `src/cli/clang_shard_tu.cpp` — per-TU timer scope; no post-signal continuation.
|
|
- `tools/audit-yams.sh` — passes `--quarantine-dir` so consecutive detector
|
|
runs share a single skip list.
|