merge/audit-and-state-map #7
6 changed files with 853 additions and 1 deletions
feat(cpp): experimental C++ front end for scoped-enum state
tool/cpp_front.py is a second front end written against the state model, not the rules: it reads a compile database with libclang (Homebrew LLVM) and writes the JSON that `pipe check --model` takes. Every rule, baseline and gate works on it unchanged. Mapping, per project `enum class` (identity from clang's resolved AST, so two enums both named QueryIntent are never confused): - a reference that produces a value (assignment, atomic store, argument, return, initialiser) is an emission; a cast from an int is an indirect one; - a `case`, `==` or `!=` is an observation; - a busy enumerator written to the method's OWN state, followed by a top-level try whose catch neither writes state nor rethrows, is a swallowed failure; - sources the build does not compile are recorded as unresolved, so findings naming them drop a confidence step. One rule-side change, optional and Dart-neutral: an EmissionSite may name its stateType. rendered-not-emitted attributed indirect emissions by owner (a bloc owns one state type), so a cast to one enum made "never emitted" unprovable for every other enum in the same C++ class. With stateType set, the suppression is scoped to that enum. Dart output is still byte identical on the fixture and all four dogfood apps. Run on ~/work/tools/yams (375 translation units from build/debug, 0 clang errors, about two minutes): 118 state enums, 82 findings. A grep-based hand check of 13 found 12 factually correct - e.g. the IPC connection FSM never enters ImplState::ReadingHeader, the downloader never reports ProgressStage::Verifying though three CLI commands render it, ExtractScopeType qualifiers are parsed but never acted on - and one false positive (a nested JSON-parse fallback taken for a stuck busy state), now excluded by requiring the method's own state and a top-level try. Bugs found on the way and fixed: ccache/compiler wrappers passed as inputs, relative -I paths resolved against the wrong directory (7 TUs silently lost third_party headers), a symlinked path putting every TU "outside" the project, and an empty compile database producing an empty "clean" model with exit 0 - all now fail loudly or are handled. test/cpp_front_test.dart runs the front end on test/fixtures/cpp_state with positive and negative controls for each rule; skipped without LLVM. 138 tests pass.
commit
4b499c1e21
35
README.md
35
README.md
|
|
@ -411,6 +411,41 @@ rule, `--baseline` ratcheting and `--fail-on` gating without touching Dart.
|
||||||
`check --model` refuses `--write-baseline`, because a model does not record
|
`check --model` refuses `--write-baseline`, because a model does not record
|
||||||
whether its front end read every file.
|
whether its front end read every file.
|
||||||
|
|
||||||
|
### Experimental: C++ state enums
|
||||||
|
|
||||||
|
`tool/cpp_front.py` is a second front end, written against the model rather than
|
||||||
|
the rules. It reads a compile database with libclang (Homebrew LLVM) and maps
|
||||||
|
each project `enum class` to a union: a reference that produces a value
|
||||||
|
(assignment, atomic `store`, argument, return, initialiser) is an emission; a
|
||||||
|
`case`, `==` or `!=` is an observation; a busy enumerator set on the method's
|
||||||
|
own state followed by a top-level `catch` that neither writes state nor
|
||||||
|
rethrows is a swallowed failure. Identity comes from clang's resolved AST, so
|
||||||
|
two enums both named `QueryIntent` are never confused.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
tool/cpp_front.py ~/work/tools/yams --build-dir ~/work/tools/yams/build/debug -o yams.json
|
||||||
|
pipe check --model yams.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Read the rules as: `rendered-not-emitted` - an enumerator something branches on
|
||||||
|
that nothing ever produces (a dead branch, or a state an FSM can never enter);
|
||||||
|
`emitted-not-rendered` - a value produced that nothing ever branches on
|
||||||
|
(computed and ignored, or reaching output only through a name table). Sources
|
||||||
|
the build does not compile are recorded as unresolved, so findings naming them
|
||||||
|
drop a confidence step. `test/cpp_front_test.dart` holds the positive and
|
||||||
|
negative controls (`test/fixtures/cpp_state`); it is skipped without LLVM.
|
||||||
|
|
||||||
|
On yams (375 translation units, 0 clang errors, about two minutes) the first
|
||||||
|
run produced 82 findings. A grep-based hand check of 13 found 12 factually
|
||||||
|
correct and 1 false positive (a nested JSON-parse fallback taken for a stuck
|
||||||
|
busy state), which the front end now excludes. Factual is not the same as a
|
||||||
|
bug: of the 12, `TransportState::Error` is covered by `!= Connected` checks and
|
||||||
|
`InboundSessionStage` is fuzzing instrumentation. The substantive ones include
|
||||||
|
the IPC connection FSM never entering `ImplState::ReadingHeader`, the downloader
|
||||||
|
never reporting `ProgressStage::Verifying` although three CLI commands render
|
||||||
|
it, and `ExtractScopeType` query qualifiers being parsed but never acted on.
|
||||||
|
n=13 from one repo bounds the false-positive rate; it does not measure it.
|
||||||
|
|
||||||
Built on `package:analyzer` with whole-project resolution, not a
|
Built on `package:analyzer` with whole-project resolution, not a
|
||||||
`custom_lint`/analyzer plugin: plugins have no project-wide index, `flutter
|
`custom_lint`/analyzer plugin: plugins have no project-wide index, `flutter
|
||||||
analyze` does not report new-system plugin diagnostics, and a CLI can pin its own
|
analyze` does not report new-system plugin diagnostics, and a CLI can pin its own
|
||||||
|
|
|
||||||
|
|
@ -523,8 +523,14 @@ FindingReport runChecks({
|
||||||
if (enabled.contains(ruleRenderedNotEmitted)) {
|
if (enabled.contains(ruleRenderedNotEmitted)) {
|
||||||
for (final coverage in flow.coverages) {
|
for (final coverage in flow.coverages) {
|
||||||
if (coverage.renderedNeverEmitted.isEmpty) continue;
|
if (coverage.renderedNeverEmitted.isEmpty) continue;
|
||||||
|
// An emission that names its state type belongs to that union only; one
|
||||||
|
// that does not (a Dart `emit(_helper())`) is attributed by owner, since a
|
||||||
|
// bloc owns one state type. A C++ class holds several enums, and a cast
|
||||||
|
// to one of them must not make every other one unprovable.
|
||||||
final ownerEmissions = flow.emissions.where(
|
final ownerEmissions = flow.emissions.where(
|
||||||
(emission) => emission.owner == coverage.union.owner,
|
(emission) => emission.stateType == null
|
||||||
|
? emission.owner == coverage.union.owner
|
||||||
|
: emission.stateType == coverage.union.name,
|
||||||
);
|
);
|
||||||
final uncertain = ownerEmissions.any(
|
final uncertain = ownerEmissions.any(
|
||||||
(emission) =>
|
(emission) =>
|
||||||
|
|
|
||||||
|
|
@ -104,6 +104,7 @@ class EmissionSite {
|
||||||
required this.confidence,
|
required this.confidence,
|
||||||
required this.variants,
|
required this.variants,
|
||||||
required this.snippet,
|
required this.snippet,
|
||||||
|
this.stateType,
|
||||||
});
|
});
|
||||||
|
|
||||||
final String path;
|
final String path;
|
||||||
|
|
@ -111,6 +112,11 @@ class EmissionSite {
|
||||||
final String owner;
|
final String owner;
|
||||||
final String member;
|
final String member;
|
||||||
|
|
||||||
|
/// The state type this emission produces, when the front end knows it even
|
||||||
|
/// though the variant is unknown (a C++ `static_cast<State>(n)`). Null means
|
||||||
|
/// "attribute by owner", which is right when an owner has one state type.
|
||||||
|
final String? stateType;
|
||||||
|
|
||||||
/// `emit`, `assign` or `notifyListeners`.
|
/// `emit`, `assign` or `notifyListeners`.
|
||||||
final String kind;
|
final String kind;
|
||||||
|
|
||||||
|
|
@ -128,6 +134,7 @@ class EmissionSite {
|
||||||
'confidence': confidence,
|
'confidence': confidence,
|
||||||
if (variants.isNotEmpty) 'variants': variants,
|
if (variants.isNotEmpty) 'variants': variants,
|
||||||
'snippet': snippet,
|
'snippet': snippet,
|
||||||
|
if (stateType != null) 'stateType': stateType,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -2039,6 +2046,7 @@ StateFlow stateFlowFromJson(Map<String, Object?> json) {
|
||||||
confidence: e['confidence']! as String,
|
confidence: e['confidence']! as String,
|
||||||
variants: strings(e['variants']),
|
variants: strings(e['variants']),
|
||||||
snippet: e['snippet']! as String,
|
snippet: e['snippet']! as String,
|
||||||
|
stateType: e['stateType'] as String?,
|
||||||
),
|
),
|
||||||
],
|
],
|
||||||
renders: [
|
renders: [
|
||||||
|
|
|
||||||
95
test/cpp_front_test.dart
Normal file
95
test/cpp_front_test.dart
Normal file
|
|
@ -0,0 +1,95 @@
|
||||||
|
import 'dart:convert';
|
||||||
|
import 'dart:io';
|
||||||
|
|
||||||
|
import 'package:pipe/pipe.dart';
|
||||||
|
import 'package:test/test.dart';
|
||||||
|
|
||||||
|
/// Contract for the experimental C++ front end (tool/cpp_front.py): it writes a
|
||||||
|
/// state model, and the unchanged rules must fire on each positive control in
|
||||||
|
/// test/fixtures/cpp_state and stay silent on each negative one.
|
||||||
|
///
|
||||||
|
/// Skipped, not passed, when Homebrew LLVM (libclang + its Python bindings) or
|
||||||
|
/// python3 is unavailable.
|
||||||
|
void main() {
|
||||||
|
final llvm = Platform.environment['PIPE_LLVM'] ?? '/opt/homebrew/opt/llvm';
|
||||||
|
final available =
|
||||||
|
File('$llvm/lib/libclang.dylib').existsSync() &&
|
||||||
|
Process.runSync('python3', ['--version']).exitCode == 0;
|
||||||
|
|
||||||
|
late FindingReport report;
|
||||||
|
|
||||||
|
setUpAll(() async {
|
||||||
|
if (!available) return;
|
||||||
|
final root = Directory.systemTemp.createTempSync('pipe-cpp');
|
||||||
|
Directory('${root.path}/src').createSync();
|
||||||
|
File(
|
||||||
|
'test/fixtures/cpp_state/src/engine.cpp',
|
||||||
|
).copySync('${root.path}/src/engine.cpp');
|
||||||
|
File('${root.path}/compile_commands.json').writeAsStringSync(
|
||||||
|
jsonEncode([
|
||||||
|
{
|
||||||
|
'directory': root.path,
|
||||||
|
'file': 'src/engine.cpp',
|
||||||
|
'arguments': ['c++', '-std=c++20', '-c', 'src/engine.cpp'],
|
||||||
|
},
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
final model = '${root.path}/model.json';
|
||||||
|
final result = Process.runSync('python3', [
|
||||||
|
'tool/cpp_front.py',
|
||||||
|
root.path,
|
||||||
|
'--include',
|
||||||
|
'src',
|
||||||
|
'-o',
|
||||||
|
model,
|
||||||
|
]);
|
||||||
|
expect(result.exitCode, 0, reason: '${result.stderr}');
|
||||||
|
expect('${result.stderr}', contains('0 clang error(s)'));
|
||||||
|
report = runChecks(
|
||||||
|
flow: stateFlowFromJson(
|
||||||
|
jsonDecode(File(model).readAsStringSync()) as Map<String, Object?>,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
Iterable<Finding> hits(String rule) =>
|
||||||
|
report.findings.where((finding) => finding.rule == rule);
|
||||||
|
|
||||||
|
test('an enum produced but never examined is reported', () {
|
||||||
|
expect(
|
||||||
|
hits(ruleEmittedNotRendered).map((f) => f.data['union']),
|
||||||
|
contains('fx::Unread'),
|
||||||
|
);
|
||||||
|
}, skip: available ? false : 'Homebrew LLVM / python3 not available');
|
||||||
|
|
||||||
|
test('an enumerator examined but never produced is a dead branch', () {
|
||||||
|
final ghost = hits(
|
||||||
|
ruleRenderedNotEmitted,
|
||||||
|
).where((f) => f.data['variant'] == 'Ghost');
|
||||||
|
expect(ghost, hasLength(1));
|
||||||
|
expect(ghost.single.file, 'src/engine.cpp');
|
||||||
|
expect(ghost.single.line, isNotNull);
|
||||||
|
}, skip: available ? false : 'Homebrew LLVM / python3 not available');
|
||||||
|
|
||||||
|
test('a cast to one enum does not mute "never emitted" for another', () {
|
||||||
|
// Casted::Y is produced only through static_cast, so it is unprovable
|
||||||
|
// and suppressed. Phase lives in the same class and must still be judged.
|
||||||
|
expect(
|
||||||
|
hits(ruleRenderedNotEmitted).map((f) => f.data['union']),
|
||||||
|
isNot(contains('fx::Casted')),
|
||||||
|
);
|
||||||
|
expect(report.suppressed.where((s) => s.contains('fx::Phase')), isEmpty);
|
||||||
|
}, skip: available ? false : 'Homebrew LLVM / python3 not available');
|
||||||
|
|
||||||
|
test(
|
||||||
|
'a swallowed failure on the method\'s own busy state fires, and only it',
|
||||||
|
() {
|
||||||
|
// startGuarded resolves the state, startNested's try is a nested local
|
||||||
|
// fallback, and Reporter::run sets a field of a local event object.
|
||||||
|
expect(hits(ruleLoadingNotReset).map((f) => f.data['member']).toList(), [
|
||||||
|
'startSwallowed',
|
||||||
|
]);
|
||||||
|
},
|
||||||
|
skip: available ? false : 'Homebrew LLVM / python3 not available',
|
||||||
|
);
|
||||||
|
}
|
||||||
104
test/fixtures/cpp_state/src/engine.cpp
vendored
Normal file
104
test/fixtures/cpp_state/src/engine.cpp
vendored
Normal file
|
|
@ -0,0 +1,104 @@
|
||||||
|
// Fixture for tool/cpp_front.py: each case is a positive or negative control
|
||||||
|
// for a rule run on the C++ state model (see test/cpp_front_test.dart).
|
||||||
|
|
||||||
|
namespace fx {
|
||||||
|
|
||||||
|
// Ghost is compared against but never produced: rendered-not-emitted.
|
||||||
|
enum class Phase { Idle, Starting, Running, Failed, Ghost };
|
||||||
|
|
||||||
|
// Produced, never compared or switched on: emitted-not-rendered.
|
||||||
|
enum class Unread { A, B };
|
||||||
|
|
||||||
|
// Y is compared against and produced only through a cast from an int, so the
|
||||||
|
// rule must suppress itself rather than call the branch dead.
|
||||||
|
enum class Casted { X, Y };
|
||||||
|
|
||||||
|
void mayThrow();
|
||||||
|
|
||||||
|
class Engine {
|
||||||
|
public:
|
||||||
|
// Negative: the catch resolves the busy state.
|
||||||
|
void startGuarded() {
|
||||||
|
phase_ = Phase::Starting;
|
||||||
|
try {
|
||||||
|
mayThrow();
|
||||||
|
phase_ = Phase::Running;
|
||||||
|
} catch (...) {
|
||||||
|
phase_ = Phase::Failed;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Positive: busy state set, then a top-level try whose catch swallows the
|
||||||
|
// failure, so Starting can outlive it (loading-not-reset).
|
||||||
|
void startSwallowed() {
|
||||||
|
phase_ = Phase::Starting;
|
||||||
|
try {
|
||||||
|
mayThrow();
|
||||||
|
phase_ = Phase::Running;
|
||||||
|
} catch (...) {
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Negative: a nested try is a local fallback, not the method's failure path.
|
||||||
|
void startNested() {
|
||||||
|
phase_ = Phase::Starting;
|
||||||
|
if (ready_) {
|
||||||
|
try {
|
||||||
|
mayThrow();
|
||||||
|
} catch (...) {
|
||||||
|
}
|
||||||
|
}
|
||||||
|
phase_ = Phase::Running;
|
||||||
|
}
|
||||||
|
|
||||||
|
bool ghost() const { return phase_ == Phase::Ghost; }
|
||||||
|
|
||||||
|
bool idle() const {
|
||||||
|
switch (phase_) {
|
||||||
|
case Phase::Idle:
|
||||||
|
return true;
|
||||||
|
case Phase::Starting:
|
||||||
|
case Phase::Running:
|
||||||
|
case Phase::Failed:
|
||||||
|
return false;
|
||||||
|
default:
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void setUnread() {
|
||||||
|
unread_ = Unread::A;
|
||||||
|
unread_ = Unread::B;
|
||||||
|
}
|
||||||
|
|
||||||
|
void load(int n) { casted_ = static_cast<Casted>(n); }
|
||||||
|
bool isY() const { return casted_ == Casted::Y; }
|
||||||
|
|
||||||
|
private:
|
||||||
|
bool ready_ = false;
|
||||||
|
Phase phase_ = Phase::Idle;
|
||||||
|
Unread unread_ = Unread::A;
|
||||||
|
Casted casted_ = Casted::X;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Negative: `ev.stage` is a field of a local event object handed to a callback,
|
||||||
|
// not state Reporter holds, so its try cannot leave Reporter stuck.
|
||||||
|
struct Event {
|
||||||
|
Phase stage;
|
||||||
|
};
|
||||||
|
void notify(const Event&);
|
||||||
|
|
||||||
|
class Reporter {
|
||||||
|
public:
|
||||||
|
void run() {
|
||||||
|
Event ev;
|
||||||
|
ev.stage = Phase::Starting;
|
||||||
|
notify(ev);
|
||||||
|
try {
|
||||||
|
mayThrow();
|
||||||
|
} catch (...) {
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace fx
|
||||||
604
tool/cpp_front.py
Normal file
604
tool/cpp_front.py
Normal file
|
|
@ -0,0 +1,604 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Experimental C++ front end: writes a pipe state model from clang's AST.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
tool/cpp_front.py <project-root> [--build-dir DIR] [--include src,include]
|
||||||
|
[--limit N] [--jobs N] [-o model.json]
|
||||||
|
pipe check --model model.json
|
||||||
|
|
||||||
|
pipe's rules read only the state model, so this front end never touches them.
|
||||||
|
It uses libclang's *resolved* AST: every enumerator reference is linked to its
|
||||||
|
EnumConstantDecl, so identity is exact rather than matched by name.
|
||||||
|
|
||||||
|
What it maps (scoped enums only, `enum class`, declared in the project):
|
||||||
|
|
||||||
|
union an `enum class`; variants are its enumerators
|
||||||
|
emission a reference that PRODUCES the value: assigned to a field or
|
||||||
|
variable, stored into an atomic, passed, returned, initialised
|
||||||
|
observation a reference that EXAMINES the value: a `case` label, `==`, `!=`
|
||||||
|
(these fill pipe's "rendered" set: something branches on it)
|
||||||
|
swallowed a member writes a busy enumerator (Starting, Loading, ...) to a
|
||||||
|
failure field, then has a try whose catch writes no state and does not
|
||||||
|
rethrow (loading-not-reset)
|
||||||
|
|
||||||
|
Consequences worth knowing before reading the findings:
|
||||||
|
- `emitted-not-rendered` here means "produced, but nothing ever branches on or
|
||||||
|
compares it". A value that reaches the user through a lookup table or
|
||||||
|
reflection (magic_enum) is invisible to this, so treat those as leads.
|
||||||
|
- `rendered-not-emitted` means "compared against, but never produced anywhere":
|
||||||
|
a dead branch, unless the value comes from a cast (`static_cast<E>(n)`), which
|
||||||
|
is recorded as an indirect emission so the rule suppresses itself.
|
||||||
|
- The rule messages still speak of "UI"; for C++ read "nothing observes it".
|
||||||
|
|
||||||
|
Requires Homebrew LLVM's libclang and its Python bindings (found automatically).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import glob
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import shlex
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
from concurrent.futures import ProcessPoolExecutor, as_completed
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
LLVM = Path(os.environ.get("PIPE_LLVM", "/opt/homebrew/opt/llvm"))
|
||||||
|
|
||||||
|
|
||||||
|
def _load_cindex():
|
||||||
|
for candidate in sorted(glob.glob(str(LLVM / "lib/python3.*/site-packages")), reverse=True):
|
||||||
|
if candidate not in sys.path:
|
||||||
|
sys.path.insert(0, candidate)
|
||||||
|
import clang.cindex as cindex # noqa: E402
|
||||||
|
|
||||||
|
# Pool workers are reused, and libclang may be configured only once.
|
||||||
|
if not cindex.Config.loaded:
|
||||||
|
cindex.Config.set_library_file(str(LLVM / "lib/libclang.dylib"))
|
||||||
|
return cindex
|
||||||
|
|
||||||
|
|
||||||
|
_SDK = None
|
||||||
|
|
||||||
|
BUSY = re.compile(
|
||||||
|
r"^(Loading|Starting|StartingUp|Initializing|Initialising|Connecting|Stopping|"
|
||||||
|
r"ShuttingDown|Busy|Pending|InProgress|Processing|Saving|Updating|Creating|"
|
||||||
|
r"Syncing|Indexing|Warming|WarmingUp|Draining|Migrating|Recovering)$"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Kinds that only wrap an expression; the classifier looks through them.
|
||||||
|
_TRANSPARENT = {
|
||||||
|
"UNEXPOSED_EXPR",
|
||||||
|
"PAREN_EXPR",
|
||||||
|
"CSTYLE_CAST_EXPR",
|
||||||
|
"CXX_FUNCTIONAL_CAST_EXPR",
|
||||||
|
"CONDITIONAL_OPERATOR",
|
||||||
|
"INIT_LIST_EXPR",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _resource_dir() -> str:
|
||||||
|
return subprocess.check_output([str(LLVM / "bin/clang"), "-print-resource-dir"], text=True).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _sdk_root() -> str | None:
|
||||||
|
"""Apple's compiler finds the macOS SDK itself; libclang needs it spelled out,
|
||||||
|
or every standard header (<chrono>, sqlite3.h) is missing."""
|
||||||
|
try:
|
||||||
|
return subprocess.check_output(["xcrun", "--show-sdk-path"], text=True).strip()
|
||||||
|
except (OSError, subprocess.CalledProcessError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _args_for(entry: dict, resource_dir: str) -> list[str]:
|
||||||
|
"""Compiler flags only: drop wrappers (ccache), the compiler, the source,
|
||||||
|
and output/dependency options that make no sense for a syntax-only parse."""
|
||||||
|
args = entry.get("arguments") or shlex.split(entry["command"])
|
||||||
|
while args and (os.path.basename(args[0]) in ("ccache", "sccache", "distcc")
|
||||||
|
or not args[0].startswith("-") and args[0] != entry["file"]
|
||||||
|
and os.path.basename(args[0]) in ("c++", "g++", "clang++", "cc", "gcc", "clang")):
|
||||||
|
args = args[1:]
|
||||||
|
out: list[str] = []
|
||||||
|
skip = False
|
||||||
|
directory = entry["directory"]
|
||||||
|
# Include paths are relative to the build directory, but libclang resolves
|
||||||
|
# them against this process's working directory: made absolute, or headers
|
||||||
|
# like third_party/... silently go missing.
|
||||||
|
path_flags = ("-I", "-isystem", "-iquote", "-idirafter", "-include")
|
||||||
|
absolute = []
|
||||||
|
pending = None
|
||||||
|
for arg in args:
|
||||||
|
if pending is not None:
|
||||||
|
absolute.append(arg if os.path.isabs(arg) else os.path.normpath(os.path.join(directory, arg)))
|
||||||
|
pending = None
|
||||||
|
continue
|
||||||
|
if arg in path_flags:
|
||||||
|
absolute.append(arg)
|
||||||
|
pending = arg
|
||||||
|
continue
|
||||||
|
for flag in path_flags:
|
||||||
|
if arg.startswith(flag) and len(arg) > len(flag) and flag != "-include":
|
||||||
|
value = arg[len(flag):]
|
||||||
|
if not os.path.isabs(value):
|
||||||
|
arg = flag + os.path.normpath(os.path.join(directory, value))
|
||||||
|
break
|
||||||
|
absolute.append(arg)
|
||||||
|
args = absolute
|
||||||
|
for arg in args:
|
||||||
|
if skip:
|
||||||
|
skip = False
|
||||||
|
continue
|
||||||
|
if arg in ("-o", "-MF", "-MT", "-MQ"):
|
||||||
|
skip = True
|
||||||
|
continue
|
||||||
|
if arg in ("-c", "-MD", "-MMD", "-fdiagnostics-color=always") or \
|
||||||
|
arg.startswith(("-o", "-MF", "-MT", "-MQ")):
|
||||||
|
continue
|
||||||
|
if arg == entry["file"] or not arg.startswith("-") and arg.endswith((".cpp", ".cc", ".cxx")):
|
||||||
|
continue
|
||||||
|
out.append(arg)
|
||||||
|
out += ["-resource-dir", resource_dir, "-Wno-everything", "-fsyntax-only"]
|
||||||
|
sdk = _SDK
|
||||||
|
if sdk and "-isysroot" not in out:
|
||||||
|
out += ["-isysroot", sdk]
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _qualified(cursor) -> str:
|
||||||
|
parts = []
|
||||||
|
c = cursor
|
||||||
|
while c is not None and c.kind.name != "TRANSLATION_UNIT":
|
||||||
|
if c.spelling:
|
||||||
|
parts.append(c.spelling)
|
||||||
|
c = c.semantic_parent
|
||||||
|
return "::".join(reversed(parts))
|
||||||
|
|
||||||
|
|
||||||
|
def _enclosing(stack, kinds):
|
||||||
|
for c in reversed(stack):
|
||||||
|
if c.kind.name in kinds:
|
||||||
|
return c
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _operator(cursor) -> str | None:
|
||||||
|
try:
|
||||||
|
op = cursor.binary_operator
|
||||||
|
return op.name if op is not None else None
|
||||||
|
except AttributeError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _top_level_try(stack) -> bool:
|
||||||
|
"""True when the try sits directly in its function's body, not nested in a
|
||||||
|
loop, branch, block or another try."""
|
||||||
|
for i in range(len(stack) - 1, -1, -1):
|
||||||
|
kind = stack[i].kind.name
|
||||||
|
if kind in ("CXX_METHOD", "FUNCTION_DECL", "CONSTRUCTOR", "LAMBDA_EXPR"):
|
||||||
|
# stack[i+1] is the body; the try must be its direct child.
|
||||||
|
return len(stack) - 1 == i + 1 and stack[i + 1].kind.name == "COMPOUND_STMT"
|
||||||
|
if kind == "CXX_TRY_STMT":
|
||||||
|
return False
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _field_owner(expr, cindex):
|
||||||
|
"""For `x.state_` / `state_` / `this->state_`: the class declaring the field."""
|
||||||
|
for node in expr.walk_preorder():
|
||||||
|
if node.kind.name == "MEMBER_REF_EXPR":
|
||||||
|
ref = node.referenced
|
||||||
|
if ref is not None and ref.kind.name == "FIELD_DECL":
|
||||||
|
return ref.semantic_parent, ref.spelling
|
||||||
|
return None, None
|
||||||
|
|
||||||
|
|
||||||
|
def analyse_tu(task):
|
||||||
|
"""Parse one translation unit; return the facts located in project files."""
|
||||||
|
file, directory, args, root, includes = task
|
||||||
|
cindex = _load_cindex()
|
||||||
|
index = cindex.Index.create()
|
||||||
|
try:
|
||||||
|
tu = index.parse(
|
||||||
|
os.path.normpath(os.path.join(directory, file)),
|
||||||
|
args=args,
|
||||||
|
options=cindex.TranslationUnit.PARSE_INCOMPLETE,
|
||||||
|
)
|
||||||
|
except cindex.TranslationUnitLoadError as error:
|
||||||
|
return {"file": file, "error": str(error)}
|
||||||
|
|
||||||
|
root = os.path.realpath(root)
|
||||||
|
prefixes = tuple(os.path.join(root, p) + os.sep for p in includes)
|
||||||
|
|
||||||
|
def in_project(cursor) -> str | None:
|
||||||
|
loc = cursor.location
|
||||||
|
if loc.file is None:
|
||||||
|
return None
|
||||||
|
path = os.path.realpath(loc.file.name)
|
||||||
|
if not path.startswith(prefixes):
|
||||||
|
return None
|
||||||
|
return os.path.relpath(path, root)
|
||||||
|
|
||||||
|
enums = {}
|
||||||
|
refs = []
|
||||||
|
busy_tries = []
|
||||||
|
|
||||||
|
def classify(stack, ref):
|
||||||
|
"""Return (role, owner, field, kind) for an enumerator reference."""
|
||||||
|
child = ref
|
||||||
|
for parent in reversed(stack):
|
||||||
|
kind = parent.kind.name
|
||||||
|
if kind in _TRANSPARENT:
|
||||||
|
child = parent
|
||||||
|
continue
|
||||||
|
if kind == "CASE_STMT":
|
||||||
|
first = next(iter(parent.get_children()), None)
|
||||||
|
if first is not None and first.hash == child.hash:
|
||||||
|
return ("observe", None, None, "case")
|
||||||
|
return ("produce", None, None, "other")
|
||||||
|
if kind == "BINARY_OPERATOR":
|
||||||
|
op = _operator(parent)
|
||||||
|
children = list(parent.get_children())
|
||||||
|
if op in ("EQ", "NE"):
|
||||||
|
return ("observe", None, None, "compare")
|
||||||
|
if op == "Assign" and len(children) == 2 and children[1].hash == child.hash:
|
||||||
|
owner, field = _field_owner(children[0], cindex)
|
||||||
|
return ("produce", owner, field, "assign")
|
||||||
|
return ("produce", None, None, "other")
|
||||||
|
if kind == "CALL_EXPR" and parent.spelling in ("store", "exchange"):
|
||||||
|
owner, field = _field_owner(parent, cindex)
|
||||||
|
return ("produce", owner, field, "assign")
|
||||||
|
if kind == "MEMBER_REF":
|
||||||
|
ref_decl = parent.referenced
|
||||||
|
if ref_decl is not None and ref_decl.kind.name == "FIELD_DECL":
|
||||||
|
return ("produce", ref_decl.semantic_parent, ref_decl.spelling, "initial")
|
||||||
|
if kind == "FIELD_DECL":
|
||||||
|
return ("produce", parent.semantic_parent, parent.spelling, "initial")
|
||||||
|
if kind in ("CALL_EXPR", "RETURN_STMT", "VAR_DECL", "COMPOUND_STMT",
|
||||||
|
"IF_STMT", "CXX_METHOD", "FUNCTION_DECL", "CONSTRUCTOR"):
|
||||||
|
return ("produce", None, None, "other")
|
||||||
|
child = parent
|
||||||
|
return ("produce", None, None, "other")
|
||||||
|
|
||||||
|
def member_name(stack):
|
||||||
|
fn = _enclosing(stack, {"CXX_METHOD", "FUNCTION_DECL", "CONSTRUCTOR",
|
||||||
|
"DESTRUCTOR", "FUNCTION_TEMPLATE", "LAMBDA_EXPR"})
|
||||||
|
if fn is None:
|
||||||
|
return "<top>", None
|
||||||
|
if fn.kind.name == "LAMBDA_EXPR":
|
||||||
|
outer = _enclosing(stack[: stack.index(fn)], {"CXX_METHOD", "FUNCTION_DECL", "CONSTRUCTOR"})
|
||||||
|
name = outer.spelling if outer is not None else "<lambda>"
|
||||||
|
return f"{name}::<lambda>", outer
|
||||||
|
return fn.spelling, fn
|
||||||
|
|
||||||
|
def scan_try(stack, try_cursor, rel):
|
||||||
|
"""A try inside a member that wrote a busy value earlier."""
|
||||||
|
body = list(try_cursor.get_children())
|
||||||
|
handlers = [c for c in body if c.kind.name == "CXX_CATCH_STMT"]
|
||||||
|
swallowed = []
|
||||||
|
for handler in handlers:
|
||||||
|
writes_state = False
|
||||||
|
rethrows = False
|
||||||
|
for node in handler.walk_preorder():
|
||||||
|
if node.kind.name == "CXX_THROW_EXPR":
|
||||||
|
rethrows = True
|
||||||
|
if node.kind.name == "DECL_REF_EXPR":
|
||||||
|
ref = node.referenced
|
||||||
|
if ref is not None and ref.kind.name == "ENUM_CONSTANT_DECL":
|
||||||
|
writes_state = True
|
||||||
|
if node.kind.name in ("CALL_EXPR",) and re.search(
|
||||||
|
r"(set|mark|fail|error|transition|reset|clear)", node.spelling or "", re.I
|
||||||
|
):
|
||||||
|
writes_state = True
|
||||||
|
if not writes_state and not rethrows:
|
||||||
|
tokens = " ".join(t.spelling for t in handler.get_tokens())
|
||||||
|
swallowed.append((handler.location.line, tokens[:117] + ("..." if len(tokens) > 117 else "")))
|
||||||
|
return swallowed
|
||||||
|
|
||||||
|
def visit(cursor, stack, busy_seen):
|
||||||
|
rel = in_project(cursor)
|
||||||
|
if rel is None and cursor.kind.name != "TRANSLATION_UNIT":
|
||||||
|
return
|
||||||
|
kind = cursor.kind.name
|
||||||
|
if kind == "ENUM_DECL" and cursor.is_scoped_enum() and cursor.is_definition():
|
||||||
|
name = _qualified(cursor)
|
||||||
|
enums[name] = {
|
||||||
|
"name": name,
|
||||||
|
"declaredIn": rel,
|
||||||
|
"line": cursor.location.line,
|
||||||
|
"variants": [c.spelling for c in cursor.get_children()
|
||||||
|
if c.kind.name == "ENUM_CONSTANT_DECL"],
|
||||||
|
}
|
||||||
|
elif kind == "DECL_REF_EXPR":
|
||||||
|
ref = cursor.referenced
|
||||||
|
if ref is not None and ref.kind.name == "ENUM_CONSTANT_DECL":
|
||||||
|
parent_enum = ref.semantic_parent
|
||||||
|
if parent_enum is not None and parent_enum.is_scoped_enum():
|
||||||
|
role, owner, field, how = classify(stack, cursor)
|
||||||
|
member, fn = member_name(stack)
|
||||||
|
klass = _enclosing(stack, {"CLASS_DECL", "STRUCT_DECL", "CLASS_TEMPLATE"})
|
||||||
|
if fn is not None and fn.semantic_parent is not None and \
|
||||||
|
fn.semantic_parent.kind.name in ("CLASS_DECL", "STRUCT_DECL", "CLASS_TEMPLATE"):
|
||||||
|
klass = fn.semantic_parent
|
||||||
|
record = {
|
||||||
|
"enum": _qualified(parent_enum),
|
||||||
|
"variant": ref.spelling,
|
||||||
|
"role": role,
|
||||||
|
"how": how,
|
||||||
|
"owner": _qualified(owner) if owner is not None else None,
|
||||||
|
"field": field,
|
||||||
|
"member": member,
|
||||||
|
"memberOwner": _qualified(klass) if klass is not None else None,
|
||||||
|
"path": rel,
|
||||||
|
"line": cursor.location.line,
|
||||||
|
"col": cursor.location.column,
|
||||||
|
}
|
||||||
|
refs.append(record)
|
||||||
|
# Only the method's OWN state counts: `ev.stage = Connecting` on a
|
||||||
|
# local event object handed to a callback is not state this
|
||||||
|
# class can leave stuck.
|
||||||
|
if role == "produce" and how == "assign" and BUSY.match(ref.spelling) \
|
||||||
|
and owner is not None and klass is not None \
|
||||||
|
and _qualified(owner) == _qualified(klass):
|
||||||
|
busy_seen.append(record)
|
||||||
|
elif kind == "CXX_STATIC_CAST_EXPR" or kind == "CSTYLE_CAST_EXPR":
|
||||||
|
target = cursor.type.get_canonical()
|
||||||
|
decl = target.get_declaration()
|
||||||
|
if decl is not None and decl.kind.name == "ENUM_DECL" and decl.is_scoped_enum():
|
||||||
|
children = list(cursor.get_children())
|
||||||
|
if children and children[-1].type.get_canonical().kind.name != "ENUM":
|
||||||
|
refs.append({
|
||||||
|
"enum": _qualified(decl), "variant": None, "role": "cast", "how": "cast",
|
||||||
|
"owner": None, "field": None, "member": member_name(stack)[0],
|
||||||
|
"memberOwner": None, "path": rel, "line": cursor.location.line,
|
||||||
|
"col": cursor.location.column,
|
||||||
|
})
|
||||||
|
elif kind == "CXX_TRY_STMT" and busy_seen and _top_level_try(stack):
|
||||||
|
# (a top-level try of the method: nested tries are local fallbacks,
|
||||||
|
# like a JSON parse guarded with `catch (...) { skip = false; }`)
|
||||||
|
member, fn = member_name(stack)
|
||||||
|
for busy in busy_seen:
|
||||||
|
if busy["member"] != member:
|
||||||
|
continue
|
||||||
|
for line, handler in scan_try(stack, cursor, rel):
|
||||||
|
busy_tries.append({
|
||||||
|
"owner": busy["owner"], "member": member, "path": rel,
|
||||||
|
"line": line, "handler": handler,
|
||||||
|
})
|
||||||
|
break
|
||||||
|
new_busy = busy_seen
|
||||||
|
if kind in ("CXX_METHOD", "FUNCTION_DECL", "CONSTRUCTOR", "LAMBDA_EXPR"):
|
||||||
|
new_busy = []
|
||||||
|
stack.append(cursor)
|
||||||
|
for child in cursor.get_children():
|
||||||
|
visit(child, stack, new_busy)
|
||||||
|
stack.pop()
|
||||||
|
|
||||||
|
visit(tu.cursor, [], [])
|
||||||
|
errors = [f"{d.location.file}:{d.location.line}: {d.spelling}"
|
||||||
|
for d in tu.diagnostics if d.severity >= 3]
|
||||||
|
return {"file": file, "enums": list(enums.values()), "refs": refs, "tries": busy_tries,
|
||||||
|
"errors": len(errors), "errorSamples": errors[:3]}
|
||||||
|
|
||||||
|
|
||||||
|
def build_model(root: Path, results: list[dict], includes: list[str]) -> dict:
|
||||||
|
enums: dict[str, dict] = {}
|
||||||
|
seen_refs = set()
|
||||||
|
refs = []
|
||||||
|
tries = {}
|
||||||
|
failed = []
|
||||||
|
for result in results:
|
||||||
|
if "error" in result:
|
||||||
|
failed.append(result["file"])
|
||||||
|
continue
|
||||||
|
for enum in result["enums"]:
|
||||||
|
enums.setdefault(enum["name"], enum)
|
||||||
|
for ref in result["refs"]:
|
||||||
|
key = (ref["path"], ref["line"], ref["col"], ref["enum"], ref["variant"])
|
||||||
|
if key in seen_refs:
|
||||||
|
continue
|
||||||
|
seen_refs.add(key)
|
||||||
|
refs.append(ref)
|
||||||
|
for t in result["tries"]:
|
||||||
|
tries.setdefault((t["path"], t["line"]), t)
|
||||||
|
|
||||||
|
by_enum = defaultdict(list)
|
||||||
|
for ref in refs:
|
||||||
|
by_enum[ref["enum"]].append(ref)
|
||||||
|
|
||||||
|
unions, coverage, emissions, renders = [], [], [], []
|
||||||
|
for name, enum in sorted(enums.items()):
|
||||||
|
if len(enum["variants"]) < 2:
|
||||||
|
continue
|
||||||
|
erefs = by_enum.get(name, [])
|
||||||
|
owners = Counter(r["owner"] for r in erefs if r["role"] == "produce" and r["owner"])
|
||||||
|
# The class whose field holds this state; without one it is a plain value
|
||||||
|
# type (a mode flag, a return code), not state anyone could leave stale.
|
||||||
|
if not owners:
|
||||||
|
continue
|
||||||
|
owner = owners.most_common(1)[0][0]
|
||||||
|
variants = enum["variants"]
|
||||||
|
emitted, initial, observed = set(), set(), set()
|
||||||
|
observe_sites = 0
|
||||||
|
emit_sites = 0
|
||||||
|
for r in erefs:
|
||||||
|
if r["role"] == "observe":
|
||||||
|
observed.add(r["variant"])
|
||||||
|
observe_sites += 1
|
||||||
|
renders.append({
|
||||||
|
"path": r["path"], "line": r["line"], "widget": r["how"],
|
||||||
|
"stateType": name,
|
||||||
|
"branches": [{"variant": r["variant"], "line": r["line"], "trivial": False}],
|
||||||
|
})
|
||||||
|
elif r["role"] in ("produce", "cast"):
|
||||||
|
emit_sites += 1
|
||||||
|
if r["variant"]:
|
||||||
|
emitted.add(r["variant"])
|
||||||
|
if r["how"] == "initial":
|
||||||
|
initial.add(r["variant"])
|
||||||
|
emissions.append({
|
||||||
|
"path": r["path"], "line": r["line"], "owner": owner,
|
||||||
|
"member": r["member"], "kind": "assign" if r["how"] == "assign" else "emit",
|
||||||
|
"confidence": "exact" if r["role"] == "produce" else "indirect",
|
||||||
|
"variants": [r["variant"]] if r["variant"] else [],
|
||||||
|
"snippet": f"{r['field'] or ''} = {name.split('::')[-1]}::{r['variant']}"
|
||||||
|
if r["variant"] else f"static_cast<{name.split('::')[-1]}>(...)",
|
||||||
|
# A class holds several enums: name the union so a cast to one
|
||||||
|
# does not make "never emitted" unprovable for the others.
|
||||||
|
"stateType": name,
|
||||||
|
})
|
||||||
|
union = {
|
||||||
|
"name": name, "declaredIn": enum["declaredIn"], "style": "enumUnion",
|
||||||
|
"owner": owner, "ownerKind": "Cubit",
|
||||||
|
"variants": [{"name": v, "declaredIn": enum["declaredIn"]} for v in variants],
|
||||||
|
}
|
||||||
|
unions.append(union)
|
||||||
|
coverage.append({
|
||||||
|
"union": name, "owner": owner, "style": "enumUnion",
|
||||||
|
"declared": sorted(variants), "emitted": sorted(emitted), "initial": sorted(initial),
|
||||||
|
"rendered": sorted(observed), "renderedInBuilder": [], "trivialInBuilder": [],
|
||||||
|
"emissionSites": emit_sites, "renderSites": observe_sites,
|
||||||
|
})
|
||||||
|
|
||||||
|
owners = {u["owner"] for u in unions}
|
||||||
|
swallowed = []
|
||||||
|
ordinals = Counter()
|
||||||
|
for (path, line), t in sorted(tries.items()):
|
||||||
|
if t["owner"] not in owners:
|
||||||
|
continue
|
||||||
|
ordinals[(t["owner"], t["member"])] += 1
|
||||||
|
swallowed.append({
|
||||||
|
"owner": t["owner"], "member": t["member"], "path": path, "line": line,
|
||||||
|
"ordinal": ordinals[(t["owner"], t["member"])], "handler": t["handler"],
|
||||||
|
})
|
||||||
|
|
||||||
|
# Files outside an owner's own declaration that mention it: pipe uses these
|
||||||
|
# to tell a dormant (unwired) union from a partly observed one.
|
||||||
|
references = defaultdict(set)
|
||||||
|
for r in refs:
|
||||||
|
if r["role"] == "observe" and r["enum"] in {u["name"] for u in unions}:
|
||||||
|
owner = next(u["owner"] for u in unions if u["name"] == r["enum"])
|
||||||
|
references[owner].add(r["path"])
|
||||||
|
|
||||||
|
return {
|
||||||
|
"root": str(root),
|
||||||
|
"directories": includes,
|
||||||
|
"frontEnd": {"name": "pipe-cpp-front (experimental)", "failedTranslationUnits": failed},
|
||||||
|
"unions": unions,
|
||||||
|
"eventUnions": [],
|
||||||
|
"emissions": emissions,
|
||||||
|
"renders": renders,
|
||||||
|
"handlers": [],
|
||||||
|
"dispatches": [],
|
||||||
|
"coverage": coverage,
|
||||||
|
"ownerReferences": {k: sorted(v) for k, v in references.items()},
|
||||||
|
"flagStates": [],
|
||||||
|
"riverpod": {"providers": [], "sites": []},
|
||||||
|
"constructedByFile": {},
|
||||||
|
"swallowedFailures": swallowed,
|
||||||
|
"unsettledFlags": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||||
|
parser.add_argument("root")
|
||||||
|
parser.add_argument("--build-dir", help="directory holding compile_commands.json")
|
||||||
|
parser.add_argument("--include", default="src,include",
|
||||||
|
help="project subdirectories whose code is analysed")
|
||||||
|
parser.add_argument("--limit", type=int)
|
||||||
|
parser.add_argument("--match", help="only translation units whose path contains this")
|
||||||
|
parser.add_argument("--jobs", type=int, default=os.cpu_count())
|
||||||
|
parser.add_argument("-o", "--output", default="-")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
root = Path(args.root).resolve()
|
||||||
|
db_dir = Path(args.build_dir) if args.build_dir else root
|
||||||
|
db_path = db_dir / "compile_commands.json"
|
||||||
|
if not db_path.is_file():
|
||||||
|
print(f"cpp-front: no compile database at {db_path}; generate one "
|
||||||
|
f"(meson/CMake write it to the build dir) and pass --build-dir",
|
||||||
|
file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
database = json.loads(db_path.read_text())
|
||||||
|
includes = [p for p in args.include.split(",") if p]
|
||||||
|
resource_dir = _resource_dir()
|
||||||
|
global _SDK
|
||||||
|
_SDK = _sdk_root()
|
||||||
|
|
||||||
|
tasks = []
|
||||||
|
seen = set()
|
||||||
|
for entry in database:
|
||||||
|
# Resolved on both sides: a symlinked path (macOS /var -> /private/var)
|
||||||
|
# otherwise lands "outside" the resolved root and the TU is skipped.
|
||||||
|
path = os.path.realpath(os.path.join(entry["directory"], entry["file"]))
|
||||||
|
if path in seen or not path.endswith((".cpp", ".cc", ".cxx")):
|
||||||
|
continue
|
||||||
|
rel = os.path.relpath(path, root)
|
||||||
|
if not any(rel.startswith(p + os.sep) for p in includes):
|
||||||
|
continue
|
||||||
|
if args.match and args.match not in rel:
|
||||||
|
continue
|
||||||
|
seen.add(path)
|
||||||
|
tasks.append((entry["file"], entry["directory"], _args_for(entry, resource_dir),
|
||||||
|
str(root), includes))
|
||||||
|
if args.limit:
|
||||||
|
tasks = tasks[: args.limit]
|
||||||
|
if not tasks:
|
||||||
|
# An empty model reads as a clean project; refuse instead.
|
||||||
|
print(f"cpp-front: no translation unit under {', '.join(includes)} in "
|
||||||
|
f"{db_dir / 'compile_commands.json'}; nothing was analysed", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
results = []
|
||||||
|
with ProcessPoolExecutor(max_workers=args.jobs) as pool:
|
||||||
|
futures = [pool.submit(analyse_tu, t) for t in tasks]
|
||||||
|
for i, future in enumerate(as_completed(futures), 1):
|
||||||
|
results.append(future.result())
|
||||||
|
if i % 25 == 0 or i == len(tasks):
|
||||||
|
print(f"cpp-front: {i}/{len(tasks)} translation units", file=sys.stderr)
|
||||||
|
|
||||||
|
model = build_model(root, results, includes)
|
||||||
|
|
||||||
|
# Sources the build does not compile (optional backends, benchmarks) were
|
||||||
|
# never analysed, so evidence in them is missing. Recorded as unresolved with
|
||||||
|
# the names they mention, pipe lowers confidence for any finding naming them.
|
||||||
|
unanalysed = []
|
||||||
|
names: set[str] = set()
|
||||||
|
for include in includes:
|
||||||
|
for path in sorted((root / include).rglob("*")):
|
||||||
|
if path.suffix not in (".cpp", ".cc", ".cxx") or os.path.realpath(path) in seen:
|
||||||
|
continue
|
||||||
|
unanalysed.append(os.path.relpath(path, root))
|
||||||
|
names.update(re.findall(r"[A-Za-z_][A-Za-z0-9_]*", path.read_text(errors="replace")))
|
||||||
|
failed_files = [os.path.normpath(f) for f in model["frontEnd"]["failedTranslationUnits"]]
|
||||||
|
if unanalysed or failed_files:
|
||||||
|
files = unanalysed + failed_files
|
||||||
|
model["unresolved"] = {
|
||||||
|
"files": files,
|
||||||
|
"analysed": len(tasks) - len(failed_files),
|
||||||
|
"first": f"{files[0]}: not compiled by this build, so not analysed",
|
||||||
|
"names": sorted(names),
|
||||||
|
}
|
||||||
|
failed = model["frontEnd"]["failedTranslationUnits"]
|
||||||
|
errors = sum(r.get("errors", 0) for r in results)
|
||||||
|
for r in results:
|
||||||
|
for sample in r.get("errorSamples", [])[:1]:
|
||||||
|
print(f"cpp-front: clang error in {r['file']}: {sample}", file=sys.stderr)
|
||||||
|
print(f"cpp-front: {len(model['unions'])} state enums, {len(model['emissions'])} "
|
||||||
|
f"emissions, {len(model['renders'])} observations, "
|
||||||
|
f"{len(model['swallowedFailures'])} swallowed failures; "
|
||||||
|
f"{len(failed)} TU(s) failed, {errors} clang error(s)", file=sys.stderr)
|
||||||
|
text = json.dumps(model, indent=2)
|
||||||
|
if args.output == "-":
|
||||||
|
print(text)
|
||||||
|
else:
|
||||||
|
Path(args.output).write_text(text)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
Loading…
Add table
Add a link
Reference in a new issue