merge/audit-and-state-map #7

Merged
trevon merged 15 commits from merge/audit-and-state-map into main 2026-09-25 00:07:04 +00:00
6 changed files with 449 additions and 24 deletions
Showing only changes of commit 0d891919e9 - Show all commits

feat(check): run the rules on a state model written by any front end

Third step of the state-map seam, and the proof that it holds.

stateFlowFromJson rebuilds a StateFlow from its toJson form (computed
members - totals, coverage differences, Riverpod gaps - are recomputed),
and `pipe check --model <file>` runs every rule on such a file instead
of analysing Dart. A front end for another language now only has to
write that JSON to get every rule, --baseline ratcheting and --fail-on
gating. StateUnion.toJson now carries uiReadsBaseFields (only when
true), which the round trip showed it had been dropping.

Checked byte for byte, through a file on disk: `pipe state --format
json` then `pipe check --model` reports exactly what `pipe check` does
on the fixture and on resolved mirrors of nuwayze (102 findings), tick,
dex and word_alive. state_map_roundtrip_test.dart asserts the same in
process (model JSON is idempotent; findings identical; every rule fires,
so the comparison is not vacuous), and model_cli_test.dart covers the
CLI path.

--model refuses --write-baseline: a model does not record whether its
front end read every file, so the baseline could silently be partial -
the hazard the source path already refuses. A malformed model is a
PipeException, not a crash.

134 tests pass.
trvon 2026-09-22 14:31:03 -06:00
Signed by: trevon
GPG key ID: 38686EA797D7A8A7

View file

@ -375,7 +375,10 @@ lib/src/index.dart resolution health, timing, diagnostics
lib/src/model.dart declaration model
lib/src/state_flow.dart the state-flow IR (unions, emissions, render sites,
events, flags, Riverpod)
lib/src/rules.dart breakage detectors, findings, baselines
lib/src/facts.dart language-neutral facts the rules judge (JSON round-trip)
lib/src/dart_facts.dart Dart front end: loading/construction recognisers
lib/src/dart_provider_facts.dart Dart front end: ChangeNotifier/provider recognisers
lib/src/rules.dart breakage detectors, findings, baselines (model only)
lib/src/recall.dart labelled-bug recall measurement
lib/src/verify.dart baseline verification (resolved / still / new)
lib/src/suggest.dart ranked fix order per file
@ -386,6 +389,28 @@ tool/adjudicate.py independent precision adjudicator
tool/build_release.sh native AOT build
```
### The state model is the seam
Rules never read a syntax tree. The Dart front end (`state_flow.dart`,
`dart_facts.dart`, `dart_provider_facts.dart`) turns resolved code into a state
model: unions and their variants, where each is emitted and rendered, event
dispatch and handling, busy paths whose failure is swallowed, observable objects
and whether their mutations are announced. `runChecks` takes only that model.
The model is plain JSON, and a model that went through JSON reports exactly
what the in-process one does - checked byte for byte on the fixture
(`test/state_map_roundtrip_test.dart`) and on four real apps:
```bash
pipe state . --format json > model.json # any front end can write this
pipe check --model model.json # every rule, baseline and gate
```
So a front end for another language only has to write that JSON; it gets every
rule, `--baseline` ratcheting and `--fail-on` gating without touching Dart.
`check --model` refuses `--write-baseline`, because a model does not record
whether its front end read every file.
Built on `package:analyzer` with whole-project resolution, not a
`custom_lint`/analyzer plugin: plugins have no project-wide index, `flutter
analyze` does not report new-system plugin diagnostics, and a CLI can pin its own

View file

@ -60,6 +60,7 @@ export 'src/state_flow.dart'
EventDispatch,
EventHandler,
EventUnion,
FlagState,
RenderBranch,
RenderSite,
StateFlow,
@ -67,7 +68,8 @@ export 'src/state_flow.dart'
StateVariant,
UnionCoverage,
UnionStyle,
buildStateFlow;
buildStateFlow,
stateFlowFromJson;
export 'src/state_flow.dart'
show RiverpodGap, RiverpodModel, RiverpodProvider, RiverpodSite;
export 'src/suggest.dart' show FilePlan, FixAction, WorkPlan, buildWorkPlan;

View file

@ -374,6 +374,12 @@ class CheckCommand extends Command<int> {
..addOption(
'write-baseline',
help: 'Write the current findings to this baseline file.',
)
..addOption(
'model',
help:
'Check a state model (`pipe state --format json`, or any front end '
'that writes it) instead of analysing Dart source.',
);
}
@ -388,12 +394,57 @@ class CheckCommand extends Command<int> {
Future<int> run() async {
final results = argResults!;
_refuseSubsetBaseline(results);
final selected = _readRules(results);
final baseline = _readBaseline(results);
final model = results['model'] as String?;
if (model != null) {
// A model does not record whether its front end read every file, so a
// baseline written from it could silently omit findings.
if (results['write-baseline'] != null) {
throw PipeException(
'--write-baseline cannot be used with --model: a model does not '
'record whether every file was read, so the baseline could be '
'partial',
remedy: 'write the baseline from a source analysis instead',
);
}
final file = File(model);
if (!file.existsSync()) {
throw PipeException('model file not found: $model');
}
final Object? json;
try {
json = jsonDecode(file.readAsStringSync());
} on FormatException catch (error) {
throw PipeException('model is not JSON: ${error.message}');
}
if (json is! Map<String, Object?>) {
throw PipeException('model must be a JSON object: $model');
}
final StateFlow flow;
try {
flow = stateFlowFromJson(json);
} on TypeError catch (error) {
throw PipeException(
'model is missing a field or has one of the wrong type: $error',
remedy:
'write it with `pipe state --format json`, or match that shape',
);
}
final report = runChecks(
flow: flow,
enabledRules: selected.isEmpty ? null : selected,
baseline: baseline,
policy: _readPolicy(results),
);
return _emitCheck(results, report);
}
final project = await loadProject(
_target(results),
directories: results['dir'] as List<String>,
);
final selected = _readRules(results);
final baseline = _readBaseline(results);
final analysis = await analyzeProject(
project,
sdkOverride: results['sdk'] as String?,
@ -405,32 +456,37 @@ class CheckCommand extends Command<int> {
policy: _readPolicy(results),
);
try {
final report = analysis.report;
if (results['format'] as String == 'json') {
stdout.writeln(_encode(report.toJson()));
} else {
final top = readPositiveInt(results, 'top', fallback: 40);
stdout.write(report.describe(top: top));
}
_writeBaseline(results, analysis, report.findings);
final threshold = switch (results['fail-on'] as String) {
'error' => PipeSeverity.error,
'warning' => PipeSeverity.warning,
'info' => PipeSeverity.info,
_ => null,
};
if (threshold != null && report.hasNewAtOrAbove(threshold)) return 1;
if ((results['fail-on-finding'] as bool) &&
report.hasNewAtOrAbove(PipeSeverity.error)) {
return 1;
}
final code = _emitCheck(results, analysis.report);
_writeBaseline(results, analysis, analysis.report.findings);
return code;
} finally {
await analysis.dispose();
}
return 0;
}
}
/// Print a check report and apply the --fail-on gate.
int _emitCheck(ArgResults results, FindingReport report) {
final top = readPositiveInt(results, 'top', fallback: 40);
if (results['format'] as String == 'json') {
stdout.writeln(_encode(report.toJson()));
} else {
stdout.write(report.describe(top: top));
}
final threshold = switch (results['fail-on'] as String) {
'error' => PipeSeverity.error,
'warning' => PipeSeverity.warning,
'info' => PipeSeverity.info,
_ => null,
};
if (threshold != null && report.hasNewAtOrAbove(threshold)) return 1;
if ((results['fail-on-finding'] as bool) &&
report.hasNewAtOrAbove(PipeSeverity.error)) {
return 1;
}
return 0;
}
/// Measure detection against a labelled set of known bugs.
class RecallCommand extends Command<int> {
RecallCommand() {

View file

@ -89,6 +89,7 @@ class StateUnion {
'ownerKind': ownerKind,
if (eventType != null) 'eventType': eventType,
'variants': [for (final variant in variants) variant.toJson()],
if (uiReadsBaseFields) 'uiReadsBaseFields': true,
};
}
@ -1971,3 +1972,184 @@ UnresolvedSummary _unresolvedSummary(
names: found.names,
);
}
// ---------------------------------------------------------------------------
// Decoding
// ---------------------------------------------------------------------------
/// Rebuild a [StateFlow] from its [StateFlow.toJson] form.
///
/// This is the seam for other front ends: anything that writes this JSON (a
/// separate process for another language, say) can be checked by every rule
/// with `runChecks(flow: stateFlowFromJson(json))`. Computed members (`totals`,
/// coverage differences, Riverpod `gaps`, `isAsync`) are ignored on input and
/// recomputed.
StateFlow stateFlowFromJson(Map<String, Object?> json) {
List<Map<String, Object?>> objects(Object? value) => [
for (final item in (value as List?) ?? const [])
item as Map<String, Object?>,
];
List<String> strings(Object? value) => [
...((value as List?) ?? const []).cast<String>(),
];
StateVariant variant(Map<String, Object?> v) => StateVariant(
name: v['name']! as String,
declaredIn: v['declaredIn'] as String?,
note: v['note'] as String?,
parent: v['parent'] as String?,
);
final unions = [
for (final u in objects(json['unions']))
StateUnion(
name: u['name']! as String,
declaredIn: u['declaredIn']! as String,
style: UnionStyle.values.byName(u['style']! as String),
owner: u['owner']! as String,
ownerKind: u['ownerKind']! as String,
eventType: u['eventType'] as String?,
variants: [for (final v in objects(u['variants'])) variant(v)],
uiReadsBaseFields: (u['uiReadsBaseFields'] as bool?) ?? false,
),
];
StateUnion unionFor(String name, String owner) =>
unions.firstWhere((union) => union.name == name && union.owner == owner);
return StateFlow(
root: json['root']! as String,
directories: strings(json['directories'] ?? const ['lib']),
unions: unions,
eventUnions: [
for (final u in objects(json['eventUnions']))
EventUnion(
name: u['name']! as String,
declaredIn: u['declaredIn']! as String,
owner: u['owner']! as String,
variants: [for (final v in objects(u['variants'])) variant(v)],
),
],
emissions: [
for (final e in objects(json['emissions']))
EmissionSite(
path: e['path']! as String,
line: e['line']! as int,
owner: e['owner']! as String,
member: e['member']! as String,
kind: e['kind']! as String,
confidence: e['confidence']! as String,
variants: strings(e['variants']),
snippet: e['snippet']! as String,
),
],
renders: [
for (final r in objects(json['renders']))
RenderSite(
path: r['path']! as String,
line: r['line']! as int,
widget: r['widget']! as String,
stateType: r['stateType']! as String,
branches: [
for (final b in objects(r['branches']))
RenderBranch(
variant: b['variant']! as String,
line: b['line']! as int,
trivial: b['trivial']! as bool,
negated: (b['negated'] as bool?) ?? false,
),
],
),
],
handlers: [
for (final h in objects(json['handlers']))
EventHandler(
bloc: h['bloc']! as String,
event: h['event']! as String,
path: h['path']! as String,
line: h['line']! as int,
),
],
dispatches: [
for (final d in objects(json['dispatches']))
EventDispatch(
event: d['event']! as String,
path: d['path']! as String,
line: d['line']! as int,
confidence: d['confidence']! as String,
snippet: d['snippet']! as String,
),
],
coverages: [
for (final c in objects(json['coverage']))
UnionCoverage(
union: unionFor(c['union']! as String, c['owner']! as String),
declared: {...strings(c['declared'])},
emitted: {...strings(c['emitted'])},
initial: {...strings(c['initial'])},
rendered: {...strings(c['rendered'])},
renderedInBuilder: {...strings(c['renderedInBuilder'])},
trivialInBuilder: {...strings(c['trivialInBuilder'])},
renderSites: c['renderSites']! as int,
emissionSites: c['emissionSites']! as int,
),
],
ownerReferences: {
for (final entry
in ((json['ownerReferences'] as Map?) ?? const {}).entries)
entry.key as String: strings(entry.value),
},
flagStates: [
for (final f in objects(json['flagStates']))
FlagState(
name: f['name']! as String,
owner: f['owner']! as String,
fields: strings(f['fields']),
defaultsTrue: f['defaultsTrue']! as bool,
),
],
riverpod: RiverpodModel(
providers: [
for (final p in objects((json['riverpod'] as Map?)?['providers']))
RiverpodProvider(
name: p['name']! as String,
kind: p['kind']! as String,
declaredIn: p['declaredIn']! as String,
line: p['line']! as int,
valueType: p['valueType'] as String?,
),
],
sites: [
for (final site in objects((json['riverpod'] as Map?)?['sites']))
RiverpodSite(
provider: site['provider']! as String,
path: site['path']! as String,
line: site['line']! as int,
watches: site['watch']! as bool,
handlesError: site['handlesError']! as bool,
handlesLoading: site['handlesLoading']! as bool,
readsValue: site['readsValue']! as bool,
),
],
),
constructedByFile: {
for (final entry
in ((json['constructedByFile'] as Map?) ?? const {}).entries)
entry.key as String: {...strings(entry.value)},
},
swallowedFailures: [
for (final f in objects(json['swallowedFailures']))
SwallowedFailure.fromJson(f),
],
unsettledFlags: [
for (final f in objects(json['unsettledFlags']))
UnsettledFlag.fromJson(f),
],
unresolved: json['unresolved'] == null
? null
: UnresolvedSummary.fromJson(
json['unresolved']! as Map<String, Object?>,
),
notifierModel: json['notifiers'] == null
? null
: NotifierModel.fromJson(json['notifiers']! as Map<String, Object?>),
);
}

116
test/model_cli_test.dart Normal file
View file

@ -0,0 +1,116 @@
import 'dart:io';
import 'package:pipe/pipe.dart';
import 'package:pipe/src/cli.dart';
import 'package:test/test.dart';
/// Collects what a command writes to stdout.
class _CapturedStdout implements Stdout {
final StringBuffer buffer = StringBuffer();
@override
void write(Object? object) => buffer.write(object);
@override
void writeln([Object? object = '']) => buffer.writeln(object);
@override
dynamic noSuchMethod(Invocation invocation) => null;
}
Future<(int?, String)> _run(List<String> args) async {
final out = _CapturedStdout();
final code = await IOOverrides.runZoned(
() => PipeRunner().run(args),
stdout: () => out,
);
return (code, out.buffer.toString());
}
const String _fixture = 'test/fixtures/stuck_loading';
void main() {
late String model;
setUpAll(() async {
final dir = Directory.systemTemp.createTempSync('pipe-model');
model = '${dir.path}/model.json';
final (code, json) = await _run([
'state',
_fixture,
'--format',
'json',
'-c',
'1',
]);
expect(code, 0);
File(model).writeAsStringSync(json);
});
test('check --model reports exactly what a source check reports', () async {
// The front-end seam as a separate process would use it: the model goes
// through a file, and the rules must not notice.
final (_, fromSource) = await _run([
'check',
_fixture,
'--format',
'json',
'-c',
'1',
]);
final (_, fromModel) = await _run([
'check',
'--model',
model,
'--format',
'json',
]);
expect(fromModel, fromSource);
});
test('check --model honours --rule and --fail-on', () async {
final (code, output) = await _run([
'check',
'--model',
model,
'--rule',
ruleEventWithoutHandler,
'--fail-on',
'error',
'--format',
'json',
]);
expect(code, 1, reason: 'event-without-handler fires at error severity');
expect(output, contains(ruleEventWithoutHandler));
expect(output, isNot(contains(ruleDeadHandler)));
});
test('check --model refuses --write-baseline', () async {
final target = '${File(model).parent.path}/baseline.json';
await expectLater(
_run(['check', '--model', model, '--write-baseline', target]),
throwsA(
isA<PipeException>().having(
(error) => error.toString(),
'message',
contains('--model'),
),
),
);
expect(File(target).existsSync(), isFalse);
});
test('check --model rejects a file that is not a model', () async {
final dir = File(model).parent.path;
File('$dir/array.json').writeAsStringSync('[1, 2]');
// An object missing required fields must be a clear error, not a crash.
File('$dir/partial.json').writeAsStringSync('{"unions": [{"name": 1}]}');
for (final bad in ['array.json', 'partial.json']) {
await expectLater(
_run(['check', '--model', '$dir/$bad']),
throwsA(isA<PipeException>()),
reason: bad,
);
}
});
}

View file

@ -0,0 +1,44 @@
import 'dart:convert';
import 'package:pipe/pipe.dart';
import 'package:test/test.dart';
/// The state-map seam: rules read only the model, so a model that went through
/// JSON - as one written by a separate front end would - must produce exactly
/// the same report as the in-process model. If a rule quietly depended on
/// something the JSON does not carry, this is where it shows.
void main() {
late ProjectAnalysis analysis;
setUpAll(() async {
analysis = await analyzeProject(
await loadProject('test/fixtures/stuck_loading'),
concurrency: 1,
);
});
tearDownAll(() async {
await analysis.dispose();
});
StateFlow throughJson(StateFlow flow) => stateFlowFromJson(
jsonDecode(jsonEncode(flow.toJson())) as Map<String, Object?>,
);
test('the model survives JSON unchanged', () {
final once = jsonEncode(analysis.flow.toJson());
final twice = jsonEncode(throughJson(analysis.flow).toJson());
expect(twice, once);
});
test('rules on the decoded model report exactly the same findings', () {
final direct = jsonEncode(runChecks(flow: analysis.flow).toJson());
final decoded = jsonEncode(
runChecks(flow: throughJson(analysis.flow)).toJson(),
);
expect(decoded, direct);
// Not vacuous: every rule fires somewhere on the fixture.
final rules = runChecks(flow: analysis.flow).findings.map((f) => f.rule);
expect(rules.toSet(), containsAll(allRules));
});
}