trevon/pipe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

merge/audit-and-state-map #7

Merged
trevon merged 15 commits from merge/audit-and-state-map into main 2026-09-25 00:07:04 +00:00
Conversation 0 Commits 15 Files changed 40 +397 -478
5 changed files with 397 additions and 478 deletions
Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit 1779da7e4d - Show all commits

docs: trim the README and split usage, rules, design and development out

Prev Next 
The README had grown to 500 lines mixing an overview, a CLI reference,
rule evidence, measurement method, architecture, build notes and
progress notes. It is now a 72-line overview (what pipe is, coverage,
quick start, the rule list, links), and the rest lives in:

- docs/usage.md: commands, flags (checked against `pipe help`), SDK and
  resolution behaviour, CI baselines, verify, fix order, agent briefs,
  remediation policy.
- docs/rules.md: each rule's evidence, the factual-vs-bug distinction,
  how precision and recall are measured, provider and other limits.
- docs/design.md: motivation, architecture, the state-model seam, the
  experimental C++ front end.
- docs/development.md: building and performance, tests, hooks.

Dropped as progress notes: the negative-result and recall-bug
narratives, before/after finding counts, review-round history, and
stale figures (a "5/5 recall" that depends on which branch a repo has
checked out, "84 tests", a duplicated recall.dart line, and the
adjudicator described as fully independent).
trvon 2026-09-23 20:40:40 -06:00
commit 1779da7e4d
Signed by: trevon
GPG key ID: 38686EA797D7A8A7

522
README.md
View file

@ -1,500 +1,72 @@
# pipe
Static state-flow analysis for Dart and Flutter apps. `pipe` resolves a project,
maps how state flows through its state management, and reports the breakages that
make an app behave worse than the code suggests — the ones that show up as a
spinner that never stops, an error the user never sees, or a tap that does
nothing.
Static state-flow analysis for Dart and Flutter apps. `pipe` maps how state
moves through an app's state management and reports the breakages that make it
behave worse than the code suggests: a spinner that never stops, an error the
user never sees, a tap that does nothing.
It is built to guide agents and reviewers on large codebases, not to score style.
It is built to guide agents and reviewers on large codebases, not to score
style. Every finding carries `file:line` evidence, a confidence, a severity and
a remediation.
## What it covers
## Coverage
| State management | Support |
| --- | --- |
| BLoC / Cubit (`on<Event>`, `emit`, sealed or unsealed state unions) | yes |
| State as a single class with a `bool loading`-style flag (`copyWith` style) | yes |
| Riverpod (`Provider`, `FutureProvider`, `StreamProvider`, `NotifierProvider`, `.family`) | yes |
| `provider` + `ChangeNotifier` | yes, with the limits below |
| GoRouter / navigation, widget layout, theming | no — out of scope |
| BLoC / Cubit (`on<Event>`, `emit`, sealed, abstract, enum or freezed unions) | yes |
| Single state class with a `bool loading`-style flag (`copyWith`) | yes |
| Riverpod (`FutureProvider`, `StreamProvider`, `NotifierProvider`, `.family`, ...) | yes |
| `provider` + `ChangeNotifier` | yes, with [limits](docs/rules.md#provider-and-changenotifier) |
| C++ `enum class` state (experimental, via libclang) | [see design](docs/design.md#c-front-end-experimental) |
| Navigation, layout, theming | no |
### `provider` / `ChangeNotifier` support and its limits
Modelled: every provider-package declaration type is collected and reported by
type in the coverage note (`Provider`, `ChangeNotifierProvider`,
`ListenableProvider`, `ValueListenableProvider`, `FutureProvider`,
`StreamProvider`, `StateProvider`, `ProxyProvider` and its variants, `.value`
constructors), along with `Consumer`, `Selector`, `ChangeNotifierProvider`,
`ListenableBuilder` and `AnimatedBuilder` as consumption sites. Declaration
detection uses the type name plus `create:`/`update:`/`value:` named arguments to
tell the provider package apart from Riverpod's positional `(ref)` callback — it
is not import-verified, so an unrelated class named `Provider` taking a `create:`
argument would be miscounted in that note.
Rules: `notifier-never-notifies`, `notifier-inert-consumer`,
`provider-read-in-build`, `notifier-field-unread`, `provider-created-in-build`.
Still not implemented, and not claimed: any analysis of `ProxyProvider`
`update` functions returning unchanged values, and rebuild-frequency reasoning
(which is why `provider-created-in-build` is info rather than a warning — the
pattern is harmless on a widget that never rebuilds and a real leak on one that
rebuilds often, and static analysis cannot tell them apart).
The `provider-read-in-build` rule does not descend into closures, because a read
inside a callback or a `Selector` builder is not made while build runs. That is a
deliberate precision choice: it costs recall on reads inside a
`Builder(builder: ...)`, which is noted rather than hidden.
Not modelled today: `bloc_concurrency` transformers, freezed unions whose
generated file is absent, and `ref.read` call sites (dispatch, not render).
## Usage
## Quick start
```bash
dart pub get
tool/build_release.sh # prints the binary's path under build/
# below, `pipe` is that binary on your PATH (or `dart run bin/pipe.dart`)
# Is the project even analyzable? (resolution health, timing, diagnostics)
pipe index <path>
# The resolved project model (declarations) as JSON
pipe graph <path>
# The state-flow graph: unions, emissions, render sites, coverage, Riverpod
pipe state <path>
# Ranked breakages with evidence and remediation
pipe check <path>
# A prompt-sized brief for an agent, optionally persisted into the yams corpus
pipe guide <path> --budget 6000 --yams
# How many labelled known bugs does pipe actually detect?
pipe recall --root <dir-with-repos> --strict
# After a fix: which accepted findings are gone?
pipe verify <path> --baseline .pipe-baseline.json --expect-resolved
# What to fix first, per file (ranked, with the scoring explained)
pipe suggest <path> --format agent --budget 3000
pipe index <path> # can the project be analysed?
pipe check <path> # ranked findings
pipe suggest <path> # what to fix first, per file
pipe guide <path> --format agent # a brief to paste into an agent prompt
```
Common flags: `--format text|json` (`agent|md|json` for `guide`), `-c N`
(concurrency, default 4), `--sdk <path>` (override SDK discovery), `--dir`
(repeatable), `--include-generated`, `--limit N`, `--rule <name>` (repeatable),
`--top N`.
Command-specific flags: `index --fail-on-error` exits 1 when the resolved project
has analyzer errors (the default reports the count and exits 0);
`suggest --budget N --max-actions N --top N` bound the agent plan and the text
report; `guide --budget N --max-findings N` bound the agent brief, and `--yams`
with `--yams-name <name>` persists that brief to the yams corpus;
`recall --manifest <path> --root <dir>` locate the labelled-bug manifest and the
directory holding the repos it names, and `recall --strict` exits non-zero when a
case it expects to still find is missed. Every numeric flag rejects a value that
is not a positive integer rather than falling back to its default.
`pipe` finds the right SDK itself: it reads `flutterRoot` from the target's
`.dart_tool/package_config.json`, then `FLUTTER_ROOT`, then `flutter` on `PATH`.
Flutter projects need Flutter's bundled Dart SDK because `dart:ui` is not part of
the standalone SDK. If the project has never been resolved, `pipe` says so and
tells you to run `pub get` rather than silently degrading every `package:` import
to `dynamic`.
A project that resolves only partly is reported, not silently analysed: an
unresolved `Foo()` is a method call rather than a construction, so evidence in
that file is invisible. `check` warns on stderr, notes how many files have
unresolved references, and lowers by one step the confidence of every finding
whose names such a file mentions (`data.confidenceLowered`). An SDK that exists
but cannot be read (for example a Flutter SDK under a privacy-protected macOS
folder, which makes `package:flutter` unresolvable) is refused up front with a
`--sdk` remedy.
## CI and ratcheting
An existing codebase will have findings on day one. Accept them once, then fail
only on what is new:
In CI, accept today's findings once and fail only on new ones:
```bash
pipe check . --write-baseline .pipe-baseline.json # accept current state
pipe check . --write-baseline .pipe-baseline.json
pipe check . --baseline .pipe-baseline.json --fail-on warning
pipe verify . --baseline .pipe-baseline.json --expect-resolved # after a fix
```
`--write-baseline` refuses a partial scan (`--rule`, `--limit`, or files that
could not be analysed): a baseline written from a subset omits every finding
outside it, and the next full run would report them all as new. `verify --rule`
likewise compares only that rule's accepted findings, instead of calling every
other rule's findings resolved.
`--fail-on none|info|warning|error` (default `none`) fails the build when a
**new** finding at or above that severity is present. Baselined findings never
trip the gate, so a ratcheted repo stays green while a regression fails.
Because today's signal is mostly warnings, `--fail-on warning` is the useful
setting; `--fail-on error` only catches the strongest rules (`--fail-on-finding`
is an alias for it).
`pipe verify` closes the other half of the loop — proving a fix rather than only
reporting a problem. It compares a fresh analysis against the baseline and prints
what is `resolved`, what `still` remains, and what is `new`, exiting non-zero with
`--expect-resolved` while anything accepted is still reported:
```text
$ pipe verify . --baseline .pipe-baseline.json
baseline 12 accepted findings
resolved 1 no longer reported — these fixes are confirmed
still 11 still reported
new 0 not in the baseline
RESOLVED
+ lib/flag_cubit.dart:25 loadUnguarded [loading-flag-not-reset]
```
Baselines are self-describing (each entry carries the rule plus a readable label),
so a resolved finding is named rather than shown as an opaque hash.
A baseline stores a **fingerprint** per finding (`rule + file + semantic
identity` such as union/variant/provider/owner/member), so moving code down a
file does not resurrect accepted findings. `--fail-on-finding` only counts new
error-severity findings, and severity counts in reports are computed over new
findings, so a ratcheted baseline does not keep a build red.
Baselines and reports are JSON, so they diff and review like any other file.
## Rules
| Rule | What it means | Verified precision (with n) |
| --- | --- | --- |
| `emitted-not-rendered` | A state the bloc emits that no UI code matches, so the user never sees it. | **79/79 (100%)** factual on a CLEAN nuwayze main via a separate regex adjudicator with passing controls (`tool/adjudicate.py`; see its independence limit below). Reported once per **root cause**, not once per variant: one finding per unobserved union (dormant feature) and one per marker class when the union is observed. When the UI reads the union's payload through **base-class fields** the finding drops to info, because the data does reach the user. On clean nuwayze main: 30 findings covering 79 variant-level claims. |
| `rendered-not-emitted` | A UI branch no state can reach: dead UI. | n=0 on nuwayze (unions whose bloc emits through helper methods are suppressed with a reason). Fires on a fixture with a passing negative control. |
| `loading-not-reset` | An in-flight state whose failure path never resolves, so the spinner sticks. | n=0 on both dogfood repos, both verified to guard correctly; proven able to fire by fixture. |
| `loading-flag-not-reset` | The same consequence for `copyWith` + `bool loading` state, which has no variant to match on. | **7/7 (100%)** on dex after hand adjudication; 4 characterised false positives were eliminated by the current gate. n=0 on nuwayze, verified correct. |
| `event-without-handler` | A dispatched event with no `on<Event>`, so the interaction does nothing. | n=0 across the corpus; verified non-vacuous (168 dispatch sites on nuwayze all map to a registered handler). |
| `dead-handler` | An `on<Event>` nothing ever dispatches or constructs. | **77/77 (100%)** on nuwayze, same adjudicator with passing controls. |
| `riverpod-async-gap` | An async provider read in the UI whose error and/or loading state is never rendered. | n=8 on word_alive, n=2 on tick, hand-adjudicated: **warning/high** only for data-carrying providers; `bool` providers read as optimistic guards are downgraded to info/medium after a confirmed false positive. |
| `notifier-never-notifies` | A `ChangeNotifier` method that changes observable state without ever notifying listeners. | Fixture contract only: **zero true positives** on the corpus (nuwayze 30 observable mutators, word_alive 36 — all of which notify). Gated on observable fields with lifecycle hooks excluded, after the first run was 9/9 false positives. |
| `notifier-inert-consumer` | A notifier consumed by a `Consumer`/`Selector` with no notifying path at all, so those consumers can never rebuild. | Fixture contract only; no real-code true positives yet. |
| `provider-read-in-build` | A `Listenable` read with `context.read` / `Provider.of(listen: false)` **directly in a build body**, so the widget never subscribes and can render stale state. | 1 finding on nuwayze from **447** raw `context.read` calls; 0 on word_alive/dex. Hand-verified: the first 4 findings were all reads inside **closures** (a `BlocListener` callback, a `Selector` builder), so a closure guard was added and only the direct-in-build case survives. |
| `notifier-field-unread` | A public getter on a consumed `ChangeNotifier` that nothing reads, so the state it exposes cannot affect what the user sees. | 3 on nuwayze (e.g. `UserProvider.isInitializing`, `AppLocaleProvider.preferredLocale`), 0 on word_alive/dex. Reported once per notifier, and only for notifiers the UI actually consumes. **medium** confidence: a name match elsewhere can mask a genuinely unread member. |
| `provider-created-in-build` | A provider constructed inside `build` without `.value`, so `create` reruns on every rebuild and state is lost. | 1 finding on nuwayze — `AppProviders.build` constructs **45** providers — from 45 raw sites, because it is reported **once per build method**. Deliberately **info**, not warning: the claim is factual, but a root `StatelessWidget` rarely rebuilds, so impact depends on rebuild frequency. |
### How precision was measured
`tool/adjudicate.py <repo>` re-derives the two highest-volume claims with its
own regex evidence instead of pipe's, and **prints controls first**. It is only
partly independent: it takes the list of unions, variants, events and owners to
check from pipe's `state` output, so a variant pipe fails to enumerate is never
checked. It exits non-zero when its controls fail or cannot discriminate, so a
CI step cannot mistake an uncertified run for a pass. It must find evidence
for every variant pipe calls *rendered* and every event pipe calls *dispatched*;
on clean nuwayze main that is 131/131 rendered (type tests, `case`/switch-
expression patterns, or base-class payload reads) and 143/162 dispatched, the
other 19 being shared event names whose construction sites a text-level checker
cannot honestly attribute — reported as *unattributable*, not as a pass.
The control is deliberately three-way — **found / unattributable / missing** — and
only *missing* fails it, because an oracle that cannot discriminate must say so
rather than report a clean bill of health. A corpus with no control material at
all exits with **NO SIGNAL** and calls precision **undefined, not perfect**. If a control is
below 100% the numbers after it are void — which is exactly what happened on the
first attempt: an earlier version of the checker matched the `on<Event>`
registration as if it were a dispatch and reported 9% precision. The tool also
classifies comment-stripped lines, because a commented-out dispatch
(`cartview.dart:512`) is not a dispatch.
Re-run the adjudicator on a **clean tree**: on a repo with uncommitted edits its
controls correctly fail and it refuses to certify, which is the behaviour you
want from an oracle.
The adjudicator deliberately judges the **factual claim only** ("never
type-tested in UI" / "never constructed elsewhere"). It does not judge UX
severity, and it runs on one repo, so it bounds false-positive risk rather than
eliminating it.
### A factually correct finding is not always a bug
`emitted-not-rendered` at high confidence means "no UI code observes this state
type". That is verifiable and true — but it can be *deliberate*. On nuwayze the
two such blocs turned out to be dormant scaffolding the repo had already
documented as unreachable:
- `AdminSettingsBloc` never had a UI consumer (the shipped admin Settings tab
uses `ArtistSettingsBloc` and reads the repository directly), and the project's
own action contract classifies `UpdateFeatureFlag` / `SaveAdminSettings` as
`"not-reachable"`.
- `CommentsBloc` was deliberately retired when a `StreamBuilder`-driven comments
UI replaced it; the last consumer was deleted in a cleanup commit.
So pipe now reports **one finding per unobserved union** instead of one per
variant (that change took nuwayze from 159 findings to 146, collapsing 15 variant
findings into 2 decisions), and the remediation says to check the repository's
docs and action contracts before deleting anything. Treat those as decisions to
confirm, not regressions to fix.
**A negative result worth recording.** The obvious next tier looked promising: a
never-matched variant that declares *no* fields beyond its base cannot hide any
data of its own, so it should be harmless. Measured on nuwayze, **all 67** of the
medium-tier findings qualified — every one was a marker state carrying no payload
of its own. A signal that is true of 100% of the population cannot separate
signal from noise, as a *discriminator*. It was still the right question though:
an audit of the same repo found the real cause, which is that the UI reads the
union's payload off the **base class** (`state.currentBooking`,
`state.isRefreshing`), so the data reaches the user without the variant ever being
matched. `uiReadsBaseFields` now records that per union and drops those findings
to info with an explanation, which removed 5 false-positive warnings on nuwayze.
**A recall bug the same audit surfaced.** The switch
`case ChatListLoaded value => ...` binds a type on a *variable* pattern, not the
object pattern `ChatListLoaded()` this matcher looked for, so an exhaustive switch
was invisible and two variants were wrongly reported as unrendered. Both the
mention collector and the branch collector now handle `DeclaredVariablePattern`.
The fixture has a regression guard (`patternView`) so this cannot come back.
This is also why the precision numbers above are stated as *factual* precision:
they say how often the claim holds, not how often the issue is worth acting on.
Every finding carries `file:line`, a `path:line: snippet` evidence string, a
confidence, a UX severity, and a remediation. If a rule cannot be validated it is
demoted or deleted, not defended.
### Recall, measured not assumed
Precision alone is not evidence. Two mechanisms pin recall:
1. **Rule contracts in CI.** `test/rule_contract_test.dart` requires every rule to
have a positive control (it must fire) *and* a negative control (an adjacent
case it must stay silent on), from fixture code written for exactly that pair —
for example `loading-flag-not-reset` must fire on `loadUnguarded` and must not
fire on `loadGuarded` or `refreshQuietly`; `event-without-handler` must fire on
`UnhandledTap` and must not fire on `NeverDispatched`, which is handled. A rule
added without both controls fails the build.
2. **Labelled real bugs.** `pipe recall` evaluates `recall/manifest.json` — real
bugs previously found in real codebases — and reports `ok/total` per rule,
failing (`--strict`) on any miss. Repos absent from the machine are reported as
**skipped**, never as passes.
Each case carries a `status`:
- `open` — the bug still exists, so pipe must **find** it.
- `fixed` — the bug was repaired, so pipe must **not** find it again. A `fixed`
case that reappears is reported as a **REGRESSION**, which turns every repair
into a permanent guard against the real code rather than a deleted
expectation.
Current result: **5/5 accounted for** (2 open bugs found, 3 fixed bugs still
quiet), 0 regressions — measured against the **working tree**, so a repo checked
out on an unmerged fix branch reports its cases as missed. The two nuwayze cases
are wired up on branches (`wire/nuwayze-*`) that are not merged yet, so they still
read as `open` here; they flip to `fixed` when those land.
Recall is still measured against a small labelled set. It bounds the risk of
false negatives; it does not prove they are absent.
### What pipe deliberately does not reimplement
`unawaited_futures`, `use_build_context_synchronously`, `cancel_subscriptions`,
`close_sinks`, BLoC naming/API hygiene (`bloc_lint`), declaration-level dead code
(`undead`, `dallow`). Those are per-file checks the ecosystem already does well.
pipe targets whole-program state reachability that per-file linters structurally
cannot express.
## Default remediation policy
**Wire it up.** An unreachable state, handler or control is usually an *unfinished
feature* rather than dead weight, so pipe's default guidance is to make it
reachable: render the missing states, add the missing branch, dispatch the event
from a control. Removal is the exception and is worded as needing explicit
approval rather than offered as an equal choice.
`--policy wire-up` (default) / `--policy remove` flips the emphasis for teams that
would rather delete. It changes the wording of every remediation that presents a
choice, the `decision:` line in the agent brief, and the heading of the
dead-handler section (`Unreachable handlers - wire up the control` versus
`Cleanly deletable`).
This is a policy, not a finding: pipe still reports the same facts either way.
## Fix order
`pipe suggest` turns a findings list into a work plan: findings are collapsed to
root causes (one action per union / provider / handler), each cause is scored, and
the plan is presented both globally and **per file**. Only new findings are
planned, so a ratcheted baseline yields an empty plan.
```text
1. [warning/high] emitted-not-rendered — CommentsState (CommentsBloc)
why: 125 = warning 50 x high (x1.0), one fix clears 6 findings (+50), user-visible failure (+25)
fix: Either render CommentsLoading in the consuming widget (or delete it and its emits) ...
where: lib/blocs/comment/comment_bloc.dart:143, ... +2 more
clears 6 finding(s) (CommentSubmitError, CommentSubmitSuccess, CommentSubmitting,
CommentsError, CommentsLoaded, CommentsLoading); verify with `pipe check --rule emitted-not-rendered`
```
The score is deliberately simple and always printed, so the order can be argued
with rather than trusted blindly:
| term | effect |
| Rule | Meaning |
| --- | --- |
| severity | `error` 100, `warning` 50, `info` 10 — dominates the **base** score |
| confidence | high ×1.0, medium ×0.7, low ×0.4 — a low-confidence finding may not be real |
| breadth | +10 per extra finding the one fix clears (capped at +50) |
| user-visible failure | +25 when the rule flagged the failure as UX-critical |
| certainty | +15 for rules whose fix needs no product judgement (dead handler, event without handler, loading not reset) |
| `emitted-not-rendered` | A state is emitted but no UI code matches it, so the user never sees it. |
| `rendered-not-emitted` | A UI branch no emitted state can reach: dead UI. |
| `loading-not-reset` | An in-progress state whose failure path never resolves, so the spinner sticks. |
| `loading-flag-not-reset` | The same, for a `bool loading` flag left set after an error. |
| `event-without-handler` | A dispatched event with no `on<Event>`, so the interaction does nothing. |
| `dead-handler` | An `on<Event>` that nothing dispatches or constructs. |
| `riverpod-async-gap` | An async provider read in the UI whose error or loading state is never handled. |
| `notifier-never-notifies` | A `ChangeNotifier` method changes observable state without notifying. |
| `notifier-inert-consumer` | A consumed notifier that never notifies, so its consumers never rebuild. |
| `provider-read-in-build` | A `Listenable` read without subscribing inside `build`, so it can render stale. |
| `notifier-field-unread` | A public member of a consumed notifier that nothing reads. |
| `provider-created-in-build` | A provider constructed in `build`, so its state is recreated on rebuild. |
Because the bonuses are additive, a wide and certain `info` group (10 base + 15
certainty + 50 breadth = 75) can outrank a narrow `warning` (35): severity
dominates the base, not the final total. That is deliberate — one fix that clears
nine findings is usually the better first move — but it is stated rather than
implied, and the `why:` line prints the arithmetic for every action.
Measured precision, recall and the limits of each rule are in
[docs/rules.md](docs/rules.md).
## Agent guidance
## Documentation
`pipe guide` emits a brief designed to be pasted into an agent prompt: findings
grouped by root cause, dense `file:line` anchors (`file:18,29,32`), a rule legend
carrying the standard fix, a `decision:` line that resolves "fix vs delete" where
the project makes that decidable, and a `verify:` command. Truncation is always
explicit — never silently dropped.
It was verified by giving independent agents *only* the brief text and no tool
access. Five rounds of that found real defects (a legend that did not match the
entries shown, hidden edit targets, a missing consumer file), all fixed. Two
limits are inherent and stated in the brief rather than hidden:
1. Deciding whether to wire up or delete an unwired bloc needs **product intent**.
pipe reports the evidence a human needs (registered in DI/providers, 0 render
sites) but cannot supply the intent.
2. Adding a missing UI branch requires reading the consumer widget, so pipe names
the file and line and does not inline widget source.
Deletion-class findings need neither, which is why they are the verified
actionable class.
## Architecture
```text
bin/pipe.dart entrypoint
lib/src/project.dart package graph validation, source discovery
lib/src/sdk.dart SDK discovery (Flutter-aware)
lib/src/analysis.dart AnalysisContextCollection lifecycle, per-file resolution
lib/src/index.dart resolution health, timing, diagnostics
lib/src/model.dart declaration model
lib/src/state_flow.dart the state-flow IR (unions, emissions, render sites,
events, flags, Riverpod)
lib/src/facts.dart language-neutral facts the rules judge (JSON round-trip)
lib/src/dart_facts.dart Dart front end: loading/construction recognisers
lib/src/dart_provider_facts.dart Dart front end: ChangeNotifier/provider recognisers
lib/src/rules.dart breakage detectors, findings, baselines (model only)
lib/src/recall.dart labelled-bug recall measurement
lib/src/verify.dart baseline verification (resolved / still / new)
lib/src/suggest.dart ranked fix order per file
lib/src/recall.dart labelled-bug recall measurement
lib/src/pipeline.dart shared analyze -> graph -> rules pipeline
lib/src/guide.dart agent brief / markdown report / yams emission
tool/adjudicate.py independent precision adjudicator
tool/build_release.sh native AOT build
```
### The state model is the seam
Rules never read a syntax tree. The Dart front end (`state_flow.dart`,
`dart_facts.dart`, `dart_provider_facts.dart`) turns resolved code into a state
model: unions and their variants, where each is emitted and rendered, event
dispatch and handling, busy paths whose failure is swallowed, observable objects
and whether their mutations are announced. `runChecks` takes only that model.
The model is plain JSON, and a model that went through JSON reports exactly
what the in-process one does - checked byte for byte on the fixture
(`test/state_map_roundtrip_test.dart`) and on four real apps:
```bash
pipe state . --format json > model.json # any front end can write this
pipe check --model model.json # every rule, baseline and gate
```
So a front end for another language only has to write that JSON; it gets every
rule, `--baseline` ratcheting and `--fail-on` gating without touching Dart.
`check --model` refuses `--write-baseline`, because a model does not record
whether its front end read every file.
### Experimental: C++ state enums
`tool/cpp_front.py` is a second front end, written against the model rather than
the rules. It reads a compile database with libclang (Homebrew LLVM) and maps
each project `enum class` to a union: a reference that produces a value
(assignment, atomic `store`, argument, return, initialiser) is an emission; a
`case`, `==` or `!=` is an observation; a busy enumerator set on the method's
own state followed by a top-level `catch` that neither writes state nor
rethrows is a swallowed failure. Identity comes from clang's resolved AST, so
two enums both named `QueryIntent` are never confused.
```bash
tool/cpp_front.py ~/work/tools/yams --build-dir ~/work/tools/yams/build/debug -o yams.json
pipe check --model yams.json
```
Read the rules as: `rendered-not-emitted` - an enumerator something branches on
that nothing ever produces (a dead branch, or a state an FSM can never enter);
`emitted-not-rendered` - a value produced that nothing ever branches on
(computed and ignored, or reaching output only through a name table). Sources
the build does not compile are recorded as unresolved, so findings naming them
drop a confidence step. `test/cpp_front_test.dart` holds the positive and
negative controls (`test/fixtures/cpp_state`); it is skipped without LLVM.
On yams (375 translation units, 0 clang errors, about two minutes) the first
run produced 82 findings. A grep-based hand check of 13 found 12 factually
correct and 1 false positive (a nested JSON-parse fallback taken for a stuck
busy state), which the front end now excludes. Factual is not the same as a
bug: of the 12, `TransportState::Error` is covered by `!= Connected` checks and
`InboundSessionStage` is fuzzing instrumentation. The substantive ones include
the IPC connection FSM never entering `ImplState::ReadingHeader`, the downloader
never reporting `ProgressStage::Verifying` although three CLI commands render
it, and `ExtractScopeType` query qualifiers being parsed but never acted on.
n=13 from one repo bounds the false-positive rate; it does not measure it.
Built on `package:analyzer` with whole-project resolution, not a
`custom_lint`/analyzer plugin: plugins have no project-wide index, `flutter
analyze` does not report new-system plugin diagnostics, and a CLI can pin its own
analyzer instead of chasing the SDK's pinned range.
Resolution mirrors Flutter's own tooling (dispose and rebuild the context
collection when `package_config.json` changes, `changeFile` +
`applyPendingFileChanges`, a fresh session after
`InconsistentAnalysisException`).
## Performance
Run pipe as a native binary, not through `dart run`. Dart JIT startup is paid on
every invocation and dominates short runs:
```bash
tool/build_release.sh # dart build cli, falling back to dart compile exe
```
Measured on the same machine (macOS arm64):
| scenario | `dart run` (JIT) | AOT binary |
| --- | --- | --- |
| 2-file project | 6.4s | **0.2s** |
| 814-file Flutter app, end to end | ~23-29s | **10.4s** |
| per-file p50 on that app | ~50-75ms | **16ms** |
The binary is ~18MB and self-contained.
### Where the remaining time goes
The first file of a large project costs ~3.1s (analyzer warmup: SDK and package
summaries), and the rest is aggregate per-file resolution. SDK *discovery* is not
the floor - it is a couple of file checks against `package_config.json` - so the
obvious "cache the SDK path" optimisation buys nothing. Cutting the warmup would
need a persistent analysis cache, which is a much larger and riskier change.
Resolution health on the 814-file app: 812/814 resolved, 0 errors, 0 unresolved
URIs. Two runs are byte-identical after dropping timing, and `pipe check` output
is byte-identical across runs.
## Tests
```bash
dart test
```
84 tests across resolution/IR extraction, rule contracts (positive *and*
negative controls per rule), Riverpod and provider modelling, agent-brief
rendering, fix-order scoring, baseline round-tripping (including legacy
fingerprints-only baselines), verification, and recall evaluation, against a
dependency-free fixture project.
- [Usage](docs/usage.md): commands, flags, SDK discovery, CI ratcheting,
verification, fix order, agent briefs.
- [Rules](docs/rules.md): what each rule claims, the evidence behind it, and how
precision and recall are measured.
- [Design](docs/design.md): why pipe exists, how it is built, the state-model
seam, and the experimental C++ front end.
- [Development](docs/development.md): building, performance, tests and hooks.

92
docs/design.md Normal file
View file

@ -0,0 +1,92 @@
# Design
## Motivation
The state bugs users feel (a spinner that never stops, an error never shown, a
tap that does nothing) are whole-program facts: a state emitted in one file and
never matched in any other, an event dispatched with no handler anywhere. A
per-file linter cannot express them, so pipe resolves the whole project and
reasons over all of it at once.
pipe is a CLI on `package:analyzer`, not a `custom_lint` or analyzer plugin:
plugins have no project-wide index, `flutter analyze` does not report
new-system plugin diagnostics, and a CLI can pin its own analyzer version.
## Architecture
```text
bin/pipe.dart entrypoint
lib/src/cli.dart commands and flags
lib/src/project.dart package validation, source discovery
lib/src/sdk.dart SDK discovery (Flutter-aware)
lib/src/analysis.dart analysis contexts, per-file resolution
lib/src/index.dart resolution health, timing, diagnostics
lib/src/model.dart declaration model (`graph`)
lib/src/state_flow.dart Dart front end: the state model and its JSON
lib/src/dart_facts.dart Dart front end: loading and construction facts
lib/src/dart_provider_facts.dart Dart front end: ChangeNotifier / provider facts
lib/src/facts.dart language-neutral facts (JSON round-trip)
lib/src/rules.dart rules, findings, baselines (model only)
lib/src/provider_rules.dart ChangeNotifier / provider rules (model only)
lib/src/pipeline.dart analyse -> model -> rules
lib/src/verify.dart baseline verification
lib/src/suggest.dart fix order
lib/src/guide.dart agent brief, markdown report, yams emission
lib/src/recall.dart labelled-bug recall
tool/adjudicate.py precision adjudicator
tool/cpp_front.py experimental C++ front end
tool/build_release.sh native AOT build
```
Resolution mirrors Flutter's own tooling: the context collection is rebuilt
when `package_config.json` changes, file changes go through `changeFile` and
`applyPendingFileChanges`, and an `InconsistentAnalysisException` gets a fresh
session.
## The state model
Rules never read a syntax tree. A front end turns code into a state model:
unions and variants, where each is emitted and matched, events dispatched and
handled, busy paths whose failure is swallowed, and observable objects with
whether their mutations are announced. `runChecks` takes only that model.
The model is plain JSON, and a model read back from JSON reports exactly what
the in-process one does (`test/state_map_roundtrip_test.dart`):
```bash
pipe state . --format json > model.json # or any front end that writes it
pipe check --model model.json # every rule, baseline and gate
```
A front end for another language only has to write that JSON.
## C++ front end (experimental)
`tool/cpp_front.py` reads a compile database with libclang (Homebrew LLVM) and
maps each project `enum class` to a union:
- **emission**: a reference that produces the value (assignment, atomic
`store`, argument, return, initialiser); a cast from an integer is an
indirect emission, which makes "never emitted" unprovable for that enum;
- **observation**: a `case`, `==` or `!=`;
- **swallowed failure**: a busy enumerator written to the method's own state,
then a top-level `catch` that neither writes state nor rethrows.
Identity comes from clang's resolved AST, so two enums with the same name are
never confused. Sources the build does not compile are recorded as unresolved.
```bash
tool/cpp_front.py <repo> --build-dir <repo>/build/debug -o model.json
pipe check --model model.json
```
For C++, read `rendered-not-emitted` as "branched on but never produced" (a
dead branch, or a state a state machine never enters) and `emitted-not-rendered`
as "produced but never branched on" (computed and ignored, or reaching output
only through a name table or reflection). Rule messages still say "UI".
On yams (375 translation units, about two minutes) a hand check of 13 findings
found 12 factually correct and 1 false positive, which the front end now
excludes. That sample bounds the false-positive rate; it does not measure it.
Controls live in `test/fixtures/cpp_state` and `test/cpp_front_test.dart`,
skipped when LLVM is absent.

54
docs/development.md Normal file
View file

@ -0,0 +1,54 @@
# Development
## Building
```bash
dart pub get
tool/build_release.sh # `dart build cli`, falling back to `dart compile exe`
```
Use the native binary rather than `dart run`: JIT startup is paid on every
invocation. On macOS arm64:
| Scenario | `dart run` | AOT binary |
| --- | --- | --- |
| 2-file project | 6.4s | 0.2s |
| 814-file Flutter app, end to end | ~23-29s | 10.4s |
| per-file p50 on that app | ~50-75ms | 16ms |
The binary is about 18MB and self-contained. On a large project the first file
costs about 3s of analyzer warm-up (SDK and package summaries). Cutting that
would need a persistent analysis cache.
## Tests
```bash
dart test
```
The suite runs against `test/fixtures/stuck_loading`, a dependency-free project
with hand-written stand-ins for BLoC, Riverpod and provider, so it resolves with
no `pub get` and no network. Do not add a `package:` import to it.
What the suite holds pipe to:
- every rule has a positive and a negative control (`rule_contract_test.dart`);
- no fingerprint depends on position, and none collide (`fingerprint_test.dart`);
- the state model survives JSON unchanged, with identical findings
(`state_map_roundtrip_test.dart`);
- partial, unreadable and unresolved scans are reported, never passed silently.
`test/cpp_front_test.dart` runs the C++ front end on `test/fixtures/cpp_state`
and is skipped without Homebrew LLVM.
## Hooks and CI
`git config core.hooksPath .githooks` enables the hooks. `pre-commit` runs
`scripts/lint.sh --hook` (format and analyze); `pre-push` runs
`scripts/pre-push-ci-gate.sh` (format, analyze and the full test suite).
Commit messages follow Conventional Commits (`commitlint.config.js`).
## Measuring precision and recall
See [rules.md](rules.md#how-precision-is-measured) for `tool/adjudicate.py` and
`pipe recall`.

82
docs/rules.md Normal file
View file

@ -0,0 +1,82 @@
# Rules
Every rule states a factual, whole-program claim ("no UI code matches this
state"). Precision below is **factual** precision: how often the claim holds,
not how often it is worth acting on. A rule that cannot be validated is demoted
or removed.
## Evidence per rule
Measured on the dogfood apps (nuwayze, dex, word_alive, tick).
| Rule | Evidence |
| --- | --- |
| `emitted-not-rendered` | 79/79 claims factual on nuwayze main (adjudicator with passing controls). One finding per unobserved union and per unobserved marker class. Drops to info when the UI reads the union's payload through base-class fields. |
| `rendered-not-emitted` | Fixture contract; suppressed with a reason when the owner emits through helper methods. |
| `loading-not-reset` | Fixture contract; the dogfood apps guard correctly. |
| `loading-flag-not-reset` | 7/7 on dex, hand-adjudicated. |
| `event-without-handler` | Fixture contract; 168 dispatch sites on nuwayze all map to a handler. Follows the event hierarchy (`on<Base>` covers subtypes) and blocs sharing an event type. |
| `dead-handler` | 77/77 on nuwayze (adjudicator). The message names the directories scanned; events built only in tests are not features. |
| `riverpod-async-gap` | 10 hand-adjudicated findings (word_alive, tick). Recognises `.when`, `AsyncError`/`AsyncLoading` patterns and values stored in locals. `bool` providers are downgraded to info. |
| `notifier-never-notifies` | Fixture contract only; no real-code true positives yet. Limited to observable fields; lifecycle hooks excluded. |
| `notifier-inert-consumer` | Fixture contract only. |
| `provider-read-in-build` | 1 finding from 447 `context.read` calls on nuwayze. Reads inside closures are excluded by design. |
| `notifier-field-unread` | 3 on nuwayze; medium confidence, since a name match elsewhere can mask an unread member. |
| `provider-created-in-build` | Info, once per build method: the claim is factual, but impact depends on how often the widget rebuilds. |
## A factual finding is not always a bug
`emitted-not-rendered` at high confidence can be deliberate: dormant scaffolding
or a retired feature. Those findings are one decision per union, and the
remediation asks to check the repo's docs and action contracts before deleting
anything.
## How precision is measured
`tool/adjudicate.py <repo>` re-derives the two highest-volume claims
(`emitted-not-rendered`, `dead-handler`) with its own regex evidence, and prints
controls first: it must find evidence for every variant pipe calls rendered and
every event pipe calls dispatched. Controls are three-way (found /
unattributable / missing), and only *missing* fails. A corpus with no control
material reports **NO SIGNAL** and undefined precision. The script exits
non-zero whenever it cannot certify.
Limits: it is only partly independent (the list of variants, events and owners
to check comes from pipe's `state` output), it judges the factual claim only,
not severity, and it needs a clean working tree.
## Recall
1. **Rule contracts.** `test/rule_contract_test.dart` requires each rule to fire
on a positive control and stay silent on an adjacent negative control. A rule
without both fails the build.
2. **Labelled real bugs.** `pipe recall` evaluates `recall/manifest.json`. An
`open` case must be found; a `fixed` case must stay quiet, and reappearing is
a **REGRESSION**. Repos missing from the machine are *skipped*, never passed.
Results reflect each repo's working tree.
The labelled set is small: it bounds the risk of false negatives, it does not
prove their absence.
## provider and ChangeNotifier
Modelled: every provider-package declaration type (`Provider`,
`ChangeNotifierProvider`, `ListenableProvider`, `ValueListenableProvider`,
`FutureProvider`, `StreamProvider`, `StateProvider`, `ProxyProvider` and
variants, `.value`), and `Consumer`, `Selector`, `ListenableBuilder` and
`AnimatedBuilder` as consumers. Declarations are told apart from Riverpod by
`create:`/`update:`/`value:` arguments, not by import, so an unrelated class
named `Provider` taking `create:` would be miscounted.
Not modelled: `ProxyProvider` `update` functions returning unchanged values, and
rebuild frequency.
## Other limits
Not modelled: `bloc_concurrency` transformers, freezed unions whose generated
file is absent, and `ref.read` call sites.
pipe deliberately does not reimplement per-file checks the ecosystem already
does well: `unawaited_futures`, `use_build_context_synchronously`,
`cancel_subscriptions`, `close_sinks`, `bloc_lint`, or declaration-level dead
code (`undead`, `dallow`).

119
docs/usage.md Normal file
View file

@ -0,0 +1,119 @@
# Usage
## Commands
| Command | What it does |
| --- | --- |
| `pipe index <path>` | Resolve the project and report analysis health, timing and diagnostics. `--fail-on-error` exits 1 on analyzer errors. |
| `pipe graph <path>` | The resolved declaration model, as JSON. |
| `pipe state <path>` | The state model: unions, emissions, render sites, events, Riverpod, notifier facts. `--format json` writes the model other commands can read back. |
| `pipe check <path>` | Ranked findings with evidence and remediation. |
| `pipe suggest <path>` | Findings collapsed to root causes and ranked, globally and per file. |
| `pipe guide <path>` | A prompt-sized brief for an agent; `--yams` persists it to the yams corpus. |
| `pipe verify <path> --baseline F` | Which accepted findings were resolved, remain, or are new. |
| `pipe recall --root <dir>` | How many labelled known bugs pipe detects (see [rules](rules.md#recall)). |
Common flags: `--format text|json` (`agent|md|json` for `guide`), `-c/--concurrency N`
(default 4), `--sdk <path>`, `--dir <dir>` (repeatable, default `lib`),
`--include-generated`, `--rule <name>` (repeatable), `--top N`, and `--limit N`
(debugging only; results are marked partial).
Command-specific: `check --fail-on`, `--baseline`, `--write-baseline`,
`--model`; `guide --budget --max-findings --yams --yams-name`;
`suggest --budget --max-actions`; `verify --expect-resolved`;
`recall --manifest --root --strict`; `--policy wire-up|remove` on `check`,
`guide` and `suggest`.
Every numeric flag rejects a value that is not a positive integer instead of
falling back to its default.
## SDK and resolution
pipe finds the SDK itself: `flutterRoot` from the target's
`.dart_tool/package_config.json`, then `FLUTTER_ROOT`, then `flutter` on `PATH`.
Flutter projects need Flutter's bundled Dart SDK, because `dart:ui` is not part
of the standalone one. Override with `--sdk`.
pipe refuses to report a clean result it did not earn:
- A project never resolved (`pub get` not run) is an error, not a scan in which
every `package:` import is `dynamic`.
- An SDK that exists but cannot be read (for example under a privacy-protected
macOS folder) is refused with a `--sdk` remedy.
- Files that fail to resolve make the results **PARTIAL**, with a warning.
- Files with unresolved references (`uri_does_not_exist`, `undefined_*`) produce
a warning and a report note, and every finding naming something such a file
mentions drops one confidence step (`data.confidenceLowered`). An unresolved
`Foo()` is a method call, not a construction, so evidence there is invisible.
## CI and baselines
```bash
pipe check . --write-baseline .pipe-baseline.json
pipe check . --baseline .pipe-baseline.json --fail-on warning
```
`--fail-on none|info|warning|error` (default `none`) fails when a **new** finding
at or above that severity is present. Baselined findings never trip it.
`--fail-on-finding` is an alias for `--fail-on error`.
A baseline stores one fingerprint per finding: rule, file and semantic identity
(union, variant, event, provider, owner, member). Fingerprints never include a
line number, so moving code does not resurrect accepted findings. Entries also
carry a readable label, so `verify` names what was resolved.
`--write-baseline` refuses a partial scan (`--rule`, `--limit`, or unreadable
files), because the baseline would omit every finding outside it. `check --model`
refuses it too, since a model does not record whether every file was read.
## Verifying a fix
```text
$ pipe verify . --baseline .pipe-baseline.json
baseline 12 accepted findings
resolved 1 no longer reported — these fixes are confirmed
still 11 still reported
new 0 not in the baseline
RESOLVED
+ lib/flag_cubit.dart:25 loadUnguarded [loading-flag-not-reset]
```
`--expect-resolved` exits non-zero while anything accepted is still reported.
`verify` refuses `--limit` and partially readable projects, and `--rule`
compares only that rule's accepted findings.
## Fix order
`pipe suggest` collapses findings to root causes (one action per union,
provider or handler), scores each, and prints the arithmetic so the order can be
argued with:
| Term | Effect |
| --- | --- |
| severity | `error` 100, `warning` 50, `info` 10 (the base score) |
| confidence | high ×1.0, medium ×0.7, low ×0.4 |
| breadth | +10 per extra finding the fix clears, capped at +50 |
| user-visible failure | +25 when the rule marks the failure as user-facing |
| certainty | +15 when the fix needs no product judgement |
The bonuses are additive, so a wide, certain `info` group can outrank a narrow
`warning`. Only new findings are planned, so a ratcheted repo yields an empty
plan.
## Agent briefs
`pipe guide` groups findings by root cause with dense anchors
(`file:18,29,32`), a rule legend carrying the standard fix, a `decision:` line
where the project makes "fix vs delete" decidable, and a `verify:` command.
Truncation is always stated.
Two limits are stated in the brief: whether to wire up or delete an unwired
bloc needs product intent, and adding a missing UI branch means reading the
consumer widget, which pipe names but does not inline.
## Remediation policy
The default is `--policy wire-up`: an unreachable state, handler or control is
usually an unfinished feature, so the remediation is to make it reachable, and
removal is worded as needing approval. `--policy remove` flips the emphasis. The
facts reported are the same either way.