main gained six PRs (#1-#6) while the audit fixes and the state-map seam
were in progress. Four of main's commits fixed the same bugs this branch
did, independently; where they overlap, the more complete version wins:
- Fingerprints: main's (a0844bb) - catchSite (`member#index` among all
catches), union/owner as identity, a legacy fingerprint so version-1
baselines still match, baseline "version": 2. This branch's `catch`
ordinal is dropped; its fingerprint_test still guards the result.
- verify --rule: main's verifyAnalysis (cf9e0b7), which also reports
findings in skipped files as NOT SCANNED.
- Unreadable SDK: both kept - this branch's up-front probe gives the
early, specific error; main's catch around the analyzer (d6b6bcb) is
the backstop.
- Riverpod: main's "a ref.listen is not a render" (f6411dc) and this
branch's AsyncError patterns / stored-value tracking compose; the
fixture keeps both sets of cases.
Ported onto the model-only rules (the seam), since main edited code this
branch had moved into front-end facts:
- excerpt.dart's literal-free, bounded quoting (a prompt-injection
guard) now also covers front-end facts: SwallowedFailure.handler and
NonSubscribingRead.excerpt, and the C++ front end elides string
literals in handler evidence the same way.
- ScanCoverage: runChecks stays source-free; the model carries coverage
(`scanCoverage` in the state JSON, written by both front ends), so
`check --model` states coverage like a source check.
- main's softened wording (rendered-not-emitted "probably unreachable",
the remove-policy dead-handler text, notifier-field-unread) and the
backticked flag message.
Docs: main's README grew the Firestore query inventory and the output
contract; the README stays short, the inventory moves to
docs/queries.md, and the contract, exit codes, verify and baseline
details join docs/usage.md.
Checked against origin/main on resolved mirrors of the dogfood apps:
nuwayze (102), dex (0) and word_alive (0) identical; tick differs only
by the intended riverpod stored-value fix. On the fixture every
difference is one of this branch's fixes. 214 tests pass.
The README had grown to 500 lines mixing an overview, a CLI reference,
rule evidence, measurement method, architecture, build notes and
progress notes. It is now a 72-line overview (what pipe is, coverage,
quick start, the rule list, links), and the rest lives in:
- docs/usage.md: commands, flags (checked against `pipe help`), SDK and
resolution behaviour, CI baselines, verify, fix order, agent briefs,
remediation policy.
- docs/rules.md: each rule's evidence, the factual-vs-bug distinction,
how precision and recall are measured, provider and other limits.
- docs/design.md: motivation, architecture, the state-model seam, the
experimental C++ front end.
- docs/development.md: building and performance, tests, hooks.
Dropped as progress notes: the negative-result and recall-bug
narratives, before/after finding counts, review-round history, and
stale figures (a "5/5 recall" that depends on which branch a repo has
checked out, "84 tests", a duplicated recall.dart line, and the
adjudicator described as fully independent).
tool/cpp_front.py is a second front end written against the state model,
not the rules: it reads a compile database with libclang (Homebrew LLVM)
and writes the JSON that `pipe check --model` takes. Every rule,
baseline and gate works on it unchanged.
Mapping, per project `enum class` (identity from clang's resolved AST,
so two enums both named QueryIntent are never confused):
- a reference that produces a value (assignment, atomic store, argument,
return, initialiser) is an emission; a cast from an int is an indirect
one;
- a `case`, `==` or `!=` is an observation;
- a busy enumerator written to the method's OWN state, followed by a
top-level try whose catch neither writes state nor rethrows, is a
swallowed failure;
- sources the build does not compile are recorded as unresolved, so
findings naming them drop a confidence step.
One rule-side change, optional and Dart-neutral: an EmissionSite may name
its stateType. rendered-not-emitted attributed indirect emissions by owner
(a bloc owns one state type), so a cast to one enum made "never emitted"
unprovable for every other enum in the same C++ class. With stateType
set, the suppression is scoped to that enum. Dart output is still byte
identical on the fixture and all four dogfood apps.
Run on ~/work/tools/yams (375 translation units from build/debug, 0 clang
errors, about two minutes): 118 state enums, 82 findings. A grep-based
hand check of 13 found 12 factually correct - e.g. the IPC connection FSM
never enters ImplState::ReadingHeader, the downloader never reports
ProgressStage::Verifying though three CLI commands render it,
ExtractScopeType qualifiers are parsed but never acted on - and one false
positive (a nested JSON-parse fallback taken for a stuck busy state),
now excluded by requiring the method's own state and a top-level try.
Bugs found on the way and fixed: ccache/compiler wrappers passed as
inputs, relative -I paths resolved against the wrong directory (7 TUs
silently lost third_party headers), a symlinked path putting every TU
"outside" the project, and an empty compile database producing an empty
"clean" model with exit 0 - all now fail loudly or are handled.
test/cpp_front_test.dart runs the front end on test/fixtures/cpp_state
with positive and negative controls for each rule; skipped without LLVM.
138 tests pass.
Third step of the state-map seam, and the proof that it holds.
stateFlowFromJson rebuilds a StateFlow from its toJson form (computed
members - totals, coverage differences, Riverpod gaps - are recomputed),
and `pipe check --model <file>` runs every rule on such a file instead
of analysing Dart. A front end for another language now only has to
write that JSON to get every rule, --baseline ratcheting and --fail-on
gating. StateUnion.toJson now carries uiReadsBaseFields (only when
true), which the round trip showed it had been dropping.
Checked byte for byte, through a file on disk: `pipe state --format
json` then `pipe check --model` reports exactly what `pipe check` does
on the fixture and on resolved mirrors of nuwayze (102 findings), tick,
dex and word_alive. state_map_roundtrip_test.dart asserts the same in
process (model JSON is idempotent; findings identical; every rule fires,
so the comparison is not vacuous), and model_cli_test.dart covers the
CLI path.
--model refuses --write-baseline: a model does not record whether its
front end read every file, so the baseline could silently be partial -
the hazard the source path already refuses. A malformed model is a
PipeException, not a crash.
134 tests pass.
Second step of the state-map seam. The five ChangeNotifier/provider rules
walked resolved units from runProviderRules. Their recognisers move to
the Dart front end (dart_provider_facts.dart) and emit a NotifierModel
of neutral facts (facts.dart):
- NotifierFact / NotifierMethod / FieldMutation: an observable object
that must announce its own changes, its methods, what each calls,
mutates, and whether it notifies directly. Deliberately general enough
for a Qt QObject or a hand-rolled observer subject.
- consumer sites, provider declarations, member uses per file.
- NonSubscribingRead: every read made while building without
subscribing; whether the type is observable is now the rule's filter,
not the recogniser's.
- ProvidersCreatedInBuild: providers constructed in a build method.
The transitive notify closure, the observable/lifecycle/wrapper
judgements and all messages stay in provider_rules.dart, which no longer
imports the analyzer.
runChecks now takes only a StateFlow: no Project, no FileOutcome. The
scanned directories ride on the model (StateFlow.directories) for the
dead-handler scope claim. facts.dart is exported publicly.
No behaviour change, checked byte for byte: `check --format json` is
identical on the fixture and on resolved mirrors of nuwayze, tick, dex
and word_alive. 128 tests pass.
First step of the state-map seam: rules should read only a
language-neutral model, so a second front end (C++) can feed them.
The two loading rules built findings inside tree visitors, and
dead-handler and the unresolved-reference check walked every unit from
runChecks. Their recognisers now live in the Dart front end
(dart_facts.dart) and emit neutral facts (facts.dart) carried on
StateFlow:
- SwallowedFailure: a busy variant is emitted, then an awaited failure is
swallowed (loading-not-reset). The "does this owner hold rendered
state" gate moved to the rule, since it is judgement, not recognition.
- UnsettledFlag: a loading flag left set across unguarded awaited work
(loading-flag-not-reset); the rule derives confidence from setsFlag and
guardedTrySeen.
- constructedByFile and UnresolvedSummary, with project-relative paths.
Every fact serialises to JSON and appears in `pipe state`. rules.dart no
longer imports any analyzer AST library; only the provider rules still
take resolved units, which is the next step.
No behaviour change, checked byte for byte: `check --format json` is
identical on the fixture and on resolved mirrors of nuwayze, tick, dex
and word_alive. 128 tests pass.
check never looked at analyzer errors, so a project that resolved only
partly was analysed as if it were whole. pipe reads names off the syntax
tree, and an unresolved `Foo()` is a method call rather than a
construction, so a dispatch, emission or construction in such a file
simply disappears - and every finding is an absence claim that the
missing evidence could refute.
Found live during this work: with the Flutter SDK unreadable, nuwayze
resolved with 40,079 analyzer errors (package:flutter unresolvable) and
check still reported 102 findings at full confidence, with no warning.
Now, when any analysed file has an unresolved-reference diagnostic
(uri_does_not_exist, undefined_*, creation_with_non_type, ...):
- check warns on stderr and adds a report note naming the first one;
- a finding is lowered one confidence step, with
data.confidenceLowered = unresolved-references, when an unresolved file
mentions one of its identity names (event, variant, union, owner,
provider, member). pipe's evidence is name-based, so a file that never
names X cannot hide evidence about X; lowering everything for one stray
error would bury the signal (a first cut did exactly that, taking all
102 nuwayze findings down for a single unresolved file).
Validated on mirrors of the dogfood repos with a readable SDK (0 analyzer
errors): nuwayze 102 findings identical to the original main, dex 0,
word_alive 1 and tick 1 (the intended riverpod change). The fixture
test adds an unresolved file that constructs NeverDispatched: its
dead-handler finding drops to medium while UnhandledTap, which that file
never names, stays high.
The fingerprint is documented as surviving line shifts, but eight of the
fixture's findings carried a line number, so moving code down a file
turned accepted findings into new ones on the next ratcheted check:
- dormant emitted-not-rendered (keyed by union + owner, neither counted
as identifying, so the line was appended)
- notifier-inert-consumer and notifier-field-unread (owner only)
- loading-not-reset, whose `catchLine` key is a line by another name
`union` and `owner` now count as identifying (each of those rules emits at
most one finding per union or notifier per file), and loading-not-reset
keys on the catch's ordinal within its member. Its `on<E>` handler
closures are also named `on<E>` like loading-flag-not-reset's, so the
ordinal cannot collide across handlers.
The existing move test only bumped `line`, not `catchLine` in the data,
so it passed while real moves did not. fingerprint_test.dart now shifts
every file of the fixture for real and requires (a) every fingerprint to
survive and (b) no two findings to share one. (a) fails on the previous
rules.dart with exactly the eight findings above; (b) passed there too, and
guards the ordinal and closure naming that replace the line.
Baselines written before this change see the affected findings once as
new; re-accept them with --write-baseline. No committed baselines exist
in the dogfood repos.
riverpod-async-gap judged handling from the text of the expression a
`ref.watch` flows into, which missed two common shapes:
- Pattern handling. A switch with `AsyncError()` / `AsyncLoading()` cases
- the form pipe's own remediation recommends - did not count as
handling, and a switch STATEMENT on the watched value was never looked
at, because the parent climb stops at the first non-expression. A
provider handled that way in one widget and read with `.value` in
another was reported as unhandled.
- A value stored in a local. `final v = ref.watch(p); ... v.when(...)`
stopped the climb at the declaration, so both the handling and the read
were invisible: a handled provider could be reported, and an unhandled
one read as `v.value` was missed (a recall gap, not only precision).
The consumer now widens to an enclosing switch statement, the uses of a
declared local are followed through the enclosing body, and
AsyncError/AsyncLoading patterns count as handling.
Fixture: patterned, switched and stored providers (each also read
directly elsewhere, so the controls are not vacuous) were all reported
before and are silent now; storedUnhandledProvider was missed and is now
reported.
Real repos: tick's categoriesStreamProvider is no longer reported; its
categories screen handles it with `catsAsync.when(...)` through a
variable, which the rule's any-site-handles definition counts as handled.
nuwayze and dex unchanged. word_alive's internalUserProvider finding comes
from a commit made to word_alive during this work, not from this change:
the old code reports it too.
freezed declares variants as redirecting factories on the base
(`const factory DocState.loading() = DocLoading;`) and blocs emit through
them. `emit(const DocState.loading())` was recorded as emitting
`DocState`, which is never a variant, so no freezed emission matched and
every variant the UI renders was reported by rendered-not-emitted as a
dead branch.
Factory redirects are now also kept by constructor name, and an instance
creation of `Type.ctor` (or an unnamed redirecting factory) is credited to
the class it redirects to, for emissions, `state =` assignments and
dispatches alike.
Reproduced on freezed_style.dart before the fix: DocLoading, DocReady
and DocStale all reported as never emitted; after, only DocStale, which
really is never emitted and stays as the positive control. No change on
nuwayze, tick, dex or word_alive.
An SDK directory that exists but cannot be read passed discovery, and the
analyzer then died with a raw FileSystemException stack trace. This
happens for real under macOS privacy protection: a Flutter SDK under
~/Documents stats fine but every open fails, and every app whose
package_config.json names that flutterRoot crashed:
Unhandled exception:
FileSystemException(path=.../dart-sdk/lib/_internal/allowed_experiments.json; ...)
The chosen SDK is now probed before the analysis collection is built, and
an unreadable one is a PipeException naming the path, where it was
discovered from, and the --sdk remedy.
The last-resort fallback to the running Dart SDK took the executable's
directory (<sdk>/bin) as the SDK root, which never contains
lib/libraries.json, so it could never succeed. sdkFromExecutable now
maps <sdk>/bin/dart to <sdk>.
The adjudicator printed "controls FAILED" or "NO SIGNAL" and then exited
0, so a CI step reading the exit status took an uncertified run as a
pass. It now exits 1 unless the controls are meaningful and pass, in both
text and json output.
It was also described as independent. Its evidence is independent (its
own regex over source), but the list of unions, variants, events and
owners it checks comes from pipe's `state` output, so a variant pipe
fails to enumerate is never checked. The docstring and README now say so
instead of "independent adjudicator".
Two more ways a partial scan produced a false claim, the same hazard class
as the --limit and unreadable-file cases verify already refuses:
- `verify --rule X` compared the fresh X-only report against the whole
baseline, so every accepted finding from any other rule was printed as
RESOLVED ("these fixes are confirmed") when it had merely not been
checked. The baseline is now filtered to the selected rules.
- `--write-baseline` (check and guide) wrote whatever the run produced.
Under --rule or --limit, or with files that could not be analysed, the
baseline silently omitted every finding outside that subset, and the
next full `check --baseline` reported them all as new. It now refuses:
--rule/--limit before analysis, unreadable files after it.
Each is covered by a test that fails against the previous cli.dart
(subset_scan_test.dart, and the unreadable-file case in
unreadable_test.dart). README documents the refusal.
The method-level check read the whole body text, closures included, for
both the `loading: true` probe and the await probe, while the try
collector skips closures. So a method whose closure set the flag and
guarded its own await was reported at high confidence as unguarded, and a
method that only registers `on<E>` closures was reported alongside each
closure. Every probe now stops at nested functions, which are analysed on
their own; the flag probe is a `field: true` named-argument visitor
instead of a regex over source.
`on<E>` handler closures were all named `<closure>`, so two handlers in one
class shared a fingerprint and encodeBaseline kept only one of them. They
are now named `on<E>`. Existing baselines that accepted a handler as
`<closure>` will see it once as new.
Fixture: scheduleGuarded (guarded closure) and registerHandlers are now
negative controls; two unguarded handlers must yield two findings with
distinct fingerprints. Before: scheduleGuarded, registerHandlers and two
colliding `<closure>` findings; after: on<FlagRefresh>, on<FlagReload>.
Two ways the event rules reported handled events as unhandled (error
severity, high confidence):
1. A handler on the event base or an intermediate group. `on<SyncEvent>`
with a switch inside handles every subclass, but handlers were matched
to dispatches by exact name, so every dispatched subclass was reported
by event-without-handler and the base handler itself by dead-handler.
Variants now record the supertype they were enumerated through, and
coverage follows it: an event is handled when it or an ancestor is
registered, and a handler is live when anything at or below it is
dispatched or constructed. A group handler still does not cover a
sibling outside the group (UploadAbandoned stays reported).
2. Blocs sharing an event type. Only the first owner of an event type was
kept, so a second bloc's handlers were ignored: an event only it handles
was reported unhandled, and its dead handlers were never examined. There
is now one event union per owning bloc. A dispatch does not record which
bloc it targets, so an event any sharing bloc handles counts as handled,
and an unhandled event is reported once per shared union rather than
once per bloc. Dead handlers are still judged per bloc.
Reproduced first on base_event.dart: 4 false findings before
(SyncStarted, UploadRetryNow unhandled; SyncEvent, UploadRetry dead) and
the shared case (ShareDismissed falsely unhandled, ShareBetaBloc's dead
ShareExpired missed). Each is now a contract control.
No change on nuwayze (102 findings) or tick (2): byte-identical before
and after, since neither uses these patterns.
