`pipe verify .` failed with "verify needs --baseline" even in a repo
whose baseline sits where the README's CI recipe writes it. nuwayze's
scripts/testing/pipe_state_flow_gate.sh commits exactly that file,
.pipe-baseline.json at the repo root, so every verify there had to
repeat the path.
Without --baseline, verify now reads <path>/.pipe-baseline.json when it
exists and says so on stderr ("pipe: no --baseline given; using ..."),
so a run never silently picks a file nobody named. With neither, the
error names both remedies: pass --baseline, or write the conventional
file with `pipe check <path> --write-baseline <path>/.pipe-baseline.json`
before the fix. pipe had no baseline-name constant; defaultBaselineName
is now the one place it is spelled.
Tests (test/verify_default_baseline_test.dart): an explicit --baseline
wins and announces nothing; the conventional file is used and
announced; with neither the error carries both remedies; and end to
end, `pipe verify <copy>` against a baseline written at the copy's
root reads it (--expect-resolved fails on the 28 still-present
findings). README and docs/usage.md show the shorter form.
On nuwayze-w3, `pipe verify /Users/.../nuwayze-w3` with no flag reads
its .pipe-baseline.json (65 accepted findings). 225 tests pass.
main gained six PRs (#1-#6) while the audit fixes and the state-map seam
were in progress. Four of main's commits fixed the same bugs this branch
did, independently; where they overlap, the more complete version wins:
- Fingerprints: main's (a0844bb) - catchSite (`member#index` among all
catches), union/owner as identity, a legacy fingerprint so version-1
baselines still match, baseline "version": 2. This branch's `catch`
ordinal is dropped; its fingerprint_test still guards the result.
- verify --rule: main's verifyAnalysis (cf9e0b7), which also reports
findings in skipped files as NOT SCANNED.
- Unreadable SDK: both kept - this branch's up-front probe gives the
early, specific error; main's catch around the analyzer (d6b6bcb) is
the backstop.
- Riverpod: main's "a ref.listen is not a render" (f6411dc) and this
branch's AsyncError patterns / stored-value tracking compose; the
fixture keeps both sets of cases.
Ported onto the model-only rules (the seam), since main edited code this
branch had moved into front-end facts:
- excerpt.dart's literal-free, bounded quoting (a prompt-injection
guard) now also covers front-end facts: SwallowedFailure.handler and
NonSubscribingRead.excerpt, and the C++ front end elides string
literals in handler evidence the same way.
- ScanCoverage: runChecks stays source-free; the model carries coverage
(`scanCoverage` in the state JSON, written by both front ends), so
`check --model` states coverage like a source check.
- main's softened wording (rendered-not-emitted "probably unreachable",
the remove-policy dead-handler text, notifier-field-unread) and the
backticked flag message.
Docs: main's README grew the Firestore query inventory and the output
contract; the README stays short, the inventory moves to
docs/queries.md, and the contract, exit codes, verify and baseline
details join docs/usage.md.
Checked against origin/main on resolved mirrors of the dogfood apps:
nuwayze (102), dex (0) and word_alive (0) identical; tick differs only
by the intended riverpod stored-value fix. On the fixture every
difference is one of this branch's fixes. 214 tests pass.
The README had grown to 500 lines mixing an overview, a CLI reference,
rule evidence, measurement method, architecture, build notes and
progress notes. It is now a 72-line overview (what pipe is, coverage,
quick start, the rule list, links), and the rest lives in:
- docs/usage.md: commands, flags (checked against `pipe help`), SDK and
resolution behaviour, CI baselines, verify, fix order, agent briefs,
remediation policy.
- docs/rules.md: each rule's evidence, the factual-vs-bug distinction,
how precision and recall are measured, provider and other limits.
- docs/design.md: motivation, architecture, the state-model seam, the
experimental C++ front end.
- docs/development.md: building and performance, tests, hooks.
Dropped as progress notes: the negative-result and recall-bug
narratives, before/after finding counts, review-round history, and
stale figures (a "5/5 recall" that depends on which branch a repo has
checked out, "84 tests", a duplicated recall.dart line, and the
adjudicator described as fully independent).
Add `pipe queries`, which lists every Firestore query a project can issue:
scope, collection, path pattern, filters with operators and value types,
composites, orderBy, limit, cursors, realtime, aggregation, in-loop, plus a
confidence and the reasons it is not exact. Receivers are matched by static
type and cloud_firestore library URI, never by name.
Queries are followed through chains, local variables (flow-insensitive with
two cuts), fields, getters and one level of helper methods with arguments
bound. An --adapter-config (YAML or JSON) maps declarative query APIs such as
a server-side gateway onto the same schema.
Inventory only: no rules, findings or baseline changes. Every emitted name is
validated against a safe character set so no source text reaches the output.
tool/cpp_front.py is a second front end written against the state model,
not the rules: it reads a compile database with libclang (Homebrew LLVM)
and writes the JSON that `pipe check --model` takes. Every rule,
baseline and gate works on it unchanged.
Mapping, per project `enum class` (identity from clang's resolved AST,
so two enums both named QueryIntent are never confused):
- a reference that produces a value (assignment, atomic store, argument,
return, initialiser) is an emission; a cast from an int is an indirect
one;
- a `case`, `==` or `!=` is an observation;
- a busy enumerator written to the method's OWN state, followed by a
top-level try whose catch neither writes state nor rethrows, is a
swallowed failure;
- sources the build does not compile are recorded as unresolved, so
findings naming them drop a confidence step.
One rule-side change, optional and Dart-neutral: an EmissionSite may name
its stateType. rendered-not-emitted attributed indirect emissions by owner
(a bloc owns one state type), so a cast to one enum made "never emitted"
unprovable for every other enum in the same C++ class. With stateType
set, the suppression is scoped to that enum. Dart output is still byte
identical on the fixture and all four dogfood apps.
Run on ~/work/tools/yams (375 translation units from build/debug, 0 clang
errors, about two minutes): 118 state enums, 82 findings. A grep-based
hand check of 13 found 12 factually correct - e.g. the IPC connection FSM
never enters ImplState::ReadingHeader, the downloader never reports
ProgressStage::Verifying though three CLI commands render it,
ExtractScopeType qualifiers are parsed but never acted on - and one false
positive (a nested JSON-parse fallback taken for a stuck busy state),
now excluded by requiring the method's own state and a top-level try.
Bugs found on the way and fixed: ccache/compiler wrappers passed as
inputs, relative -I paths resolved against the wrong directory (7 TUs
silently lost third_party headers), a symlinked path putting every TU
"outside" the project, and an empty compile database producing an empty
"clean" model with exit 0 - all now fail loudly or are handled.
test/cpp_front_test.dart runs the front end on test/fixtures/cpp_state
with positive and negative controls for each rule; skipped without LLVM.
138 tests pass.
- reject a second project path as a usage error instead of analysing
only the first
- resolve the default recall manifest from the working directory, else
from the pipe checkout the binary was built from, so recall works
outside the repo
- write --write-baseline even when guide --yams fails to persist
- drain the yams child's stdout so a chatty yams cannot block pipe
- report any unexpected exception in one line with exit 2; PIPE_DEBUG=1
prints the stack trace
Third step of the state-map seam, and the proof that it holds.
stateFlowFromJson rebuilds a StateFlow from its toJson form (computed
members - totals, coverage differences, Riverpod gaps - are recomputed),
and `pipe check --model <file>` runs every rule on such a file instead
of analysing Dart. A front end for another language now only has to
write that JSON to get every rule, --baseline ratcheting and --fail-on
gating. StateUnion.toJson now carries uiReadsBaseFields (only when
true), which the round trip showed it had been dropping.
Checked byte for byte, through a file on disk: `pipe state --format
json` then `pipe check --model` reports exactly what `pipe check` does
on the fixture and on resolved mirrors of nuwayze (102 findings), tick,
dex and word_alive. state_map_roundtrip_test.dart asserts the same in
process (model JSON is idempotent; findings identical; every rule fires,
so the comparison is not vacuous), and model_cli_test.dart covers the
CLI path.
--model refuses --write-baseline: a model does not record whether its
front end read every file, so the baseline could silently be partial -
the hazard the source path already refuses. A malformed model is a
PipeException, not a crash.
134 tests pass.
check never looked at analyzer errors, so a project that resolved only
partly was analysed as if it were whole. pipe reads names off the syntax
tree, and an unresolved `Foo()` is a method call rather than a
construction, so a dispatch, emission or construction in such a file
simply disappears - and every finding is an absence claim that the
missing evidence could refute.
Found live during this work: with the Flutter SDK unreadable, nuwayze
resolved with 40,079 analyzer errors (package:flutter unresolvable) and
check still reported 102 findings at full confidence, with no warning.
Now, when any analysed file has an unresolved-reference diagnostic
(uri_does_not_exist, undefined_*, creation_with_non_type, ...):
- check warns on stderr and adds a report note naming the first one;
- a finding is lowered one confidence step, with
data.confidenceLowered = unresolved-references, when an unresolved file
mentions one of its identity names (event, variant, union, owner,
provider, member). pipe's evidence is name-based, so a file that never
names X cannot hide evidence about X; lowering everything for one stray
error would bury the signal (a first cut did exactly that, taking all
102 nuwayze findings down for a single unresolved file).
Validated on mirrors of the dogfood repos with a readable SDK (0 analyzer
errors): nuwayze 102 findings identical to the original main, dex 0,
word_alive 1 and tick 1 (the intended riverpod change). The fixture
test adds an unresolved file that constructs NeverDispatched: its
dead-handler finding drops to medium while UnhandledTap, which that file
never names, stays high.
The adjudicator printed "controls FAILED" or "NO SIGNAL" and then exited
0, so a CI step reading the exit status took an uncertified run as a
pass. It now exits 1 unless the controls are meaningful and pass, in both
text and json output.
It was also described as independent. Its evidence is independent (its
own regex over source), but the list of unions, variants, events and
owners it checks comes from pipe's `state` output, so a variant pipe
fails to enumerate is never checked. The docstring and README now say so
instead of "independent adjudicator".
Two more ways a partial scan produced a false claim, the same hazard class
as the --limit and unreadable-file cases verify already refuses:
- `verify --rule X` compared the fresh X-only report against the whole
baseline, so every accepted finding from any other rule was printed as
RESOLVED ("these fixes are confirmed") when it had merely not been
checked. The baseline is now filtered to the selected rules.
- `--write-baseline` (check and guide) wrote whatever the run produced.
Under --rule or --limit, or with files that could not be analysed, the
baseline silently omitted every finding outside that subset, and the
next full `check --baseline` reported them all as new. It now refuses:
--rule/--limit before analysis, unreadable files after it.
Each is covered by a test that fails against the previous cli.dart
(subset_scan_test.dart, and the unreadable-file case in
unreadable_test.dart). README documents the refusal.
7ee8020 added the caution that the order is a score, not a mandate, but the
summary line two lines below still told the agent to work them in order.
The README fix-order example was hand-written with invented counts; it is now
the real output of the plan for the stuck_loading fixture.
Drift, corrected:
- The suggest example showed a "fix: Either render ..." line that no current
remediation produces; it now shows the current agent-plan shape
(caution line, coverage line, wire-up remediation), marked illustrative
because it could not be regenerated from one run.
- Common flags: suggest's `text|agent|json` formats, `recall -c` defaulting to
8 rather than 4, `--[no-]include-generated` defaulting on, and a missing
--dir being an error.
- `--yams` persists the full markdown report, not the brief.
- The architecture list named recall.dart twice and omitted cli.dart,
errors.dart, pool.dart and provider_rules.dart (and now excerpt.dart).
- The hardcoded "84 tests" is gone; the recall result and the word_alive
manifest note are dated "as of 2026-09-20", since both describe branches that
were unmerged at that date.
Documented from the previous PR: stdout carries only the requested output,
the `coverage` block and one-line coverage note, exit codes (0/1/2/64),
verify's --rule filter and NOT SCANNED category, version-2 baselines and how
version-1 fingerprints are still matched, packing that stops at the first
entry that does not fit, bounded source excerpts, pasteable verify commands,
and the policy's effect on the legend and on suggest's certainty bonus.
Four legend epithets claimed more than their rules establish:
- rendered-not-emitted (medium): "so it is dead code". The rule sees only the
scanned code and suppresses owners whose helper-method emissions it cannot
resolve; it now says the branch is probably unreachable.
- riverpod-async-gap: "a failure shows a blank or stale screen". It can; the
rule downgrades `bool` guard providers to info precisely because it often
does not.
- notifier-field-unread (medium): "can never affect what the user sees". The
check is a by-name read search in the scanned code; the finding message and
the epithet now say "likely".
- provider-created-in-build: "on every rebuild and state is lost". The rule is
info because rebuild frequency is unknown; it now says "if the widget
rebuilds".
The README rules table and the rendered-not-emitted and notifier-field-unread
messages are aligned with the same wording.
An execution sweep comparing README flags against what each command accepts found
no documented flag that the CLI rejects, but five working flags the README never
mentioned: index --fail-on-error, suggest --max-actions/--budget,
guide --max-findings/--yams-name, recall --manifest. Each is now described with
its verified behaviour (e.g. index exits 1 with analyzer errors, 0 without), plus
a note that numeric flags reject non-positive-integer values instead of falling
back to a default.
The independent precision adjudicator reported a pipe defect that did not exist.
Three defects in the checker, all found by chasing its own output against the
source:
1. It matched variant names by UNQUALIFIED name, so `is ExportSuccess` in
analytics_tab.dart was credited to ArtistStoreManagementState.ExportSuccess -
the very defect pipe's own per-union scoping had just fixed. Those three files
reference AdminAnalyticsState, ContentModerationState and
HelpCenterManagementState, and none of them mentions
ArtistStoreManagementState, so pipe's finding was correct and the checker's
"contradiction" was false. Evidence about a variant now requires a file that
also names the union or its owner - unless the name is declared by only one
union, in which case it is unambiguous and needs no scoping (event
construction sites carry no type annotation, and demanding one rejected sound
evidence: the dispatched control fell from 162/162 to 143/162).
2. Its notion of "rendered" was type-tests only, missing switch-expression arms
(`ChatListOperationInProgress value =>` has no `case` keyword in Dart 3) and
base-class payload reads (`state.upcomingBookings`, where BookingsLoaded
passes `required super.upcomingBookings`). Both are real ways a variant reaches
the screen; pipe models both.
3. `controls_pass` was `len(control1) == len(rendered) and ...`, which on a
corpus with no control material is `0 == 0` - a vacuous pass. word_alive
printed "controls pass: True" with 0/0 and a pristine-looking 100%. The control
is now three-way (found / unattributable / missing), only `missing` fails, and
an empty corpus exits NO SIGNAL with precision reported as UNDEFINED rather
than perfect.
Also fixed: the scoping pass looked for union names in `all_names`, which holds
variant names only, so `union in file_ids` could never be true and every control
silently read 0.
Result on clean nuwayze main: controls 131/131 rendered, 143/162 dispatched (19
unattributable, 0 missing) - controls pass, and precision is 79/79 + 77/77 with
zero contradictions. No pipe defect was found this round; the adjudicator's one
apparent finding was its own.
84 tests still pass; dart analyze clean.
An independent Gemini 3.8 Flash audit of this repo found real problems. Each was
reproduced before being fixed.
Confirmed defects:
- tool/adjudicate.py crashed with KeyError once R001 started grouping variants
under 'variants' instead of 'variant'. Fixing that exposed a second bug of my
own: it then divided verified VARIANTS by FINDINGS and reported 216% precision.
Counts are now coherent (findings vs variant-level checks) and a clean repo
reports 0/0 rather than a fake 100%.
- GuideCommand called _readPolicy but never registered --policy, so
exited 64 despite the README documenting it.
- loading-flag-not-reset treated a constructor-invoked method as inheriting the
loading flag without checking flag.defaultsTrue, so a default-FALSE flag was
reported as in flight.
- Finding.fingerprint embedded the line number for rules whose only identity was
, so moving code broke baseline entries. is now a
discriminating key; only loading-not-reset, which can emit two findings from one
method, carries an explicit catchLine.
- suggest grouped unrelated provider-read-in-build findings by file; the group key,
label and target list now use valueType/call.
Recall defect found while checking the auditor's own claim: UI variant mentions
were collected project-wide by unqualified name, so ExportSuccess - which appears
in FOUR unions on nuwayze - was cross-credited. Mentions are now scoped per union
to files that also name the state type or its owner, which surfaced
ArtistStoreManagementState[ExportSuccess] as a real finding.
Five negative controls could not fail (an event that is never handled can never be
reported as unhandled; a union with no render sites can never be reported as
rendered-not-emitted). Added fixture cases that make each one bite: HandledTap
(handled AND dispatched), PatternState render sites, GuardedCubit (guarded loading
path), QuietCubit._warmUp (constructor-invoked await on a default-false flag).
Root cause of a recurring mystery: this repo EXCLUDED test/fixtures/** from
analysis, which made any analyzer running in the repo context parse the fixture
without resolving its imports and report phantom undefined-class errors. I had
dismissed these as stale cache several times; they were my own exclusion. The
exclude is gone and the fixture governs itself. A new test asserts the fixture has
zero analyzer errors, not merely that it resolves, so IR distortion cannot hide.
84 tests (was 81).
An unreachable state, handler or control is usually an unfinished feature rather
than dead weight, so pipe now leads with making it reachable: render the missing
states, add the missing branch, dispatch the event from a control. Removal is the
exception and is worded as needing explicit approval instead of being offered as
an equal alternative.
New FixPolicy (wire-up default | remove), threaded through runChecks,
runProviderRules, analyzeProject and buildGuidance, with --policy on check, guide
and suggest. It changes every remediation that presents a choice, the brief's
decision line, and the dead-handler section heading ('Unreachable handlers - wire
up the control' vs 'Cleanly deletable').
It is a policy, not a finding: the same facts are reported either way.
81 tests, including 6 new ones asserting the default and that remove flips it.
Two real defects found by an audit of nuwayze, both confirmed against the code.
1. Recall: the switch 'case ChatListLoaded value => value' binds the type on a
DeclaredVariablePattern, not the object pattern ChatListLoaded() this matcher
looked for. An exhaustive switch was therefore invisible and two variants were
wrongly reported as never rendered. Both the mention collector and the branch
collector now handle declared variable patterns.
2. Precision: the UI often reads a union's payload off the BASE class
(state.currentBooking, state.isRefreshing), so the data reaches the user even
though the variant is never matched. StateUnion now records uiReadsBaseFields
and those findings drop to info with an explanation instead of warning. That
removed 5 false-positive warnings on nuwayze.
3. recall/manifest.json gains status=open|fixed. A fixed case that is reported
again is a REGRESSION, so repaired bugs become permanent guards against the
real code instead of deleted expectations. The three bugs fixed earlier today
(dex x2, word_alive) are now fixed-status guards:
recall --strict -> 5/5 accounted for, 0 regressions.
nuwayze: 111 findings, 11 warnings (was 16). 75 tests, fixture regression guard
patternView covers the typed-pattern case.
Completes the provider/ChangeNotifier rule set:
- notifier-field-unread: a public getter on a notifier the UI consumes that
nothing reads, so the state it exposes cannot reach the user. Once per
notifier. nuwayze: 3 (UserProvider.isInitializing,
AppLocaleProvider.preferredLocale, and _AgentHubController with 8 members).
- provider-created-in-build: a provider constructed in build without .value, so
create reruns on every rebuild. Reported once per build method, not once per
provider: nuwayze's AppProviders.build constructs 45 of them and that is one
decision, not 45 bugs. Deliberately info, not warning, because a root
StatelessWidget rarely rebuilds and static analysis cannot tell rebuild
frequency apart.
Also fixes a real defect the budget test caught: the brief's 'by rule:' line
grew with the rule count and crowded out the findings it summarises. It is now
capped at the top 6 with a '+N more' suffix, the omission note is reserved for
up front so the artifact stays inside its budget, and the overflow message is
shorter.
75 tests. Verified the fixture with pipe's own resolver (0 errors) and validated
that check by injecting a type error into a copy (errors=1).
Adds the provider half of the state-management surface:
- collects every provider-package declaration type (Provider,
ChangeNotifierProvider, ListenableProvider, ValueListenableProvider,
FutureProvider, StreamProvider, StateProvider, ProxyProvider family) and
reports the counts by type in the coverage note
- new rule provider-read-in-build: a Listenable read with context.read or
Provider.of(listen: false) directly in a build body, so the widget never
subscribes and can render stale state
447 raw context.read calls in nuwayze reduce to 1 finding. The first four were
hand-checked and every one was a read inside a closure (a BlobListener-style
listener callback or a Selector builder), so the collector now refuses to
descend into nested functions; that costs recall on reads inside Builder and is
recorded in the README rather than hidden.
Also fixes a third fingerprint collision: valueType/call were missing from the
finding identity, so two provider-read-in-build findings in one file shared a
baseline fingerprint. Identity now requires a discriminating key and falls back
to the line only when there is none.
The union-observed-but-variant-never-matched tier was one finding per variant,
so 67 findings described the same fix across 21 unions. Group by marker class
instead: failures and loading markers stay warnings because an unrendered
failure means the user gets no feedback, other markers drop to info.
nuwayze: 146 findings -> 108, and the actionable core is now visible: 14
warnings (2 dormant decisions + 12 missing-feedback groups), down from 49.
Also records a negative result in the README: tiering these by 'variant declares
no fields beyond its base' was tried and rejected, because 67/67 qualified and a
signal true of the whole population cannot discriminate.
Dogfooding after fixing the three reported bugs showed that high-confidence
emitted-not-rendered findings can be deliberately dormant scaffolding rather
than bugs: on nuwayze both such blocs were already documented as unreachable
in the repository's own action contract, and the CommentsBloc consumer had
been deleted on purpose when a StreamBuilder UI replaced it.
Reporting nine near-identical findings for one product decision buries the
findings that are real regressions, so an unobserved union now yields a single
finding listing its variants, and the remediation says to check the
repository's docs and action contracts before deleting anything.
nuwayze: 159 findings -> 146 (15 variant findings collapse to 2 decisions).
