Found by dogfooding on nuwayze: `pipe queries lib` from the repo root
failed with
no resolved package graph at .dart_tool/package_config.json
remedy: run `flutter pub get` or `dart pub get` in .../nuwayze-w3/lib
It failed even with the package resolved, because the path was taken
as the project root: pipe looked for lib/.dart_tool/package_config.json,
which never exists, and the remedy sent the user to run pub get in a
directory that has no pubspec.
enclosingPackage() (project.dart) splits a path into the package that
contains it (the path itself when it holds a pubspec.yaml, else the
nearest ancestor that does) and the directory it names under that
root. `pipe queries` loads the package root and scans only that
directory, so `pipe queries lib/services` is `--dir lib/services` on
the package; an explicit --dir with such a path is relative to it.
Without a resolved graph the error now names the package root.
loadProject's own remedy uses the same lookup, so `pipe check lib`
(and the other commands, which still take the path as the root) now
says the path is inside the package at <root> and suggests passing
that with `--dir lib`, instead of pub get in lib.
Tests: enclosingPackage on a root, a nested directory, a nested
package and a missing path; loadProject's remedy for a directory
inside a package; and `pipe queries` on the fixture's lib/data
(root reported as the package, one file discovered), with
`lib --dir data`, and on an unresolved package where the remedy names
the root, not lib.
On nuwayze-w3: `pipe queries lib` exits 0 with 313 queries over 847
files, identical to `pipe queries .`; `pipe queries lib/services` scans
161 files and every one of its 212 queries sits under lib/services.
233 tests pass.
`pipe verify .` failed with "verify needs --baseline" even in a repo
whose baseline sits where the README's CI recipe writes it. nuwayze's
scripts/testing/pipe_state_flow_gate.sh commits exactly that file,
.pipe-baseline.json at the repo root, so every verify there had to
repeat the path.
Without --baseline, verify now reads <path>/.pipe-baseline.json when it
exists and says so on stderr ("pipe: no --baseline given; using ..."),
so a run never silently picks a file nobody named. With neither, the
error names both remedies: pass --baseline, or write the conventional
file with `pipe check <path> --write-baseline <path>/.pipe-baseline.json`
before the fix. pipe had no baseline-name constant; defaultBaselineName
is now the one place it is spelled.
Tests (test/verify_default_baseline_test.dart): an explicit --baseline
wins and announces nothing; the conventional file is used and
announced; with neither the error carries both remedies; and end to
end, `pipe verify <copy>` against a baseline written at the copy's
root reads it (--expect-resolved fails on the 28 still-present
findings). README and docs/usage.md show the shorter form.
On nuwayze-w3, `pipe verify /Users/.../nuwayze-w3` with no flag reads
its .pipe-baseline.json (65 accepted findings). 225 tests pass.
Found by dogfooding on nuwayze: `pipe check` reported
_AgentHubController is consumed in the UI but 8 of its public
member(s) are read by name nowhere outside the class ...
for a private ChangeNotifier whose widget, declared in the same file
(lib/screens/settings/agent_hub_screen.dart), renders every one of
those getters (`controller.authorizations`, `controller.hasLoadError`,
...).
The rule claims "outside the class" but discounted the notifier's
whole file: member reads were collected per file and the file that
declares the notifier was skipped. A private controller can only be
rendered from its own library, so that exclusion hid exactly the reads
that matter.
Member reads are now kept per scope: reads inside a ChangeNotifier
class body go under `memberUseScope(file, Class)`, everything else
under the file. The rule discounts only the notifier's own scope, so a
widget in the same file, or in a `part` of the library, counts as a
reader, while a notifier reading itself (`this.x`, `other.x`) still
does not. The model JSON keeps its shape (a map of name sets); only
the keys gained the class scope.
Tests (test/field_unread_scope_test.dart, a generated project): a
same-file widget read and a part-file read are not reported; a member
read nowhere and a member read only inside its own class are.
On nuwayze-w3 the finding is gone and nothing else moves: 65 findings
with the current release, 64 with this change, the only difference
being _AgentHubController. 221 tests pass.
Found by dogfooding on nuwayze main: `pipe verify` against the ratchet
baseline printed
resolved 1 no longer reported - these fixes are confirmed
+ service_browse_bloc.dart:20 LoadServicesByArtist [dead-handler]
but nothing was fixed. nuwayze commit ccca9bd1e ("delete dead firestore
query code") deleted the event and its handler. Under pipe's default
wire-up policy, deleting an unreachable handler is the exception that
needs approval, so calling it a confirmed fix overstated what happened.
verify now separates the two. A gone finding is REMOVED when its
subject no longer exists:
- its file is not among the project's files;
- for rules labelled by a declaration (the event/state/union/provider or
owner class), that name is no longer declared anywhere;
- for dead-handler, the on<Event> registration itself is gone.
Everything else stays RESOLVED. Member-labelled rules (a method or
build) are classed by file only, never guessed. Removed entries still
count as gone for --expect-resolved; they are only named honestly, in
text ("removed N gone because the code was deleted - not wired up",
a REMOVED section) and JSON (removed, removedEntries).
On nuwayze main the entry now reads REMOVED. Tests: a unit test over
hand-built entries (a wired handler stays resolved, a deleted handler
and a deleted file are removed, a member label is not guessed), and an
end-to-end test that deletes a fixture's dead handler and runs
verifyAnalysis. 216 tests pass.
The front end hardcoded /opt/homebrew/opt/llvm and lib/libclang.dylib, so
on Linux it died before parsing anything:
FileNotFoundError: [Errno 2] No such file or directory:
'/opt/homebrew/opt/llvm/bin/clang'
PIPE_LLVM could move the prefix but not the file name, so no value of it
made the tool run there.
Discovery is now per platform: Homebrew's LLVM on macOS, the system prefix
elsewhere, PIPE_LLVM still winning over both. The library is looked up as
.dylib or .so, exact name first and then highest version, and under
lib/*-linux-gnu as well, because distributions ship
libclang.so.<major>.<minor> without always adding the unversioned symlink
and Debian keeps it in a multiarch subdirectory. -print-resource-dir falls
back to clang on PATH when the prefix ships only the library. When nothing
matches, the conventional name is handed to clang.cindex so its own error
names the library rather than a path this code invented.
_sdk_root already degrades to None off macOS, since xcrun is absent.
test/cpp_front_test.dart looked for the same Homebrew dylib, so on Linux
it reported "All tests skipped" rather than failing. That is why the gap
went unnoticed: the suite was green because it ran nothing. Its detection
now mirrors the front end's, and on this Linux machine the four cases go
from skipped to run and passing.
Verified end to end against an unrelated 87-translation-unit C++ project:
87/87 parsed, 0 TUs failed, 0 clang errors, 26 state enums and 879
emissions recovered.
Both suites fail test/stdout_json_test.dart under the parallel runner on
this machine, pristine main included, and it passes in isolation on both
branches; a serial run failed a different test instead. Pre-existing and
unrelated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
main gained six PRs (#1-#6) while the audit fixes and the state-map seam
were in progress. Four of main's commits fixed the same bugs this branch
did, independently; where they overlap, the more complete version wins:
- Fingerprints: main's (a0844bb) - catchSite (`member#index` among all
catches), union/owner as identity, a legacy fingerprint so version-1
baselines still match, baseline "version": 2. This branch's `catch`
ordinal is dropped; its fingerprint_test still guards the result.
- verify --rule: main's verifyAnalysis (cf9e0b7), which also reports
findings in skipped files as NOT SCANNED.
- Unreadable SDK: both kept - this branch's up-front probe gives the
early, specific error; main's catch around the analyzer (d6b6bcb) is
the backstop.
- Riverpod: main's "a ref.listen is not a render" (f6411dc) and this
branch's AsyncError patterns / stored-value tracking compose; the
fixture keeps both sets of cases.
Ported onto the model-only rules (the seam), since main edited code this
branch had moved into front-end facts:
- excerpt.dart's literal-free, bounded quoting (a prompt-injection
guard) now also covers front-end facts: SwallowedFailure.handler and
NonSubscribingRead.excerpt, and the C++ front end elides string
literals in handler evidence the same way.
- ScanCoverage: runChecks stays source-free; the model carries coverage
(`scanCoverage` in the state JSON, written by both front ends), so
`check --model` states coverage like a source check.
- main's softened wording (rendered-not-emitted "probably unreachable",
the remove-policy dead-handler text, notifier-field-unread) and the
backticked flag message.
Docs: main's README grew the Firestore query inventory and the output
contract; the README stays short, the inventory moves to
docs/queries.md, and the contract, exit codes, verify and baseline
details join docs/usage.md.
Checked against origin/main on resolved mirrors of the dogfood apps:
nuwayze (102), dex (0) and word_alive (0) identical; tick differs only
by the intended riverpod stored-value fix. On the fixture every
difference is one of this branch's fixes. 214 tests pass.
Add `pipe queries`, which lists every Firestore query a project can issue:
scope, collection, path pattern, filters with operators and value types,
composites, orderBy, limit, cursors, realtime, aggregation, in-loop, plus a
confidence and the reasons it is not exact. Receivers are matched by static
type and cloud_firestore library URI, never by name.
Queries are followed through chains, local variables (flow-insensitive with
two cuts), fields, getters and one level of helper methods with arguments
bound. An --adapter-config (YAML or JSON) maps declarative query APIs such as
a server-side gateway onto the same schema.
Inventory only: no rules, findings or baseline changes. Every emitted name is
validated against a safe character set so no source text reaches the output.
tool/cpp_front.py is a second front end written against the state model,
not the rules: it reads a compile database with libclang (Homebrew LLVM)
and writes the JSON that `pipe check --model` takes. Every rule,
baseline and gate works on it unchanged.
Mapping, per project `enum class` (identity from clang's resolved AST,
so two enums both named QueryIntent are never confused):
- a reference that produces a value (assignment, atomic store, argument,
return, initialiser) is an emission; a cast from an int is an indirect
one;
- a `case`, `==` or `!=` is an observation;
- a busy enumerator written to the method's OWN state, followed by a
top-level try whose catch neither writes state nor rethrows, is a
swallowed failure;
- sources the build does not compile are recorded as unresolved, so
findings naming them drop a confidence step.
One rule-side change, optional and Dart-neutral: an EmissionSite may name
its stateType. rendered-not-emitted attributed indirect emissions by owner
(a bloc owns one state type), so a cast to one enum made "never emitted"
unprovable for every other enum in the same C++ class. With stateType
set, the suppression is scoped to that enum. Dart output is still byte
identical on the fixture and all four dogfood apps.
Run on ~/work/tools/yams (375 translation units from build/debug, 0 clang
errors, about two minutes): 118 state enums, 82 findings. A grep-based
hand check of 13 found 12 factually correct - e.g. the IPC connection FSM
never enters ImplState::ReadingHeader, the downloader never reports
ProgressStage::Verifying though three CLI commands render it,
ExtractScopeType qualifiers are parsed but never acted on - and one false
positive (a nested JSON-parse fallback taken for a stuck busy state),
now excluded by requiring the method's own state and a top-level try.
Bugs found on the way and fixed: ccache/compiler wrappers passed as
inputs, relative -I paths resolved against the wrong directory (7 TUs
silently lost third_party headers), a symlinked path putting every TU
"outside" the project, and an empty compile database producing an empty
"clean" model with exit 0 - all now fail loudly or are handled.
test/cpp_front_test.dart runs the front end on test/fixtures/cpp_state
with positive and negative controls for each rule; skipped without LLVM.
138 tests pass.
- reject a second project path as a usage error instead of analysing
only the first
- resolve the default recall manifest from the working directory, else
from the pipe checkout the binary was built from, so recall works
outside the repo
- write --write-baseline even when guide --yams fails to persist
- drain the yams child's stdout so a chatty yams cannot block pipe
- report any unexpected exception in one line with exit 2; PIPE_DEBUG=1
prints the stack trace
riverpod-async-gap treated any site that read an AsyncValue as a UI read,
including ref.listen. A listener mirrors the value into a side effect and
draws nothing, so it cannot leave a blank or stale screen. Dogfooding on
word_alive flagged an analytics user-property listener this way.
A gap now needs a ref.watch that reads the value. Listeners still count as
places that handle error or loading.
Third step of the state-map seam, and the proof that it holds.
stateFlowFromJson rebuilds a StateFlow from its toJson form (computed
members - totals, coverage differences, Riverpod gaps - are recomputed),
and `pipe check --model <file>` runs every rule on such a file instead
of analysing Dart. A front end for another language now only has to
write that JSON to get every rule, --baseline ratcheting and --fail-on
gating. StateUnion.toJson now carries uiReadsBaseFields (only when
true), which the round trip showed it had been dropping.
Checked byte for byte, through a file on disk: `pipe state --format
json` then `pipe check --model` reports exactly what `pipe check` does
on the fixture and on resolved mirrors of nuwayze (102 findings), tick,
dex and word_alive. state_map_roundtrip_test.dart asserts the same in
process (model JSON is idempotent; findings identical; every rule fires,
so the comparison is not vacuous), and model_cli_test.dart covers the
CLI path.
--model refuses --write-baseline: a model does not record whether its
front end read every file, so the baseline could silently be partial -
the hazard the source path already refuses. A malformed model is a
PipeException, not a crash.
134 tests pass.
Second step of the state-map seam. The five ChangeNotifier/provider rules
walked resolved units from runProviderRules. Their recognisers move to
the Dart front end (dart_provider_facts.dart) and emit a NotifierModel
of neutral facts (facts.dart):
- NotifierFact / NotifierMethod / FieldMutation: an observable object
that must announce its own changes, its methods, what each calls,
mutates, and whether it notifies directly. Deliberately general enough
for a Qt QObject or a hand-rolled observer subject.
- consumer sites, provider declarations, member uses per file.
- NonSubscribingRead: every read made while building without
subscribing; whether the type is observable is now the rule's filter,
not the recogniser's.
- ProvidersCreatedInBuild: providers constructed in a build method.
The transitive notify closure, the observable/lifecycle/wrapper
judgements and all messages stay in provider_rules.dart, which no longer
imports the analyzer.
runChecks now takes only a StateFlow: no Project, no FileOutcome. The
scanned directories ride on the model (StateFlow.directories) for the
dead-handler scope claim. facts.dart is exported publicly.
No behaviour change, checked byte for byte: `check --format json` is
identical on the fixture and on resolved mirrors of nuwayze, tick, dex
and word_alive. 128 tests pass.
check never looked at analyzer errors, so a project that resolved only
partly was analysed as if it were whole. pipe reads names off the syntax
tree, and an unresolved `Foo()` is a method call rather than a
construction, so a dispatch, emission or construction in such a file
simply disappears - and every finding is an absence claim that the
missing evidence could refute.
Found live during this work: with the Flutter SDK unreadable, nuwayze
resolved with 40,079 analyzer errors (package:flutter unresolvable) and
check still reported 102 findings at full confidence, with no warning.
Now, when any analysed file has an unresolved-reference diagnostic
(uri_does_not_exist, undefined_*, creation_with_non_type, ...):
- check warns on stderr and adds a report note naming the first one;
- a finding is lowered one confidence step, with
data.confidenceLowered = unresolved-references, when an unresolved file
mentions one of its identity names (event, variant, union, owner,
provider, member). pipe's evidence is name-based, so a file that never
names X cannot hide evidence about X; lowering everything for one stray
error would bury the signal (a first cut did exactly that, taking all
102 nuwayze findings down for a single unresolved file).
Validated on mirrors of the dogfood repos with a readable SDK (0 analyzer
errors): nuwayze 102 findings identical to the original main, dex 0,
word_alive 1 and tick 1 (the intended riverpod change). The fixture
test adds an unresolved file that constructs NeverDispatched: its
dead-handler finding drops to medium while UnhandledTap, which that file
never names, stays high.
The fingerprint is documented as surviving line shifts, but eight of the
fixture's findings carried a line number, so moving code down a file
turned accepted findings into new ones on the next ratcheted check:
- dormant emitted-not-rendered (keyed by union + owner, neither counted
as identifying, so the line was appended)
- notifier-inert-consumer and notifier-field-unread (owner only)
- loading-not-reset, whose `catchLine` key is a line by another name
`union` and `owner` now count as identifying (each of those rules emits at
most one finding per union or notifier per file), and loading-not-reset
keys on the catch's ordinal within its member. Its `on<E>` handler
closures are also named `on<E>` like loading-flag-not-reset's, so the
ordinal cannot collide across handlers.
The existing move test only bumped `line`, not `catchLine` in the data,
so it passed while real moves did not. fingerprint_test.dart now shifts
every file of the fixture for real and requires (a) every fingerprint to
survive and (b) no two findings to share one. (a) fails on the previous
rules.dart with exactly the eight findings above; (b) passed there too, and
guards the ordinal and closure naming that replace the line.
Baselines written before this change see the affected findings once as
new; re-accept them with --write-baseline. No committed baselines exist
in the dogfood repos.
riverpod-async-gap judged handling from the text of the expression a
`ref.watch` flows into, which missed two common shapes:
- Pattern handling. A switch with `AsyncError()` / `AsyncLoading()` cases
- the form pipe's own remediation recommends - did not count as
handling, and a switch STATEMENT on the watched value was never looked
at, because the parent climb stops at the first non-expression. A
provider handled that way in one widget and read with `.value` in
another was reported as unhandled.
- A value stored in a local. `final v = ref.watch(p); ... v.when(...)`
stopped the climb at the declaration, so both the handling and the read
were invisible: a handled provider could be reported, and an unhandled
one read as `v.value` was missed (a recall gap, not only precision).
The consumer now widens to an enclosing switch statement, the uses of a
declared local are followed through the enclosing body, and
AsyncError/AsyncLoading patterns count as handling.
Fixture: patterned, switched and stored providers (each also read
directly elsewhere, so the controls are not vacuous) were all reported
before and are silent now; storedUnhandledProvider was missed and is now
reported.
Real repos: tick's categoriesStreamProvider is no longer reported; its
categories screen handles it with `catsAsync.when(...)` through a
variable, which the rule's any-site-handles definition counts as handled.
nuwayze and dex unchanged. word_alive's internalUserProvider finding comes
from a commit made to word_alive during this work, not from this change:
the old code reports it too.
freezed declares variants as redirecting factories on the base
(`const factory DocState.loading() = DocLoading;`) and blocs emit through
them. `emit(const DocState.loading())` was recorded as emitting
`DocState`, which is never a variant, so no freezed emission matched and
every variant the UI renders was reported by rendered-not-emitted as a
dead branch.
Factory redirects are now also kept by constructor name, and an instance
creation of `Type.ctor` (or an unnamed redirecting factory) is credited to
the class it redirects to, for emissions, `state =` assignments and
dispatches alike.
Reproduced on freezed_style.dart before the fix: DocLoading, DocReady
and DocStale all reported as never emitted; after, only DocStale, which
really is never emitted and stays as the positive control. No change on
nuwayze, tick, dex or word_alive.
An SDK directory that exists but cannot be read passed discovery, and the
analyzer then died with a raw FileSystemException stack trace. This
happens for real under macOS privacy protection: a Flutter SDK under
~/Documents stats fine but every open fails, and every app whose
package_config.json names that flutterRoot crashed:
Unhandled exception:
FileSystemException(path=.../dart-sdk/lib/_internal/allowed_experiments.json; ...)
The chosen SDK is now probed before the analysis collection is built, and
an unreadable one is a PipeException naming the path, where it was
discovered from, and the --sdk remedy.
The last-resort fallback to the running Dart SDK took the executable's
directory (<sdk>/bin) as the SDK root, which never contains
lib/libraries.json, so it could never succeed. sdkFromExecutable now
maps <sdk>/bin/dart to <sdk>.
Two more ways a partial scan produced a false claim, the same hazard class
as the --limit and unreadable-file cases verify already refuses:
- `verify --rule X` compared the fresh X-only report against the whole
baseline, so every accepted finding from any other rule was printed as
RESOLVED ("these fixes are confirmed") when it had merely not been
checked. The baseline is now filtered to the selected rules.
- `--write-baseline` (check and guide) wrote whatever the run produced.
Under --rule or --limit, or with files that could not be analysed, the
baseline silently omitted every finding outside that subset, and the
next full `check --baseline` reported them all as new. It now refuses:
--rule/--limit before analysis, unreadable files after it.
Each is covered by a test that fails against the previous cli.dart
(subset_scan_test.dart, and the unreadable-file case in
unreadable_test.dart). README documents the refusal.
The method-level check read the whole body text, closures included, for
both the `loading: true` probe and the await probe, while the try
collector skips closures. So a method whose closure set the flag and
guarded its own await was reported at high confidence as unguarded, and a
method that only registers `on<E>` closures was reported alongside each
closure. Every probe now stops at nested functions, which are analysed on
their own; the flag probe is a `field: true` named-argument visitor
instead of a regex over source.
`on<E>` handler closures were all named `<closure>`, so two handlers in one
class shared a fingerprint and encodeBaseline kept only one of them. They
are now named `on<E>`. Existing baselines that accepted a handler as
`<closure>` will see it once as new.
Fixture: scheduleGuarded (guarded closure) and registerHandlers are now
negative controls; two unguarded handlers must yield two findings with
distinct fingerprints. Before: scheduleGuarded, registerHandlers and two
colliding `<closure>` findings; after: on<FlagRefresh>, on<FlagReload>.
Two ways the event rules reported handled events as unhandled (error
severity, high confidence):
1. A handler on the event base or an intermediate group. `on<SyncEvent>`
with a switch inside handles every subclass, but handlers were matched
to dispatches by exact name, so every dispatched subclass was reported
by event-without-handler and the base handler itself by dead-handler.
Variants now record the supertype they were enumerated through, and
coverage follows it: an event is handled when it or an ancestor is
registered, and a handler is live when anything at or below it is
dispatched or constructed. A group handler still does not cover a
sibling outside the group (UploadAbandoned stays reported).
2. Blocs sharing an event type. Only the first owner of an event type was
kept, so a second bloc's handlers were ignored: an event only it handles
was reported unhandled, and its dead handlers were never examined. There
is now one event union per owning bloc. A dispatch does not record which
bloc it targets, so an event any sharing bloc handles counts as handled,
and an unhandled event is reported once per shared union rather than
once per bloc. Dead handlers are still judged per bloc.
Reproduced first on base_event.dart: 4 false findings before
(SyncStarted, UploadRetryNow unhandled; SyncEvent, UploadRetry dead) and
the shared case (ShareDismissed falsely unhandled, ShareBetaBloc's dead
ShareExpired missed). Each is now a contract control.
No change on nuwayze (102 findings) or tick (2): byte-identical before
and after, since neither uses these patterns.
ae412a7 routed filesystem failures to exit 2, but the analyzer throws its own
FileSystemException (package:analyzer/file_system), which the dart:io catch
in bin/pipe.dart never matched. A project whose package_config still named a
moved Flutter SDK therefore crashed with exit 255 and a stack trace. Found
by dogfooding on word_alive after its SDK moved.
Analyzer.create now turns that into a PipeException naming the SDK path, with
the remedy to re-run flutter pub get or pass --sdk.
The markdown report's summary line counted every finding while its heading and
the severity counts beside it counted only new ones, so a fully baselined run
read "20 findings ({error: 0, warning: 0, info: 0})" above "## Findings (0)".
The summary now counts new findings and notes how many are baselined.
Also: loading-flag-not-reset rendered its flag as `SettingsState`loading``, a
missing `.` that produced two adjacent code spans. It now reads
`SettingsState.loading`.
Every brief entry and plan action ended with `verify: pipe check --rule X`.
Pasted, that checks the current directory, over lib/ only, against no
baseline - not the run that produced the finding. Run from anywhere but the
project root, or after a scan with --dir or --baseline, it answered a
different question.
The hint is now built by checkCommand(): `pipe check <root> --rule X`, plus
`--baseline <path>` and every `--dir` when the run used them, with paths
shell-quoted when needed.
Three ways the agent plan claimed more than it knew:
- It opened with "Work them in this order" and nothing else. It now opens with
"Static heuristics; confirm each against the cited file. Order is a score,
not a mandate."
- Under the default wire-up policy a dead handler still scored "no product
judgement needed (+15)". Wiring one up means deciding which control should
dispatch the event, which is product judgement; the bonus now applies only
under --policy remove, and buildWorkPlan takes the policy.
- A grouped action used its first finding's remediation, so a union with five
dead handlers said "dispatch LoadItems from the UI" and nothing about the
other four. A group now gets an action built from all of its targets
("dispatch each of A, B, C ..."), and targets include the variant and member
lists that grouped findings carry.
The brief said "nothing outside the bloc references it" and, under
--policy remove, that removing a dead handler "is safe even without product
context", while the scan defaults to --dir lib. The dead-handler finding
itself was scoped in f0fc57e; these three statements were not. A handler
dispatched only from test/ still read as safe to delete.
The decision line and the dead-handler section now say "nothing under <dirs>"
and tell the reader to rule out unscanned directories (rerun with --dir)
before wiring or removing; the remove-policy remediation in rules.dart says
the same. buildGuidance takes the scanned directories, which the guide
command passes from the project.
`ruleFix` was a constant map that ignored `--policy`. Under the default
wire-up policy the brief's legend still said "Remove the dead handler, or
dispatch the event" and "delete the variant and its emits" as equal choices,
directly above findings whose remediation said removal needs explicit
approval - the legend contradicted the policy the README describes.
ruleFixFor(policy) now returns the legend's fixes for a policy: under wire-up
the choice-bearing rules (emitted-not-rendered, rendered-not-emitted,
event-without-handler, dead-handler, notifier-field-unread) lead with making
the code reachable and say removal needs explicit approval; under remove they
keep offering both. `ruleFix` remains, as the default policy's map.
The plan sorted by score alone. List.sort is not stable, so equal-score
actions came out in an order nothing guaranteed, and two runs over the same
code could hand an agent a different first action. Ties now fall back to file,
line, rule and label, both in the global order and within each file.
With several --dir values, one that did not exist was skipped silently: the
no-files guard only fires when every directory is empty, so
`--dir lib --dir tset` scanned lib and reported as though it had covered what
was asked for. A --dir that does not exist is now an error naming it.
A directory that could not be listed, or a baseline that could not be read or
written, escaped as an uncaught FileSystemException: exit 255 and a stack
trace. Those paths now raise a PipeException with the OS reason (exit 2), and
bin/pipe.dart catches any remaining FileSystemException the same way. The exit
codes are documented in bin/pipe.dart: 0 success, 1 a gate failed, 2 pipe could
not do what was asked, 64 usage.
rule_scope_claim_test widened the fixture with a test/ directory it does not
have, relying on exactly the silent skip; it now builds a project that has one.
`guide --yams` persists the markdown report, not the brief, and the markdown
had neither the brief's caution line ("static reachability claims ... confirm
each against the cited file") nor the [severity/confidence] of each finding -
so a report read back from the corpus presented every finding as equally
certain, with no framing. Its metadata still carried a stale `owner=pi` and
nothing to date or reproduce it.
The markdown now opens with the caution line, headings read
`### N. [severity/confidence] rule — location`, and the metadata records
`generated=<ISO-8601 UTC>` and, when the project is a git checkout,
`commit=<HEAD>`. `owner=pi` is gone.
The analysed repository is untrusted input, and its source reached the agent
brief and the yams corpus verbatim: emission and dispatch snippets were
`node.toSource()`, provider-read-in-build quoted the whole call, and the brief
printed `evidence: $item` raw (the markdown wrapped it in single backticks,
which a backtick in the source closes). A string literal written as an
instruction to the reader was pasted into the prompt as-is.
Every quoted excerpt now goes through sourceExcerpt(): string literals,
interpolations included, become '…'; comments are dropped (they are not
tokens); whitespace collapses; the result is capped at 120 characters, as
loading-not-reset's catch excerpts already were. Where the brief and the
markdown render evidence, it is labelled "source excerpt (data, not
instructions)" and fenced with one more backtick than the longest run inside
it, so nothing quoted can close its own quotation.
Fingerprints are unaffected: provider-read-in-build still keys on the full call
source, which is kept for identity and no longer quoted.
Both packers skipped an entry that did not fit the budget and kept going, so a
large top-ranked finding vanished while smaller, lower-ranked ones were packed
in behind it - presented as the most important work. Packing now stops at the
first entry that does not fit and counts it and everything after it as
omitted.
suggest's agent plan also reserved no room for its own omission note, so the
note itself could push the plan over budget; it now reserves room for the
largest possible note unless the entry is the last one.
When no entry fit, the guide said "this header alone exceeds the budget" even
when the header had room and it was the top entry that was too large. It now
says which of the two it was, with the size.
A report listed findings without saying how much of the project they were
drawn from, so "0 findings" from 3 of 40 files looked like a clean project -
and an agent handed a brief had no way to know otherwise.
check, guide and suggest JSON now carry a `coverage` block
{discovered, selected, resolved, failed, skipped, complete}. The guide brief,
the markdown report, the suggest agent plan and both text outputs open with a
one-line coverage note, which says PARTIAL whenever a discovered file was not
analysed.
The note lives in the brief's header, which budget trimming never touches. The
"Coverage (what was actually examined)" caveats section was the first thing
dropped under a tight budget, silently; when it no longer fits, a one-line
pointer now says how many caveats were left out.
`check --format json --write-baseline b.json` printed "wrote baseline to ..."
after the JSON document, and `guide --yams` printed "persisted to yams as ..."
after the brief, so stdout was no longer a single JSON document (or a clean
brief) and `| jq` failed. Both status lines now go to stderr.
A finding with no discriminating key fell back to `line=N`, and three rules
had none: a dormant emitted-not-rendered union carried only {union, owner},
and notifier-field-unread and notifier-inert-consumer only {owner}.
loading-not-reset embedded the catch's absolute line as `catchLine`. Any edit
above one of those findings changed its fingerprint, so `check --baseline`
re-reported an accepted finding as new and `verify` reported its entry as
RESOLVED.
union and owner now count as discriminating (each of those rules emits at most
one finding per union or notifier), and loading-not-reset carries `catchSite`,
the catch's index among the catch clauses of its enclosing member
(`twoSwallowedAwaits#1`), which does not move when code outside the member
does.
The test that guarded this shifted only findings.first. It now shifts every
finding, and a second test really inserts lines at the top of every fixture
file, re-analyses, and requires the same fingerprint set.
Migration: fingerprints change for emitted-not-rendered (dormant unions),
notifier-field-unread, notifier-inert-consumer and loading-not-reset. Old
baselines stay readable and keep working - both `check --baseline` and
`verify` also match the version-1 fingerprint (`legacyFingerprint`) - but
those entries stay line-sensitive until the baseline is rewritten with
--write-baseline, which now writes `"version": 2`.
The analyzer skips a file that analysis_options `exclude:` names (contextFor
throws, which pipe records as `skipped`). The verify gate refused a scan only
when files FAILED, and partialScanWarning named only failures, so a skipped
file's accepted findings were simply absent from the fresh report - and verify
listed them as RESOLVED, "these fixes are confirmed", for code nobody read.
Excluded files are a normal part of a project, so they do not abort verify.
Instead each accepted finding whose file was not analysed - skipped, failed,
filtered out by --no-include-generated, or outside every scanned --dir - is
reported as NOT SCANNED: neither resolved nor still present. allResolved (and
so --expect-resolved) now requires that list to be empty too.
partialScanWarning counts skipped files, so a scan with excluded files is
announced as PARTIAL on stderr like one with failures.
`verify --rule X` compared the fresh report against the whole baseline. Every
accepted finding of every other rule was absent from the report, because those
rules never ran, so verify listed them as RESOLVED - "these fixes are
confirmed" for fixes nobody made.
Baseline entries are now filtered to the enabled rules before comparison. An
entry's rule is its `rule` field or, for a fingerprints-only baseline, the
fingerprint's first `|` segment. Entries set aside are counted as `outOfScope`
in JSON and named in the text summary, so the filter is visible rather than
silent.
The finding said "nothing anywhere ever dispatches or constructs X" while the
analysis only ever scanned --dir (default lib). On a real app that overclaim was
false for 67 of its 77 findings: their events are constructed by TEST code, which
pipe never read. The claim, not the count, was the bug:
$ pipe check . --rule dead-handler # 77 findings
$ pipe check . --dir lib --dir test --rule dead-handler # 10 findings
67 features were one reading away from being "wired up" as missing UI, and the
same overclaim nearly justified deleting a handler covered by five seeded fuzz
tests - pipe said nothing anywhere constructed the event, because it never looked
in test/.
The message now names the scope it actually searched ("nothing under lib ever
dispatches or constructs X"), `Project` records the scanned directories so any
finding can do the same, and the remediation says to check for a dispatch in a
directory that was not scanned before wiring, since test-only handlers are not
missing features.
Found while auditing 77 "dead handlers" for a UI-wiring task: the first deletion
candidate turned out to be referenced by test/blocs/.../artist_dashboard_orders_fuzz_test.dart.
112 tests (was 109), dart analyze clean.
The previous commit over-corrected. It skipped EVERY case for a repo that had an
unreadable file, so a repo with one unreadable file silently dropped the
regression guard for all of its cases - a worse failure than the false MISSED it
replaced, because a skipped case guards nothing:
$ pipe recall --repo-with-one-unreadable-file
pipe: recall evaluated none of its 2 case(s): ... # both cases lost
Only a NEGATIVE verdict is unsafe when files could not be read. A hit stands,
because evidence that was actually read is evidence; a miss becomes "unknown"
rather than a false regression. So:
recall 1/1 known bugs accounted for (1 skipped)
by rule {dead-handler: 1/1}
skipped: evidence-in-UNREADABLE-item_bloc: 1 file(s) could not be analysed in /repo
The case with readable evidence is still evaluated and still found; only the case
whose evidence lives in the unreadable file is skipped, with the reason.
Also hardened, from the same round's probes: the "analysed nothing" guard tested
only `resolved == 0 && failed > 0`, so a run where every file was SKIPPED
(outside any analysis context) could still report success having analysed
nothing. It is not reachable through --dir (a path that does not resolve under
the root now hits the no-files guard first), but it was open by construction, so
the test is now simply "nothing was resolved".
Probes that came back clean this round, so they are recorded rather than
re-tested: all six commands on a healthy project produce ZERO bytes on stderr and
the same 20 findings (no false positives from the last three fixes); `contexts 0`
is not reachable; and a baseline written from a partially readable project does
warn on stderr at write time, which is the mitigation for the reverse
cross-baseline direction (`new 6 not in the baseline` is then factually correct
rather than a spurious regression claim).
109 tests (was 108), dart analyze clean.
The all-unreadable case was fixed last commit; the PARTIAL case still produced
two false claims.
1. `verify` reported unread findings as fixes. With two unreadable files it
printed:
baseline 20 accepted findings
resolved 8 no longer reported — these fixes are confirmed
RESOLVED
+ lib/item_bloc.dart:48 LoadItems [dead-handler]
item_bloc.dart was one of the files it could not read. Those 8 findings were
absent from the fresh report because they were never read, not because they
were fixed. `verify` now refuses a scan in which any file failed, mirroring
the --limit guard: a claim about someone's fixes must rest on a complete scan.
2. `recall` reported such a case as MISSED, a false regression claim that also
drove `--strict` to exit non-zero. A case whose evidence lives in a file that
could not be read cannot be judged either way, so it is now recorded as
skipped with the reason, exactly like a repo that is not present:
"1 file(s) could not be analysed in /repo".
3. Consequential accuracy fix: the vacuous-run guard added two commits ago said
"no repo was present under <root>" whenever nothing was evaluated. With the
new skip reason that diagnosis was wrong - the repo WAS present, its files
were unreadable - and the remedy pointed at the wrong problem. The message now
reports the actual skip reasons.
Verified: verify exits 2 on a partially readable project and still works on a
fully readable one; recall skips the unreadable case, keeps evaluating the
others, and reports no MISSED. The normal fixture is unaffected. 108 tests (was
106); the new tests fail without these changes (+0 -2) and skip honestly if
chmod cannot restrict the process.
Found by trying to FALSIFY the claim that the silent-success family was closed.
It was not - for the second iteration running, the falsification attempt found a
real defect.
An unreadable file is handed back by the analyzer as a ResolvedUnitResult with
EMPTY content and zero declarations - not an error, not a failure. pipe counted
that as a successful resolution, so a project whose files could not be read
produced:
$ pipe index /tmp/unreadable
files 12/12 resolved (12 discovered, 0 generated, 0 skipped, 0 failed)
$ pipe check /tmp/unreadable
findings 0 exit 0
Verified directly: the same 12 files give `contentLen=0 decls=0` when
unreadable and `contentLen=722 decls=2` when readable, with `failure=null` in
both cases. So `check` reported a clean bill of health for a project it had
extracted nothing from - the same silent-success family as the empty scan and
the truncated scan.
Two changes, both at points where the information exists exactly once:
- the resolution loop detects a unit with empty content for a file that has bytes
on disk (a genuinely empty file is still fine) and records it as a failure, so
`filesFailed` becomes reachable and `index` accounts for it:
`11/12 resolved (12 discovered, 0 generated, 0 skipped, 1 failed)`;
- the outcome summary warns when some files failed and REFUSES when every file
failed, so a partial result can never be mistaken for a complete one:
pipe: WARNING 1 file(s) could not be analysed (11 resolved), so results are PARTIAL.
pipe: no file could be analysed: all 12 file(s) failed to resolve
The normal fixture is byte-for-byte unaffected (no warning, same 20 findings),
so this adds no false positive on readable projects.
Found by probing whether an all-files-fail run was reachable; a chmod-000 corpus
made it reachable and exposed that it had been silently scored as success.
106 tests (was 104), dart analyze clean.
The committed versions were not in canonical format; they were reformatted after
the commit, which would have made the pre-commit hook rewrite them on the next
commit and block it.
Continuing the "where else can pipe report success without doing the work?"
line, which produced the --limit false verification last iteration.
1. The empty-scan guard from 51929e2 had a hole at a different layer. It fires in
loadProject, which counts DISCOVERED files, so a project whose only source is
generated still had one file:
$ pipe check . --no-include-generated
files 0/0 resolved (1 discovered, 0 selected, 1 generated, ...)
findings 0 exit 0
Files were discovered, then every one was filtered out of the selection, and
the analysis ran over nothing. The guard now also fires where selection
happens - the single place a file can be dropped - naming the cause:
"nothing to analyse: all 1 discovered file(s) were excluded by
--no-include-generated".
2. `recall` reported "0/0 known bugs accounted for" and exited 0 - including
under --strict - when it had verified nothing: an empty manifest, or a --root
under which none of the repos exist. "0/0 accounted for" reads as a pass, so
both are now failures that name why:
recall evaluated none of its 5 case(s): no repo was present under /corpus
recall manifest has no cases: recall/manifest.json
The per-case `skipped:` lines are kept; this only changes the case where there
is no result at all to report.
Verified: the two new tests fail without these changes and pass with them; a
healthy recall run still reports 3/5 with its notes, a project with generated
files included still exits 0, and `check --baseline <unrelated>` is still clearly
distinguishable from a clean project.
Also probed and CLEAN, so they are not re-tested blindly: `--budget 1` discloses
omissions in both text and JSON ("All 20 findings omitted ... Raise --budget"),
and a baseline matching nothing prints `findings 20 new=20` against a clean
project's `findings 0 new=0`.
104 tests (was 102), dart analyze clean.
`--limit` is documented as a debugging aid, but a truncated analysis looked
exactly like a complete one, and `verify` interpreted the truncation as work that
had been done:
$ pipe verify . --baseline b.json --limit 2
baseline 20 accepted findings
resolved 19 no longer reported — these fixes are confirmed
still 1 still reported
RESOLVED
19 findings were reported as FIXED because they were never scanned. That is a
false verification of fixes nobody made - the worst output this tool can produce,
because someone could use it to claim a defect was repaired.
Two changes:
- `verify` refuses a partial scan outright: "verify needs a complete scan: with
--limit, findings that were never scanned are reported as resolved", with a
remedy pointing at `check --limit` for debugging.
- Every other command now warns, because a truncated `check --limit 2` reporting
1 finding instead of 20 was equally mistakable for a clean result:
pipe: WARNING --limit 2 is a debugging aid: 2 of 12 file(s) were analysed,
so every result below is PARTIAL.
The warning is emitted at the single place truncation can happen (the file
selection in resolveProject) rather than in each command, so a future command
cannot forget it - forgetting to wire something up is precisely the failure mode
this whole iteration kept finding.
Found by asking whether any other path could report success without doing the
work, after the empty-scan fix. 102 tests (was 100), dart analyze clean.
Analysing zero files exited 0 with no message, so the tool passed while having
checked nothing:
$ pipe check . --dir libs # typo: analysed 0 files, exit 0
$ pipe check . --dir <empty dir> # 0 files, exit 0
That is the same defect class as a test that cannot fail, at the level of the
whole tool: a CI job running `pipe check` "passed" having inspected no code at
all. A scan that finds no Dart files is now an actionable error:
pipe: no Dart files found under libs in /project; pass --dir to choose which
directories to scan
This also covers the nonexistent `--dir` case, which was previously skipped
silently by the directory walk.
Second defect: `--no-include-generated` printed
files 12/12 resolved (13 discovered, 1 generated, 0 skipped, 0 failed)
The 13-versus-12 gap could not be reconciled from the line - it read as though a
file had vanished or failed silently. `filesSelected` existed in the model and in
the JSON all along; the text line simply omitted it. It now prints `12 selected`
whenever it differs from the discovered count, so the arithmetic resolves, and
stays quiet when nothing was filtered.
Found by running the remaining unexercised flags (`--dir`, `--include-generated`)
end to end, as the execution-first pass did for the rest.
100 tests (was 95), dart analyze clean.
An execution sweep comparing README flags against what each command accepts found
no documented flag that the CLI rejects, but five working flags the README never
mentioned: index --fail-on-error, suggest --max-actions/--budget,
guide --max-findings/--yams-name, recall --manifest. Each is now described with
its verified behaviour (e.g. index exits 1 with analyzer errors, 0 without), plus
a note that numeric flags reject non-positive-integer values instead of falling
back to a default.
