The adjudicator printed "controls FAILED" or "NO SIGNAL" and then exited
0, so a CI step reading the exit status took an uncertified run as a
pass. It now exits 1 unless the controls are meaningful and pass, in both
text and json output.
It was also described as independent. Its evidence is independent (its
own regex over source), but the list of unions, variants, events and
owners it checks comes from pipe's `state` output, so a variant pipe
fails to enumerate is never checked. The docstring and README now say so
instead of "independent adjudicator".
The independent precision adjudicator reported a pipe defect that did not exist.
Three defects in the checker, all found by chasing its own output against the
source:
1. It matched variant names by UNQUALIFIED name, so `is ExportSuccess` in
analytics_tab.dart was credited to ArtistStoreManagementState.ExportSuccess -
the very defect pipe's own per-union scoping had just fixed. Those three files
reference AdminAnalyticsState, ContentModerationState and
HelpCenterManagementState, and none of them mentions
ArtistStoreManagementState, so pipe's finding was correct and the checker's
"contradiction" was false. Evidence about a variant now requires a file that
also names the union or its owner - unless the name is declared by only one
union, in which case it is unambiguous and needs no scoping (event
construction sites carry no type annotation, and demanding one rejected sound
evidence: the dispatched control fell from 162/162 to 143/162).
2. Its notion of "rendered" was type-tests only, missing switch-expression arms
(`ChatListOperationInProgress value =>` has no `case` keyword in Dart 3) and
base-class payload reads (`state.upcomingBookings`, where BookingsLoaded
passes `required super.upcomingBookings`). Both are real ways a variant reaches
the screen; pipe models both.
3. `controls_pass` was `len(control1) == len(rendered) and ...`, which on a
corpus with no control material is `0 == 0` - a vacuous pass. word_alive
printed "controls pass: True" with 0/0 and a pristine-looking 100%. The control
is now three-way (found / unattributable / missing), only `missing` fails, and
an empty corpus exits NO SIGNAL with precision reported as UNDEFINED rather
than perfect.
Also fixed: the scoping pass looked for union names in `all_names`, which holds
variant names only, so `union in file_ids` could never be true and every control
silently read 0.
Result on clean nuwayze main: controls 131/131 rendered, 143/162 dispatched (19
unattributable, 0 missing) - controls pass, and precision is 79/79 + 77/77 with
zero contradictions. No pipe defect was found this round; the adjudicator's one
apparent finding was its own.
84 tests still pass; dart analyze clean.
An independent Gemini 3.8 Flash audit of this repo found real problems. Each was
reproduced before being fixed.
Confirmed defects:
- tool/adjudicate.py crashed with KeyError once R001 started grouping variants
under 'variants' instead of 'variant'. Fixing that exposed a second bug of my
own: it then divided verified VARIANTS by FINDINGS and reported 216% precision.
Counts are now coherent (findings vs variant-level checks) and a clean repo
reports 0/0 rather than a fake 100%.
- GuideCommand called _readPolicy but never registered --policy, so
exited 64 despite the README documenting it.
- loading-flag-not-reset treated a constructor-invoked method as inheriting the
loading flag without checking flag.defaultsTrue, so a default-FALSE flag was
reported as in flight.
- Finding.fingerprint embedded the line number for rules whose only identity was
, so moving code broke baseline entries. is now a
discriminating key; only loading-not-reset, which can emit two findings from one
method, carries an explicit catchLine.
- suggest grouped unrelated provider-read-in-build findings by file; the group key,
label and target list now use valueType/call.
Recall defect found while checking the auditor's own claim: UI variant mentions
were collected project-wide by unqualified name, so ExportSuccess - which appears
in FOUR unions on nuwayze - was cross-credited. Mentions are now scoped per union
to files that also name the state type or its owner, which surfaced
ArtistStoreManagementState[ExportSuccess] as a real finding.
Five negative controls could not fail (an event that is never handled can never be
reported as unhandled; a union with no render sites can never be reported as
rendered-not-emitted). Added fixture cases that make each one bite: HandledTap
(handled AND dispatched), PatternState render sites, GuardedCubit (guarded loading
path), QuietCubit._warmUp (constructor-invoked await on a default-false flag).
Root cause of a recurring mystery: this repo EXCLUDED test/fixtures/** from
analysis, which made any analyzer running in the repo context parse the fixture
without resolving its imports and report phantom undefined-class errors. I had
dismissed these as stale cache several times; they were my own exclusion. The
exclude is gone and the fixture governs itself. A new test asserts the fixture has
zero analyzer errors, not merely that it resolves, so IR distortion cannot hide.
84 tests (was 81).
