pipe/docs/queries.md

# Firestore query inventory

`pipe queries` lists every Firestore query the scanned code can issue, as
typed, machine-readable records. It is an **inventory only**: no rules, no
findings, no baseline, and deliberately no index analysis. Deciding whether a
query needs a composite index, and probing a live project, belong to the
consumer of this output, which can compare it against `firestore.indexes.json`.

The path may be a package root or a directory inside one: `pipe queries lib/data`
resolves the package that contains `lib/data` (the nearest `pubspec.yaml`) and
scans only that directory, as `--dir lib/data` would. `--dir` given with such a
path is relative to it.

What it recognises:

- **cloud_firestore chains.** Receivers are identified by **static type and
  library**: an `InterfaceType` named `FirebaseFirestore`, `CollectionReference`,
  `Query`, `DocumentReference`, `AggregateQuery`, `Filter` or `FieldPath` whose
  library URI starts with `package:cloud_firestore/` or
  `package:cloud_firestore_platform_interface/`. Nothing is matched by name, so
  `Iterable.where`/`take`, or a project class called `Query`, never is.
- **One record per terminal call**: `get()` (`realtime: false`), `snapshots()`
  (`realtime: true`), `count()` / `aggregate(...)` (`aggregate: count|sum|average`)
  on a `Query` or `CollectionReference`. `DocumentReference.get/snapshots` are
  point reads that need no index: they are counted in `totals.documentReads`
  and never listed.
- `collection(path)`, `collectionGroup(id)` and `doc(...)` segments become
  `scope`, `collection` (the last collection id) and a `path` pattern such as
  `stores/*/products`. Document ids are expected to be dynamic and cost
  nothing; a dynamic **collection** id makes the record `partial`, and a fully
  dynamic one also sets `collection: null`.
- `where(...)` with every named operator (`isEqualTo` `==`, `isNotEqualTo` `!=`,
  `isLessThan` `<`, `isLessThanOrEqualTo` `<=`, `isGreaterThan` `>`,
  `isGreaterThanOrEqualTo` `>=`, `arrayContains` `array-contains`,
  `arrayContainsAny` `array-contains-any`, `whereIn` `in`, `whereNotIn`
  `not-in`, `isNull: true` `==` null, `isNull: false` `!=` null); a field may be
  a string, `FieldPath([...])` (dotted) or `FieldPath.documentId` (`__name__`).
  `where(Filter.or(...))` / `Filter.and(...)` become a `composite` node.
- `orderBy(field, descending:)`, `limit`, `limitToLast` (`limit.last: true`),
  and any cursor (`startAt`, `startAfter`, `endAt`, `endBefore` and their
  `...Document` forms) as `cursor: true`. `withConverter` passes through.
- Strings resolve through literals, adjacent and interpolated strings (`*` for
  each unknown part), `+`, `const` and `final` variables with a literal
  initializer, and one-return path helpers (`storePath(id) => 'stores/$id'`)
  with their arguments bound.
- **Built queries.** A local `Query` variable is the union of its definitions in
  its function body; a clause added under an `if`, ternary, `switch`, loop,
  `catch` or closure is `conditional: true`. Two cuts keep that union honest:
  an assignment textually after the terminal is ignored unless both sit in one
  loop, and an `if` that encloses both the assignment and the terminal does not
  make the clause conditional. Fields are followed through their initializer,
  constructor initializers and assignments in their class.
- **Helpers.** A getter, method or function in the project that returns a query
  is summarised and inlined at its call sites with the call's arguments bound.
  Methods are inlined **one level deep**; getters and fields take no arguments
  and do not count against that. A query that arrives as a parameter, or from a
  call pipe cannot follow, is `origin: unknown`, `confidence: unknown`, with
  the clauses added locally still recorded.
- `inLoop: true` when the terminal is inside a `for`/`for-in`/`while`/`do` body
  or a collection `for`, or inside a closure passed to `Iterable.map`/
  `forEach`/`expand`, `Stream.asyncMap`, or within `Future.wait(...)`. A fixed
  `Future.wait([a.get(), b.get()])` is not a loop.

## Schema

```json
{
  "version": 1,
  "root": "relative path, or the directory name when outside the cwd",
  "coverage": {"discovered": 827, "selected": 827, "resolved": 825, "failed": 0, "skipped": 2, "complete": false},
  "totals": {
    "queries": 372,
    "byConfidence": {"exact": 336, "partial": 28, "unknown": 8},
    "byScope": {"collection": 297, "collectionGroup": 66, "unknown": 9},
    "byOrigin": {"chain": 189, "helper": 47, "unknown": 8, "variable": 128},
    "realtime": 28, "inLoop": 37, "documentReads": 165,
    "collections": {"bookings": 23, "...": 0}
  },
  "unresolvedFirestoreUris": [{"site": "lib/a.dart:3", "uri": "package:cloud_firestore/cloud_firestore.dart"}],
  "notes": ["why the inventory may be incomplete; never empty when queries is"],
  "queries": [{
    "site": "lib/services/category_service.dart:23",
    "owner": "CategoryService", "member": "getCategories",
    "scope": "collection", "collection": "categories", "path": "categories",
    "filters": [{"field": "isActive", "op": "==", "conditional": false, "valueType": "bool"}],
    "orderBy": [{"field": "name", "dir": "asc", "conditional": false}],
    "limit": {"kind": "none", "conditional": false},
    "cursor": false, "realtime": false, "inLoop": false,
    "confidence": "exact", "origin": "chain", "reasons": []
  }]
}
```

- `filters[].valueType`: `string`, `int`, `double`, `num`, `bool`, `timestamp`
  (`DateTime` or `Timestamp`), `null`, `list`, `map`, `reference`, or
  `unknown`; `inCount` is the length of a literal list operand.
- `composite` (optional): `{op: or|and, filters: [filter | {composite}]}`.
- `limit`: `{kind: literal|variable|none, value?, conditional, last?}`.
- `orderBy[].dir`: `asc`, `desc`, or `unknown` when `descending:` is not a
  constant.
- `confidence`: `exact`, `partial` (the source is known but something is not:
  every cause is in `reasons`), or `unknown` (the query's source could not be
  followed). `origin`: `chain`, `variable`, `helper`, `unknown`, `adapter`.
- `queries` is sorted by site. Status lines and notes go to stderr, so stdout
  is exactly the document. `--format md` renders the same data grouped by
  collection, with the coverage line first.

**Injection policy.** The analysed repository is untrusted (see
[agent briefs](usage.md#agent-briefs)). The inventory never quotes source: every
collection, path and field name must match `^[A-Za-z0-9_.*/\-]+$` or it is
replaced with `*` and a reason is added; owners and members are identifiers.

**Unresolved packages.** If `cloud_firestore` is not in the package graph, or
an import of it does not resolve, the output says so in `notes` (and
`unresolvedFirestoreUris`) rather than presenting an empty list as a clean
result; `--require-firestore` turns that into exit 2.

## Adapter config

Server-side code often wraps Firestore in a declarative API that takes a query
as data, so there is no chain to follow. `--adapter-config` (YAML, or JSON,
which is a subset of YAML) tells pipe what such an API's arguments mean. The
resulting records use the same schema with `origin: adapter`. For a gateway
like this:

```dart
abstract class FirestoreGateway {
  Future<List<QueriedDoc>> queryCollection(String collectionPath,
      {List<WhereEquals> filters = const [], WhereArrayContains? arrayContains,
       List<WhereRange> ranges = const [], List<OrderBy> orderBy = const [],
       String? startAfterId, int? limit});
  Future<List<QueriedDoc>> queryCollectionGroup(String collectionId,
      {List<WhereEquals> filters = const [], WhereIn? inFilter,
       WhereArrayContainsAny? arrayContainsAny, List<WhereRange> ranges = const [],
       List<OrderBy> orderBy = const [], int? limit});
}
abstract class FirestoreTransaction {
  Future<List<QueriedDoc>> queryCollection(String collectionPath,
      {List<WhereEquals> filters = const [], int? limit});
}
// WhereEquals(field, value), WhereIn(field, values), WhereArrayContains(field, value),
// WhereArrayContainsAny(field, values), WhereRange(field, {gte, gt, lte, lt}),
// OrderBy(field, {descending = false})
```

the config is:

```json
{
  "adapters": [{
    "interface": ["FirestoreGateway", "FirestoreTransaction"],
    "library": "package:nuwayze_functions_dart/src/data/firestore_gateway.dart",
    "methods": {
      "queryCollection": {"scope": "collection", "path": 0},
      "queryCollectionGroup": {"scope": "collectionGroup", "path": 0}
    },
    "args": {
      "filters": "filters", "arrayContains": "filters", "arrayContainsAny": "filters",
      "inFilter": "filters", "ranges": "filters",
      "orderBy": "orderBy", "limit": "limit", "startAfterId": "cursor"
    },
    "types": {
      "WhereEquals": {"field": 0, "op": "==", "value": 1},
      "WhereIn": {"field": 0, "op": "in", "value": 1},
      "WhereArrayContains": {"field": 0, "op": "array-contains", "value": 1},
      "WhereArrayContainsAny": {"field": 0, "op": "array-contains-any", "value": 1},
      "WhereRange": {"field": 0, "ops": {"gte": ">=", "gt": ">", "lte": "<=", "lt": "<"}},
      "OrderBy": {"kind": "orderBy", "field": 0, "descending": "descending"}
    }
  }]
}
```

- `interface` (a name or a list) + `library`: a call matches when the invoked
  method's enclosing class is one of these interfaces from that library, or a
  class implementing one. Names alone never match.
- `methods.<name>`: `scope` (`collection` | `collectionGroup`), `path` (the
  argument holding the path or group id), optional `realtime: true`.
- `args.<named argument>`: a role - `filters` and `orderBy` (a value object, a
  list literal with `if`/`for`/spread elements, a ternary, or a local list built
  with `add`/`addAll`; elements under an `if` are conditional), `limit`, or
  `cursor` (any non-null value).
- `types.<class>`: how to read one value object, matched by class name and
  library (the adapter's, or `library:`). `field` and `value` are a positional
  index or a named argument. The operator is either fixed (`op`), one per
  present named bound (`ops`), or read from an argument (`opArg`) and mapped
  through `opMap`, keyed by enum constant name or string value
  (`{"opArg": 1, "opMap": {"lessThan": "<", "greaterThan": ">"}}`).
  `kind: orderBy` with `descending` reads an ordering instead of a filter.
- Every operator must be one of `==`, `!=`, `<`, `<=`, `>`, `>=`,
  `array-contains`, `array-contains-any`, `in`, `not-in`; a config that names
  anything else, an unknown role, or a bad scope is refused with exit 2.

## Limits

- Flow-insensitive within a function, apart from the two cuts above: a clause
  on a path that returns early is still in the union (as conditional).
- Helper methods are inlined one level deep. A query-consuming wrapper - a
  method that takes a `Query` parameter and calls `get()` itself - is reported
  at its own terminal as `unknown`; the clauses its callers pass in are not
  attributed to it.
- A collection name taken from a loop variable over a constant list, a map
  lookup or a runtime value is `*`, with the record `partial`.
- No index analysis, by design: the output carries what an index check needs
  (scope, collection group, equality vs. range vs. array operators, order
  directions, `in` sizes) and leaves the decision to the consumer.

## Calibration

Calibrated on the NuWayze app (Flutter, `lib/`, 827 files) and its Dart backend
(`functions-dart`, 147 files, with the config above), 2026-09-23:

| package | extracted | independent count | exact / partial / unknown |
| --- | --- | --- | --- |
| app | 372 query terminals + 165 document reads | 372 `get`/`snapshots`/`count`/`aggregate` calls whose receiver resolves to `Query`/`CollectionReference` (336 + 36) | 336 / 28 / 8 |
| backend | 136 adapter calls | 136 `queryCollection(`/`queryCollectionGroup(` call sites by grep | 121 / 15 / 0 |

Every `partial` is a genuinely dynamic collection or field name; the 8
`unknown` are query-consuming wrappers (a `Query` parameter, or a wrapper that
applies caller-supplied builder closures). Zero false positives were found:
records exist only where the receiver type resolves into cloud_firestore.