446 lines
19 KiB
Markdown
446 lines
19 KiB
Markdown
# fold CLI Design Philosophy
|
||
|
||
## Purpose
|
||
|
||
This document defines the required design methodology for the `fold`
|
||
CLI. It exists to prevent command sprawl, flattening of unrelated
|
||
concerns, weak help output, and error messages that leave users or
|
||
agents at dead ends.
|
||
|
||
This is not a wishlist. Any change to CLI commands, help text,
|
||
aliases, examples, error copy, or user flow is reviewed against this
|
||
document.
|
||
|
||
This is a *policy* document. Specific contributor TODO items live in
|
||
plan files; this document defines how future CLI changes are designed
|
||
and reviewed.
|
||
|
||
## Goals
|
||
|
||
- Keep the CLI learnable without memorizing examples.
|
||
- Make the command graph reflect fold's real product mental model:
|
||
scan code → triage findings → confirm or fuzz promising ones →
|
||
export a report.
|
||
- Prefer layered workflows over oversized command groups.
|
||
- Make non-interactive and agent use first-class — fold is invoked by
|
||
CI runners, IDE plugins, MCP clients, and worker fanouts more often
|
||
than by humans at a prompt.
|
||
- Ensure every failure teaches the next valid action.
|
||
- Keep one canonical path per user intent.
|
||
- Favor recognition over recall: users and agents discover valid next
|
||
actions from names, help, and machine metadata instead of memorized
|
||
examples.
|
||
|
||
## fold's Workflow Stages and Canonical Verbs
|
||
|
||
fold's product shape is a four-stage scan-and-promote pipeline. The
|
||
verb table below is the canonical vocabulary; CLI changes that
|
||
introduce verbs outside this table need an explicit justification.
|
||
|
||
| Stage | Verb | Use |
|
||
|--------------------------------|-------------------------------|---------------------------------------------------------------------------|
|
||
| Discover | `scan` | Walk code (source / binary / Rust), run detectors, emit Findings |
|
||
| Inspect (future) | `show` | Read one Finding's evidence, counterexample, related summaries |
|
||
| Promote to Proved | `confirm` | Re-prove a SAT Finding with deeper bounds, record verification metadata |
|
||
| Promote to ConcreteWitness | `fuzz prep` + `fuzz run` | Emit libFuzzer harness + Z3-derived corpus → run under ASan → ingest crash |
|
||
| Export (future) | `report` | Combine baseline + new Findings; emit SARIF / markdown / JSON |
|
||
| Long-lived integration servers | `serve mcp` / `serve lsp` (future) | Run the JSON-RPC stdio MCP server or the LSP server |
|
||
| Internal | (hidden) `worker` | Subprocess fanout target; not user-facing |
|
||
|
||
Notes:
|
||
|
||
- `fuzz prep` and `fuzz run` are the two stages of one promotion.
|
||
Grouping them under `fuzz` makes the order obvious; flat
|
||
`fuzz-prep` / `fuzz-run` at the root hides the relationship.
|
||
- `serve` is the canonical home for long-running integration
|
||
endpoints. Until the Phase-2 rename lands, top-level `mcp` and
|
||
`lsp` remain — call sites and IDE configs depend on those paths.
|
||
- Do not introduce synonyms (`get`, `fetch`, `view`, `describe`,
|
||
`analyze`, `find`) when one of the above already names the stage.
|
||
|
||
## Design Principles
|
||
|
||
### 1. One canonical path per intent
|
||
|
||
Each user intent has one primary command path. Aliases may exist for
|
||
migration, but they do not compete with the canonical command in help
|
||
or docs. Hidden aliases are acceptable for backwards compatibility.
|
||
|
||
New features are introduced under their canonical path, not behind an
|
||
alias.
|
||
|
||
Review question: if a user asks "what command do I run for X?", is
|
||
there exactly one preferred answer?
|
||
|
||
### 2. Layer commands by workflow stage
|
||
|
||
Top-level commands represent stable domains, not implementation
|
||
detail. Within a domain, subcommands follow the lifecycle order:
|
||
|
||
1. Discover what exists
|
||
2. Inspect details
|
||
3. Promote / confirm / mutate
|
||
4. Export
|
||
|
||
If a command family needs many examples to explain itself, the
|
||
command graph is carrying too much at one level — split the group
|
||
instead of adding more help prose.
|
||
|
||
### 3. Separate scan-time discovery from promote-time confirmation
|
||
|
||
fold's two-stage shape is a load-bearing distinction:
|
||
|
||
- `scan` is fast, broad, and over-approximate. Findings can be any
|
||
confidence tier; many are Heuristic.
|
||
- `confirm` and `fuzz prep` / `fuzz run` are slow, narrow, and
|
||
confidence-promoting. They take an existing Finding (or set of
|
||
Findings) and produce a higher-confidence one.
|
||
|
||
CLI surfaces never blur these. A flag on `scan` does not silently
|
||
trigger confirmation. A flag on `confirm` does not silently re-run
|
||
the broad scan.
|
||
|
||
### 4. Help teaches the graph, not compensates for it
|
||
|
||
`--help` is enough to discover the next layer. Examples reinforce the
|
||
command structure rather than acting as the only way to understand
|
||
it. Per-flag descriptions distinguish neighboring commands.
|
||
|
||
Required behavior:
|
||
|
||
- Every command group makes its immediate next options obvious.
|
||
- Help for sibling commands explains why each exists.
|
||
- Root help explains top-level domains.
|
||
- Group help shows the lifecycle order inside that subtree.
|
||
- Leaf help includes preconditions, output flags, and at most a few
|
||
examples that can be run against the built CLI.
|
||
- Long-running or mutating leaf help names the safe dry-run /
|
||
inspection command when one exists (`scan --paths foo.c -f json`
|
||
before `fuzz run`, for example).
|
||
|
||
### 5. Errors must be guiding, actionable, deterministic
|
||
|
||
Every user-facing runtime error does three things:
|
||
|
||
1. State what failed.
|
||
2. State why it failed, if known.
|
||
3. Give 1-2 exact next commands to recover.
|
||
|
||
Good error shape:
|
||
|
||
```
|
||
binary mode requires --binary <path>.
|
||
Try:
|
||
fold scan --mode=binary --binary path/to/exe
|
||
fold scan path/to.c # source scan instead
|
||
```
|
||
|
||
Recoverable runtime errors use a stable shape; long-term, include a
|
||
stable `error[code]:` token so scripts and agents can switch on it:
|
||
|
||
```
|
||
error[no_compile_db]: No compilation database found and no --paths supplied.
|
||
Try:
|
||
fold scan --compile-db build/compile_commands.json
|
||
fold scan src/foo.c # synthesize from a single file
|
||
Retry: safe after the precondition is satisfied.
|
||
```
|
||
|
||
Rules:
|
||
|
||
- Do not emit vague failures like "not found", "invalid input", or
|
||
"unexpected response" without context.
|
||
- If valid candidates are known (detector names, file paths,
|
||
configuration keys), print them.
|
||
- Keep stdout reserved for requested output; write diagnostics and
|
||
recovery guidance to stderr unless JSON output explicitly includes
|
||
an `error` object.
|
||
- State whether retry is safe.
|
||
|
||
### 6. Validate structured input at parse time
|
||
|
||
Prefer typed enums and CLI11 validators (`CLI::IsMember`) over
|
||
free-form strings. Invalid modes, formats, detector names, search
|
||
scopes, and similar structured inputs fail during argument parsing,
|
||
not downstream.
|
||
|
||
Invalid invocations return a non-zero exit status. fold does not
|
||
accept invalid input and then print a warning while exiting
|
||
successfully.
|
||
|
||
### 7. Non-interactive behavior is a first-class requirement
|
||
|
||
fold's primary callers are CI runners, MCP clients, IDE plugins,
|
||
hooks, and worker subprocesses. Interactive use is the secondary
|
||
mode.
|
||
|
||
Rules:
|
||
|
||
- No implicit prompts when stdin or stdout is not a TTY.
|
||
- Ambiguous selection in non-interactive mode fails with the exact
|
||
selector flags or IDs to use next.
|
||
- Destructive or externally-mutating commands provide `--dry-run`
|
||
when the effect can be previewed and `--yes` only as an explicit
|
||
confirmation bypass.
|
||
- Progress, prompts, and warnings go on stderr so stdout stays
|
||
parseable.
|
||
|
||
### 8. Machine-readable output is stable and complete
|
||
|
||
- `--format json` (and `--output` to file) exposes identifiers and
|
||
state needed for follow-up automation. Display names are not
|
||
enough where canonical IDs exist.
|
||
- If human output distinguishes Heuristic vs Proved vs
|
||
ConcreteWitness, JSON exposes the confidence tier explicitly.
|
||
- JSON responses include a schema or output version for long-lived
|
||
automation surfaces.
|
||
- Booleans, timestamps, IDs, and partial-failure arrays are
|
||
explicit — not prose-only status fields.
|
||
- Do not mix human commentary into `--format json` output.
|
||
- Long-running commands that need streaming machine output use a
|
||
documented framing (RFC 7464 JSON Text Sequences for new
|
||
surfaces; existing NDJSON workers remain stable).
|
||
|
||
### 9. Derive metadata from the real command graph
|
||
|
||
Do not maintain separate, stale command taxonomies by hand when they
|
||
can drift from CLI11 definitions. Any helper used for command
|
||
discovery, categorization, docs generation, or MCP-tool routing is
|
||
driven by the same canonical CLI structure.
|
||
|
||
Agent-facing metadata exposes the canonical path, aliases,
|
||
deprecation status, workflow stage, prerequisites, interactivity
|
||
class, side-effect level, and machine-readable output support.
|
||
|
||
### 10. Verify the built CLI surface, not just source
|
||
|
||
Do not update help text, docs, migration notes, or error suggestions
|
||
for a command path until that path is confirmed in the built CLI.
|
||
CLI11 source changes are not sufficient evidence on their own.
|
||
|
||
Prefer direct checks: `./build/release/src/cli/fold <command> --help`
|
||
while the change is in flight. Recovery commands named in error
|
||
copy must be testable against the built binary.
|
||
|
||
### 11. Preserve backwards compatibility deliberately
|
||
|
||
Aliases and deprecated paths are intentional, documented, and
|
||
temporary. Today fold has:
|
||
|
||
- `--input-type source|binary` — deprecated alias for `--mode`. Kept
|
||
for backwards compat; emits a warning if both are provided.
|
||
- `fuzz-prep` / `fuzz-run` — top-level aliases for `fuzz prep` /
|
||
`fuzz run`. Kept for backwards compat (existing scripts depend on
|
||
them); hidden from `--help`. Errors and docs point at the canonical
|
||
path.
|
||
|
||
When keeping a legacy path, mark the canonical replacement in help
|
||
or errors. Do not add new duplicate entrypoints unless there is a
|
||
written migration plan.
|
||
|
||
### 12. Stdout, stderr, and exit status stay scriptable
|
||
|
||
Scripts and agents depend on predictable process contracts:
|
||
|
||
- stdout is for the primary requested data (the report).
|
||
- stderr is for diagnostics, progress, prompts, warnings, and
|
||
recovery guidance.
|
||
- A failed requested action returns a non-zero exit status.
|
||
- `--quiet` suppresses human chatter, not required machine-readable
|
||
data.
|
||
- Exit-status classes are documented when callers need to
|
||
distinguish parse errors, missing preconditions, not-found errors,
|
||
internal failures.
|
||
|
||
Today's exit-status contract (held stable as a public surface):
|
||
|
||
| Exit code | Meaning |
|
||
|-----------|-------------------------------------------------------------|
|
||
| 0 | Scan completed, no actionable findings |
|
||
| 1 | Scan completed, at least one actionable finding remains |
|
||
| 2 | Invalid input, configuration error, or scan exception |
|
||
|
||
### 13. Treat literature and standards as design constraints
|
||
|
||
CLI design changes are justified by product fit first, grounded in
|
||
recognized HCI and standards guidance when the change is broad
|
||
enough to affect future contributors.
|
||
|
||
This methodology does not claim its verb taxonomy is universal —
|
||
it is fold's consistency policy.
|
||
|
||
See the Evidence Base section for citations.
|
||
|
||
## Required Review Checklist
|
||
|
||
For every CLI surface change:
|
||
|
||
- [ ] Is there one canonical command path for the user intent?
|
||
- [ ] Does the command sit at the correct lifecycle stage?
|
||
- [ ] Does it use a verb from fold's canonical table (or document the
|
||
exception)?
|
||
- [ ] Does `--help` make the next step obvious without reading
|
||
source or docs?
|
||
- [ ] Are structured inputs typed and parser-validated?
|
||
- [ ] Does the failure path print exact next commands?
|
||
- [ ] Does the failure path include stable error codes for
|
||
script-handled cases?
|
||
- [ ] Is there a deterministic non-interactive path?
|
||
- [ ] Was the actual built CLI surface verified for any new or
|
||
renamed command paths?
|
||
- [ ] Does `--format json` expose the canonical identifiers and
|
||
state required for automation?
|
||
- [ ] Does stdout/stderr behavior remain scriptable?
|
||
- [ ] Are aliases or deprecated paths minimized and clearly treated
|
||
as legacy?
|
||
- [ ] Did this change reduce, preserve, or worsen top-level command
|
||
sprawl?
|
||
|
||
If any answer is "worsen" or "no", redesign before merging unless
|
||
there is an explicitly documented exception.
|
||
|
||
## Preferred CLI Flow Patterns
|
||
|
||
### Discovery-first flow
|
||
|
||
Use when a resource may or may not exist locally.
|
||
|
||
```
|
||
scan -> (review findings) -> confirm <finding-id>
|
||
or fuzz prep -> fuzz run
|
||
```
|
||
|
||
### Mutation flow
|
||
|
||
Use when changing remote or governed state. fold today has no
|
||
externally-mutating commands; this slot is reserved for future
|
||
surfaces (publishing baselines, etc.).
|
||
|
||
```
|
||
inspect -> validate preconditions -> dry-run/confirm -> mutate
|
||
```
|
||
|
||
### Machine-output flow
|
||
|
||
Use when a command is invoked by agents, hooks, or scripts.
|
||
|
||
```
|
||
validate flags -> run action ->
|
||
print one complete JSON object on stdout ->
|
||
diagnostics on stderr
|
||
```
|
||
|
||
For streaming workflows (worker fanout, future `fuzz run --tail`):
|
||
|
||
```
|
||
validate flags -> emit framed events ->
|
||
finish with explicit success/failure event
|
||
```
|
||
|
||
### Server flow
|
||
|
||
For long-running integration endpoints (`mcp`, `lsp`):
|
||
|
||
```
|
||
parse flags -> framing handshake (LSP Content-Length / JSON-RPC) ->
|
||
request/response loop until peer closes
|
||
```
|
||
|
||
## Anti-Patterns
|
||
|
||
Avoid:
|
||
|
||
- Multiple top-level nouns for the same concept.
|
||
- Large flat command enums that mix discovery, promotion, and
|
||
serving at one level.
|
||
- Free-form `string` flags for structured modes — use CLI11
|
||
validators.
|
||
- Error messages that require the user to guess the next command.
|
||
- Help text that depends on long example dumps to explain basic
|
||
navigation.
|
||
- Interactive-only selection for workflows that agents or scripts
|
||
need.
|
||
- Separate manual command category lists that can drift from CLI11.
|
||
- Human diagnostics mixed into `--format json` stdout.
|
||
- Implicit prompts in non-TTY execution.
|
||
- Recovery text that names commands not verified in the built binary.
|
||
- Subprocess-implementation-detail subcommands (`worker`) shown at
|
||
the same level as user-facing ones.
|
||
- Two flags that overlap on the same decision (`--mode` and
|
||
`--input-type` both selecting the backend).
|
||
- Enum-typed flags whose validator accepts values the implementation
|
||
silently ignores (today `--hybrid-strategy` accepts five values,
|
||
four of which warn and fall back to `merge-only`).
|
||
|
||
## When to Split a Command Group
|
||
|
||
Split or refactor when any of the following become true:
|
||
|
||
- The group mixes more than one lifecycle stage at the same level.
|
||
- Sibling commands need long doc comments to explain their
|
||
differences.
|
||
- Most commands in the group need cross-references to each other to
|
||
be usable.
|
||
- The handler module has grown large enough that policy and flow
|
||
drift are hard to review.
|
||
- The same identifiers, flags, or recovery copy are repeated across
|
||
many subcommands.
|
||
|
||
## Audit: fold today vs the policy
|
||
|
||
This section names the anti-patterns observable in fold's current
|
||
CLI and links each to the resolution (or explicit deferral).
|
||
|
||
| # | Anti-pattern | Resolution |
|
||
|---|--------------------------------------------------------------------------------------------------|--------------------------------------------------|
|
||
| 1 | `worker` exposed at top level despite being a subprocess-fanout implementation detail | Commit C1 — hide via CLI11 `group("")` |
|
||
| 2 | `fuzz-prep` and `fuzz-run` flat at root; relationship invisible from help | Commit C1 — add `fuzz` parent + hidden aliases |
|
||
| 3 | No `--quiet` flag; `INFO`-level chatter always leaks to stderr | Commit C1 — `--quiet` lowers spdlog level |
|
||
| 4 | `--detectors` accepts any string; invalid names silently dropped downstream | Commit C2 — `CLI::IsMember` against registry |
|
||
| 5 | Top error sites give the *what* but rarely the *how to recover* | Commit C3 — `errorWithRecovery` helper |
|
||
| 6 | radare2's `INFO:` / `WARN:` analysis chatter leaks through fold's stderr | Commit C4 — filter unless `--trace` set |
|
||
| 7 | `--input-type source\|binary` is a deprecated alias for `--mode` but stays prominent in help | Tracked — narrow help blurb in Phase 2 |
|
||
| 8 | `--hybrid-strategy` accepts `auto`/`binary-first`/`source-first`/`bidirectional` but only `merge-only` is implemented | Tracked — narrow validator in Phase 2 |
|
||
| 9 | `mcp` and `lsp` at top level instead of under `serve` | Tracked — rename in Phase 2 (IDE-config break) |
|
||
| 10 | `--format json` has no schema-version field | Tracked — add `output_version` field |
|
||
| 11 | Workers emit NDJSON; `--format json` emits one document. Two streaming framings on one CLI | Tracked — migrate to RFC 7464 in Phase 3 |
|
||
| 12 | No `fold show <finding-id>` for inspecting a single finding | Tracked — Phase 2 implementation |
|
||
| 13 | No `fold report` for combining baseline + new findings (today done as `scan --baseline`) | Tracked — Phase 2 implementation |
|
||
|
||
Phase 1 (this round) addresses #1–#6. Phase 2 work is tracked in the
|
||
project's task list.
|
||
|
||
## Relationship to Other Docs
|
||
|
||
- `docs/USAGE.md` — user-facing command reference. Updated to use
|
||
the canonical verb table.
|
||
- `docs/positioning.md` — what fold is and isn't. The CLI shape
|
||
reflects the positioning.
|
||
- `docs/research/cli-router.md` — architectural design for the
|
||
mode-resolution router. Implementation reference, not user guide.
|
||
- This document supersedes ad-hoc decisions in PR descriptions; when
|
||
in doubt, the checklist is the merge gate.
|
||
|
||
## Evidence Base
|
||
|
||
This methodology is a policy document, not a literature survey. The
|
||
constraints above are grounded in work on learnability,
|
||
novice-to-expert transition, actionable error messages, reproducible
|
||
workflow tools, and machine-readable interchange standards:
|
||
|
||
- Grossman, Fitzmaurice, and Attar, "A survey of software
|
||
learnability", CHI 2009, DOI `10.1145/1518701.1518803`.
|
||
- Cockburn, Gutwin, Scarr, and Malacria, "Supporting Novice to
|
||
Expert Transitions in User Interfaces", ACM Computing Surveys
|
||
2014, DOI `10.1145/2659796`.
|
||
- Traver, "On Compiler Error Messages: What They Say and What They
|
||
Mean", Advances in Human-Computer Interaction 2010, DOI
|
||
`10.1155/2010/602570`.
|
||
- Bolyen et al., "Reproducible, interactive, scalable and
|
||
extensible microbiome data science using QIIME 2", Nature
|
||
Biotechnology 2019, DOI `10.1038/s41587-019-0209-9`.
|
||
- Otasek et al., "Cytoscape Automation", Genome Biology 2019, DOI
|
||
`10.1186/s13059-019-1758-4`.
|
||
- RFC 8259, "The JavaScript Object Notation (JSON) Data Interchange
|
||
Format".
|
||
- RFC 7464, "JavaScript Object Notation (JSON) Text Sequences".
|