# Gem objective completion audit This audit keeps the project objective broader than the latest successful optimization. A status is complete only when the named current-state evidence covers the full requirement; planned commands and historical intent are not proof. | Requirement | Status | Authoritative evidence | Remaining proof | |---|---|---|---| | Standalone C inference server; production never links ggml/libllama or launches llama-server | Complete | `architecture.md`; native/bench/profile dependency-guard CTests; `otool` receipts in benchmark artifacts | Re-run the complete guards after every new production target is added. | | Fixed dense Gemma 4 31B graph and strict GGUF/tokenizer ownership | Complete | 60-layer/833-tensor contract in `model.c`; authentic header/tokenizer tests; malformed topology/extent tests | A new GGUF layout is out of scope until explicitly added and gated. | | M4 Max-specific Metal execution and benchmark/profiling scaffold | Complete | `../build/benchmark-protocol.md`; `gem-bench`, stage/decode/attention profilers; raw three-repetition artifacts; adjacent backend oracle receipts | Continue applying the protocol to every promoted axis. | | Small hax-compatible offline server and one-command lifecycle | Complete | `gem agent`; `gem.hax_project_memory`, `gem.hax_native_server`, and lifecycle tests; real-hax artifacts including tool loops and MTP autodiscovery; final selector-19 8K+ smoke returned exactly `OK` and reclaimed its port | None for the current kernel decision. | | Exact multi-turn long-prompt reuse | Complete through 128K | Exact token-boundary cache implementation; 32K/64K receipts; audited `2026-08-26-server-gate-31b-split16-131072` with three fresh processes, exact 131,072-token reuse, and 26-token extensions | Re-run if cache identity or request tokenization changes. | | 128K memory capacity | Complete | `artifacts/2026-08-24-context-capacity-31b`: 31B F32 capacity and request probes on the physical 128 GiB host | Capacity alone does not prove cold TTFT or current-kernel speed. | | Current 31B/F32 long-decode value optimization | Complete | Promoted selector 19; independent split screens; boundary/tail/repeat correctness; authentic 64K/128K whole-token gains of 22.62%/26.79%; 128K KV gain 32.20% | Re-open value-side work only if a new profile shows another justified ceiling. | | Long-prompt progress and disconnect cancellation | Complete | Batch-boundary callback implementation; CPU fixture; authentic `2026-08-26-prefill-cancel-31b-131072` receipt with four keepalives, curl status 28, health recovery, and a successful same-process request | Re-run if batch boundaries or server streaming semantics change. | | Fastest justified production prefill kernel through 64K | Complete | Selector 19 routes 31B/F32 post-4K global prefill to BQ8/BK64; isolated attention improved about 59% at 4K/64K/128K, evaluated graph improved 7.12% at 4K and 33.38% at 64K, and the 64K logit gate retained 0.99999958 correlation with 8.35e-7 NMSE | The current BQ8 binary still needs an uncontaminated 128K server acceptance gate. | | Complete post-change test state | Complete for the retained diagnostic | All 69 normal and all 69 freshly rebuilt ASan/UBSan tests pass with BK128 available only through diagnostic selector 23; the adjacent backend-op oracle remains 4,793/4,793 | Repeat after any production routing change. | | End-to-end 128K production acceptance | Incomplete for current BQ8 production | One current BQ8 cold request and exact-prefix extension completed, but extreme unrelated host load contaminated the timings; the older audited three-run receipt predates the BQ8 prefill promotion | Run three fresh schema-4 repetitions on a host that passes load, one-second system-busy, AC-power, and thermal preflight, preserving raw JSON, `gem print-config`, digests, SSE/progress, and continuous host-monitor receipts. | | Latest 128K prefill kernel cycle | Diagnostic open; production unchanged | BQ8/BK64 is promoted. Exact conditional rescaling was bit-identical but rejected and removed after sub-threshold results. BQ8/BK128 is correct, deterministic, uses 20,544 bytes of threadgroup memory, and is exposed only through diagnostic selector 23; its initial screen was host-contaminated | Run the quiet-host operator gate. If it advances, run the prepared selector-19/selector-23 evaluated-prefix/logit gate; otherwise remove BK128. Resolve it before rerunning the current-production 128K server gate. | | Latest long-decode score/K cycle | Diagnostic ready; production unchanged | Selector 20 changes only global-decode K staging from tile1 to tile8 while retaining selector 19's split16 value path; threshold/tail parity and a fail-closed ABBA runner exist. The parked D512 fused split-KV design now records the measured baseline, ownership, bounded scratch, resource formula, and fail-closed gates without adding a selector. | After closing BK128 and current-production 128K acceptance, run the schema-2 64K/128K score-tile8 gate. Advance to authentic logits/model timing only if projected whole-token gain exceeds 3% at both lengths; otherwise remove selector 20 before implementing fused split-KV. | ## Current critical path 1. Preserve selector 19 with BQ8/BK64 as production and retain its completed 8K, 32K, and 64K server gates. 2. On a host that passes the frozen load/power preflight, run the seeded, interleaved BK64/BK128 operator gate with identical inputs and enough warmup to avoid frequency-ramp bias. Operator-gate schema 2 also requires AC power, no thermal/performance warning, complete digest/linkage receipts, and a positive paired 95% confidence lower bound at both long contexts. The clean-host launcher can capture the first sustained quiet window without killing or reprioritizing unrelated work; its observation receipt supplements rather than replaces the gate's host receipts. 3. Promote BK128 only if it clears the protocol threshold consistently. Otherwise remove the diagnostic pipeline. The evaluated-graph step is frozen in `scripts/run-bq8-bk128-graph-gate.sh` and independently revalidates the advancing operator artifacts, provenance, continuous host state, logits, paired timing, and configuration. A promotion also requires evaluated-graph/logit evidence, all local tests, and the adjacent backend-op oracle before a performance claim. 4. Run the schema-4 three-repetition 128K cold/prefix server gate against the resulting production binary, including raw JSON, `gem print-config`, digests, progress/SSE validation, and host receipts. 5. Run the already-implemented selector-20 score-tile8 diagnostic under its schema-2 clean-host gate. Remove it if it misses the model admission floor. If it passes, complete authentic logits and populated-model timing before any routing change. Only then consider the larger fused split-KV GQA score/K design. The standalone engine is complete enough to serve hax natively, and BQ8/BK64 is the justified production prefill choice through the completed 64K gate. Current-production 128K acceptance is not yet complete: the only finished BQ8 run was host-contaminated, and BK128 is routed only through diagnostic selector 23 pending its clean operator and evaluated-graph gates. Seeded profiles cannot substitute for authentic prompt throughput or TTFT.