2 KiB
Gem native engine workflow
This project is a standalone C inference server specialized for Gemma 4 12B and 31B on Apple
Silicon. Production binaries must not link libllama/ggml or launch llama-server. The adjacent
DeepSeek-V4-enabled llama.cpp checkout is a reference and benchmark oracle only; algorithms and
Metal kernels may be independently adapted with attribution.
Before changing engine behavior, retrieve project memory first:
scripts/yams search --cwd --query "<task or symbol>"
Use scripts/yams, never bare yams, so the corpus, config, and session state stay scoped to this
project. Start its local daemon with scripts/yams daemon start before its first use. Index durable
research records, benchmark configurations, benchmark results, and profiling
findings with scripts/memory-index.sh; do not index model weights, generated build output, API
keys, or traces.
Performance changes require a one-axis benchmark comparison under the protocol in
docs/build/benchmark-protocol.md. Do not promote a setting without raw JSON, a recorded gem print-config, and three repetitions. Kernel changes belong in the adjacent llama.cpp checkout and
need its backend-op correctness test before the local benchmark gate.
The native ownership boundary is GGUF loading, tokenizer, fixed Gemma 4 graph, KV cache, sampler, Metal dispatch, and hax-compatible HTTP/SSE. Keep model-specific code explicit: only 48-layer 12B and 60-layer 31B GGUFs are accepted. The existing llama.cpp launcher is legacy transition and must not gain new production responsibilities.
For offline hax use, prefer build/gem agent --model <GGUF> so the C launcher waits for model
readiness and owns server cleanup. With F32 KV, that command automatically uses a regular matching
mtp-<GGUF filename> sidecar in the same directory; pass --no-mtp for direct-path diagnostics.
The two-terminal serve plus hax form remains available for debugging. Hax does not use MCP; it
receives this instruction file and invokes YAMS through the project CLI wrapper when it needs memory.