34 lines
2 KiB
Markdown
34 lines
2 KiB
Markdown
# Gem native engine workflow
|
|
|
|
This project is a standalone C inference server specialized for Gemma 4 12B and 31B on Apple
|
|
Silicon. Production binaries must not link libllama/ggml or launch `llama-server`. The adjacent
|
|
DeepSeek-V4-enabled llama.cpp checkout is a reference and benchmark oracle only; algorithms and
|
|
Metal kernels may be independently adapted with attribution.
|
|
|
|
Before changing engine behavior, retrieve project memory first:
|
|
|
|
```sh
|
|
scripts/yams search --cwd --query "<task or symbol>"
|
|
```
|
|
|
|
Use `scripts/yams`, never bare `yams`, so the corpus, config, and session state stay scoped to this
|
|
project. Start its local daemon with `scripts/yams daemon start` before its first use. Index durable
|
|
research records, benchmark configurations, benchmark results, and profiling
|
|
findings with `scripts/memory-index.sh`; do not index model weights, generated build output, API
|
|
keys, or traces.
|
|
|
|
Performance changes require a one-axis benchmark comparison under the protocol in
|
|
`docs/build/benchmark-protocol.md`. Do not promote a setting without raw JSON, a recorded `gem
|
|
print-config`, and three repetitions. Kernel changes belong in the adjacent llama.cpp checkout and
|
|
need its backend-op correctness test before the local benchmark gate.
|
|
|
|
The native ownership boundary is GGUF loading, tokenizer, fixed Gemma 4 graph, KV cache, sampler,
|
|
Metal dispatch, and hax-compatible HTTP/SSE. Keep model-specific code explicit: only 48-layer 12B
|
|
and 60-layer 31B GGUFs are accepted. The existing llama.cpp launcher is legacy transition and must
|
|
not gain new production responsibilities.
|
|
|
|
For offline hax use, prefer `build/gem agent --model <GGUF>` so the C launcher waits for model
|
|
readiness and owns server cleanup. With F32 KV, that command automatically uses a regular matching
|
|
`mtp-<GGUF filename>` sidecar in the same directory; pass `--no-mtp` for direct-path diagnostics.
|
|
The two-terminal `serve` plus `hax` form remains available for debugging. Hax does not use MCP; it
|
|
receives this instruction file and invokes YAMS through the project CLI wrapper when it needs memory.
|