[Native] Serve many sequences from one ggml engine KV cache - #25007

Stack

Off-graph KV cache for the native ggml engine, through serving Muse Glimmer on it. Review bottom-up; each PR builds and passes its tests on its own.

• [Native] Launch the Muse Glimmer server on the native backend #24957
• [Native] Serve Muse Glimmer on the native ggml engine #24956
• [Native] Select the logits rows an off-graph export returns #25009
• [Native] Gather the rows of x[:, rows] in the ggml engine #25008
• [Native] Serve many sequences from one ggml engine KV cache #25007 (this PR)
• [Native] Separate the ggml engine's K/V storage from its sequence cache #25006
• [Native] Publish the context length with off-graph packages #24955
• [Native] Share the off-graph cache geometry with other callers #24954
• [Native] Export Muse Glimmer through the native backend #24951
• [Native] Replay sample sets without decoding their tokens #24950
• [Native] Map Muse Glimmer GGUF tensors onto its HF model #24949
• [Native] Run GGUF Q5_K weights in the ggml engine #24948
• [Native] Load the first layers of a GGUF with --layers #24947
• [Native] Run the HF LLM export on newer transformers #24946
• [Native] Fuse broadcast RoPE frequencies and lower tanh in ggml #24945
• [Native] Support sliding-window attention with the off-graph KV cache #24944
• [Native] Carry named metadata in native packages #24943
• [Native] Add chunked prefill and interactive chat to the LLM runner #24639
• [Native] Run off-graph KV cache packages in the LLM runner #24638
• [Native] Serve kvcache::update_and_attend in the ggml engine #24637
• [Native] Add load-time options to engine contexts #24636
• [Native] Export HF LLMs with an off-graph KV cache #24635
• Move off-graph HF attention to extension/llm/cache #24634

Summary

The ggml engine's off-graph cache held one sequence, so a batching runtime couldn't put several sessions in one forward, clone a prefix, or share a pool between sessions.

• Cache: GgmlCellCache is the neutral CellCache over a GgmlKvPool whose rows are its cells, registered for kind::kBatched and kind::kBatchedCell. It offers BatchControl to the control plane.
• Engine: a context accepts either cache through llm_cache_registry_key. With a cell cache, the executor:
  • places the forward's tokens at their positions before building the graph (the control plane declared their sequences);
  • grows storage to the step's read extent;
  • fills each window's cache rows and mask straight from the cache's step: rows are the step's cells, and the mask keeps each query to its own sequence's earlier cells within its window.
• Windows: a windowed layer keeps every cell, so a rewind or clone never needs history the cache dropped.
• Failure: a forward that fails after placement leaves its cells claimed, as a failed MLX forward does.

Test plan

New engine tests against a multi-sequence reference (naive attention per sequence):

• two sequences prefilling separately, then decoding in one forward, with and without graph caching, on fp32 and fp16 storage, starting from one row so storage regrows;
• a clone sharing its source's prefix as both continue, plus rewinds and closing the clone;
• sliding windows, including inside a prefill wider than the window;
• undeclared steps and positions that don't continue a sequence are refused.

All pass on CPU, and on Metal through a temporary local change. Making the executor ignore the cache's mask fails three of the four.

Authored with Claude Code.