Garbled / repeating <unused49> in LM Studio? Known Gemma 4 client bug — not the weights (fixes inside)
Hi everyone — a few people have reported garbled output, repeating tokens, or leaked <|channel> tags when
running v2 (and v1) in LM Studio. I dug into it carefully and want to be fully transparent, because it's easy to
misread this as "broken quantization." It isn't.
TL;DR — this is a known upstream Gemma 4 + client bug, not a problem with these weights or quants. The exact same GGUF
that floods in LM Studio runs perfectly clean under llama.cpp. It's a client/runtime config issue, and the
fixes are below.
It's not the quant — proof
I took the published Q6_K file and asked it the same kind of question that was failing ("why does cv2 fail on
Chinese-character paths?") under llama.cpp --jinja. It answered cleanly and correctly — proper thinking, correct code
(np.fromfile + cv2.imdecode), zero , zero leaked tokens. Same file, same prompt: clean in llama.cpp, broken
in LM Studio. So the weights and the quant are fine.
What's actually happening
This is a Gemma-4-family-wide issue that hits Google's own official models too, not just fine-tunes:
- Google's official gemma-4-26B-A4B-it (via unsloth GGUF) produces the exact same <|channel> →
unsloth discussion (https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/discussions/2). Quote: "once the eval bug
was fixed in llama.cpp, even the old quants worked." - Tracked at the model level across LM Studio / Ollama / Cloudflare → google-deepmind/gemma #622
(https://github.com/google-deepmind/gemma/issues/622), llama.cpp #21516
(https://github.com/ggml-org/llama.cpp/issues/21516). - LM Studio root cause: its reasoning parser expects /, but Gemma 4 thinks in <|channel>thought …
<channel|>. LM Studio doesn't auto-detect that, so the thinking tokens leak or collapse into → LM Studio
bug #2013 (https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/2013).
How to fix it
✅ Most reliable — llama.cpp --jinja (what I test and recommend):
llama-server -m gemma4-v2-Q4_K_M.gguf --jinja -ngl 99 -fa on -c 16384
--temp 1.0 --top-p 0.95 --top-k 64
Use a recent build (I verified b9553). --jinja applies Gemma 4's real template + thinking/tool parser, so none of this
happens.
🛠️ If you want to stay in LM Studio:
- Update LM Studio and its runtime (the llama.cpp / MLX runtime under the hood) to the latest — the upstream eval bug
behind the floods is fixed in newer builds. - Set the reasoning-parsing tokens so LM Studio knows where thinking begins/ends:
- Inference Settings → Reasoning Parsing
- Start: <|channel>thought · End: <channel|> - Sampler: temp 1.0, top_p 0.95, top_k 64, repeat_penalty 1.1 — a missing repetition penalty is what turns a hiccup
into an endless 0000… / stream. - KV cache: prefer FP16 for coding quality (avoid q4_0).
- If it still loops mid-chat, reload the model / clear the chat to rebuild the KV cache, and prefer Q5_K_M / Q6_K /
Q8_0.
Two related client issues (same family)
- Endless 0000… → no repetition penalty. Set repeat_penalty 1.1, temp 1.0 (koboldcpp defaults to no penalty — the #1
cause). - Leaked <|tool_call> / <|channel> tokens, tools not firing → your front-end isn't parsing Gemma 4's native tool
format. Use llama.cpp --jinja (returns clean structured tool-calls).
Thanks to everyone who reported this with screenshots — genuinely helpful. This really is a client/runtime config
issue, and the model is solid once the runtime handles Gemma 4 correctly. Anything not covered here, open a discussion
and I'll help. 🙏
I wanna know how stay in ollama，just like LM Studio?
Hi @link921 — the reason it "just works" in LM Studio but not Ollama is that LM Studio reads the jinja chat template
embedded in the GGUF, while Ollama uses its own Go templates and doesn't always pick up a custom GGUF's template on
import — so a raw import can hit "does not support chat". Two things fix it:
- Give it a template Ollama understands. I didn't change the tokenizer, so the official Gemma 4 template works as-is.
Pull the official model once and copy its template:
ollama pull gemma4
ollama show --modelfile gemma4 # copy the TEMPLATE """...""" block from the output
- Pin the context so Ollama doesn't pre-allocate KV for the full 256K window — that's the other common load-time OOM.
Then make a Modelfile next to the GGUF:
FROM ./gemma-4-12B-agentic-...-v2-Q4_K_M.gguf
TEMPLATE """<paste the TEMPLATE block from ollama show above>"""
PARAMETER num_ctx 8192
PARAMETER temperature 1.0
PARAMETER top_p 0.95
PARAMETER top_k 64
PARAMETER repeat_penalty 1.1
and build it:
ollama create gemma4-v2 -f Modelfile
ollama run gemma4-v2
Use a recent Ollama (≥ 0.21) if you also want tool-calling to work.
If you'd rather not touch templates at all, the most reliable route is to skip Ollama's import and run llama.cpp
directly — it reads the embedded jinja template correctly and exposes an OpenAI-compatible endpoint any UI can connect
to:
llama-server -m gemma-4-12B-...-v2-Q4_K_M.gguf --jinja -c 8192 -ngl 99
Same model either way; the Modelfile route keeps you inside Ollama like you asked.