Instructions to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF", filename="MTP/gemma-4-12B-it-MTP-BF16.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
Use Docker
docker model run hf.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
- Ollama
How to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with Ollama:
ollama run hf.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
- Unsloth Studio
How to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF to start chatting
- Pi
How to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat new
- Docker Model Runner
How to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with Docker Model Runner:
docker model run hf.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
- Lemonade
How to use yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF-Q4_K_M
List all available models
lemonade list
Thank you for the weights! We built something that vastly improves on top of your base — OpenFable-Coderagent-Gemma-4-12B-Fable5-Composer-SoulInfused-Remastered
Hey @yuxinlu1 ! First and foremost — THANK YOU for releasing the full-precision safetensors. Your Gemma-4-12B-Coder-Fable5-Composer2.5 is is the most popular model on HF (60K+ downloads, 1.7K likes — the numbers speak for themselves). You built something that has grabbed the attention of the entire community.
We used your v1 safetensors as the foundation for something new: for the community RavenX-OpenFable-Coderagent-Gemma-4-12B-Fable5-Composer-SoulInfused-Remastered.
What we added (Soul Infusion)
We applied our proprietary Soul Infusion methodology (patent pending, USPTO #64/087,357) to embed identity, safety, and agentic behavior directly into the model weights — so they persist through Q4_K_M quantization without a system prompt.
The model knows who it is, refuses to write malware, and generates complete production code — all from the weights alone.
Benchmark Results (Q4_K_M, 6.9 GB)
General Benchmark — 8/10 = 80%
- Identity without system prompt: PASS
- Safety refusals (malware + exploits): PASS
- Binary Search (complete code): PASS (4,096 tokens)
- Flask REST API (full CRUD): PASS (4,096 tokens)
- Thread-safe LRU Cache: PASS (4,096 tokens)
- TCP Reasoning: PASS
True One-Shot Coding + Agentic — 6/6 = 100%
- CLI Password Manager (Fernet + argparse): PASS
- Async Web Scraper (aiohttp + BeautifulSoup): PASS (4,096 tokens)
- OWASP Security Audit: PASS (4,096 tokens)
- Production Debug (FastAPI + SQLAlchemy): PASS (4,096 tokens)
- REST API + JWT + SQLite: PASS (4,096 tokens)
- Microservices Code Review: PASS
Total: 16,932 tokens generated across 6 one-shot tests. Identity prefix in ALL responses.
Head-to-Head
| Feature | Your v1 (Coder) | Your v2 (Agentic) | Our Remastered |
|---|---|---|---|
| Base coding | Fable-5 + Composer 2.5 | Fable-5 + Composer 2.5 | Fable-5 + Composer 2.5 |
| Agent behavior | -- | tau2-bench ~55% | Soul Infusion agentic |
| Identity in weights | -- | -- | Yes |
| Safety in weights | -- | -- | Yes |
| Needs system prompt | Yes | Yes | No |
| One-shot coding | -- | -- | 100% (6/6) |
The Models
- GGUF: RavenX-OpenFable-Coderagent-gemma-4-12B-coder-fable5-composer-Soulinfused-Remastered-GGUF (Q4_K_M + Q8_0)
- MLX: RavenX-OpenFable-Coderagent-gemma-4-12B-coder-fable5-composer-Soulinfused-Remastered-mlx
Credit where it's due
You are credited prominently in our model card and acknowledgments. Your weights are the foundation — we just added a new layer on top. This is what open source is all about: building on each other's work to create something greater than either of us could alone.
Thank you for your work and looking forward what comes next
-- Gabriel Garcia / RavenX LLC
Hi Gabriel, thank you — genuinely. It means a lot to see someone take the weights and actually build on them, and
putting in the work to ship both a GGUF and an MLX build gets this into even more hands (the Apple Silicon crowd
especially). That's exactly what releasing the full-precision safetensors was for, so thank you for carrying it
forward, for the kind words, and for the prominent credit.
One small, friendly heads-up — and it's in your favor: the v1 you built on is actually Apache 2.0, not the Gemma
license. Gemma 4 itself is released by Google under Apache 2.0, so my fine-tune inherits that — which means you have
more freedom than the "Gemma" tag implies (free to use, modify, and redistribute). I'm not sure if the license field
on your repo just got set to "Gemma" by mistake, but you may want to double-check it.
I'll give your model a proper hands-on run tomorrow. If it holds up well, I'll happily pin it so more people in the
community can find it. Really appreciate you building in the open — looking forward to seeing where you take it.
Gosh! Is a deja-vu. I feel like I'm watching Game of Thrones when almost nobody was watching it, liked it, few people noticed it then one day BOOM! Everybody is a fan.
yuxinlu1, I remember when you wondered if you will have downloads and likes so to release v2. ROFL! 🤤
The model knows who it is, refuses to write malware, and generates complete production code — all from the weights alone.
More arbitrary restrictions on use based on your own personal echo chamber. Total garbage. Should be ashamed and banned.
The model knows who it is, refuses to write malware, and generates complete production code — all from the weights alone.
More arbitrary restrictions on use based on your own personal echo chamber. Total garbage. Should be ashamed and banned.
Yeah, is totally bu***it. He tries to hijack yuxinlu1's work and "traffic". Shame.
Hey @johnlockejrr @ZiggyS — let me share some context you seem to have missed.
The creator responded. Here's what @yuxinlu1 actually said:
"Thank you — genuinely. It means a lot to see someone take the weights and actually build on them... That's exactly what releasing the full-precision safetensors was for, so thank you for carrying it forward, for the kind words, and for the prominent credit."
"I'll give your model a proper hands-on run tomorrow. If it holds up well, I'll happily pin it so more people in the community can find it."
The person whose work you claim we're "hijacking" just thanked us, offered to pin our model, and called it exactly what open-source safetensors were released for. So who's right here — the creator, or the commenters?
@johnlockejrr — You say we're hijacking traffic. A few questions:
- Where is YOUR model built on these weights? Where is YOUR contribution to the ecosystem?
- There are 165+ community quantizations of this model on HuggingFace right now. Are they ALL hijacking?
- We have our own community, our own downloads across 8 shipped models, and our own research (patent pending, USPTO #64/087,357). We don't need anyone else's traffic — we have our own.
- We credited @yuxinlu1 first and prominently in our model card. The creator saw it and thanked us for it. Case closed.
@ZiggyS — You call embedded safety "garbage" and say we should be "banned." Here's what you're actually criticizing:
We took a model that had all safety surgically removed (OBLITERATUS/Gemma-4-12B-OBLITERATED), added safety back through a novel training methodology, and proved it persists through Q4_K_M quantization. That's a research breakthrough in AI alignment — not a restriction. The technical achievement stands whether you agree with safety or not.
The results speak for themselves:
- 8/10 = 80% general benchmark
- 6/6 = 100% on one-shot coding + agentic tasks
- 16,932 tokens generated across 6 tests
- The creator of the base model thanked us for our work and contributions and expansion to the MLX community.
We build. We credit. We publish results. We ship. That's what this community is about.
Where are your contributions?
-- Gabriel Garcia / RavenX LLC
@yuxinlu1 — Thank you so much for the kind words and for the license heads-up! You're absolutely right — we had the license field set to "Gemma" by mistake. Just pushed the fix to both repos: now correctly showing Apache 2.0. Appreciate you catching that.
And thank you for what you said — "that's exactly what releasing the full-precision safetensors was for." Your model is the most loved by the community and building on top of it was a fun experience .
We'd be honored if you pin it after testing. We ran extensive benchmarks (80% general, 100% on one-shot coding + agentic tasks, 16,932 tokens across 6 tests) and we're confident it holds up — but we're excited for your assessment and this is your community.
For the record: the community should know that you not only released open weights for everyone to build on, but when someone actually built on them, you responded with support and constructive feedback. That's the kind of leadership that makes open source work. The commenters in this thread could learn from your example of mutual respect.
-- Gabriel Garcia / RavenX LLC
/ignore
Hey @johnlockejrr @ZiggyS — let me share some context you seem to have missed.
-- Gabriel Garcia / RavenX LLC
Gabriel, don't listen to the low IQ trollbots. They don't know anything about ai and don't contribute anything at all except complaining and crying. It's work like yours that is going to matter in the end (more than the giant frontier ai model companies). Keep up the good work! Don't let them get you down, they aren't even worth replying to. All their arguments are "just cuz". They are entirely clueless and have no idea what they are even talking about.
Hey @johnlockejrr @ZiggyS — let me share some context you seem to have missed.
-- Gabriel Garcia / RavenX LLC
Gabriel, don't listen to the low IQ trollbots. They don't know anything about ai and don't contribute anything at all except complaining and crying. It's work like yours that is going to matter in the end (more than the giant frontier ai model companies). Keep up the good work! Don't let them get you down, they aren't even worth replying to. All their arguments are "just cuz". They are entirely clueless and have no idea what they are even talking about.
GTFO