Instructions to use Zorix-official/Zorix-Nano-2B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Zorix-official/Zorix-Nano-2B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Zorix-official/Zorix-Nano-2B-GGUF",
	filename="Zorix-Nano-2B-Q5_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Zorix-official/Zorix-Nano-2B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M
# Run inference directly in the terminal:
./llama-cli -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

Use Docker

docker model run hf.co/Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

LM Studio
Jan

vLLM

How to use Zorix-official/Zorix-Nano-2B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Zorix-official/Zorix-Nano-2B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Zorix-official/Zorix-Nano-2B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

Ollama
How to use Zorix-official/Zorix-Nano-2B-GGUF with Ollama:
```
ollama run hf.co/Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M
```

Unsloth Studio

How to use Zorix-official/Zorix-Nano-2B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Zorix-official/Zorix-Nano-2B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Zorix-official/Zorix-Nano-2B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Zorix-official/Zorix-Nano-2B-GGUF to start chatting

How to use Zorix-official/Zorix-Nano-2B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Zorix-official/Zorix-Nano-2B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use Zorix-official/Zorix-Nano-2B-GGUF with Docker Model Runner:
```
docker model run hf.co/Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M
```

Lemonade

How to use Zorix-official/Zorix-Nano-2B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Zorix-official/Zorix-Nano-2B-GGUF:Q5_K_M

Run and chat with the model

lemonade run user.Zorix-Nano-2B-GGUF-Q5_K_M

List all available models

lemonade list

Zorix Nano 2B

Zorix Nano 2B is a compact multilingual language model released by Zorix.

Official website: https://zorix.it
Developer: Zorix
Architecture: zorix-nano
Model size: approximately 2B parameters
Format: GGUF
Quantization: Q5_K_M
Languages: Chinese, English and Italian
Context length: 262,144 tokens

Architecture

The model uses the custom GGUF architecture identifier:

general.architecture = zorix-nano

Main architecture parameters:

Parameter| Value
Hidden size| 2048
Layers| 24
Feed-forward size| 6144
Attention heads| 8
KV heads| 2
Context length| 262144
RoPE base frequency| 10000000
SSM state size| 128
Full-attention interval| 4

Runtime requirement

This model is officially supported only by:

Zorix GGUF Run

It is not officially supported by:

- llama.cpp
- Ollama
- LM Studio
- GPT4All
- Transformers
- Other standard GGUF runtimes

Other runtimes may return:

unknown model architecture: zorix-nano

Use this model

Android ARM64 and Termux

Create the required directories:

mkdir -p "$HOME/.zorix/bin"
mkdir -p "$HOME/.zorix/models"

Download the Android runtime:

curl -L \
  "https://huggingface.co/Zorix-official/Zorix-Nano-2B-GGUF/resolve/main/runtime/android-arm64/zorix-runtime" \
  -o "$HOME/.zorix/bin/zorix-runtime"

Download the launcher:

curl -L \
  "https://huggingface.co/Zorix-official/Zorix-Nano-2B-GGUF/resolve/main/runtime/android-arm64/zorix-gguf-run" \
  -o "$HOME/.zorix/bin/zorix-gguf-run"

Download Zorix Nano 2B:

curl -L \
  "https://huggingface.co/Zorix-official/Zorix-Nano-2B-GGUF/resolve/main/Zorix-Nano-2B-Q5_K_M.gguf" \
  -o "$HOME/.zorix/models/Zorix-Nano-2B-Q5_K_M.gguf"

Enable the commands:

chmod +x \
  "$HOME/.zorix/bin/zorix-runtime" \
  "$HOME/.zorix/bin/zorix-gguf-run"

export PATH="$HOME/.zorix/bin:$PATH"

Run the model by specifying its path:

zorix-gguf-run \
  -m "$HOME/.zorix/models/Zorix-Nano-2B-Q5_K_M.gguf"

Run one prompt:

zorix-gguf-run \
  -m "$HOME/.zorix/models/Zorix-Nano-2B-Q5_K_M.gguf" \
  -p "你好，请介绍一下你自己。"

Linux ARM64, Debian and Ubuntu

The Linux runtime supports ARM64/AArch64 systems using glibc.

Create the directories:

mkdir -p "$HOME/.zorix/bin"
mkdir -p "$HOME/.zorix/models"

Download the Linux ARM64 runtime:

curl -L \
  "https://huggingface.co/Zorix-official/Zorix-Nano-2B-GGUF/resolve/main/runtime/linux-arm64/zorix-runtime-linux-arm64" \
  -o "$HOME/.zorix/bin/zorix-runtime"

Download the launcher:

curl -L \
  "https://huggingface.co/Zorix-official/Zorix-Nano-2B-GGUF/resolve/main/runtime/linux-arm64/zorix-gguf-run" \
  -o "$HOME/.zorix/bin/zorix-gguf-run"

Download the model:

curl -L \
  "https://huggingface.co/Zorix-official/Zorix-Nano-2B-GGUF/resolve/main/Zorix-Nano-2B-Q5_K_M.gguf" \
  -o "$HOME/.zorix/models/Zorix-Nano-2B-Q5_K_M.gguf"

Install required libraries on Debian or Ubuntu:

sudo apt update
sudo apt install -y libc6 libstdc++6 libgomp1 libgcc-s1

Enable and run:

chmod +x "$HOME/.zorix/bin/zorix-runtime"
chmod +x "$HOME/.zorix/bin/zorix-gguf-run"

export PATH="$HOME/.zorix/bin:$PATH"

zorix-gguf-run \
  -m "$HOME/.zorix/models/Zorix-Nano-2B-Q5_K_M.gguf"

Supported platforms

Platform| Architecture| Status
Android Termux| ARM64| Supported
Linux| ARM64/AArch64| Supported
Debian| ARM64/AArch64| Supported
Ubuntu| ARM64/AArch64| Supported
Linux AMD64| Not currently supported| 
Windows| Not currently supported| 
macOS| Not currently supported| 

Model file

File| Description
"Zorix-Nano-2B-Q5_K_M.gguf"| Zorix Nano 2B Q5_K_M model
"config.json"| Model configuration and architecture information
"LICENSE"| Model license

Limitations

Zorix Nano 2B may:

- Generate incorrect information
- Misunderstand complex instructions
- Produce repetitive responses
- Generate unsuitable or biased content
- Perform worse than larger language models

Review important outputs before relying on them.

The model is not a substitute for qualified medical, legal, financial or safety professionals.

Developer

Zorix

Official website:

https://zorix.it

Model repository:

https://huggingface.co/Zorix-official/Zorix-Nano-2B-GGUF

Downloads last month: 167

GGUF

Model size

2B params

Architecture

zorix-nano

Hardware compatibility

5-bit