Instructions to use nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx"
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx
Run Hermes
hermes
- MLX LM
How to use nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx", "messages": [ {"role": "user", "content": "Hello"} ] }'
Hunyuan-0.5B-Instruct-dwq5-mlx
this model comes in Strabberry flavor:
How many R are in Strawberry
<think>
Okay, let's see. The user is asking how many "R" are in the word "Strawberry." First, I need to spell out the word correctly. Let me check: S-T-R-A-B-B-E-R-R-Y. Wait, let's break it down letter by letter. S, T, R, A, B, B, E, R, R, Y. Now, I need to count the "R"s. Let's go through each letter:
1. S
2. T
3. R (first R)
4. A
5. B
6. B
7. E
8. R (second R)
9. R (third R)
10. Y
So that's three "R"s. Let me double-check to make sure I didn't miss any. The word is "Strawberry." Let's spell it again: S-T-R-A-B-B-E-R-R-Y. Yep, positions 3, 8, and 9 are R. So three R's. That should be the answer.
</think>
<answer>
The word "Strawberry" has three "R" letters.
Right, for the wrong reason
479.826 tok/sec
This model Hunyuan-0.5B-Instruct-dwq5-mlx was converted to MLX format from tencent/Hunyuan-0.5B-Instruct using mlx-lm version 0.26.3.
Use with mlx
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("Hunyuan-0.5B-Instruct-dwq5-mlx")
prompt = "hello"
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True
)
response = generate(model, tokenizer, prompt=prompt, verbose=True)
- Downloads last month
- 6
Model size
0.1B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
5-bit
Model tree for nightmedia/Hunyuan-0.5B-Instruct-dwq5-mlx
Base model
tencent/Hunyuan-0.5B-Pretrain Finetuned
tencent/Hunyuan-0.5B-Instruct