OpenEnv documentation

Unity ML-Agents Environment

OpenEnv

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.4.1).

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

Unity ML-Agents Environment

OpenEnv wrapper for Unity ML-Agents environments. This environment provides access to Unity’s reinforcement learning environments through a standardized HTTP/WebSocket interface.

Supported Environments

Environment	Action Type	Description
PushBlock	Discrete (7)	Push a block to a goal position
3DBall	Continuous (2)	Balance a ball on a platform
3DBallHard	Continuous (2)	Harder version of 3DBall
GridWorld	Discrete (5)	Navigate a grid to find goals
Basic	Discrete (3)	Simple left/right movement

More environments may be available depending on the ML-Agents registry version.

Installation

Option 1: Non-Docker Installation (Local Development)

Prerequisites

Python 3.10+
uv (recommended) or pip

Install from OpenEnv Repository

# Clone the OpenEnv repository (if not already done)
git clone https://github.com/your-org/OpenEnv.git
cd OpenEnv

# Install the unity_env package with dependencies
cd envs/unity_env
uv pip install -e .

# Or with pip
pip install -e .

Install Dependencies Only

cd envs/unity_env

# Using uv (recommended)
uv sync

# Or using pip
pip install -r requirements.txt  # if available
pip install mlagents-envs numpy pillow fastapi uvicorn pydantic

Verify Installation

# Test the installation
cd envs/unity_env
python -c "from client import UnityEnv; print('Installation successful!')"

Note: The first run will download Unity environment binaries (~500MB). These are cached in ~/.mlagents-cache/ for future use.

Option 2: Docker Installation

Prerequisites

Docker installed and running
Python 3.10+ (for running the client)

Build the Docker Image

cd envs/unity_env

# Build the Docker image
docker build -f server/Dockerfile -t unity-env:latest .

# Verify the build
docker images | grep unity-env

Note for Apple Silicon (M1/M2/M3/M4) users: Docker mode is not supported on Apple Silicon because Unity’s Mono runtime crashes under x86_64 emulation. Use direct mode (--direct) or server mode (--url) instead, which run native macOS binaries. See Troubleshooting for details.

Run the Docker Container

# Run with default settings (graphics enabled, 800x600)
docker run -p 8000:8000 unity-env:latest

# Run with custom settings
docker run -p 8000:8000 \
  -e UNITY_NO_GRAPHICS=0 \
  -e UNITY_WIDTH=1280 \
  -e UNITY_HEIGHT=720 \
  -e UNITY_TIME_SCALE=1.0 \
  unity-env:latest

# Run in headless mode (faster for training)
docker run -p 8000:8000 \
  -e UNITY_NO_GRAPHICS=1 \
  -e UNITY_TIME_SCALE=20 \
  unity-env:latest

# Run with persistent cache (avoid re-downloading binaries)
docker run -p 8000:8000 \
  -v ~/.mlagents-cache:/root/.mlagents-cache \
  unity-env:latest

Install Client Dependencies

To connect to the Docker container, install the client on your host machine:

cd envs/unity_env
pip install requests websockets

Quick Start

Option 1: Direct Mode (Fastest for Testing)

Run the Unity environment directly without a server:

# From the OpenEnv repository root:

# Run with graphics (default: 1280x720)
python examples/unity_simple.py --direct

# Run with custom window size
python examples/unity_simple.py --direct --width 800 --height 600

# Run headless (faster for training)
python examples/unity_simple.py --direct --no-graphics --time-scale 20

# Run 3DBall environment
python examples/unity_simple.py --direct --env 3DBall --episodes 5

Option 2: Server Mode

Start the server and connect with a client:

# Terminal 1: Start the server (graphics enabled by default)
cd envs/unity_env
uv run uvicorn server.app:app --host 0.0.0.0 --port 8000

# Terminal 2: Run the example client (from repo root)
python examples/unity_simple.py --url http://localhost:8000
python examples/unity_simple.py --url http://localhost:8000 --env 3DBall --episodes 5

Option 3: Docker Mode

Run via Docker container (auto-starts and connects):

# Build the Docker image first
cd envs/unity_env
docker build -f server/Dockerfile -t unity-env:latest .

# Run examples from repo root:
cd ../..

# Run with default settings
python examples/unity_simple.py --docker

# Run with custom window size
python examples/unity_simple.py --docker --width 1280 --height 720

# Run headless (faster for training)
python examples/unity_simple.py --docker --no-graphics --time-scale 20

# Run 3DBall for 10 episodes
python examples/unity_simple.py --docker --env 3DBall --episodes 10

# Use a custom Docker image
python examples/unity_simple.py --docker --docker-image my-unity-env:v1

Example Scripts

Basic Usage Examples

1. Direct Mode - Quick Testing

# Run PushBlock with graphics (default)
python examples/unity_simple.py --direct

# Output:
# ============================================================
# Unity ML-Agents Environment - Direct Mode
# ============================================================
# Environment: PushBlock
# Episodes: 3
# Max steps: 500
# Window size: 1280x720
# Graphics: Enabled
# ...

2. Direct Mode - Training Configuration

# Headless mode with fast simulation (20x speed)
python examples/unity_simple.py --direct --no-graphics --time-scale 20 --episodes 10 --max-steps 1000

# This is ideal for training - no graphics overhead, faster simulation

3. Direct Mode - 3DBall with Custom Window

# Run 3DBall (continuous actions) with larger window
python examples/unity_simple.py --direct --env 3DBall --width 1280 --height 720 --episodes 5

4. Docker Mode - Production-like Testing

# Build the image first
cd envs/unity_env
docker build -f server/Dockerfile -t unity-env:latest .

# Run via Docker with graphics
python examples/unity_simple.py --docker --width 1280 --height 720

# Run via Docker in headless mode for training
python examples/unity_simple.py --docker --no-graphics --time-scale 20 --episodes 20

5. Server Mode - Separate Server and Client

# Terminal 1: Start server with specific settings
UNITY_WIDTH=1280 UNITY_HEIGHT=720 uv run uvicorn server.app:app --port 8000

# Terminal 2: Connect and run episodes
python examples/unity_simple.py --url http://localhost:8000 --env PushBlock --episodes 5
python examples/unity_simple.py --url http://localhost:8000 --env 3DBall --episodes 5

6. Alternating Environments

# Run alternating episodes between PushBlock and 3DBall
python examples/unity_simple.py --direct --env both --episodes 6
# Episodes 1,3,5 = PushBlock; Episodes 2,4,6 = 3DBall

Command Line Options

Option	Default	Description
`--direct`	-	Run environment directly (no server)
`--docker`	-	Run via Docker container
`--url`	localhost:8000	Server URL for server mode
`--docker-image`	unity-env:latest	Docker image name
`--env`	PushBlock	Environment: PushBlock, 3DBall, both
`--episodes`	3	Number of episodes
`--max-steps`	500	Max steps per episode
`--width`	1280	Window width in pixels
`--height`	720	Window height in pixels
`--no-graphics`	-	Headless mode (faster)
`--time-scale`	1.0	Simulation speed multiplier
`--quality-level`	5	Graphics quality 0-5
`--quiet`	-	Reduce output verbosity

Python Client Usage

Connect to Server

from envs.unity_env import UnityEnv, UnityAction

# Connect to the server
with UnityEnv(base_url="http://localhost:8000") as client:
    # Reset to PushBlock environment
    result = client.reset(env_id="PushBlock")
    print(f"Observation dims: {len(result.observation.vector_observations)}")

    # Take actions
    for _ in range(100):
        # PushBlock actions: 0=noop, 1=forward, 2=backward,
        # 3=rotate_left, 4=rotate_right, 5=strafe_left, 6=strafe_right
        action = UnityAction(discrete_actions=[1])  # Move forward
        result = client.step(action)
        print(f"Reward: {result.reward}, Done: {result.done}")

        if result.done:
            result = client.reset()

Connect via Docker

from envs.unity_env import UnityEnv, UnityAction

# Automatically start Docker container and connect
client = UnityEnv.from_docker_image(
    "unity-env:latest",
    environment={
        "UNITY_NO_GRAPHICS": "0",
        "UNITY_WIDTH": "1280",
        "UNITY_HEIGHT": "720",
    }
)

try:
    result = client.reset(env_id="PushBlock")
    for _ in range(100):
        action = UnityAction(discrete_actions=[1])
        result = client.step(action)
finally:
    client.close()

Switch Environments Dynamically

# Start with PushBlock
result = client.reset(env_id="PushBlock")
# ... train on PushBlock ...

# Switch to 3DBall (continuous actions)
result = client.reset(env_id="3DBall")
action = UnityAction(continuous_actions=[0.5, -0.3])
result = client.step(action)

Direct Mode (Embedded Server)

from envs.unity_env.client import UnityEnv
from envs.unity_env.models import UnityAction

# Create client with embedded local server (no separate server needed)
client = UnityEnv.from_direct(
    env_id="PushBlock",
    no_graphics=False,  # Show graphics window
    width=1280,
    height=720,
    time_scale=1.0,
)

try:
    result = client.reset()
    print(f"Observation: {len(result.observation.vector_observations)} dimensions")

    for step in range(100):
        action = UnityAction(discrete_actions=[1])  # Move forward
        result = client.step(action)
        print(f"Step {step}: reward={result.reward}, done={result.done}")

        if result.done:
            result = client.reset()
finally:
    client.close()

Action Spaces

PushBlock (Discrete)

7 discrete actions:

0: No operation
1: Move forward
2: Move backward
3: Rotate left
4: Rotate right
5: Strafe left
6: Strafe right

action = UnityAction(discrete_actions=[1])  # Move forward

3DBall (Continuous)

2 continuous actions in range [-1, 1]:

Action 0: X-axis rotation
Action 1: Z-axis rotation

action = UnityAction(continuous_actions=[0.5, -0.3])

Observations

All environments provide vector observations. The size depends on the environment:

PushBlock: 70 dimensions (14 ray-casts detecting walls/goals/blocks)
3DBall: 8 dimensions (rotation and ball position/velocity)
GridWorld: Visual observations (grid view)

result = client.reset()
obs = result.observation

# Access observations
print(f"Vector obs: {obs.vector_observations}")
print(f"Behavior: {obs.behavior_name}")
print(f"Action spec: {obs.action_spec_info}")

Visual Observations (Optional)

Some environments support visual observations. Enable with include_visual=True:

result = client.reset(include_visual=True)
if result.observation.visual_observations:
    # Base64-encoded PNG images
    for img_b64 in result.observation.visual_observations:
        # Decode and use the image
        import base64
        img_bytes = base64.b64decode(img_b64)

Configuration

Direct Mode Arguments

When using UnityEnv.from_direct() to run with an embedded server:

from envs.unity_env.client import UnityEnv

client = UnityEnv.from_direct(
    env_id="PushBlock",      # Unity environment to load
    no_graphics=False,       # False = show graphics window
    width=1280,              # Window width in pixels
    height=720,              # Window height in pixels
    time_scale=1.0,          # Simulation speed (20.0 for fast training)
    quality_level=5,         # Graphics quality 0-5
    port=8765,               # Port for embedded server
)

Environment Variables

For Docker deployment, configure via environment variables:

Variable	Default	Description
`UNITY_ENV_ID`	PushBlock	Default Unity environment
`UNITY_NO_GRAPHICS`	0	Set to 1 for headless mode
`UNITY_WIDTH`	1280	Window width in pixels
`UNITY_HEIGHT`	720	Window height in pixels
`UNITY_TIME_SCALE`	1.0	Simulation speed multiplier
`UNITY_QUALITY_LEVEL`	5	Graphics quality 0-5
`UNITY_CACHE_DIR`	~/.mlagents-cache	Binary cache directory

Environment State

Access detailed environment information:

state = client.state()
print(f"Environment: {state.env_id}")
print(f"Episode ID: {state.episode_id}")
print(f"Step count: {state.step_count}")
print(f"Available envs: {state.available_envs}")
print(f"Action spec: {state.action_spec}")
print(f"Observation spec: {state.observation_spec}")

Troubleshooting

Docker Mode Fails on Apple Silicon (M1/M2/M3/M4)

Symptom: When running with --docker on Apple Silicon Macs, you see an error like:

Error running with Docker: Server error: The Unity environment took too long to respond...

Or in Docker logs:

* Assertion: should not be reached at tramp-amd64.c:605
Environment shut down with return code -6 (SIGABRT)

Cause: Unity ML-Agents binaries are x86_64 (Intel) only. When Docker runs the x86_64 Linux container on Apple Silicon, it uses QEMU emulation. The Mono runtime inside Unity has architecture-specific code that crashes under emulation.

Solutions:

Use Direct Mode (recommended for macOS):
```
python examples/unity_simple.py --direct --no-graphics
```
Direct mode downloads native macOS binaries which work on Apple Silicon.

Use Server Mode with a local server:

# Terminal 1: Start server (uses native macOS binaries)
uvicorn server.app:app --host 0.0.0.0 --port 8000

# Terminal 2: Run client
python examples/unity_simple.py --url http://localhost:8000

Use an x86_64 Linux machine for Docker mode: The Docker image works correctly on native x86_64 Linux machines (cloud VMs, dedicated servers, etc.).

First Run is Slow

The first run downloads Unity binaries (~500MB). This is normal and only happens once. Binaries are cached in ~/.mlagents-cache/.

Graphics Not Showing

Ensure --no-graphics is NOT set
On Linux, ensure X11 is available
For Docker, you may need to set up X11 forwarding

Docker Container Fails to Start

# Check Docker logs
docker logs <container_id>

# Ensure the image is built
docker images | grep unity-env

# Rebuild if necessary
cd envs/unity_env
docker build -f server/Dockerfile -t unity-env:latest .

Import Errors

# Ensure you're in the correct directory
cd envs/unity_env

# Install dependencies
uv sync
# or
pip install -e .

mlagents-envs Installation Issues

The mlagents-envs and mlagents packages are installed from source by default (via the GitHub repository). If you encounter issues or want to install manually:

# Clone the ml-agents repository
git clone https://github.com/Unity-Technologies/ml-agents.git
cd ml-agents

# Install mlagents-envs from source
pip install -e ./ml-agents-envs

# Install the full ml-agents package
pip install -e ./ml-agents

This approach is useful when:

You need to modify the mlagents source code
You want to use a specific branch or commit
The git dependency in pyproject.toml is causing issues

Caveats

First Run Download: Unity binaries (~500MB) are downloaded on first use
Platform-Specific: Binaries are platform-specific (macOS, Linux, Windows)
Apple Silicon + Docker: Docker mode does not work on Apple Silicon Macs due to x86_64 emulation issues with Unity’s Mono runtime. Use direct mode or server mode instead.
Single Worker: Unity environments are not thread-safe; use workers=1
Graphics Mode: Some features require X11/display for graphics mode
Multi-Agent: Currently uses first agent only; full multi-agent support planned

Dependencies

mlagents-envs (installed from source via git)
mlagents (installed from source via git)
numpy>=1.20.0
pillow>=9.0.0 (for visual observations)
openenv>=0.2.0

References

Update on GitHub

←TBench2 Wildfire→