Deploying GLM-4.5-Air with SGLang on Dedicated GPUs

Running large language models on dedicated hardware grants enterprise developers complete control over data privacy, serving configuration, and hardware utilization. In this guide, we explore how to deploy the GLM-4.5-Air-FP8 mixture-of-experts (MoE) model using SGLang on an NVIDIA GPU server, exposing a high-performance OpenAI-compatible API endpoint.

While self-hosting provides deep control, managing dedicated GPU infrastructure can introduce operational overhead and high capital expenses. For developers seeking instant, scalable access without managing server clusters, platforms like n1n.ai provide unified, high-speed API access to state-of-the-art models like GLM-4.5-Air, Claude 3.5 Sonnet, and DeepSeek-V3 with latency < 50ms.

Understanding GLM-4.5-Air-FP8 Architecture

GLM-4.5-Air is an advanced MoE model engineered for advanced reasoning, complex coding workflows, and autonomous agent tasks. Featuring approximately 106B total parameters with roughly 12B active parameters per token during inference, it delivers exceptional performance-to-compute ratios.

When deploying the FP8 quantized checkpoint (zai-org/GLM-4.5-Air-FP8), hardware sizing is critical. Based on Z.ai benchmarks, recommended hardware configurations include:

Step 1: Server Provisioning and Driver Setup

Begin by updating your Ubuntu environment and installing essential compilation tools and NVIDIA drivers:

sudo apt update && sudo apt upgrade -y
sudo apt install -y curl git wget build-essential
ubuntu-drivers devices
sudo ubuntu-drivers install
sudo reboot

After rebooting, verify your GPU topology and driver status:

nvidia-smi
nvidia-smi -L
nvidia-smi topo -m

Step 2: Environment Setup with SGLang

We will use uv for lightning-fast Python package management and isolate our virtual environment:

curl -LsSf https://astral.sh/uv/install.sh | sh
source "$HOME/.local/bin/env"
mkdir -p ~/glm-deployment && cd ~/glm-deployment
uv venv --python 3.12
source .venv/bin/activate
uv pip install --prerelease=allow sglang

Configure your Hugging Face cache directory on high-capacity local NVMe storage:

sudo install -d -o "$USER" -g "$(id -gn)" /data/huggingface
echo 'export HF_HOME=/data/huggingface' >> ~/.bashrc
source ~/.bashrc

Step 3: Launching the SGLang Inference Server

Run the SGLang launch script to spin up the serving engine. Binding to 127.0.0.1 secures the endpoint locally before setting up your reverse proxy:

python -m sglang.launch_server \
    --model-path zai-org/GLM-4.5-Air-FP8 \
    --served-model-name glm-4.5-air-fp8 \
    --host 127.0.0.1 \
    --port 30000

Verify that the service is running and correctly listing the model:

curl http://127.0.0.1:30000/v1/models

Step 4: Interfacing via OpenAI Python SDK

Because SGLang provides an OpenAI-compatible API layer, you can seamlessly integrate your self-hosted model using standard libraries:

from openai import OpenAI

client = OpenAI(
    api_key="local-development",
    base_url="http://127.0.0.1:30000/v1"
)

response = client.chat.completions.create(
    model="glm-4.5-air-fp8",
    messages=[{"role": "user", "content": "Explain the role of GPU memory in LLM inference."}],
    max_tokens=300,
)

print(response.choices[0].message.content)

Production Pro-Tips & Security

1. Systemd Automation: Always wrap your SGLang command inside a systemd service unit to ensure automatic recovery on server reboots or OOM terminations.
2. Network Security: Never expose raw HTTP inference ports directly to the public internet. Put them behind an Nginx or Caddy reverse proxy equipped with TLS termination, API key authentication, and rate-limiting.
3. API Aggregation Alternative: If maintaining cluster uptime, GPU monitoring, and quantization compatibility becomes too burdensome, enterprise engineering teams often route production traffic through n1n.ai to eliminate infrastructure overhead while retaining top-tier model performance.

Get a free API key at n1n.ai