SGLang diffusion is an inference framework for accelerated image/video generation.
SGLang diffusion features an end-to-end unified pipeline for accelerating diffusion models. It is designed to be modular and extensible, allowing users to easily add new models and optimizations.
Key Features
SGLang Diffusion has the following features:
- Broad model support: Wan series, FastWan series, Hunyuan, LTX-2, Qwen-Image, Qwen-Image-Edit, Flux, Z-Image, GLM-Image
- Fast inference speed: enpowered by highly optimized kernel from sgl-kernel and efficient scheduler loop
- Ease of use: OpenAI-compatible api, CLI, and python sdk support
- Multi-platform support:
- NVIDIA GPUs (H100, H200, A100, B200, 4090)
- AMD GPUs (MI300X, MI325X)
- Ascend NPU (A2, A3)
- Apple Silicon (M-series via MPS)
- Moore Threads GPUs (MTT S5000)
AMD/ROCm Support
SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the installation guide for setup instructions.
Moore Threads/MUSA Support
SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software stack. On MUSA platforms, we use the Torch SDPA backend for attention. See the installation guide for setup instructions.
Apple MPS Support
SGLang Diffusion supports Apple Silicon (M-series) via the MPS backend. Since Triton is Linux-only, all Triton kernels are replaced with PyTorch-native fallbacks on MPS. Norm operations can be optionally accelerated with MLX fused Metal kernels (SGLANG_USE_MLX=1). See the installation guide for setup instructions.
Getting Started
uv pip install 'sglang[diffusion]' --prerelease=allow
For more installation methods (e.g. pypi, uv, docker, ROCm/AMD, MUSA/Moore Threads), check install.md.
Inference
Here's a minimal example to generate a video using the default settings:
from sglang.multimodal_gen import DiffGenerator
def main():
# Create a diff generator from a pre-trained model
generator = DiffGenerator.from_pretrained(
model_path="Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
num_gpus=1, # Adjust based on your hardware
)
# Generate the video
video = generator.generate(
sampling_params_kwargs=dict(
prompt="A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes wide with interest.",
return_frames=True, # Also return frames from this call (defaults to False)
output_path="my_videos/", # Controls where videos are saved
save_output=True
)
)
if __name__ == '__main__':
main()
Or, more simply, with the CLI:
sglang generate --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
--text-encoder-cpu-offload --pin-cpu-memory \
--prompt "A curious raccoon" \
--save-output
For LTX-2 two-stage generation, use --pipeline-class-name LTX2TwoStagePipeline. The
spatial upsampler and distilled LoRA are auto-resolved from the same model snapshot by
default, and can still be overridden with --spatial-upsampler-path and
--distilled-lora-path when needed.
LoRA support
Apply LoRA adapters via --lora-path:
sglang generate \
--model-path Qwen/Qwen-Image-Edit-2511 \
--lora-path prithivMLmods/Qwen-Image-Edit-2511-Anime \
--prompt "Transform into anime." \
--image-path "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png" \
--save-output
For more usage examples (e.g. OpenAI compatible API, server mode), check cli.md.
Contributing
All contributions are welcome. The contribution guide is available here.
Acknowledgement
We learnt and reused code from the following projects: