Environment Variables

Apple MPS

Environment Variable	Default	Description
`SGLANG_USE_MLX`	not set	Set to `1` to enable MLX fused Metal kernels for norm ops on MPS

Caching Acceleration

These variables configure caching acceleration for Diffusion Transformer (DiT) models. SGLang supports multiple caching strategies - see caching documentation for an overview.

Cache-DiT Configuration

See cache-dit documentation for detailed configuration.

Environment Variable	Default	Description
`SGLANG_CACHE_DIT_ENABLED`	false	Enable Cache-DiT acceleration
`SGLANG_CACHE_DIT_FN`	1	First N blocks to always compute
`SGLANG_CACHE_DIT_BN`	0	Last N blocks to always compute
`SGLANG_CACHE_DIT_WARMUP`	4	Warmup steps before caching
`SGLANG_CACHE_DIT_RDT`	0.24	Residual difference threshold
`SGLANG_CACHE_DIT_MC`	3	Max continuous cached steps
`SGLANG_CACHE_DIT_TAYLORSEER`	false	Enable TaylorSeer calibrator
`SGLANG_CACHE_DIT_TS_ORDER`	1	TaylorSeer order (1 or 2)
`SGLANG_CACHE_DIT_SCM_PRESET`	none	SCM preset (none/slow/medium/fast/ultra)
`SGLANG_CACHE_DIT_SCM_POLICY`	dynamic	SCM caching policy
`SGLANG_CACHE_DIT_SCM_COMPUTE_BINS`	not set	Custom SCM compute bins
`SGLANG_CACHE_DIT_SCM_CACHE_BINS`	not set	Custom SCM cache bins

Cloud Storage

These variables configure S3-compatible cloud storage for automatically uploading generated images and videos.

Environment Variable	Default	Description
`SGLANG_CLOUD_STORAGE_TYPE`	not set	Set to `s3` to enable cloud storage
`SGLANG_S3_BUCKET_NAME`	not set	The name of the S3 bucket
`SGLANG_S3_ENDPOINT_URL`	not set	Custom endpoint URL (for MinIO, OSS, etc.)
`SGLANG_S3_REGION_NAME`	us-east-1	AWS region name
`SGLANG_S3_ACCESS_KEY_ID`	not set	AWS Access Key ID
`SGLANG_S3_SECRET_ACCESS_KEY`	not set	AWS Secret Access Key

CUDA Crash Debugging

These variables enable kernel API logging and optional input/output dumps around diffusion CUDA kernel call boundaries. They are useful when tracking down CUDA crashes such as illegal memory access, device-side assert, or shape mismatches in custom kernels.

Environment Variable	Default	Description
`SGLANG_KERNEL_API_LOGLEVEL`	`0`	Controls crash-debug kernel API logging. `1` logs API names, `3` logs tensor metadata, `5` adds tensor statistics, and `10` also writes dump snapshots.
`SGLANG_KERNEL_API_LOGDEST`	`stdout`	Destination for crash-debug kernel API logs. Use `stdout`, `stderr`, or a file path. `%i` is replaced with the process PID.
`SGLANG_KERNEL_API_DUMP_DIR`	`sglang_kernel_api_dumps`	Output directory for level-10 kernel API dumps. `%i` is replaced with the process PID.
`SGLANG_KERNEL_API_DUMP_INCLUDE`	not set	Comma-separated wildcard patterns for kernel API names to include in level-10 dumps.
`SGLANG_KERNEL_API_DUMP_EXCLUDE`	not set	Comma-separated wildcard patterns for kernel API names to exclude from level-10 dumps.

4.0 KiB Raw Blame History