Files

Gahow Wang 445e491123 Add vLLM v0.18.1 source tree with KV transfer abort fix

third_party/vllm/ now tracked in git for direct patch management.
Based on vLLM v0.18.1 release with one patch applied:

  vllm/v1/core/sched/scheduler.py:
    Replace fatal assert with graceful skip when KV transfer callback
    arrives for an already-aborted request during PD disaggregated serving.

Future vLLM modifications should be made directly in third_party/vllm/
and committed normally. The patches/ directory is kept as documentation
of what changed from upstream.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-22 00:30:38 +08:00

quantization

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

speculative_decoding

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

automatic_prefix_caching.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

batch_invariance.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

custom_arguments.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

custom_logitsprocs.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

disagg_encoder.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

disagg_prefill.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

interleaved_thinking.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

lora.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

mooncake_connector_usage.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

multimodal_inputs.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

nixl_connector_usage.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

prompt_embeds.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

README.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

reasoning_outputs.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

sleep_mode.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

structured_outputs.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

tool_calling.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

README.md

Features

Compatibility Matrix

The tables below show mutually exclusive features and the support on some hardware.

The symbols used have the following meanings:

✅ = Full compatibility
🟠 = Partial compatibility
❌ = No compatibility
❔ = Unknown or TBD

!!! note Check the ❌ or 🟠 with links to see tracking issue for unsupported feature/hardware combination.

Feature x Feature

Feature	CP	APC	LoRA	SD	CUDA graph	pooling	enc-dec	logP	prmpt logP	async output	multi-step	mm	best-of	beam-search	prompt-embeds
CP	✅
APC	✅	✅
LoRA	✅	✅	✅
SD	✅	✅	❌	✅
CUDA graph	✅	✅	✅	✅	✅
pooling	🟠*	🟠*	✅	❌	✅	✅
enc-dec	❌	❌	❌	❌	✅	✅	✅
logP	✅	✅	✅	✅	✅	❌	✅	✅
prmpt logP	✅	✅	✅	✅	✅	❌	✅	✅	✅
async output	✅	✅	✅	❌	✅	❌	❌	✅	✅	✅
multi-step	❌	✅	❌	❌	✅	❌	❌	✅	✅	✅	✅
mm	✅	✅	🟠^{^}	❔	✅	✅	✅	✅	✅	✅	❔	✅
best-of	✅	✅	✅	❌	✅	❌	✅	✅	✅	❔	❌	✅	✅
beam-search	✅	✅	✅	❌	✅	❌	✅	✅	✅	❔	❌	❔	✅	✅
prompt-embeds	✅	✅	✅	❌	✅	❌	❌	✅	❌	❔	❔	❌	❔	❔	✅

* Chunked prefill and prefix caching are only applicable to last-token or all pooling with causal attention.
^{^} LoRA is only applicable to the language backbone of multimodal models.

Feature x Hardware

Feature	Volta	Turing	Ampere	Ada	Hopper	CPU	AMD	Intel GPU
CP	❌	✅	✅	✅	✅	✅	✅	✅
APC	❌	✅	✅	✅	✅	✅	✅	✅
LoRA	✅	✅	✅	✅	✅	✅	✅	✅
SD	✅	✅	✅	✅	✅	❌	✅	✅
CUDA graph	✅	✅	✅	✅	✅	❌	✅	❌
pooling	✅	✅	✅	✅	✅	✅	✅	✅
enc-dec	✅	✅	✅	✅	✅	✅	❌	✅
mm	✅	✅	✅	✅	✅	✅	✅	✅
prompt-embeds	✅	✅	✅	✅	✅	✅	❔	✅
logP	✅	✅	✅	✅	✅	✅	✅	✅
prmpt logP	✅	✅	✅	✅	✅	✅	✅	✅
async output	✅	✅	✅	✅	✅	❌	❌	✅
multi-step	✅	✅	✅	✅	✅	❌	✅	✅
best-of	✅	✅	✅	✅	✅	✅	✅	✅
beam-search	✅	✅	✅	✅	✅	✅	✅	✅

!!! note For information on feature support on Google TPU, please refer to the TPU-Inference Recommended Models and Features documentation.