Files

Gahow Wang 445e491123 Add vLLM v0.18.1 source tree with KV transfer abort fix

third_party/vllm/ now tracked in git for direct patch management.
Based on vLLM v0.18.1 release with one patch applied:

  vllm/v1/core/sched/scheduler.py:
    Replace fatal assert with graceful skip when KV transfer callback
    arrives for an already-aborted request during PD disaggregated serving.

Future vLLM modifications should be made directly in third_party/vllm/
and committed normally. The patches/ directory is kept as documentation
of what changed from upstream.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-22 00:30:38 +08:00

mooncake_connector

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

disagg_proxy_demo.py

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

kv_events.sh

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

moriio_toy_proxy_server.py

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

README.md

Add vLLM v0.18.1 source tree with KV transfer abort fix

2026-05-22 00:30:38 +08:00

README.md

Disaggregated Serving

This example contains scripts that demonstrate the disaggregated serving features of vLLM.

Files

disagg_proxy_demo.py - Demonstrates XpYd (X prefill instances, Y decode instances).
kv_events.sh - Demonstrates KV cache event publishing.
mooncake_connector - A proxy demo for MooncakeConnector.