Files

Gahow Wang 445e491123 Add vLLM v0.18.1 source tree with KV transfer abort fix

third_party/vllm/ now tracked in git for direct patch management.
Based on vLLM v0.18.1 release with one patch applied:

  vllm/v1/core/sched/scheduler.py:
    Replace fatal assert with graceful skip when KV transfer callback
    arrives for an already-aborted request during PD disaggregated serving.

Future vLLM modifications should be made directly in third_party/vllm/
and committed normally. The patches/ directory is kept as documentation
of what changed from upstream.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-22 00:30:38 +08:00

2.2 KiB

Raw Blame History

Summary

Configuration

API documentation for vLLM's configuration classes.

[vllm.config.ModelConfig][]
[vllm.config.CacheConfig][]
[vllm.config.LoadConfig][]
[vllm.config.ParallelConfig][]
[vllm.config.SchedulerConfig][]
[vllm.config.DeviceConfig][]
[vllm.config.SpeculativeConfig][]
[vllm.config.LoRAConfig][]
[vllm.config.MultiModalConfig][]
[vllm.config.PoolerConfig][]
[vllm.config.StructuredOutputsConfig][]
[vllm.config.ProfilerConfig][]
[vllm.config.ObservabilityConfig][]
[vllm.config.KVTransferConfig][]
[vllm.config.CompilationConfig][]
[vllm.config.VllmConfig][]

Offline Inference

LLM Class.

[vllm.LLM][]

LLM Inputs.

[vllm.inputs.PromptType][]
[vllm.inputs.TextPrompt][]
[vllm.inputs.TokensPrompt][]

vLLM Engines

Engine classes for offline and online inference.

[vllm.LLMEngine][]
[vllm.AsyncLLMEngine][]

Inference Parameters

Inference parameters for vLLM APIs.

[vllm.SamplingParams][]
[vllm.PoolingParams][]

Multi-Modality

vLLM provides experimental support for multi-modal models through the [vllm.multimodal][] package.

Multi-modal inputs can be passed alongside text and token prompts to supported models via the multi_modal_data field in [vllm.inputs.PromptType][].

Looking to add your own multi-modal model? Please follow the instructions listed here.

[vllm.multimodal.MULTIMODAL_REGISTRY][]

Inputs

User-facing inputs.

[vllm.multimodal.inputs.MultiModalDataDict][]

Internal data structures.

[vllm.multimodal.inputs.PlaceholderRange][]
[vllm.multimodal.inputs.NestedTensors][]
[vllm.multimodal.inputs.MultiModalFieldElem][]
[vllm.multimodal.inputs.MultiModalFieldConfig][]
[vllm.multimodal.inputs.MultiModalKwargsItem][]
[vllm.multimodal.inputs.MultiModalKwargsItems][]
[vllm.multimodal.inputs.MultiModalInputs][]

Data Parsing

[vllm.multimodal.parse][]

Data Processing

[vllm.multimodal.processing][]

Registry

[vllm.multimodal.registry][]

Model Development

[vllm.model_executor.models.interfaces_base][]
[vllm.model_executor.models.interfaces][]
[vllm.model_executor.models.adapters][]

2.2 KiB Raw Blame History