Gahow Wang
445e491123
third_party/vllm/ now tracked in git for direct patch management.
Based on vLLM v0.18.1 release with one patch applied:
vllm/v1/core/sched/scheduler.py:
Replace fatal assert with graceful skip when KV transfer callback
arrives for an already-aborted request during PD disaggregated serving.
Future vLLM modifications should be made directly in third_party/vllm/
and committed normally. The patches/ directory is kept as documentation
of what changed from upstream.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Benchmarks
This directory used to contain vLLM's benchmark scripts and utilities for performance testing and evaluation.
Contents
- Serving benchmarks: Scripts for testing online inference performance (latency, throughput)
- Throughput benchmarks: Scripts for testing offline batch inference performance
- Specialized benchmarks: Tools for testing specific features like structured output, prefix caching, long document QA, request prioritization, and multi-modal inference
- Dataset utilities: Framework for loading and sampling from various benchmark datasets (ShareGPT, HuggingFace datasets, synthetic data, etc.)
Usage
For detailed usage instructions, examples, and dataset information, see the Benchmark CLI documentation.
For full CLI reference see: