Files

Gahow Wang 445e491123 Add vLLM v0.18.1 source tree with KV transfer abort fix

third_party/vllm/ now tracked in git for direct patch management.
Based on vLLM v0.18.1 release with one patch applied:

  vllm/v1/core/sched/scheduler.py:
    Replace fatal assert with graceful skip when KV transfer callback
    arrives for an already-aborted request during PD disaggregated serving.

Future vLLM modifications should be made directly in third_party/vllm/
and committed normally. The patches/ directory is kept as documentation
of what changed from upstream.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-22 00:30:38 +08:00

2.3 KiB

Raw Blame History

Monitoring Dashboards

This directory contains monitoring dashboard configurations for vLLM, providing comprehensive observability for your vLLM deployments.

Dashboard Platforms

We provide dashboards for two popular observability platforms:

Grafana
Perses

Dashboard Format Approach

All dashboards are provided in native formats that work across different deployment methods:

Grafana (JSON)

✅ Works with any Grafana instance (cloud, self-hosted, Docker)
✅ Direct import via Grafana UI or API
✅ Can be wrapped in Kubernetes operators when needed
✅ No vendor lock-in or deployment dependencies

Perses (YAML)

✅ Works with standalone Perses instances
✅ Compatible with Perses API and CLI
✅ Supports Dashboard-as-Code workflows
✅ Can be wrapped in Kubernetes operators when needed

Dashboard Contents

Both platforms provide equivalent monitoring capabilities:

Dashboard	Description
Performance Statistics	Tracks latency, throughput, and performance metrics
Query Statistics	Monitors request volume, query performance, and KPIs

Quick Start

First, navigate to this example's directory:

cd examples/online_serving/dashboards

Grafana

Import the JSON directly into the Grafana UI, or use the API:

curl -X POST http://grafana/api/dashboards/db \
  -H "Content-Type: application/json" \
  -d @grafana/performance_statistics.json

Perses

Import via the Perses CLI:

percli apply -f perses/performance_statistics.yaml

Requirements

Prometheus metrics from your vLLM deployment
Data source configured in your monitoring platform
vLLM metrics enabled and accessible

Platform-Specific Documentation

For detailed deployment instructions and platform-specific options, see:

Grafana Documentation - JSON dashboards, operator usage, manual import
Perses Documentation - YAML specs, CLI usage, operator wrapping

Contributing

When adding new dashboards, please:

Provide native formats (JSON for Grafana, YAML specs for Perses)
Update platform-specific README files
Ensure dashboards work across deployment methods
Test with the latest platform versions

2.3 KiB Raw Blame History