Gahow Wang
445e491123
third_party/vllm/ now tracked in git for direct patch management.
Based on vLLM v0.18.1 release with one patch applied:
vllm/v1/core/sched/scheduler.py:
Replace fatal assert with graceful skip when KV transfer callback
arrives for an already-aborted request during PD disaggregated serving.
Future vLLM modifications should be made directly in third_party/vllm/
and committed normally. The patches/ directory is kept as documentation
of what changed from upstream.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
15 lines
815 B
Markdown
15 lines
815 B
Markdown
# KubeAI
|
|
|
|
[KubeAI](https://github.com/substratusai/kubeai) is a Kubernetes operator that enables you to deploy and manage AI models on Kubernetes. It provides a simple and scalable way to deploy vLLM in production. Functionality such as scale-from-zero, load based autoscaling, model caching, and much more is provided out of the box with zero external dependencies.
|
|
|
|
Please see the Installation Guides for environment specific instructions:
|
|
|
|
- [Any Kubernetes Cluster](https://www.kubeai.org/installation/any/)
|
|
- [AKS](https://www.kubeai.org/installation/aks/)
|
|
- [EKS](https://www.kubeai.org/installation/eks/)
|
|
- [GKE](https://www.kubeai.org/installation/gke/)
|
|
|
|
Once you have KubeAI installed, you can
|
|
[configure text generation models](https://www.kubeai.org/how-to/configure-text-generation-models/)
|
|
using vLLM.
|