gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e5761fa6f3332d42b7ac8be08346587ce1a86778
agentic-kvc/third_party/vllm/vllm
History
Gahow Wang e13391eeab Evict migrated blocks from prefix cache after KV send completes 
After a session migrates from C to D via offload, C's blocks were freed
to the LRU tail (most-recently-used position), making them the last to
be evicted. Since the session won't return to C, these blocks are dead
weight occupying cache capacity.

Now capture block IDs before _free_blocks and call evict_blocks to
remove them from the prefix cache hash table, so they can be reused
sooner for active sessions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-24 16:56:34 +08:00
..
assets
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
benchmarks
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
compilation
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
config
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
device_allocator
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
distributed
Fix multi-turn replay fidelity: track realized output tokens across all components
2026-05-24 14:47:51 +08:00
engine
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
entrypoints
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
inputs
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
kernels
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
logging_utils
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
lora
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
model_executor
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
multimodal
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
parser
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
platforms
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
plugins
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
profiler
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
ray
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
reasoning
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
renderers
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
third_party
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
tokenizers
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
tool_parsers
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
tracing
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
transformers_utils
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
triton_utils
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
usage
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
utils
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
v1
Evict migrated blocks from prefix cache after KV send completes
2026-05-24 16:56:34 +08:00
vllm_flash_attn
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
__init__.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
_aiter_ops.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
_custom_ops.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
_oink_ops.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
_xpu_ops.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
beam_search.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
collect_env.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
connections.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
env_override.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
envs.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
exceptions.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
forward_context.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
logger.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
logits_process.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
logprobs.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
model_inspection.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
outputs.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
pooling_params.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
py.typed
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
sampling_params.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
scalar_type.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
scripts.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
sequence.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
tasks.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00
version.py
Add vLLM v0.18.1 source tree with KV transfer abort fix
2026-05-22 00:30:38 +08:00