Gahow Wang 1d2148cf65 Remove second push_new gate that caused downgrade-to-cold-LOCAL ...

After _push_allowed was relaxed, the cost model correctly chose push
for high-cache sessions on overloaded instances. But a second gate at
execution time (push_new < heavy_threshold) blocked the actual offload,
downgrading to LOCAL on the target instance — which had no cache.
Worse, session affinity was already updated to the target, so all
subsequent turns also hit cold prefill.

This was the root cause of relaxed gate's performance regression:
affinity broken + push blocked = worst of both worlds.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-25 00:42:31 +08:00

analysis

Add comprehensive research findings document

2026-05-23 07:16:31 +08:00

docs

Add migration policy design doc with A/B experiment results

2026-05-24 18:24:49 +08:00

experiments

Add elastic PS evaluation plan for production-realistic trace

2026-05-23 15:56:05 +08:00

patches

Add vLLM patches directory for version-controlled patch management

2026-05-22 00:26:14 +08:00

replayer

Fix multi-turn replay fidelity: track realized output tokens across all components

2026-05-24 14:47:51 +08:00

scripts

Remove second push_new gate that caused downgrade-to-cold-LOCAL

2026-05-25 00:42:31 +08:00

tests

proxy: Settings dataclass + cache-ratio gate + P-pick offload penalty (B4, M2, M3, D5)

2026-05-23 21:11:17 +08:00

third_party/vllm

Evict migrated blocks from prefix cache after KV send completes

2026-05-24 16:56:34 +08:00

.gitignore

Phase 1 milestone: system-level analysis + reproducible report

2026-05-22 16:17:41 +08:00

FIXES.md

Add FIXES.md with prioritized repo cleanup checklist

2026-05-23 20:35:56 +08:00

pyproject.toml

tests: add minimal coverage for percentile + proxy routing (S1)

2026-05-23 21:07:14 +08:00

REPORT.md

Fix A+C: real cache sync + cached-prefill-on-C architecture

2026-05-24 11:22:38 +08:00

TODO.md

LMetric routing policy (OSDI'26) + A/B results vs linear baseline

2026-05-22 16:57:32 +08:00

uv.lock

Fix NONE_HASH import: use module ref instead of from-import (value binding bug)

2026-05-24 01:32:19 +08:00

Description

No description provided

48 MiB

Languages

Python 82.9%

Shell 17.1%