After E3 exposed massive session-level eviction (90 trims × avg
67K tokens/evict = 6.1M tokens trashed in 1h12min), we have to
acknowledge the local-patch sequence (E2→load-floor→Fix A →
proposed disable-migration → proposed disable-admission) was a
KVC-to-DP collapse trajectory, not a fix.
The fundamental issue: SessionAwareCache merged two responsibilities
that should be separate.
1. Session lifecycle tracking (legitimate — streaming sessions
reuse KV across turns and need per-session metadata).
2. Eviction granularity decision (wrong — sessions should not be
the eviction unit).
`release_session` frees the session-exclusive range
[cache_protected_len, kv_allocated_len), which is the post-radix-
commit tail accumulated over decode/extend. On Inferact's
50-session workload this is 35-87K tokens per session. The radix
tree never gets a chance to do block-level leaf-LRU on that range
because it was never committed there.
Effect: evict-revisit cycle forces full 50-90K re-prefill per
session per evict — which is exactly the per-request cost of naive
PD-disagg. KVC's direct-to-D fast-path advantage collapses.
The right fix is structural (not a patch): progressively commit
streaming-session decode output to the radix tree so SGLang's
block-level LRU can shed only the deepest leaves, preserving the
recent prefix that next-turn requests are most likely to match.
SessionSlot becomes pure metadata. Scope is ~1-2 weeks of vendored
SGLang refactor, orthogonal-and-complementary to the D→P sync work
proposed in RESEED_SLOW_PATH_AND_D_TO_P_GAP §4.
Doc lists five anti-patterns the next agent should avoid (tuning
migration_reject_threshold, disabling migration/admission, etc) —
all of those are local symptoms downstream of the eviction
granularity choice.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
