f2b: regenerate CDF from production trace (1.3M sessions on dash0)
Pulls 456 (rank%, cum%) sample points from the raw production trace at dash0:/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl, cached locally so the figure is reproducible without ssh access. Sampled anchors match the precomputed summary exactly: top 1% = 46.5%, top 5% = 66.5%, top 10% = 74.6% plus newly readable points: top 25% = 87.5%, top 50% = 96.0% Workload characterization is now consistent with the production distribution rather than the small replay subset. Replay window CDF kept as an overlay to show the same hockey-stick shape on the data §5 actually uses. - analysis/characterization/data/production_session_skew_cdf.json: cached sample points (29 KB), so the figure rebuilds locally - scripts/plot_session_skew_cdf.py: now plots from the cache + replay raw - MEETING.md / PAPER_OUTLINE.md: revert numbers to production trace, add top-25%/50% data points Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -26,7 +26,7 @@ L = Λ · N · W_turn(L) # agentic, T_human≈0
|
||||
| | 数据 | 图 |
|
||||
|---|---|---|
|
||||
| KV reuse 几乎只在 session 内 | intra 93.2% / cross 5.7% / shared 1.1% |  |
|
||||
| Session 极度偏斜 | replay 上 top 1% / 5% / 10% = 24% / 62% / 76% input mass(production 全 trace 更陡,top 1% = 46.5%) |  |
|
||||
| Session 极度偏斜 | production trace 上 top 1% / 5% / 10% / 25% / 50% = **46.5% / 66.5% / 74.6% / 87.5% / 96.0%** input mass |  |
|
||||
| 单请求 KV footprint 已经很大 | p99 = 11.8 GiB ≈ H20 12% |  |
|
||||
|
||||
理论 APC 上界 = intra-session 79.6% / any-session 80.3%,差 <1pp。**任何不 affinity 的调度都丢绝大部分 reuse。**
|
||||
@@ -58,7 +58,7 @@ agentic 平均请求 33.6k token 需 3.3GB KV;4P+4D / 6P+2D 在 agentic regime
|
||||
| sticky | **20.3s** | 55.4s | **34.6s** |
|
||||
| unified | **10.3s** | 37.7s | **18.0s** |
|
||||
|
||||
机制:top 5% 的 session 占 ~62% input 量、且 hot session 数量远多于 instance 数(8 个),sticky 的 hash 绑定让 **每个 worker 都自己承接一份 hot session**,median worker 也被拖慢。Unified 用 LMetric fallback 把 cold/new session 重路由到非 hot worker,保留 7/8 worker 的速度。系统 p90 由大多数请求决定,所以 unified 几乎 2x 快。
|
||||
机制:production trace 上 top 1% 的 session 占 46.5% input 量、且 hot session 数量远多于 instance 数(8 个),sticky 的 hash 绑定让 **每个 worker 都自己承接一份 hot session**,median worker 也被拖慢。Unified 用 LMetric fallback 把 cold/new session 重路由到非 hot worker,保留 7/8 worker 的速度。系统 p90 由大多数请求决定,所以 unified 几乎 2x 快。
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user