gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v2 exp(d): expand figure to 6 panels (TTFT/E2E mean+p90, TPS, per-worker GPU util)

Browse Source 
Per request: TTFT mean+p90, E2E mean+p90, decode TPS (output goodput; total/
prefill TPS omitted as cache-miss-inflated), and per-worker GPU-util boxplots
(8 workers/arm, tracets vs thinktime) showing utilization level + balance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-05-30 21:10:27 +08:00
parent 9b6091fe6e
commit 0b180c191e
3 changed files with 65 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
v2/exp_d_policy_dispatch/README.md
View File

@@ -27,6 +27,13 @@ Analyzer: `scripts/bench_report.py` (summaries in `results/`).
## Result (ms; `figs/exp_d_policy_dispatch.png`)
The figure has 6 panels — TTFT mean/p90, E2E mean/p90, decode TPS (output
goodput), and the per-worker GPU-util box (8 workers/arm). Decode TPS is the
honest throughput metric (total/prefill TPS is inflated by cache-miss recompute,
e.g. LMetric); thinktime ≥ tracets on it everywhere (the system drains faster
with real think-time). The GPU-util box shows LPWL also keeps the tightest
worker balance.
| policy | mode | TTFT p90 | E2E mean | E2E p90 | E2E p99 | TPOT p90 | APC | req-bal |
|---|---|---:|---:|---:|---:|---:|---:|---:|
| **LPWL** | tracets | 11099 | 9827 | 25366 | 93929 | 33 | 0.650 | **1.49×** |