Full v3 trace re-profile with layer-wise: matched migrations improve

Browse Source

1213/1214 success; matched migrations (4 common) improved -2.6 to -7.2s,
scaling with prefill hidden behind transfer. Trace-level TTFT p90 -6% / p99
-5% (modest: migrations are 2% of reqs and partly queue-bound). Confirms
layer-wise removes the transfer half of migration overhead but not the
control-plane/queue residual. DESIGN.md updated with results.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

...

This commit is contained in:

Gahow Wang

2026-05-28 19:16:37 +08:00

parent 21db2affb4

commit 63387f614d

5 changed files with 2467 additions and 1 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1

microbench/connector_tax/layerwise/results/trace/layerwise_breakdown.json Normal file

File diff suppressed because one or more lines are too long