Gahow Wang e77bdcac5a Layerwise under load: overlap benefit survives (bg=16) ...

mb7 with background decode load (8/instance). Critical-path transfer overhead
stays ~constant ~90ms for layerwise vs 158/239/749ms baseline (up to 7.9x at
32k), prefill not slowed, KV correct. Confirms the overlap holds on busy
instances. DESIGN.md updated with idle-vs-load table + the two blockers
(chunk-safety, concurrent-transfer safety) that the full 1200-req trace needs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-28 16:30:14 +08:00

..

connector_tax

Layerwise under load: overlap benefit survives (bg=16)

2026-05-28 16:30:14 +08:00

fresh_setup

MB5 driver fixes: bash env-prefix + replayer flag names + python date math

2026-05-27 23:23:23 +08:00

interference

Microbench 1 plots: prefill-decode interference heatmap + lines

2026-05-26 14:21:30 +08:00

lifecycle

PD-sep server-side profiling: vLLM patches + per-request breakdown

2026-05-26 13:59:09 +08:00

patches

PD-sep server-side profiling: vLLM patches + per-request breakdown

2026-05-26 13:59:09 +08:00

__init__.py

Microbench 3 (connector_tax): infrastructure for KV connector substrate tax

2026-05-26 17:27:41 +08:00

ANALYSIS.md

Microbench: prefill-decode interference + PD transfer lifecycle

2026-05-26 00:57:06 +08:00

interference_microbench_design.md

Microbench: prefill-decode interference + PD transfer lifecycle

2026-05-26 00:57:06 +08:00

plot_breakdown_real.py

PD-sep server-side profiling: vLLM patches + per-request breakdown

2026-05-26 13:59:09 +08:00

plot_breakdown.py

PD-sep server-side profiling: vLLM patches + per-request breakdown

2026-05-26 13:59:09 +08:00

plot_interference.py

Microbench 1 plots: prefill-decode interference heatmap + lines

2026-05-26 14:21:30 +08:00

TODO.md

Microbench: prefill-decode interference + PD transfer lifecycle

2026-05-26 00:57:06 +08:00

transfer_lifecycle_design.md

Microbench: prefill-decode interference + PD transfer lifecycle

2026-05-26 00:57:06 +08:00