MB5 PD-disagg pipeline: working end-to-end
Three independent bugs were blocking PD-disagg smoke; each fix is
isolated so the next PD experiment doesn't re-hit them.
1. mb5_launch.sh
- stop_all() also kills mb5_pd_proxy.py (our vendored copy),
not just the upstream filename, and asserts ports 8000-8007 +
PROXY_PORT are free before launching — stale proxies were
silently passing the readiness check.
- Proxy readiness uses a generic "any HTTP response" probe;
mooncake_connector_proxy only exposes /v1/completions so
/v1/models 404 is expected.
2. mb5_pd_proxy.py (vendored from third_party so deploy.sh ships it)
- Force min_tokens=1 on the prefill leg. Clients that set
min_tokens == max_tokens (our replayer does) collide with
vLLM's min_tokens<=max_tokens check after the proxy caps
max_tokens=1.
3. instrument_kv_snapshot.py
- Adds a second patch target: initialize
MooncakeConnectorWorker.bootstrap_server = None in __init__.
vLLM 0.18.1 only sets it under the is_kv_producer branch, so
kv_consumer hits AttributeError as soon as the first remote
prefill request lands.
- apply/revert refactored to iterate over (path, patches) pairs.
plot_kv_pool_timeline.py also handles snapshot files that never
captured a running request (would otherwise IndexError on an empty
stackplot input).
Smoke: 4P+4D × 20 reqs → 20/20 success, mean 3.9s, p99 17s, 8 PIDs
all writing snapshots (601 total), well above the 8C baseline.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -70,6 +70,29 @@ def plot_one_instance(snaps: list[dict], out: Path, title: str) -> None:
|
||||
# Sort by first-seen time so the band order follows arrival
|
||||
all_req_ids.sort(key=lambda r: req_first_seen[r])
|
||||
|
||||
if not all_req_ids:
|
||||
# No requests ever ran on this instance; plot a flat used_blocks line
|
||||
# instead of the stackplot (which can't handle empty input).
|
||||
fig, ax1 = plt.subplots(figsize=(13, 4))
|
||||
used = [s["used_blocks"] for s in snaps]
|
||||
ax1.plot(times, used, color="#888", lw=1.5, label="used_blocks (no running reqs sampled)")
|
||||
ax1.axhline(total_blocks, color="#444", lw=1.5, ls="-",
|
||||
label=f"pool total = {total_blocks} blocks")
|
||||
ax1.axhline(total_blocks * 0.9, color="#c44e52", lw=1.2, ls="--", alpha=0.7,
|
||||
label="90% capacity")
|
||||
ax1.set_ylabel("KV blocks")
|
||||
ax1.set_ylim(0, total_blocks * 1.05)
|
||||
ax1.set_xlabel("wall-clock since first snapshot (s)")
|
||||
ax1.set_title(title + " [no per-request data; instance idle?]")
|
||||
ax1.legend(loc="upper right", fontsize=9)
|
||||
ax1.grid(True, alpha=0.3)
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
fig.tight_layout()
|
||||
fig.savefig(out, dpi=120)
|
||||
plt.close(fig)
|
||||
print(f"wrote {out} (n_snapshots={len(snaps)}, 0 running reqs ever)")
|
||||
return
|
||||
|
||||
matrix = np.zeros((len(all_req_ids), len(times)), dtype=np.int64)
|
||||
req_to_row = {r: i for i, r in enumerate(all_req_ids)}
|
||||
for j, s in enumerate(snaps):
|
||||
|
||||
Reference in New Issue
Block a user