gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
fec50fa45d4ab0bcdea78568ecb44d24a44a440d
agentic-kvc/analysis/mb2/A_intra_kvboth.jsonl
Gahow Wang de164e5a64 MB2: pure KV-transfer cost on dash1 intra-node — Mooncake ~9.7 GB/s steady 
Full sweep result on dash1 GPU 0+1 with vanilla vLLM 0.18.1 +
mooncake-transfer-engine 0.3.11, kv_both connector. Per-stage decomposition
via the instrumentation patch (analyze_mb2.py pairs A's send_blocks with
B's receive_kv enter/finish by time window).

Steady-state (1k..32k tokens, 96 MiB..3 GiB KV):
   pure_transfer ≈ size / 9.7 GB/s
   rx_overhead   ≈ 2–3 ms (ZMQ handshake + P-side setup)
   bandwidth     ≈ 9.6–10.1 GB/s, very stable

Large-size regime (65k..131k tokens, 6..12 GiB):
   p50 bandwidth collapses to 3.4–4.5 GB/s
   max bandwidth still hits ~9.7 GB/s (some runs achieve it)
   p99 agentic request (11.5 GiB) lands here

Implication for §3.2 PD-disaggregation cost argument:
   median agentic decode = 50–200 ms (tool-call JSON output)
   median agentic-tail KV transfer (p99 11.5 GiB):
     best case (9.7 GB/s)  ≈ 1.19 s
     observed range         1.5 – 10 s
   ⇒ KV transfer is 8–100× larger than the decode it enables.

This is intra-node — the lower-bound transfer cost. Inter-node RDMA
will be slower; that's MB2 phase 2.

Adds:
- analyze_mb2.py: pair A.send_blocks ↔ B.receive_kv by time window;
  per-size aggregation (n, ms_p50, ms_min/max, GB/s_p50/max)
- plot_mb2.py: log-log transfer-time chart + bandwidth-vs-size chart
- analysis/mb2/A_intra_kvboth.jsonl, B_intra_kvboth.jsonl: raw events
  (51 + 102 events including the sanity preamble)
- analysis/mb2/intra_kvboth_breakdown.json: paired and aggregated
- figs/mb2_transfer_time_intra.png, figs/mb2_transfer_bw_intra.png

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:04:03 +08:00

52 lines
11 KiB
JSON
Raw Blame History

{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.023202952987048775, "t_start_unix": 1779879143.174031, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879143.1972432}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.005375694017857313, "t_start_unix": 1779879143.2982283, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879143.3036084}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.021170366962905973, "t_start_unix": 1779879143.5159554, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879143.5371296}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.020726953051052988, "t_start_unix": 1779879143.6974514, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879143.7181835}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08536655298667029, "t_start_unix": 1779879144.3294952, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879144.4148676}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08367906499188393, "t_start_unix": 1779879145.0419943, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879145.125678}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.0004059679922647774, "t_start_unix": 1779879221.7078288, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879221.7082384}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.000346789020113647, "t_start_unix": 1779879221.7838593, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879221.7842083}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.005353622022084892, "t_start_unix": 1779879221.8607252, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879221.8660822}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.005279594974126667, "t_start_unix": 1779879221.9432015, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879221.9484842}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.0053006180096417665, "t_start_unix": 1779879222.0243337, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.0296378}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010396577999927104, "t_start_unix": 1779879222.130936, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.141335}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010438029014039785, "t_start_unix": 1779879222.2442062, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.2546473}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010436972021125257, "t_start_unix": 1779879222.3581295, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.3685696}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010396371013484895, "t_start_unix": 1779879222.4725878, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.4829867}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010352785000577569, "t_start_unix": 1779879222.5837166, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.5940716}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.00034007197245955467, "t_start_unix": 1779879222.7521152, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.752458}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.00041691696969792247, "t_start_unix": 1779879222.9143836, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.914805}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.020633380976505578, "t_start_unix": 1779879223.0778644, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879223.098502}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.020639199996367097, "t_start_unix": 1779879223.2603853, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879223.2810278}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.020575353992171586, "t_start_unix": 1779879223.4418828, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879223.4624615}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.041439525957684964, "t_start_unix": 1779879223.7544343, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879223.7958782}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.04152030003024265, "t_start_unix": 1779879224.0914912, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879224.133016}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.04148670402355492, "t_start_unix": 1779879224.4262393, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879224.4677298}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.04146742797456682, "t_start_unix": 1779879224.7617002, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879224.8031723}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.04143296502297744, "t_start_unix": 1779879225.0978234, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879225.1392617}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.0003991159610450268, "t_start_unix": 1779879225.760789, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879225.7611918}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.00041423802031204104, "t_start_unix": 1779879226.3864496, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879226.3868673}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08309489500243217, "t_start_unix": 1779879227.0107942, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879227.0938945}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08372796402545646, "t_start_unix": 1779879227.7207224, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879227.8044555}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08398396399570629, "t_start_unix": 1779879228.4314566, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879228.5154452}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.16950496198842302, "t_start_unix": 1779879230.1334376, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879230.3029544}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.16713789198547602, "t_start_unix": 1779879231.8981037, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879232.0652575}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.16713115200400352, "t_start_unix": 1779879233.6608078, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879233.827945}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.16709016199456528, "t_start_unix": 1779879235.419875, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879235.5869706}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.166486973001156, "t_start_unix": 1779879237.1821773, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879237.34867}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.31926770601421595, "t_start_unix": 1779879241.9880297, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879242.3073065}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.3197040680097416, "t_start_unix": 1779879246.9779432, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879247.2976692}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.32088329299585894, "t_start_unix": 1779879251.9643052, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879252.285209}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.5439103110111319, "t_start_unix": 1779879256.9989722, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879257.5428913}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.5193864739849232, "t_start_unix": 1779879262.2562187, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879262.7756212}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 1.9844180009677075, "t_start_unix": 1779879278.199048, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879280.1834915}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 2.1099297259934247, "t_start_unix": 1779879295.6967168, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879297.8066647}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 1.8950715209939517, "t_start_unix": 1779879313.3236735, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879315.2187643}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 0.9277855920372531, "t_start_unix": 1779879330.6715357, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879331.599329}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 0.6652462020283565, "t_start_unix": 1779879346.9950044, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879347.6602724}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 1.3330365709844045, "t_start_unix": 1779879402.7169023, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879404.04997}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 5.839069904992357, "t_start_unix": 1779879459.0566247, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879464.8957155}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 9.862486142024864, "t_start_unix": 1779879519.9567635, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879529.8192694}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 2.8350498770014383, "t_start_unix": 1779879584.9780834, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879587.813154}
{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 1.485496642999351, "t_start_unix": 1779879642.639775, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879644.1252885}
Reference in New Issue View Git Blame Copy Permalink