Chunk-safe + concurrent layer-wise connector (per-step incremental shipping)

Browse Source

Scheduler tracks per-producer block_ids (accumulated from scheduler_output)
and emits per-step LWSendMeta with cumulative computed_tokens. Worker
lw_wait_for_save records a CUDA event per step and enqueues progress; the
sender-loop ship loop drains it, shipping only computed+dst-wanted+unshipped
blocks in order (correct under chunked prefill). Per-transfer state =
concurrent-safe. Keeps v1 single-transfer version as reference.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

...

This commit is contained in:

Gahow Wang

2026-05-28 17:15:54 +08:00

parent e77bdcac5a

commit 4242bba034

2 changed files with 1892 additions and 179 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1685

microbench/connector_tax/layerwise/mooncake_connector.LAYERWISE.v1_singlexfer.py Normal file

File diff suppressed because it is too large Load Diff