74 lines
3.9 KiB
Markdown
74 lines
3.9 KiB
Markdown
# Phase-aware telemetry intervention-response v3 protocol
|
|
|
|
Status: **FROZEN AFTER A NON-COMPARATIVE OPERATIONAL FAILURE AND BEFORE V3 RUNS**.
|
|
|
|
Date: 2026-07-14 (Asia/Singapore).
|
|
|
|
## Why v2 was invalid
|
|
|
|
The first v2 session completed the 300-second arrival windows at 1.5 and 2.125
|
|
requests/s/GPU. At 3.125 requests/s/GPU, MNS=16 could not drain the admitted
|
|
requests before the 450-second client timeout. The session produced no result
|
|
and no MNS=64 action endpoint was run. V2 is therefore an invalid operational
|
|
attempt, not evidence for or against the telemetry hypothesis.
|
|
|
|
This failure was observed before any MNS action comparison. V3 excludes only
|
|
the unmeasurable overload point and reruns every retained point on fresh
|
|
servers; it does not reuse the completed v2 low/mid results.
|
|
|
|
## Question and hypothesis
|
|
|
|
Question: after enough replay time for queue, batch, and KV state to develop,
|
|
does an MNS intervention create telemetry responses that exceed workload-repeat
|
|
noise, and does any such response predict whether the intervention repairs the
|
|
full-run SLO outcome better than external prefix outcomes alone?
|
|
|
|
Hypothesis: increasing MNS from 16 to 64 has little value at the 1.5
|
|
requests/s/GPU control load but can repair the 2.125 requests/s/GPU pressure
|
|
load. Queue, running-set, batch, or KV telemetry should expose the difference
|
|
at stable replay phases. Label balance is an assumption to test, not a fact.
|
|
|
|
## Frozen setup
|
|
|
|
- Solo placement on dash0 GPU 0-3: 4 NVIDIA H20 GPUs, Qwen3-30B-A3B, patched
|
|
vLLM `0.24.1.dev3+opprof`, fixed TP=4.
|
|
- Action: MNS `16 -> 64`; all other engine and workload parameters fixed.
|
|
- Workload: `chat_w20260312_1000`, replay-time scale 0.5, hence 300 seconds.
|
|
- Loads per GPU: 1.5 control and 2.125 pressure requests/s. The failed 3.125
|
|
overload point is excluded from V3 and retained only as a failure artifact.
|
|
- Three disjoint request bands. Each MNS action pair has exact request,
|
|
arrival, and input-length hashes. Endpoint order is A/B, B/A, A/B; load
|
|
order is low/mid, mid/low, low/mid.
|
|
- Every session starts a fresh server, then runs the accepted 16-request long
|
|
warm-up and bounded burn-in before measured runs.
|
|
- SLO-unrecoverable early stop is disabled. Measured results must cover the
|
|
full 300-second arrival window and must not be early-stopped.
|
|
- Cumulative checkpoints are 10%, 25%, 50%, 75%, and 100%; non-overlapping
|
|
quarter blocks are also reported.
|
|
- A Layer-1 interval is complete only if timestamps are monotonic and its
|
|
start, end, and maximum internal record gaps are each at most one second.
|
|
- Incremental V3 cap is the unused portion of the original 8 H20-hour cap.
|
|
The exact prior cost and V3 cap are machine-recorded in the manifest.
|
|
|
|
## Frozen gates
|
|
|
|
Data validity requires six uncensored sessions, six action pairs, eight
|
|
same-config repeat pairs, exact action-pair hashes, full request accounting,
|
|
zero Layer-1 drops, continuous coverage, all stream/footer invariants, no
|
|
co-resident compute process, idle GPUs before and after sessions, nonnegative
|
|
counters, bounded ratios, non-identical per-config state, and monotonic request
|
|
coverage across checkpoints. Any red flag stops analysis.
|
|
|
|
Mechanism evidence requires at least two telemetry features to exceed the
|
|
unchanged v1 repeat-noise thresholds at the same pair of adjacent checkpoints.
|
|
Those features must have a direction consistent in both retained load regimes.
|
|
|
|
Decision evidence additionally requires at least two positive and two negative
|
|
full-run action-efficacy labels, valid leave-one-repetition-out folds, and at
|
|
least one phase-stable mechanism feature whose balanced accuracy is at least
|
|
0.75 and at least 0.15 above the best external prefix-outcome feature at two
|
|
adjacent predeclared checkpoints from 25% onward.
|
|
|
|
V3 remains a development mechanism pilot. Even `OPEN_E2E_POLICY_TEST` opens a
|
|
held-out tuning-policy experiment; it is not itself a paper performance claim.
|