95 lines
4.8 KiB
Markdown
95 lines
4.8 KiB
Markdown
# Phase-aware telemetry intervention-response v2 protocol
|
|
|
|
Status: **INVALID OPERATIONAL ATTEMPT; SUPERSEDED BY V3**.
|
|
|
|
Date: 2026-07-14 (Asia/Singapore).
|
|
|
|
The first MNS=16 session timed out while draining the 3.125 requests/s/GPU
|
|
workload after its 300-second arrival window. It produced no high-load result,
|
|
and no MNS=64 endpoint was run. No comparative conclusion is drawn from this
|
|
attempt; see `intervention-response-v3-two-load-protocol-20260714.md`.
|
|
|
|
## Correction to v0/v1
|
|
|
|
The 5/10-second analyses tested an ultra-early verifier. They did not test
|
|
whether telemetry observed after the engine has developed queue, batch, and KV
|
|
state can guide tuning. The P1 replay lasts 60 seconds after time scaling, and
|
|
the 5/10-second prefixes contain only a small fraction of its requests.
|
|
|
|
V2 therefore replaces absolute cutoffs with replay phase. The old audits and
|
|
their negative decisions remain immutable, but their claim is narrowed to the
|
|
first 5/10 seconds.
|
|
|
|
## Historical corrective audit
|
|
|
|
The historical audit is development-only and cannot become confirmatory after
|
|
the horizon concern was observed.
|
|
|
|
- Infer each trial's intended replay duration as selected requests divided by
|
|
offered requests per second. All trials must agree.
|
|
- Find every complete 10% replay decile supported by every trial. Analyze all
|
|
such deciles; selecting only the best horizon is forbidden.
|
|
- At each decile report both:
|
|
- cumulative state from replay start to the checkpoint; and
|
|
- the non-overlapping 10%-wide state block ending at the checkpoint.
|
|
- Report admitted/completed request coverage, response-versus-repeat statistics,
|
|
telemetry versus external-outcome efficacy, and per-feature trajectory drift.
|
|
- Reuse the frozen v1 action and repeat pairs and the frozen response and
|
|
incremental-efficacy thresholds. These thresholds are descriptive in V2;
|
|
passing one post-hoc horizon does not open a contribution claim.
|
|
|
|
If early stopping prevents complete observation of the replay phases, the
|
|
historical decision is `REQUIRES_UNCENSORED_PHASE_AWARE_PILOT`, independent of
|
|
which early decile looks best.
|
|
|
|
## Uncensored matched pilot
|
|
|
|
The pilot is a mechanism gate, not paper evidence.
|
|
|
|
- Hardware/engine/model: solo placement on dash0, 4 NVIDIA H20 GPUs, patched
|
|
vLLM `0.24.1.dev3+opprof`, Qwen3-30B-A3B, fixed `TP=4`.
|
|
- Action: `MNS 16 -> 64`; topology, model, engine build, workload, arrival
|
|
sequence, offered load, and all other settings remain fixed.
|
|
- Workload: `chat_w20260312_1000` at replay-time scale `0.5`, hence 300 seconds.
|
|
- Offered loads per GPU: `1.5`, `2.125`, and `3.125` requests/s. These supply a
|
|
low control and the two already-observed P1 pressure regimes.
|
|
- Repetitions: three disjoint session bands, exact request sequence matched
|
|
across action endpoints. Endpoint order alternates `A/B`, `B/A`, `A/B`;
|
|
load order is counter-rotated across repetitions.
|
|
- A fresh server receives the accepted long-request warm-up and a bounded
|
|
burn-in before each measured session.
|
|
- SLO-unrecoverable early stop is disabled. Every run must observe the full
|
|
300-second arrival window; a separate 360-second safety deadline may mark a
|
|
run invalid but cannot manufacture a full-run label.
|
|
- Cumulative checkpoints: 10%, 25%, 50%, 75%, and 100%, or 30/75/150/225/300
|
|
seconds. Quarter blocks are analyzed separately from cumulative means.
|
|
- A measured Layer-1 interval is complete only when its start-boundary,
|
|
end-boundary, and maximum internal record gaps are each at most one second;
|
|
timestamps must be monotonic.
|
|
- Placement is serialized. Co-location remains forbidden because Phase 6
|
|
observed material co-location-induced outcome shifts.
|
|
- Hard cap: 8 H20-hours including startup, warm-up, burn-in, invalid attempts,
|
|
and cleanup.
|
|
|
|
## Gates
|
|
|
|
Data validity requires complete 300-second Layer-1 coverage, zero dropped
|
|
records, exact request/arrival/length hashes across action endpoints, monotonic
|
|
timestamps, full request accounting, idle GPUs before and after each session,
|
|
and no co-resident GPU process.
|
|
|
|
Mechanism evidence requires at least two telemetry features whose matched action
|
|
response exceeds same-config repeat noise at the same pair of consecutive
|
|
checkpoints under the unchanged v1 response thresholds. Those features must
|
|
also have a consistent direction in at least two of the three load regimes.
|
|
|
|
Decision evidence additionally requires both action-efficacy classes and at
|
|
least one of the phase-stable mechanism features to reach
|
|
leave-one-repetition-out balanced accuracy at least 0.75 and exceed the best
|
|
external prefix outcome by at least 0.15 at two adjacent predeclared
|
|
checkpoints from 25% onward. Without label balance the pilot can adjudicate
|
|
mechanism evidence only.
|
|
|
|
No H20 run is launched if the local analyzer/tests, manifest preflight, GPU
|
|
probe, command dry-run, projected cost, or cleanup plan fails.
|