Add agentic workload characterization audit scaffold
This commit is contained in:
@@ -0,0 +1,66 @@
|
||||
# Main-Claim Allowed Runs
|
||||
|
||||
Status: current audit gate
|
||||
Date: 2026-05-25
|
||||
|
||||
## Allowed For Workload-Shape Claims
|
||||
|
||||
These artifacts can support trace/workload characterization claims:
|
||||
|
||||
- `dash0:/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl`
|
||||
- Compact formatted full trace.
|
||||
- CPU summary recorded in `full_trace_summary.json`.
|
||||
- Supports long-input/short-output and session token-mass skew claims.
|
||||
- Does not prove runtime cache hits or online sequentiality.
|
||||
- `traces/w600_r0.0015_st30.jsonl`
|
||||
- Local sampled trace.
|
||||
- Useful for local dry runs and figure generation.
|
||||
- Not the canonical full-trace source.
|
||||
|
||||
## Allowed For Legacy Baseline Sanity Claims
|
||||
|
||||
These existing runs can support sanity-level comparisons, but not final
|
||||
paper-grade SRR claims:
|
||||
|
||||
- `outputs/gpu_ab_combined`
|
||||
- `outputs/gpu_ab_pdsep`
|
||||
- `outputs/contention_16s_ts10`
|
||||
- `outputs/contention_16s_elastic`
|
||||
- `outputs/combined_1000req`
|
||||
- `outputs/exp3_pd_sep_tp1_mooncake`
|
||||
|
||||
Allowed claims:
|
||||
|
||||
- Static PD-sep was worse than combined in these existing fixed-request runs.
|
||||
- Elastic transfer-based migration did not improve the summarized 500-request
|
||||
high-contention run.
|
||||
- GPU-util imbalance exists in these artifacts.
|
||||
|
||||
Disallowed claims:
|
||||
|
||||
- Online SRR.
|
||||
- Per-session sequentiality.
|
||||
- Causal attribution of prefill/decode interference.
|
||||
- Causal attribution of session hot spots from GPU utilization alone.
|
||||
|
||||
## Not Yet Allowed For Main Claims
|
||||
|
||||
The following need fresh instrumentation or fresh runs:
|
||||
|
||||
- Batch 2 prefill/decode interference.
|
||||
- Batch 3 session hot-spot root cause.
|
||||
- Batch 4 sustainable request rate.
|
||||
- Batch 5 failure attribution near SRR boundary.
|
||||
|
||||
## Required Upgrade Before Paper-Grade Claims
|
||||
|
||||
Future main-claim runs must include:
|
||||
|
||||
- per-request actual dispatch timestamp;
|
||||
- per-request finish/error timestamp;
|
||||
- route decision and selected worker;
|
||||
- per-worker queue delay;
|
||||
- per-worker KV occupancy;
|
||||
- per-worker APC/cache-hit snapshot;
|
||||
- attempted/completed/error/goodput counters;
|
||||
- session-causal load generation.
|
||||
Reference in New Issue
Block a user