Add agentic workload characterization audit scaffold
This commit is contained in:
@@ -0,0 +1,10 @@
|
||||
# Figures Index
|
||||
|
||||
No generated figures are committed by this script. Batch-specific figures should be generated from:
|
||||
|
||||
- `analysis/characterization/analyze.py` for Batch 0/1 trace figures.
|
||||
- future Batch 2 step-timeline artifacts for interference plots.
|
||||
- future Batch 3 per-worker/session artifacts for hot-spot plots.
|
||||
- future Batch 4 arrival-rate sweep artifacts for SRR curves.
|
||||
|
||||
This file exists so the audit package has a stable placeholder until fresh figures are generated.
|
||||
@@ -0,0 +1,11 @@
|
||||
# Characterization Claim Matrix
|
||||
|
||||
| Claim | Status | Supporting Data | Needed Next | Reviewer Risk |
|
||||
|---|---|---|---|---|
|
||||
| Batch 0 substrate audit is only partially complete for existing runs. | `partially_supported` | metrics.jsonl lacks actual dispatch/finish timestamps in current artifacts. | Add request dispatch and finish/error timestamps to future replayer/proxy metrics. | Cannot use these runs to prove online per-session sequentiality. |
|
||||
| Batch 1 workload shape can be characterized from formatted traces and metrics. | `supported_for_trace_shape` | Full compact trace CPU summary in `full_trace_summary.json`: input p50/p90/p99 = 20k/87.9k/125.5k, output p50/p90/p99 = 80/811/6.6k, top 1% sessions hold 46.5% of input-token mass. | Add cache-hit joined records for actual reuse decomposition. | Actual cache reuse decomposition needs cached_tokens joined with hash_ids. |
|
||||
| Static PD separation is worse than combined in existing 200-request GPU A/B. | `supported_by_existing_artifact` | outputs/gpu_ab_combined vs outputs/gpu_ab_pdsep metrics.summary.json. | Refresh with PD matrix, multiple seeds, cudagraph-enabled methodology. | Legacy run has no per-stage TTFT breakdown and no step-level KV occupancy. |
|
||||
| Elastic transfer-based migration does not improve high-contention 500-request run. | `supported_by_existing_artifact` | outputs/contention_16s_ts10 vs outputs/contention_16s_elastic metrics.summary.json and gpu_util.csv. | Attribute whether failure is trigger quality, transfer overhead, or wrong load regime. | Existing metrics lack actual sequentiality proof and per-request transfer waterfall. |
|
||||
| PD-colo prefill/decode interference is not yet directly proven by step-level data in this package. | `not_yet_supported` | No decode-step and prefill-overlap timestamp artifact found in summarized runs. | Run Batch 2 controlled same-worker/different-worker injection with step timestamps. | Cannot claim interference as causal without Batch 2. |
|
||||
| Session hot-spot residual imbalance is suggested but not fully attributed. | `partially_supported` | gpu_util.csv shows per-GPU mean-util imbalance in existing runs. | Collect per-worker queue delay, session-to-worker map, and per-session token mass per worker. | GPU util imbalance alone is not enough to prove session hot-spot. |
|
||||
| SRR is not measured by existing fixed-request runs. | `not_yet_supported` | No arrival-rate sweep artifacts found. | Implement Batch 4 Poisson session-arrival SRR sweep. | Latency-at-one-load cannot support sustainable throughput claim. |
|
||||
51
analysis/characterization/current_results/claim_matrix.json
Normal file
51
analysis/characterization/current_results/claim_matrix.json
Normal file
@@ -0,0 +1,51 @@
|
||||
[
|
||||
{
|
||||
"claim": "Batch 0 substrate audit is only partially complete for existing runs.",
|
||||
"needed_next": "Add request dispatch and finish/error timestamps to future replayer/proxy metrics.",
|
||||
"reviewer_risk": "Cannot use these runs to prove online per-session sequentiality.",
|
||||
"status": "partially_supported",
|
||||
"supporting_data": "metrics.jsonl lacks actual dispatch/finish timestamps in current artifacts."
|
||||
},
|
||||
{
|
||||
"claim": "Batch 1 workload shape can be characterized from formatted traces and metrics.",
|
||||
"needed_next": "Add cache-hit joined records for actual reuse decomposition.",
|
||||
"reviewer_risk": "Actual cache reuse decomposition needs cached_tokens joined with hash_ids.",
|
||||
"status": "supported_for_trace_shape",
|
||||
"supporting_data": "Full compact trace CPU summary in full_trace_summary.json: input p50/p90/p99 = 20k/87.9k/125.5k, output p50/p90/p99 = 80/811/6.6k, top 1% sessions hold 46.5% of input-token mass."
|
||||
},
|
||||
{
|
||||
"claim": "Static PD separation is worse than combined in existing 200-request GPU A/B.",
|
||||
"needed_next": "Refresh with PD matrix, multiple seeds, cudagraph-enabled methodology.",
|
||||
"reviewer_risk": "Legacy run has no per-stage TTFT breakdown and no step-level KV occupancy.",
|
||||
"status": "supported_by_existing_artifact",
|
||||
"supporting_data": "outputs/gpu_ab_combined vs outputs/gpu_ab_pdsep metrics.summary.json."
|
||||
},
|
||||
{
|
||||
"claim": "Elastic transfer-based migration does not improve high-contention 500-request run.",
|
||||
"needed_next": "Attribute whether failure is trigger quality, transfer overhead, or wrong load regime.",
|
||||
"reviewer_risk": "Existing metrics lack actual sequentiality proof and per-request transfer waterfall.",
|
||||
"status": "supported_by_existing_artifact",
|
||||
"supporting_data": "outputs/contention_16s_ts10 vs outputs/contention_16s_elastic metrics.summary.json and gpu_util.csv."
|
||||
},
|
||||
{
|
||||
"claim": "PD-colo prefill/decode interference is not yet directly proven by step-level data in this package.",
|
||||
"needed_next": "Run Batch 2 controlled same-worker/different-worker injection with step timestamps.",
|
||||
"reviewer_risk": "Cannot claim interference as causal without Batch 2.",
|
||||
"status": "not_yet_supported",
|
||||
"supporting_data": "No decode-step and prefill-overlap timestamp artifact found in summarized runs."
|
||||
},
|
||||
{
|
||||
"claim": "Session hot-spot residual imbalance is suggested but not fully attributed.",
|
||||
"needed_next": "Collect per-worker queue delay, session-to-worker map, and per-session token mass per worker.",
|
||||
"reviewer_risk": "GPU util imbalance alone is not enough to prove session hot-spot.",
|
||||
"status": "partially_supported",
|
||||
"supporting_data": "gpu_util.csv shows per-GPU mean-util imbalance in existing runs."
|
||||
},
|
||||
{
|
||||
"claim": "SRR is not measured by existing fixed-request runs.",
|
||||
"needed_next": "Implement Batch 4 Poisson session-arrival SRR sweep.",
|
||||
"reviewer_risk": "Latency-at-one-load cannot support sustainable throughput claim.",
|
||||
"status": "not_yet_supported",
|
||||
"supporting_data": "No arrival-rate sweep artifacts found."
|
||||
}
|
||||
]
|
||||
95
analysis/characterization/current_results/comparisons.json
Normal file
95
analysis/characterization/current_results/comparisons.json
Normal file
@@ -0,0 +1,95 @@
|
||||
[
|
||||
{
|
||||
"baseline": "outputs/gpu_ab_combined",
|
||||
"e2e_p50_delta_pct": 40.870329127661,
|
||||
"e2e_p90_delta_pct": 15.206416995091814,
|
||||
"error_count": [
|
||||
2,
|
||||
13
|
||||
],
|
||||
"gpu_imbalance_ratio": [
|
||||
3.2445157838416265,
|
||||
11.149056603773586
|
||||
],
|
||||
"gpu_mean_util": [
|
||||
30.541666666666664,
|
||||
12.367081447963802
|
||||
],
|
||||
"name": "combined_vs_pdsep_200",
|
||||
"request_count": [
|
||||
200,
|
||||
200
|
||||
],
|
||||
"success_count": [
|
||||
198,
|
||||
187
|
||||
],
|
||||
"tpot_p90_delta_pct": 1.3481309269699875,
|
||||
"ttft_p50_delta_pct": 98.06752892925572,
|
||||
"ttft_p90_delta_pct": 44.79649177751278,
|
||||
"variant": "outputs/gpu_ab_pdsep",
|
||||
"wall_clock_delta_pct": 142.27736808267244
|
||||
},
|
||||
{
|
||||
"baseline": "outputs/contention_16s_ts10",
|
||||
"e2e_p50_delta_pct": 11.538788125232664,
|
||||
"e2e_p90_delta_pct": -5.080083318118138,
|
||||
"error_count": [
|
||||
2,
|
||||
2
|
||||
],
|
||||
"gpu_imbalance_ratio": [
|
||||
2.310775410408662,
|
||||
2.600767754318618
|
||||
],
|
||||
"gpu_mean_util": [
|
||||
23.030492424242425,
|
||||
26.349561403508773
|
||||
],
|
||||
"name": "contention_baseline_vs_elastic_500",
|
||||
"request_count": [
|
||||
500,
|
||||
500
|
||||
],
|
||||
"success_count": [
|
||||
498,
|
||||
498
|
||||
],
|
||||
"tpot_p90_delta_pct": 13.63098996823875,
|
||||
"ttft_p50_delta_pct": 12.433589435386224,
|
||||
"ttft_p90_delta_pct": 13.412576920999959,
|
||||
"variant": "outputs/contention_16s_elastic",
|
||||
"wall_clock_delta_pct": -0.5645626396767849
|
||||
},
|
||||
{
|
||||
"baseline": "outputs/combined_1000req",
|
||||
"e2e_p50_delta_pct": 202.85189980479385,
|
||||
"e2e_p90_delta_pct": 128.274511020719,
|
||||
"error_count": [
|
||||
2,
|
||||
204
|
||||
],
|
||||
"gpu_imbalance_ratio": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"gpu_mean_util": [
|
||||
null,
|
||||
null
|
||||
],
|
||||
"name": "combined_1000_vs_pdsep_mooncake",
|
||||
"request_count": [
|
||||
1000,
|
||||
1000
|
||||
],
|
||||
"success_count": [
|
||||
998,
|
||||
796
|
||||
],
|
||||
"tpot_p90_delta_pct": -34.83638659447109,
|
||||
"ttft_p50_delta_pct": 781.9835547522864,
|
||||
"ttft_p90_delta_pct": 1030.68607857992,
|
||||
"variant": "outputs/exp3_pd_sep_tp1_mooncake",
|
||||
"wall_clock_delta_pct": 119.18997774599991
|
||||
}
|
||||
]
|
||||
77
analysis/characterization/current_results/current_results.md
Normal file
77
analysis/characterization/current_results/current_results.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# Current Characterization Results
|
||||
|
||||
Generated: 2026-05-25T06:52:18.096448+00:00
|
||||
Git commit: `21ffb3d4f77956d008b1815a3c0d46e0188ac390`
|
||||
|
||||
## Canonical Full-Trace CPU Summary
|
||||
|
||||
Source: `dash0:/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl`.
|
||||
This is CPU-only parsing of the compact formatted trace with session IDs
|
||||
reconstructed from `parent_chat_id` chains.
|
||||
|
||||
| Metric | Value |
|
||||
|---|---:|
|
||||
| Requests | 2,114,220 |
|
||||
| Sessions | 1,307,276 |
|
||||
| Trace span | 7,199.975 s |
|
||||
| Input tokens p50/p90/p99 | 20,030 / 87,855 / 125,527 |
|
||||
| Output tokens p50/p90/p99 | 80 / 811 / 6,615 |
|
||||
| Input/output ratio p50/p90/p99 | 217.8 / 1,204.4 / 4,251.6 |
|
||||
| Turns/session p50/p90/p99/max | 1 / 1 / 18 / 3,091 |
|
||||
| Session input tokens p50/p90/p99/max | 12,486 / 72,676 / 974,934 / 156,756,974 |
|
||||
| Top 1% / 5% / 10% sessions by input-token mass | 46.5% / 66.5% / 74.6% |
|
||||
|
||||
Immediate reading: the full trace strongly supports long-input/short-output
|
||||
and heavy-tailed session token mass. It does **not** by itself prove online
|
||||
sequentiality or actual cache-hit reuse; those require runtime timestamps and
|
||||
cache-hit fields.
|
||||
|
||||
## Existing Run Summaries
|
||||
|
||||
| Run | OK/Req | TTFT p50/p90 | E2E p50/p90 | TPOT p90 | GPU mean util | GPU imbalance |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| outputs/gpu_ab_combined | 198/200 | 1.01/9.36 | 5.05/30.2 | 0.0732 | 30.5 | 3.24 |
|
||||
| outputs/gpu_ab_pdsep | 187/200 | 1.99/13.5 | 7.11/34.8 | 0.0742 | 12.4 | 11.1 |
|
||||
| outputs/contention_16s_ts10 | 498/500 | 0.826/9.71 | 5.8/51 | 0.103 | 23 | 2.31 |
|
||||
| outputs/contention_16s_elastic | 498/500 | 0.929/11 | 6.47/48.4 | 0.117 | 26.3 | 2.6 |
|
||||
| outputs/combined_1000req | 998/1000 | 0.393/2.57 | 3.22/28 | 0.113 | n/a | n/a |
|
||||
| outputs/exp3_pd_sep_tp1_mooncake | 796/1000 | 3.47/29 | 9.75/63.9 | 0.0739 | n/a | n/a |
|
||||
|
||||
## Pairwise Comparisons
|
||||
|
||||
| Comparison | TTFT p50 Δ | TTFT p90 Δ | E2E p50 Δ | E2E p90 Δ | TPOT p90 Δ | Wall-clock Δ |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| combined_vs_pdsep_200 | +98.1% | +44.8% | +40.9% | +15.2% | +1.3% | +142.3% |
|
||||
| contention_baseline_vs_elastic_500 | +12.4% | +13.4% | +11.5% | -5.1% | +13.6% | -0.6% |
|
||||
| combined_1000_vs_pdsep_mooncake | +782.0% | +1030.7% | +202.9% | +128.3% | -34.8% | +119.2% |
|
||||
|
||||
## What We Can Say Now
|
||||
|
||||
- **partially_supported**: Batch 0 substrate audit is only partially complete for existing runs.
|
||||
Supporting data: metrics.jsonl lacks actual dispatch/finish timestamps in current artifacts.
|
||||
Next: Add request dispatch and finish/error timestamps to future replayer/proxy metrics.
|
||||
- **supported_for_trace_shape**: Batch 1 workload shape can be characterized from formatted traces and metrics.
|
||||
Supporting data: full compact trace CPU summary in `full_trace_summary.json`: input p50/p90/p99 = 20k/87.9k/125.5k, output p50/p90/p99 = 80/811/6.6k, top 1% sessions hold 46.5% of input-token mass.
|
||||
Next: add cache-hit joined records for actual reuse decomposition.
|
||||
- **supported_by_existing_artifact**: Static PD separation is worse than combined in existing 200-request GPU A/B.
|
||||
Supporting data: outputs/gpu_ab_combined vs outputs/gpu_ab_pdsep metrics.summary.json.
|
||||
Next: Refresh with PD matrix, multiple seeds, cudagraph-enabled methodology.
|
||||
- **supported_by_existing_artifact**: Elastic transfer-based migration does not improve high-contention 500-request run.
|
||||
Supporting data: outputs/contention_16s_ts10 vs outputs/contention_16s_elastic metrics.summary.json and gpu_util.csv.
|
||||
Next: Attribute whether failure is trigger quality, transfer overhead, or wrong load regime.
|
||||
- **not_yet_supported**: PD-colo prefill/decode interference is not yet directly proven by step-level data in this package.
|
||||
Supporting data: No decode-step and prefill-overlap timestamp artifact found in summarized runs.
|
||||
Next: Run Batch 2 controlled same-worker/different-worker injection with step timestamps.
|
||||
- **partially_supported**: Session hot-spot residual imbalance is suggested but not fully attributed.
|
||||
Supporting data: gpu_util.csv shows per-GPU mean-util imbalance in existing runs.
|
||||
Next: Collect per-worker queue delay, session-to-worker map, and per-session token mass per worker.
|
||||
- **not_yet_supported**: SRR is not measured by existing fixed-request runs.
|
||||
Supporting data: No arrival-rate sweep artifacts found.
|
||||
Next: Implement Batch 4 Poisson session-arrival SRR sweep.
|
||||
|
||||
## Main Reviewer Risks
|
||||
|
||||
- **high**: Session sequentiality not proven - Add dispatch/finish timestamps and run Batch 0 before SRR claims.
|
||||
- **medium**: Legacy PD-sep data may not match final methodology - Use fresh PD matrix for paper-grade claims.
|
||||
- **medium**: GPU util is not a sufficient hot-spot proof - Add route-decision and per-worker queue logs for Batch 3.
|
||||
- **medium**: Cache reuse decomposition is incomplete without joined hash/cache-hit data - Emit hash_ids/session_id/cached_tokens in the same per-request record.
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"input": {
|
||||
"count": 2114220,
|
||||
"max": 202371,
|
||||
"mean": 33637.38370084476,
|
||||
"p50": 20030.0,
|
||||
"p90": 87855.1000000001,
|
||||
"p95": 104738.0,
|
||||
"p99": 125527.0
|
||||
},
|
||||
"input_output_ratio": {
|
||||
"count": 2108130,
|
||||
"max": 143664.0,
|
||||
"mean": 534.3516074828406,
|
||||
"p50": 217.8,
|
||||
"p90": 1204.3769610389616,
|
||||
"p95": 1814.3478327228322,
|
||||
"p99": 4251.585499999998
|
||||
},
|
||||
"output": {
|
||||
"count": 2114220,
|
||||
"max": 132665,
|
||||
"mean": 444.97059624826176,
|
||||
"p50": 80.0,
|
||||
"p90": 811.0,
|
||||
"p95": 2213.0,
|
||||
"p99": 6614.810000000056
|
||||
},
|
||||
"path": "/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl",
|
||||
"records": 2114220,
|
||||
"session_input_tokens": {
|
||||
"count": 1307276,
|
||||
"max": 156756974,
|
||||
"mean": 54400.77639916896,
|
||||
"p50": 12486.0,
|
||||
"p90": 72676.0,
|
||||
"p95": 108523.25,
|
||||
"p99": 974933.75
|
||||
},
|
||||
"sessions": 1307276,
|
||||
"top_session_input_fraction": {
|
||||
"top10pct": 0.7464402483455778,
|
||||
"top1pct": 0.46456810581415175,
|
||||
"top5pct": 0.6651718740752172
|
||||
},
|
||||
"trace_span_s": 7199.975,
|
||||
"turns_per_session": {
|
||||
"count": 1307276,
|
||||
"max": 3091,
|
||||
"mean": 1.6172713336739908,
|
||||
"p50": 1.0,
|
||||
"p90": 1.0,
|
||||
"p95": 2.0,
|
||||
"p99": 18.0
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
# Main-Claim Allowed Runs
|
||||
|
||||
Status: current audit gate
|
||||
Date: 2026-05-25
|
||||
|
||||
## Allowed For Workload-Shape Claims
|
||||
|
||||
These artifacts can support trace/workload characterization claims:
|
||||
|
||||
- `dash0:/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl`
|
||||
- Compact formatted full trace.
|
||||
- CPU summary recorded in `full_trace_summary.json`.
|
||||
- Supports long-input/short-output and session token-mass skew claims.
|
||||
- Does not prove runtime cache hits or online sequentiality.
|
||||
- `traces/w600_r0.0015_st30.jsonl`
|
||||
- Local sampled trace.
|
||||
- Useful for local dry runs and figure generation.
|
||||
- Not the canonical full-trace source.
|
||||
|
||||
## Allowed For Legacy Baseline Sanity Claims
|
||||
|
||||
These existing runs can support sanity-level comparisons, but not final
|
||||
paper-grade SRR claims:
|
||||
|
||||
- `outputs/gpu_ab_combined`
|
||||
- `outputs/gpu_ab_pdsep`
|
||||
- `outputs/contention_16s_ts10`
|
||||
- `outputs/contention_16s_elastic`
|
||||
- `outputs/combined_1000req`
|
||||
- `outputs/exp3_pd_sep_tp1_mooncake`
|
||||
|
||||
Allowed claims:
|
||||
|
||||
- Static PD-sep was worse than combined in these existing fixed-request runs.
|
||||
- Elastic transfer-based migration did not improve the summarized 500-request
|
||||
high-contention run.
|
||||
- GPU-util imbalance exists in these artifacts.
|
||||
|
||||
Disallowed claims:
|
||||
|
||||
- Online SRR.
|
||||
- Per-session sequentiality.
|
||||
- Causal attribution of prefill/decode interference.
|
||||
- Causal attribution of session hot spots from GPU utilization alone.
|
||||
|
||||
## Not Yet Allowed For Main Claims
|
||||
|
||||
The following need fresh instrumentation or fresh runs:
|
||||
|
||||
- Batch 2 prefill/decode interference.
|
||||
- Batch 3 session hot-spot root cause.
|
||||
- Batch 4 sustainable request rate.
|
||||
- Batch 5 failure attribution near SRR boundary.
|
||||
|
||||
## Required Upgrade Before Paper-Grade Claims
|
||||
|
||||
Future main-claim runs must include:
|
||||
|
||||
- per-request actual dispatch timestamp;
|
||||
- per-request finish/error timestamp;
|
||||
- route decision and selected worker;
|
||||
- per-worker queue delay;
|
||||
- per-worker KV occupancy;
|
||||
- per-worker APC/cache-hit snapshot;
|
||||
- attempted/completed/error/goodput counters;
|
||||
- session-causal load generation.
|
||||
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Rebuild this current-results audit package.
|
||||
python3 analysis/characterization/summarize_runs.py --output-dir analysis/characterization/current_results --runs outputs/gpu_ab_combined outputs/gpu_ab_pdsep outputs/contention_16s_ts10 outputs/contention_16s_elastic outputs/combined_1000req outputs/exp3_pd_sep_tp1_mooncake
|
||||
|
||||
# Example Batch 0/1 local trace analysis.
|
||||
python3 analysis/characterization/analyze.py \
|
||||
--trace traces/w600_r0.0015_st30.jsonl \
|
||||
--kv-bytes-per-token 98304 \
|
||||
--task-name w600_local_full_trace \
|
||||
--overwrite
|
||||
|
||||
# CPU-only full compact trace summary was computed on dash0 from:
|
||||
# /home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl
|
||||
# Recompute either by running analyze.py on dash0, or by copying that compact
|
||||
# formatted JSONL locally. Do not use the 487G raw file directly.
|
||||
@@ -0,0 +1,26 @@
|
||||
[
|
||||
{
|
||||
"evidence": "Current metrics include trace timestamp and latency but not actual dispatch/finish wall-clock timestamps.",
|
||||
"mitigation": "Add dispatch/finish timestamps and run Batch 0 before SRR claims.",
|
||||
"risk": "Session sequentiality not proven",
|
||||
"severity": "high"
|
||||
},
|
||||
{
|
||||
"evidence": "PD matrix scaffold exists separately; some old runs used earlier flags/methodology.",
|
||||
"mitigation": "Use fresh PD matrix for paper-grade claims.",
|
||||
"risk": "Legacy PD-sep data may not match final methodology",
|
||||
"severity": "medium"
|
||||
},
|
||||
{
|
||||
"evidence": "Existing artifacts have gpu_util.csv but lack per-worker queue and session ownership.",
|
||||
"mitigation": "Add route-decision and per-worker queue logs for Batch 3.",
|
||||
"risk": "GPU util is not a sufficient hot-spot proof",
|
||||
"severity": "medium"
|
||||
},
|
||||
{
|
||||
"evidence": "Trace has hash_ids; metrics have cached_tokens; request IDs may not join across all artifacts.",
|
||||
"mitigation": "Emit hash_ids/session_id/cached_tokens in the same per-request record.",
|
||||
"risk": "Cache reuse decomposition is incomplete without joined hash/cache-hit data",
|
||||
"severity": "medium"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,8 @@
|
||||
# Reviewer Risk Register
|
||||
|
||||
| Risk | Severity | Evidence | Mitigation |
|
||||
|---|---|---|---|
|
||||
| Session sequentiality not proven | `high` | Current metrics include trace timestamp and latency but not actual dispatch/finish wall-clock timestamps. | Add dispatch/finish timestamps and run Batch 0 before SRR claims. |
|
||||
| Legacy PD-sep data may not match final methodology | `medium` | PD matrix scaffold exists separately; some old runs used earlier flags/methodology. | Use fresh PD matrix for paper-grade claims. |
|
||||
| GPU util is not a sufficient hot-spot proof | `medium` | Existing artifacts have gpu_util.csv but lack per-worker queue and session ownership. | Add route-decision and per-worker queue logs for Batch 3. |
|
||||
| Cache reuse decomposition is incomplete without joined hash/cache-hit data | `medium` | Trace has hash_ids; metrics have cached_tokens; request IDs may not join across all artifacts. | Emit hash_ids/session_id/cached_tokens in the same per-request record. |
|
||||
720
analysis/characterization/current_results/run_summaries.json
Normal file
720
analysis/characterization/current_results/run_summaries.json
Normal file
@@ -0,0 +1,720 @@
|
||||
[
|
||||
{
|
||||
"apc_summary": {
|
||||
"reason": "apc.txt missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"artifact_availability": {
|
||||
"apc_txt": false,
|
||||
"breakdown_json": false,
|
||||
"gpu_util_csv": true,
|
||||
"metrics_jsonl": true,
|
||||
"metrics_summary_json": true
|
||||
},
|
||||
"breakdown_summary": {
|
||||
"reason": "breakdown.json missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"error_count": 2,
|
||||
"exists": true,
|
||||
"external_cache_hit_ratio": null,
|
||||
"gpu_summary": {
|
||||
"gpu_count": 8,
|
||||
"max_mean_util_pct": 63.166666666666664,
|
||||
"max_min_ratio": 3.2445157838416265,
|
||||
"mean_util_pct": 30.541666666666664,
|
||||
"min_mean_util_pct": 19.46875,
|
||||
"per_gpu_mean_util_pct": {
|
||||
"0": 29.145833333333332,
|
||||
"1": 20.041666666666668,
|
||||
"2": 25.0,
|
||||
"3": 63.166666666666664,
|
||||
"4": 21.927083333333332,
|
||||
"5": 34.90625,
|
||||
"6": 19.46875,
|
||||
"7": 30.677083333333332
|
||||
},
|
||||
"status": "available",
|
||||
"stddev_across_gpu_mean_util_pct": 13.337857305429534
|
||||
},
|
||||
"latency_stats_s": {
|
||||
"count": 198.0,
|
||||
"mean": 13.01780862723021,
|
||||
"p50": 5.048548387829214,
|
||||
"p90": 30.18109704903327,
|
||||
"p99": 119.01174414204434
|
||||
},
|
||||
"metrics_jsonl_rows": 200,
|
||||
"metrics_summary_available": true,
|
||||
"prefix_cache_hit_ratio": 0.0,
|
||||
"request_count": 200,
|
||||
"run": "outputs/gpu_ab_combined",
|
||||
"session_summary": {
|
||||
"request_cached_tokens": {
|
||||
"count": 200,
|
||||
"max": 0.0,
|
||||
"mean": 0.0,
|
||||
"p50": 0.0,
|
||||
"p90": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0
|
||||
},
|
||||
"request_input_tokens": {
|
||||
"count": 200,
|
||||
"max": 111927.0,
|
||||
"mean": 29318.375,
|
||||
"p50": 21376.0,
|
||||
"p90": 81218.19999999998,
|
||||
"p95": 87571.2,
|
||||
"p99": 101619.27999999994
|
||||
},
|
||||
"request_output_tokens": {
|
||||
"count": 200,
|
||||
"max": 5083.0,
|
||||
"mean": 257.675,
|
||||
"p50": 72.0,
|
||||
"p90": 664.3,
|
||||
"p95": 1063.9999999999993,
|
||||
"p99": 3300.1799999999994
|
||||
},
|
||||
"session_count": 145,
|
||||
"session_input_tokens": {
|
||||
"count": 145,
|
||||
"max": 1567423.0,
|
||||
"mean": 40439.137931034486,
|
||||
"p50": 10879.0,
|
||||
"p90": 72438.39999999997,
|
||||
"p95": 106934.39999999988,
|
||||
"p99": 374927.28
|
||||
},
|
||||
"status": "available",
|
||||
"top_session_input_fraction": {
|
||||
"top_10pct": 0.6588294883328288,
|
||||
"top_1pct": 0.33276622595897626,
|
||||
"top_5pct": 0.5534430199490934
|
||||
},
|
||||
"turns_per_session": {
|
||||
"count": 145,
|
||||
"max": 21.0,
|
||||
"mean": 1.3793103448275863,
|
||||
"p50": 1.0,
|
||||
"p90": 1.0,
|
||||
"p95": 2.0,
|
||||
"p99": 10.560000000000002
|
||||
}
|
||||
},
|
||||
"success_count": 198,
|
||||
"tpot_stats_s": {
|
||||
"count": 198.0,
|
||||
"mean": 0.04929455188644857,
|
||||
"p50": 0.03717198147904128,
|
||||
"p90": 0.07317714408040046,
|
||||
"p99": 0.10039294634234945
|
||||
},
|
||||
"ttft_stats_s": {
|
||||
"count": 198.0,
|
||||
"mean": 3.8987488178453984,
|
||||
"p50": 1.0068706551101059,
|
||||
"p90": 9.355209570843726,
|
||||
"p99": 33.855437273858115
|
||||
},
|
||||
"wall_clock_s": 483.543720243033
|
||||
},
|
||||
{
|
||||
"apc_summary": {
|
||||
"reason": "apc.txt missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"artifact_availability": {
|
||||
"apc_txt": false,
|
||||
"breakdown_json": false,
|
||||
"gpu_util_csv": true,
|
||||
"metrics_jsonl": true,
|
||||
"metrics_summary_json": true
|
||||
},
|
||||
"breakdown_summary": {
|
||||
"reason": "breakdown.json missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"error_count": 13,
|
||||
"exists": true,
|
||||
"external_cache_hit_ratio": null,
|
||||
"gpu_summary": {
|
||||
"gpu_count": 8,
|
||||
"max_mean_util_pct": 26.737556561085974,
|
||||
"max_min_ratio": 11.149056603773586,
|
||||
"mean_util_pct": 12.367081447963802,
|
||||
"min_mean_util_pct": 2.3981900452488687,
|
||||
"per_gpu_mean_util_pct": {
|
||||
"0": 26.737556561085974,
|
||||
"1": 14.44343891402715,
|
||||
"2": 19.036199095022624,
|
||||
"3": 7.4389140271493215,
|
||||
"4": 2.3981900452488687,
|
||||
"5": 8.841628959276019,
|
||||
"6": 11.963800904977376,
|
||||
"7": 8.076923076923077
|
||||
},
|
||||
"status": "available",
|
||||
"stddev_across_gpu_mean_util_pct": 7.15857427072345
|
||||
},
|
||||
"latency_stats_s": {
|
||||
"count": 187.0,
|
||||
"mean": 15.366858840781827,
|
||||
"p50": 7.111906730104238,
|
||||
"p90": 34.77056052000262,
|
||||
"p99": 119.2064151619561
|
||||
},
|
||||
"metrics_jsonl_rows": 200,
|
||||
"metrics_summary_available": true,
|
||||
"prefix_cache_hit_ratio": 0.0,
|
||||
"request_count": 200,
|
||||
"run": "outputs/gpu_ab_pdsep",
|
||||
"session_summary": {
|
||||
"request_cached_tokens": {
|
||||
"count": 200,
|
||||
"max": 0.0,
|
||||
"mean": 0.0,
|
||||
"p50": 0.0,
|
||||
"p90": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0
|
||||
},
|
||||
"request_input_tokens": {
|
||||
"count": 200,
|
||||
"max": 111927.0,
|
||||
"mean": 29318.375,
|
||||
"p50": 21376.0,
|
||||
"p90": 81218.19999999998,
|
||||
"p95": 87571.2,
|
||||
"p99": 101619.27999999994
|
||||
},
|
||||
"request_output_tokens": {
|
||||
"count": 200,
|
||||
"max": 5097.0,
|
||||
"mean": 290.18,
|
||||
"p50": 75.5,
|
||||
"p90": 685.8,
|
||||
"p95": 1102.5999999999997,
|
||||
"p99": 3433.6599999999844
|
||||
},
|
||||
"session_count": 145,
|
||||
"session_input_tokens": {
|
||||
"count": 145,
|
||||
"max": 1567423.0,
|
||||
"mean": 40439.137931034486,
|
||||
"p50": 10879.0,
|
||||
"p90": 72438.39999999997,
|
||||
"p95": 106934.39999999988,
|
||||
"p99": 374927.28
|
||||
},
|
||||
"status": "available",
|
||||
"top_session_input_fraction": {
|
||||
"top_10pct": 0.6588294883328288,
|
||||
"top_1pct": 0.33276622595897626,
|
||||
"top_5pct": 0.5534430199490934
|
||||
},
|
||||
"turns_per_session": {
|
||||
"count": 145,
|
||||
"max": 21.0,
|
||||
"mean": 1.3793103448275863,
|
||||
"p50": 1.0,
|
||||
"p90": 1.0,
|
||||
"p95": 2.0,
|
||||
"p99": 10.560000000000002
|
||||
}
|
||||
},
|
||||
"success_count": 187,
|
||||
"tpot_stats_s": {
|
||||
"count": 187.0,
|
||||
"mean": 0.05175559044923285,
|
||||
"p50": 0.04020531923068981,
|
||||
"p90": 0.07416366779122173,
|
||||
"p99": 0.10770365060609463
|
||||
},
|
||||
"ttft_stats_s": {
|
||||
"count": 187.0,
|
||||
"mean": 5.3933139261814524,
|
||||
"p50": 1.9942838260903955,
|
||||
"p90": 13.546015257015824,
|
||||
"p99": 39.30951087013818
|
||||
},
|
||||
"wall_clock_s": 1171.516998933861
|
||||
},
|
||||
{
|
||||
"apc_summary": {
|
||||
"reason": "apc.txt missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"artifact_availability": {
|
||||
"apc_txt": false,
|
||||
"breakdown_json": false,
|
||||
"gpu_util_csv": true,
|
||||
"metrics_jsonl": true,
|
||||
"metrics_summary_json": true
|
||||
},
|
||||
"breakdown_summary": {
|
||||
"reason": "breakdown.json missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"error_count": 2,
|
||||
"exists": true,
|
||||
"external_cache_hit_ratio": 0.0,
|
||||
"gpu_summary": {
|
||||
"gpu_count": 8,
|
||||
"max_mean_util_pct": 40.095454545454544,
|
||||
"max_min_ratio": 2.310775410408662,
|
||||
"mean_util_pct": 23.030492424242425,
|
||||
"min_mean_util_pct": 17.35151515151515,
|
||||
"per_gpu_mean_util_pct": {
|
||||
"0": 22.243939393939392,
|
||||
"1": 22.798484848484847,
|
||||
"2": 22.09090909090909,
|
||||
"3": 40.095454545454544,
|
||||
"4": 17.484848484848484,
|
||||
"5": 20.225757575757576,
|
||||
"6": 17.35151515151515,
|
||||
"7": 21.953030303030303
|
||||
},
|
||||
"status": "available",
|
||||
"stddev_across_gpu_mean_util_pct": 6.752783259358802
|
||||
},
|
||||
"latency_stats_s": {
|
||||
"count": 498.0,
|
||||
"mean": 21.908606036002105,
|
||||
"p50": 5.798741589998826,
|
||||
"p90": 51.00022796400299,
|
||||
"p99": 201.6967467680006
|
||||
},
|
||||
"metrics_jsonl_rows": 500,
|
||||
"metrics_summary_available": true,
|
||||
"prefix_cache_hit_ratio": 0.0,
|
||||
"request_count": 500,
|
||||
"run": "outputs/contention_16s_ts10",
|
||||
"session_summary": {
|
||||
"request_cached_tokens": {
|
||||
"count": 500,
|
||||
"max": 0.0,
|
||||
"mean": 0.0,
|
||||
"p50": 0.0,
|
||||
"p90": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0
|
||||
},
|
||||
"request_input_tokens": {
|
||||
"count": 500,
|
||||
"max": 134865.0,
|
||||
"mean": 29190.97,
|
||||
"p50": 17756.0,
|
||||
"p90": 77365.70000000006,
|
||||
"p95": 87790.89999999997,
|
||||
"p99": 111360.44
|
||||
},
|
||||
"request_output_tokens": {
|
||||
"count": 500,
|
||||
"max": 16427.0,
|
||||
"mean": 404.286,
|
||||
"p50": 75.0,
|
||||
"p90": 863.7000000000004,
|
||||
"p95": 2384.45,
|
||||
"p99": 4928.559999999999
|
||||
},
|
||||
"session_count": 347,
|
||||
"session_input_tokens": {
|
||||
"count": 347,
|
||||
"max": 1929072.0,
|
||||
"mean": 42061.916426512966,
|
||||
"p50": 11342.0,
|
||||
"p90": 68297.20000000004,
|
||||
"p95": 116213.79999999993,
|
||||
"p99": 521898.36000000057
|
||||
},
|
||||
"status": "available",
|
||||
"top_session_input_fraction": {
|
||||
"top_10pct": 0.6824335059780473,
|
||||
"top_1pct": 0.32352388426969025,
|
||||
"top_5pct": 0.579125805000656
|
||||
},
|
||||
"turns_per_session": {
|
||||
"count": 347,
|
||||
"max": 29.0,
|
||||
"mean": 1.440922190201729,
|
||||
"p50": 1.0,
|
||||
"p90": 1.0,
|
||||
"p95": 3.0,
|
||||
"p99": 10.54000000000002
|
||||
}
|
||||
},
|
||||
"success_count": 498,
|
||||
"tpot_stats_s": {
|
||||
"count": 498.0,
|
||||
"mean": 0.07772806444638436,
|
||||
"p50": 0.06774563665097065,
|
||||
"p90": 0.10325221968416468,
|
||||
"p99": 0.503443661631568
|
||||
},
|
||||
"ttft_stats_s": {
|
||||
"count": 498.0,
|
||||
"mean": 3.5127640336366928,
|
||||
"p50": 0.8264882279981975,
|
||||
"p90": 9.71474659699743,
|
||||
"p99": 34.40133859900379
|
||||
},
|
||||
"wall_clock_s": 1472.1331836190002
|
||||
},
|
||||
{
|
||||
"apc_summary": {
|
||||
"line_count": 8,
|
||||
"preview": "inst_0: prefix=45.7% ext=24.7%\ninst_1: prefix=33.1% ext=13.3%\ninst_2: prefix=14.2% ext=0.0%\ninst_3: prefix=50.7% ext=6.3%\ninst_4: prefix=33.6% ext=0.0%\ninst_5: prefix=66.6% ext=9.2%\ninst_6: prefix=42.1% ext=18.8%\ninst_7: prefix=36.7% ext=7.9%",
|
||||
"status": "available"
|
||||
},
|
||||
"artifact_availability": {
|
||||
"apc_txt": true,
|
||||
"breakdown_json": true,
|
||||
"gpu_util_csv": true,
|
||||
"metrics_jsonl": true,
|
||||
"metrics_summary_json": true
|
||||
},
|
||||
"breakdown_summary": {
|
||||
"field_sample": [
|
||||
"cache_hit",
|
||||
"estimated_new_tokens",
|
||||
"input_length",
|
||||
"policy",
|
||||
"request_id",
|
||||
"route_class",
|
||||
"routed_to",
|
||||
"t_done",
|
||||
"t_first_token",
|
||||
"t_proxy_recv"
|
||||
],
|
||||
"mode_counts": {
|
||||
"HEAVY_COLO": 111,
|
||||
"HEAVY_OFFLOAD": 13,
|
||||
"MEDIUM": 163,
|
||||
"WARM": 213
|
||||
},
|
||||
"route_counts": {
|
||||
"linear": 487
|
||||
},
|
||||
"row_count": 500,
|
||||
"status": "available"
|
||||
},
|
||||
"error_count": 2,
|
||||
"exists": true,
|
||||
"external_cache_hit_ratio": 0.0,
|
||||
"gpu_summary": {
|
||||
"gpu_count": 8,
|
||||
"max_mean_util_pct": 47.54385964912281,
|
||||
"max_min_ratio": 2.600767754318618,
|
||||
"mean_util_pct": 26.349561403508773,
|
||||
"min_mean_util_pct": 18.280701754385966,
|
||||
"per_gpu_mean_util_pct": {
|
||||
"0": 22.410526315789475,
|
||||
"1": 26.57894736842105,
|
||||
"2": 23.54035087719298,
|
||||
"3": 47.54385964912281,
|
||||
"4": 18.96842105263158,
|
||||
"5": 26.403508771929825,
|
||||
"6": 18.280701754385966,
|
||||
"7": 27.07017543859649
|
||||
},
|
||||
"status": "available",
|
||||
"stddev_across_gpu_mean_util_pct": 8.6079068381278
|
||||
},
|
||||
"latency_stats_s": {
|
||||
"count": 498.0,
|
||||
"mean": 21.780466573040204,
|
||||
"p50": 6.467846095998539,
|
||||
"p90": 48.40937389100145,
|
||||
"p99": 196.93401125499804
|
||||
},
|
||||
"metrics_jsonl_rows": 500,
|
||||
"metrics_summary_available": true,
|
||||
"prefix_cache_hit_ratio": 0.0,
|
||||
"request_count": 500,
|
||||
"run": "outputs/contention_16s_elastic",
|
||||
"session_summary": {
|
||||
"request_cached_tokens": {
|
||||
"count": 500,
|
||||
"max": 0.0,
|
||||
"mean": 0.0,
|
||||
"p50": 0.0,
|
||||
"p90": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0
|
||||
},
|
||||
"request_input_tokens": {
|
||||
"count": 500,
|
||||
"max": 134865.0,
|
||||
"mean": 29190.97,
|
||||
"p50": 17756.0,
|
||||
"p90": 77365.70000000006,
|
||||
"p95": 87790.89999999997,
|
||||
"p99": 111360.44
|
||||
},
|
||||
"request_output_tokens": {
|
||||
"count": 500,
|
||||
"max": 16427.0,
|
||||
"mean": 395.466,
|
||||
"p50": 75.0,
|
||||
"p90": 809.8000000000013,
|
||||
"p95": 2264.2999999999943,
|
||||
"p99": 4928.559999999999
|
||||
},
|
||||
"session_count": 347,
|
||||
"session_input_tokens": {
|
||||
"count": 347,
|
||||
"max": 1929072.0,
|
||||
"mean": 42061.916426512966,
|
||||
"p50": 11342.0,
|
||||
"p90": 68297.20000000004,
|
||||
"p95": 116213.79999999993,
|
||||
"p99": 521898.36000000057
|
||||
},
|
||||
"status": "available",
|
||||
"top_session_input_fraction": {
|
||||
"top_10pct": 0.6824335059780473,
|
||||
"top_1pct": 0.32352388426969025,
|
||||
"top_5pct": 0.579125805000656
|
||||
},
|
||||
"turns_per_session": {
|
||||
"count": 347,
|
||||
"max": 29.0,
|
||||
"mean": 1.440922190201729,
|
||||
"p50": 1.0,
|
||||
"p90": 1.0,
|
||||
"p95": 3.0,
|
||||
"p99": 10.54000000000002
|
||||
}
|
||||
},
|
||||
"success_count": 498,
|
||||
"tpot_stats_s": {
|
||||
"count": 498.0,
|
||||
"mean": 0.08306021258238128,
|
||||
"p50": 0.06973490711122092,
|
||||
"p90": 0.117326519391297,
|
||||
"p99": 0.5879216773072795
|
||||
},
|
||||
"ttft_stats_s": {
|
||||
"count": 498.0,
|
||||
"mean": 3.757858794331448,
|
||||
"p50": 0.9292503809992922,
|
||||
"p90": 11.017744456999935,
|
||||
"p99": 34.20241540799907
|
||||
},
|
||||
"wall_clock_s": 1463.8220696580029
|
||||
},
|
||||
{
|
||||
"apc_summary": {
|
||||
"reason": "apc.txt missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"artifact_availability": {
|
||||
"apc_txt": false,
|
||||
"breakdown_json": false,
|
||||
"gpu_util_csv": false,
|
||||
"metrics_jsonl": true,
|
||||
"metrics_summary_json": true
|
||||
},
|
||||
"breakdown_summary": {
|
||||
"reason": "breakdown.json missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"error_count": 2,
|
||||
"exists": true,
|
||||
"external_cache_hit_ratio": null,
|
||||
"gpu_summary": {
|
||||
"reason": "gpu_util.csv missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"latency_stats_s": {
|
||||
"count": 998.0,
|
||||
"mean": 15.744418202954922,
|
||||
"p50": 3.220371481962502,
|
||||
"p90": 28.005282410187647,
|
||||
"p99": 204.00479479599744
|
||||
},
|
||||
"metrics_jsonl_rows": 1000,
|
||||
"metrics_summary_available": true,
|
||||
"prefix_cache_hit_ratio": 0.5443149471989938,
|
||||
"request_count": 1000,
|
||||
"run": "outputs/combined_1000req",
|
||||
"session_summary": {
|
||||
"request_cached_tokens": {
|
||||
"count": 1000,
|
||||
"max": 0.0,
|
||||
"mean": 0.0,
|
||||
"p50": 0.0,
|
||||
"p90": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0
|
||||
},
|
||||
"request_input_tokens": {
|
||||
"count": 1000,
|
||||
"max": 171427.0,
|
||||
"mean": 31611.14,
|
||||
"p50": 19798.0,
|
||||
"p90": 82584.20000000001,
|
||||
"p95": 93305.64999999997,
|
||||
"p99": 111947.83999999998
|
||||
},
|
||||
"request_output_tokens": {
|
||||
"count": 1000,
|
||||
"max": 41233.0,
|
||||
"mean": 392.579,
|
||||
"p50": 70.0,
|
||||
"p90": 685.8000000000002,
|
||||
"p95": 1834.6999999999975,
|
||||
"p99": 4584.459999999996
|
||||
},
|
||||
"session_count": 671,
|
||||
"session_input_tokens": {
|
||||
"count": 671,
|
||||
"max": 2320064.0,
|
||||
"mean": 47110.49180327869,
|
||||
"p50": 12394.0,
|
||||
"p90": 71875.0,
|
||||
"p95": 112984.5,
|
||||
"p99": 760591.4999999925
|
||||
},
|
||||
"status": "available",
|
||||
"top_session_input_fraction": {
|
||||
"top_10pct": 0.6915149849072194,
|
||||
"top_1pct": 0.36223103627392117,
|
||||
"top_5pct": 0.5945521736957288
|
||||
},
|
||||
"turns_per_session": {
|
||||
"count": 671,
|
||||
"max": 36.0,
|
||||
"mean": 1.4903129657228018,
|
||||
"p50": 1.0,
|
||||
"p90": 1.0,
|
||||
"p95": 3.0,
|
||||
"p99": 12.599999999999909
|
||||
}
|
||||
},
|
||||
"success_count": 998,
|
||||
"tpot_stats_s": {
|
||||
"count": 998.0,
|
||||
"mean": 0.05681003092240363,
|
||||
"p50": 0.04029440155459775,
|
||||
"p90": 0.11343103184021618,
|
||||
"p99": 0.2962149563411783
|
||||
},
|
||||
"ttft_stats_s": {
|
||||
"count": 998.0,
|
||||
"mean": 1.0351265607308877,
|
||||
"p50": 0.39304836583323777,
|
||||
"p90": 2.5655168020166457,
|
||||
"p99": 8.210839052917436
|
||||
},
|
||||
"wall_clock_s": 1340.061015482992
|
||||
},
|
||||
{
|
||||
"apc_summary": {
|
||||
"reason": "apc.txt missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"artifact_availability": {
|
||||
"apc_txt": false,
|
||||
"breakdown_json": false,
|
||||
"gpu_util_csv": false,
|
||||
"metrics_jsonl": true,
|
||||
"metrics_summary_json": true
|
||||
},
|
||||
"breakdown_summary": {
|
||||
"reason": "breakdown.json missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"error_count": 204,
|
||||
"exists": true,
|
||||
"external_cache_hit_ratio": null,
|
||||
"gpu_summary": {
|
||||
"reason": "gpu_util.csv missing",
|
||||
"status": "unavailable"
|
||||
},
|
||||
"latency_stats_s": {
|
||||
"count": 796.0,
|
||||
"mean": 23.561836449842698,
|
||||
"p50": 9.752956213895231,
|
||||
"p90": 63.928921481827274,
|
||||
"p99": 168.2179710320197
|
||||
},
|
||||
"metrics_jsonl_rows": 1000,
|
||||
"metrics_summary_available": true,
|
||||
"prefix_cache_hit_ratio": 0.0,
|
||||
"request_count": 1000,
|
||||
"run": "outputs/exp3_pd_sep_tp1_mooncake",
|
||||
"session_summary": {
|
||||
"request_cached_tokens": {
|
||||
"count": 1000,
|
||||
"max": 0.0,
|
||||
"mean": 0.0,
|
||||
"p50": 0.0,
|
||||
"p90": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0
|
||||
},
|
||||
"request_input_tokens": {
|
||||
"count": 1000,
|
||||
"max": 171427.0,
|
||||
"mean": 31611.14,
|
||||
"p50": 19798.0,
|
||||
"p90": 82584.20000000001,
|
||||
"p95": 93305.64999999997,
|
||||
"p99": 111947.83999999998
|
||||
},
|
||||
"request_output_tokens": {
|
||||
"count": 1000,
|
||||
"max": 41233.0,
|
||||
"mean": 412.176,
|
||||
"p50": 74.0,
|
||||
"p90": 759.7000000000008,
|
||||
"p95": 1834.6999999999975,
|
||||
"p99": 4928.559999999999
|
||||
},
|
||||
"session_count": 671,
|
||||
"session_input_tokens": {
|
||||
"count": 671,
|
||||
"max": 2320064.0,
|
||||
"mean": 47110.49180327869,
|
||||
"p50": 12394.0,
|
||||
"p90": 71875.0,
|
||||
"p95": 112984.5,
|
||||
"p99": 760591.4999999925
|
||||
},
|
||||
"status": "available",
|
||||
"top_session_input_fraction": {
|
||||
"top_10pct": 0.6915149849072194,
|
||||
"top_1pct": 0.36223103627392117,
|
||||
"top_5pct": 0.5945521736957288
|
||||
},
|
||||
"turns_per_session": {
|
||||
"count": 671,
|
||||
"max": 36.0,
|
||||
"mean": 1.4903129657228018,
|
||||
"p50": 1.0,
|
||||
"p90": 1.0,
|
||||
"p95": 3.0,
|
||||
"p99": 12.599999999999909
|
||||
}
|
||||
},
|
||||
"success_count": 796,
|
||||
"tpot_stats_s": {
|
||||
"count": 796.0,
|
||||
"mean": 0.05211825330315642,
|
||||
"p50": 0.06622354774330945,
|
||||
"p90": 0.07391575907026088,
|
||||
"p99": 0.10499966285609896
|
||||
},
|
||||
"ttft_stats_s": {
|
||||
"count": 796.0,
|
||||
"mean": 10.753033336598003,
|
||||
"p50": 3.4666219488717616,
|
||||
"p90": 29.00794132403098,
|
||||
"p99": 81.7531874559354
|
||||
},
|
||||
"wall_clock_s": 2937.2794416199904
|
||||
}
|
||||
]
|
||||
Reference in New Issue
Block a user