feat(analyzer): surface TTFT/TPOT as features and CDF figures

The gateway records three epoch-ms stamps per request (admitted, forwarded to
backend, backend first token). They reached *-raw.jsonl but stopped there:
features.csv had only total latency, and none of the figures covered
time-to-first-token.

Derive ttft_ms, ttft_e2e_ms and tpot_ms in one helper shared by the streaming
path (preparation.py) and the in-memory path (features.py) so the two cannot
drift, and add figures 14 (TTFT, backend vs end-to-end) and 15 (TPOT). The two
TTFT series are plotted together because their offset is the gateway admission
delay, which is then readable straight off the chart.

Rows missing a backend stamp and single-token responses are recorded as 0 and
filtered at plot time; left in, they would stack at the origin and pull every
percentile down. Figure 15 is a single panel: TPOT is near-deterministic
(p50 21.9, p90 23.0), so a p90 zoom renders a flat line and a vertical edge.

Verified by recomputing ttft_ms straight from the original SLS rows for all
2190 requests of a sample: 0 mismatches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-11 06:31:48 +00:00
parent 7cb4cac56d
commit 825062f554
8 changed files with 95 additions and 6 deletions

View File

@@ -5,7 +5,7 @@ import json
import os
from pathlib import Path
from trace_analyzer.helpers import percentile
from trace_analyzer.helpers import latency_breakdown, percentile
from trace_analyzer.parser import get_adapter
from tqdm.auto import tqdm
@@ -98,6 +98,7 @@ def stream_prepare(input_path: str | Path, output_dir: str | Path, *, show_progr
"latency_ms": record.meta.total_cost_time_ms,
"ms_per_input_token": _safe_div(record.meta.total_cost_time_ms, record.usage.input_tokens),
"ms_per_output_token": _safe_div(record.meta.total_cost_time_ms, record.usage.output_tokens),
**latency_breakdown(record.meta, record.usage.output_tokens),
"long_context": 1 if record.usage.input_tokens >= 32000 else 0,
"high_cache": 1 if cache_hit_ratio >= 0.8 else 0,
"tool_burst_alert": 1 if max_tool_burst >= 4 else 0,