The gateway records three epoch-ms stamps per request (admitted, forwarded to
backend, backend first token). They reached *-raw.jsonl but stopped there:
features.csv had only total latency, and none of the figures covered
time-to-first-token.
Derive ttft_ms, ttft_e2e_ms and tpot_ms in one helper shared by the streaming
path (preparation.py) and the in-memory path (features.py) so the two cannot
drift, and add figures 14 (TTFT, backend vs end-to-end) and 15 (TPOT). The two
TTFT series are plotted together because their offset is the gateway admission
delay, which is then readable straight off the chart.
Rows missing a backend stamp and single-token responses are recorded as 0 and
filtered at plot time; left in, they would stack at the origin and pull every
percentile down. Figure 15 is a single panel: TPOT is near-deterministic
(p50 21.9, p90 23.0), so a p90 zoom renders a flat line and a vertical edge.
Verified by recomputing ttft_ms straight from the original SLS rows for all
2190 requests of a sample: 0 mismatches.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
request_model values like qwen-plus matched no registered family, so
resolve_model_family fell through to its "glm5" default and the formatter
rendered canonical_prompt with the GLM-5 chat template and tokenized release
blocks with the GLM-5 tokenizer. The fallback is silent, so nothing surfaced it.
Register the family, ship Qwen3-plus tokenizer/chat_template (the template was
embedded in tokenizer_config.json upstream and is extracted here), and add a
QwenPlusTraceAdapter ahead of the GLM catch-all in get_raw_adapter -- the
registry entry alone is not enough, since the adapter is what picks the
template. The shared message/tool normalization is factored out of
build_glm5_canonical_prompt so both builders use one copy.
Measured against provider-reported usage.input_tokens on a 400-request sample,
median tokenized/reported ratio goes 0.9449 -> 1.0000, and unique release block
count on a 2190-request sample moves 5886 -> 5701.
Note deepseek-v4-flash is still unregistered and still resolves to glm5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three one-off scripts kept outside the two packages: a formatter for the
qwenapp Aquila access-log shape, its characterization data generator, and a
session-sticky routing imbalance simulation over a formatted trace.
Local tokenization does not always agree with the input_length the provider
reports, which left the release row carrying a block count derived from our own
count. Truncate or pad digest_hexes to the block count implied by the provider
value. Padding digests are derived per request, so a padded block can never
collide with another request and be scored as shared prefix reuse.
Collapse chunk_*.jsonl in rounds with a bounded merge fan-in so the number of
simultaneously open chunk iterators stays flat regardless of how many chunks a
trace produces, and write a resume manifest so --resume-tmp-dir can continue
from an existing formatter temp directory without rescanning the raw trace.
