DR-fix A/B: env-gate hash sync drops slope from +81 to -0.7 μs/1k blocks
Adds an env-gated skip for the per-step `set(cache.keys())` walk in MooncakeConnectorScheduler.build_connector_meta() that was introduced in our own commita7df84b(Direct RDMA read). Re-runs the cache_sweep A/B with three configs: plain (control), mooncake_both (baseline), and mooncake_both_drfix (VLLM_MOONCAKE_DISABLE_DIRECT_READ_SYNC=1). Files: apply_direct_read_fix.py one-line env-gate patch (markered revert) run_drfix.sh orchestrator for plain + mooncake_both + drfix analyze.py extended to compare mooncake_both_drfix vs plain and mooncake_both vs mooncake_both_drfix REPORT_DRFIX.md findings results/20260526_1543_drfix/ run artifacts Headline: config | slope (μs/1k blocks) | step_dur p50 @ 16.6k ----------------------|----------------------|--------------------- mooncake_both | +81.0 | 1 550 μs mooncake_both_drfix | -0.7 (≈ 0) | 95 μs plain (control) | -1.8 (≈ 0) | 72 μs build_meta p50 @ 16.6k blocks: mooncake_both = 1 459 μs mooncake_both_drfix = 6 μs (residual loop bookkeeping) worker get_finished p50: mooncake_both = 178 μs (unchanged; this fix doesn't touch it) mooncake_both_drfix = 183 μs The fix recovers 1 453 μs (99.6 %) of the scheduler-side cost at |cache|=16.6k blocks. drfix's per-bin step_dur tracks plain within ±50 μs across the full cache range — that's noise-level. The slope goes from +81 to essentially zero. Worker-side get_finished (180 μs constant) is unchanged because the DR-fix touches scheduler.build_connector_meta only. That's the next target if we want to bring kv_both fully back to plain-level. Extrapolation to trace-replay (|cache|≈13k, APC≈79%): before: build_meta 1 060 μs + get_finished 180 μs = 1.24 ms/step after DR-fix: build_meta 6 μs + get_finished 180 μs = ~0.19 ms/step → 85% reduction in per-step connector cost → TPOT inflation drops from ~+18% to ~+3% on a 7 ms decode step Confirms: the entire O(|cache|) slope was introduced by our own direct-RDMA-read implementation (commita7df84b), not upstream Mooncake. Production fix: gate the sync on the presence of any direct_read consumer, or replace per-step diff with an incremental delta listener fed by block_pool add/remove callbacks. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
127
microbench/connector_tax/cache_sweep/REPORT_DRFIX.md
Normal file
127
microbench/connector_tax/cache_sweep/REPORT_DRFIX.md
Normal file
@@ -0,0 +1,127 @@
|
||||
# DR-fix A/B: hash-sync skip eliminates the O(|cache|) slope
|
||||
|
||||
Run: `results/20260526_1543_drfix/`
|
||||
Compares three configs in a single orchestrated run (same vLLM
|
||||
process lifecycle order, same machine, same patch stack):
|
||||
|
||||
| config | what it does |
|
||||
|---|---|
|
||||
| `plain` | no kv connector — control |
|
||||
| `mooncake_both` | `kv_role=kv_both`, hash sync ON (baseline) |
|
||||
| `mooncake_both_drfix` | same launcher, but `VLLM_MOONCAKE_DISABLE_DIRECT_READ_SYNC=1` |
|
||||
|
||||
The DR-fix patch (`apply_direct_read_fix.py`) replaces the line
|
||||
`if self._block_pool is not None:` at
|
||||
`mooncake_connector.py:433` with an env-gated variant that lets us
|
||||
A/B that single code path without recompiling vLLM. The
|
||||
`set(self._block_pool.cache.keys())` walk is the only thing it gates.
|
||||
|
||||
## Headline
|
||||
|
||||
| config | slope (μs / 1k blocks) | step_dur p50 @ \|cache\|=16.6k | build_meta p50 |
|
||||
|---|---:|---:|---:|
|
||||
| mooncake_both | **+81.0** | **1 550 μs** | **1 459 μs** |
|
||||
| mooncake_both_drfix | **−0.7** (≈ 0) | **95 μs** | **6 μs** |
|
||||
| plain (control) | **−1.8** (≈ 0) | 72 μs | 0 |
|
||||
|
||||
The DR-fix kills the slope. mooncake_both_drfix's curve overlays
|
||||
plain across the full \|cache\| range — see `figure.png`.
|
||||
|
||||
|
||||
|
||||
## Savings at the cache ceiling
|
||||
|
||||
At \|cache\| ≈ 16.6 k blocks (where the prior run sat for ~ 80 % of
|
||||
its lifetime, and where the trace-replay run with APC≈79 % would
|
||||
sit on a same-config H20):
|
||||
|
||||
| component | baseline mooncake_both | drfix | saved |
|
||||
|---|---:|---:|---:|
|
||||
| `build_connector_meta` p50 | 1 459 μs | 6 μs | **−1 453 μs (−99.6 %)** |
|
||||
| total step_duration p50 | 1 550 μs | 95 μs | **−1 455 μs (−94 %)** |
|
||||
| worker `get_finished` p50 | 178 μs | 183 μs | unchanged (this fix doesn't touch it) |
|
||||
| worker `start_load_kv` p50 | 2 μs | 2 μs | unchanged |
|
||||
|
||||
So the patch did **exactly** what the source-code reading
|
||||
predicted: the O(\|cache\|) walk was the entire scheduler-side cost,
|
||||
and turning it off recovers all of it. `get_finished` overhead is
|
||||
untouched — that's a separate fix candidate.
|
||||
|
||||
## Throughput (sanity check, not the focus)
|
||||
|
||||
| config | requests completed in 241 s | effective rate |
|
||||
|---|---:|---:|
|
||||
| plain | 322 | 1.34 req/s |
|
||||
| mooncake_both | 365 | 1.51 req/s |
|
||||
| mooncake_both_drfix | 384 | 1.59 req/s |
|
||||
|
||||
Note the plain run had a transient inflight spike (`t+90s inflight=15`)
|
||||
that other configs did not — this is Poisson-arrival variance, not
|
||||
a real ordering. The per-step measurements (n ≥ 15 k decode steps
|
||||
per config) are far more reliable than the request-count totals
|
||||
for comparing across configs.
|
||||
|
||||
## Slope decomposition at each cache bin
|
||||
|
||||
| bin | cache mid | plain p50 | mooncake_both p50 | mooncake_both_drfix p50 | drfix tax vs plain |
|
||||
|---:|---:|---:|---:|---:|---:|
|
||||
| 1 | 2 629 | 71 | — | 85 | +14 μs |
|
||||
| 2 | 4 382 | 124 | 655 | 94 | −30 μs |
|
||||
| 3 | 6 135 | 134 | 809 | 121 | −13 μs |
|
||||
| 4 | 7 888 | 90 | 1 157 | 101 | +11 μs |
|
||||
| 5 | 9 640 | 134 | 981 | 150 | +16 μs |
|
||||
| 6 | 11 393 | 109 | 1 052 | 160 | +51 μs |
|
||||
| 7 | 13 146 | 124 | 1 228 | 158 | +34 μs |
|
||||
| 8 | 14 899 | 128 | 1 298 | 132 | +4 μs |
|
||||
| 9 | 16 652 | 72 | 1 550 | 95 | +23 μs |
|
||||
|
||||
mooncake_both_drfix sits **within ±50 μs of plain** at every bin —
|
||||
that's noise-level. The mooncake_both column rises monotonically
|
||||
with bin, drfix doesn't. This is the cleanest possible "ablation".
|
||||
|
||||
## What this means for the trace-replay 45 %
|
||||
|
||||
The prior cache_sweep extrapolation said at \|cache\|≈13 k blocks
|
||||
(APC≈79 %) the per-step cost is ~ 1.24 ms (1 060 μs build_meta + 180 μs
|
||||
get_finished). With the DR fix:
|
||||
|
||||
```
|
||||
build_meta (drfix) ≈ 6 μs ← reduced from ~1 060 μs
|
||||
get_finished ≈ 180 μs ← unchanged
|
||||
total ≈ 186 μs
|
||||
```
|
||||
|
||||
So the DR fix alone takes the per-step connector cost from
|
||||
**~1.24 ms to ~0.19 ms** — an **85 % reduction**. On a ~7 ms decode
|
||||
step that's TPOT inflation dropping from +18 % to +3 %.
|
||||
|
||||
If we also fix the `get_finished` constant (the second fix
|
||||
candidate listed in REPORT.md), per-step cost goes to plain-level
|
||||
~0 — recovering the entire substrate tax in `kv_both` mode.
|
||||
|
||||
## Reproducibility
|
||||
|
||||
```
|
||||
cd microbench/connector_tax/cache_sweep
|
||||
bash run_drfix.sh # ~22 min on H20
|
||||
```
|
||||
|
||||
The orchestrator applies v1+v2+DR_FIX patches, runs the three
|
||||
configs sequentially (the third with the env var set), reverts
|
||||
all patches on exit, and produces SUMMARY.md + figure.png.
|
||||
|
||||
## Implications
|
||||
|
||||
1. **The +85 μs / 1k blocks slope was 100 % from our own
|
||||
`a7df84b` direct-RDMA-read implementation**, not Mooncake's
|
||||
upstream design. Disabling it via env var fully recovers the
|
||||
tax.
|
||||
2. **Direct-read is opt-in by request**: the synthetic workload
|
||||
here never sets `direct_read=True`, so the hash sync was doing
|
||||
no useful work. Production should gate the sync on
|
||||
`direct_read_consumers_present`, or do it incrementally via
|
||||
block_pool callbacks rather than per-step diff.
|
||||
3. **Worker `get_finished` is the next target**: still 180 μs/step
|
||||
constant in both mooncake_both and mooncake_both_drfix. Caused
|
||||
by two `run_coroutine_threadsafe(...).result()` blocking waits
|
||||
in `kv_both` mode even when both queues are empty.
|
||||
@@ -210,12 +210,16 @@ def render(root: Path, all_cfg: dict):
|
||||
# Tax vs cache for mc vs plain
|
||||
plain = all_cfg.get("plain")
|
||||
mc = all_cfg.get("mooncake_both")
|
||||
mc_dr = all_cfg.get("mooncake_both_drfix")
|
||||
noop = all_cfg.get("noop_connector")
|
||||
if plain and mc:
|
||||
lines.append("\n## Connector tax(cache_size) — mooncake_both vs plain\n")
|
||||
lines.append("| bin | cache mid | plain step p50 | mc step p50 | tax (μs) | tax (%) |")
|
||||
|
||||
def _tax_table(label, baseline, target):
|
||||
if not (baseline and target):
|
||||
return
|
||||
lines.append(f"\n## {label}\n")
|
||||
lines.append("| bin | cache mid | baseline step p50 | target step p50 | tax (μs) | tax (%) |")
|
||||
lines.append("|---:|---:|---:|---:|---:|---:|")
|
||||
for bp, bm in zip(plain["per_bin"], mc["per_bin"]):
|
||||
for bp, bm in zip(baseline["per_bin"], target["per_bin"]):
|
||||
if bp["step_duration_us_p50"] and bm["step_duration_us_p50"]:
|
||||
tax = bm["step_duration_us_p50"] - bp["step_duration_us_p50"]
|
||||
pct = tax / bp["step_duration_us_p50"] * 100
|
||||
@@ -224,6 +228,12 @@ def render(root: Path, all_cfg: dict):
|
||||
f"{bp['step_duration_us_p50']} | {bm['step_duration_us_p50']} | "
|
||||
f"{tax:+d} | {pct:+.1f} |"
|
||||
)
|
||||
|
||||
_tax_table("Connector tax(cache_size) — mooncake_both vs plain", plain, mc)
|
||||
_tax_table("Connector tax(cache_size) — mooncake_both_drfix vs plain", plain, mc_dr)
|
||||
_tax_table("DR-fix savings — mooncake_both vs mooncake_both_drfix",
|
||||
mc_dr, mc) # baseline=fixed, target=original → "tax" = savings
|
||||
|
||||
if plain and noop:
|
||||
# Framework cost: noop_connector tax = pure dispatch
|
||||
lines.append("\n## Framework cost — noop_connector vs plain\n")
|
||||
@@ -252,7 +262,8 @@ def render(root: Path, all_cfg: dict):
|
||||
import matplotlib.pyplot as plt
|
||||
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(13, 5))
|
||||
colors = {"plain": "tab:blue", "noop_connector": "tab:orange",
|
||||
"mooncake_both": "tab:red"}
|
||||
"mooncake_both": "tab:red",
|
||||
"mooncake_both_drfix": "tab:green"}
|
||||
for cfg, r in all_cfg.items():
|
||||
if r is None: continue
|
||||
xs = [b["cache_size_mid"] for b in r["per_bin"]]
|
||||
|
||||
95
microbench/connector_tax/cache_sweep/apply_direct_read_fix.py
Executable file
95
microbench/connector_tax/cache_sweep/apply_direct_read_fix.py
Executable file
@@ -0,0 +1,95 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Env-gate the direct-RDMA-read hash sync in MooncakeConnectorScheduler.
|
||||
|
||||
The `set(self._block_pool.cached_block_hash_to_block._cache.keys())` walk
|
||||
at `mooncake_connector.py:434-450` was introduced in our own commit
|
||||
`a7df84b` ("Direct RDMA read: D reads cached KV from C's GPU without
|
||||
C's scheduler") and runs unconditionally every scheduler step regardless
|
||||
of whether any direct-read consumer is actually attached.
|
||||
|
||||
This patch lets us turn it off at runtime via:
|
||||
|
||||
VLLM_MOONCAKE_DISABLE_DIRECT_READ_SYNC=1
|
||||
|
||||
so we can measure the slope contribution of that block in isolation. With
|
||||
the env var set, the entire hash-table sync block is skipped and the
|
||||
bootstrap server's `_hash_table` stays empty (which is fine as long as
|
||||
no direct_read consumer ever queries — and our synthetic bench never
|
||||
does).
|
||||
|
||||
Usage:
|
||||
python apply_direct_read_fix.py --apply [--vllm-root PATH]
|
||||
python apply_direct_read_fix.py --revert [--vllm-root PATH]
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
MARKER = "# CT_DR_FIX"
|
||||
DEFAULT_VLLM_ROOT = Path(
|
||||
"/home/admin/cpfs/wjh/agentic-kv/.venv/lib/python3.12/site-packages/vllm"
|
||||
)
|
||||
|
||||
CONN_REL = "distributed/kv_transfer/kv_connector/v1/mooncake/mooncake_connector.py"
|
||||
|
||||
# The exact line we replace
|
||||
ORIG_LINE = " if self._block_pool is not None:"
|
||||
PATCHED_LINE = (
|
||||
" if self._block_pool is not None and "
|
||||
"__import__('os').environ.get('VLLM_MOONCAKE_DISABLE_DIRECT_READ_SYNC', '0') != '1':"
|
||||
" " + MARKER
|
||||
)
|
||||
|
||||
|
||||
def apply_to_file(path: Path) -> bool:
|
||||
if not path.exists():
|
||||
print(f" SKIP {path} (not found)")
|
||||
return False
|
||||
text = path.read_text()
|
||||
if MARKER in text:
|
||||
print(f" already patched: {path}")
|
||||
return False
|
||||
if ORIG_LINE not in text:
|
||||
raise RuntimeError(
|
||||
f"Cannot find target line `{ORIG_LINE.strip()}` in {path}. "
|
||||
"File may have been refactored."
|
||||
)
|
||||
new_text = text.replace(ORIG_LINE, PATCHED_LINE, 1)
|
||||
path.write_text(new_text)
|
||||
print(f" patched: {path}")
|
||||
return True
|
||||
|
||||
|
||||
def revert_file(path: Path) -> bool:
|
||||
if not path.exists():
|
||||
return False
|
||||
text = path.read_text()
|
||||
if MARKER not in text:
|
||||
print(f" no marker: {path}")
|
||||
return False
|
||||
new_text = text.replace(PATCHED_LINE, ORIG_LINE, 1)
|
||||
path.write_text(new_text)
|
||||
print(f" reverted: {path}")
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--apply", action="store_true")
|
||||
ap.add_argument("--revert", action="store_true")
|
||||
ap.add_argument("--vllm-root", type=Path, default=DEFAULT_VLLM_ROOT)
|
||||
args = ap.parse_args()
|
||||
if not (args.apply ^ args.revert):
|
||||
ap.error("Specify exactly one of --apply / --revert")
|
||||
conn = args.vllm_root / CONN_REL
|
||||
if args.apply:
|
||||
print(f"Applying CT_DR_FIX to {args.vllm_root}")
|
||||
apply_to_file(conn)
|
||||
else:
|
||||
print(f"Reverting CT_DR_FIX from {args.vllm_root}")
|
||||
revert_file(conn)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,99 @@
|
||||
# Cache-size sweep — summary
|
||||
|
||||
Run root: `microbench/connector_tax/cache_sweep/results/20260526_1543_drfix`
|
||||
|
||||
## Per-config fit (`step_duration_us ≈ a + b · cache_size`)
|
||||
|
||||
| config | n steps | cache max | step_dur p50 (μs) | build_meta p50 (μs) | slope (μs / 1k blocks) | intercept (μs) |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| mooncake_both | 16194 | 17528 | 1550 | 1459 | 81.0 | 310.3 |
|
||||
| mooncake_both_drfix | 14980 | 17528 | 95 | 6 | -0.7 | 113.4 |
|
||||
| plain | 17826 | 17530 | 72 | 0 | -1.8 | 112.9 |
|
||||
|
||||
### mooncake_both — per-bin
|
||||
|
||||
| bin | cache mid | n | step_dur p50 | step_dur p90 | build_meta p50 | build_meta p90 |
|
||||
|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 2 | 4382 | 23 | 655 | 955 | 541 | 810 |
|
||||
| 3 | 6135 | 127 | 809 | 1141 | 673 | 963 |
|
||||
| 4 | 7888 | 138 | 1157 | 1284 | 976 | 1112 |
|
||||
| 5 | 9640 | 357 | 981 | 1423 | 873 | 1272 |
|
||||
| 6 | 11393 | 273 | 1052 | 1500 | 958 | 1384 |
|
||||
| 7 | 13146 | 298 | 1228 | 1707 | 1117 | 1574 |
|
||||
| 8 | 14899 | 388 | 1298 | 1806 | 1210 | 1686 |
|
||||
| 9 | 16652 | 14247 | 1550 | 2245 | 1459 | 2094 |
|
||||
|
||||
*worker side (n=16694)* — get_finished p50/p90/p99 = 178/251/346 μs; start_load_kv p50/p90 = 2/6 μs
|
||||
|
||||
|
||||
### mooncake_both_drfix — per-bin
|
||||
|
||||
| bin | cache mid | n | step_dur p50 | step_dur p90 | build_meta p50 | build_meta p90 |
|
||||
|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 0 | 876 | 17 | 60 | 73 | 6 | 6 |
|
||||
| 1 | 2629 | 297 | 85 | 105 | 6 | 7 |
|
||||
| 2 | 4382 | 90 | 94 | 129 | 6 | 8 |
|
||||
| 3 | 6135 | 297 | 121 | 156 | 6 | 7 |
|
||||
| 4 | 7888 | 152 | 101 | 131 | 6 | 7 |
|
||||
| 5 | 9640 | 95 | 150 | 173 | 6 | 7 |
|
||||
| 6 | 11393 | 108 | 160 | 184 | 6 | 7 |
|
||||
| 7 | 13146 | 71 | 158 | 188 | 6 | 7 |
|
||||
| 8 | 14899 | 239 | 132 | 165 | 6 | 7 |
|
||||
| 9 | 16652 | 13242 | 95 | 145 | 6 | 7 |
|
||||
|
||||
*worker side (n=15480)* — get_finished p50/p90/p99 = 183/262/352 μs; start_load_kv p50/p90 = 2/2 μs
|
||||
|
||||
|
||||
### plain — per-bin
|
||||
|
||||
| bin | cache mid | n | step_dur p50 | step_dur p90 | build_meta p50 | build_meta p90 |
|
||||
|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 1 | 2630 | 51 | 70 | 93 | 0 | 0 |
|
||||
| 2 | 4382 | 314 | 60 | 99 | 0 | 0 |
|
||||
| 3 | 6136 | 147 | 138 | 163 | 0 | 0 |
|
||||
| 4 | 7888 | 291 | 96 | 129 | 0 | 0 |
|
||||
| 5 | 9642 | 124 | 134 | 168 | 0 | 0 |
|
||||
| 6 | 11394 | 213 | 114 | 142 | 0 | 0 |
|
||||
| 7 | 13148 | 67 | 108 | 130 | 0 | 0 |
|
||||
| 8 | 14900 | 34 | 142 | 166 | 0 | 0 |
|
||||
| 9 | 16654 | 16259 | 72 | 115 | 0 | 0 |
|
||||
|
||||
## Connector tax(cache_size) — mooncake_both vs plain
|
||||
|
||||
| bin | cache mid | baseline step p50 | target step p50 | tax (μs) | tax (%) |
|
||||
|---:|---:|---:|---:|---:|---:|
|
||||
| 1 | 2630 | 70 | 655 | +585 | +835.7 |
|
||||
| 2 | 4382 | 60 | 809 | +749 | +1248.3 |
|
||||
| 3 | 6136 | 138 | 1157 | +1019 | +738.4 |
|
||||
| 4 | 7888 | 96 | 981 | +885 | +921.9 |
|
||||
| 5 | 9642 | 134 | 1052 | +918 | +685.1 |
|
||||
| 6 | 11394 | 114 | 1228 | +1114 | +977.2 |
|
||||
| 7 | 13148 | 108 | 1298 | +1190 | +1101.9 |
|
||||
| 8 | 14900 | 142 | 1550 | +1408 | +991.5 |
|
||||
|
||||
## Connector tax(cache_size) — mooncake_both_drfix vs plain
|
||||
|
||||
| bin | cache mid | baseline step p50 | target step p50 | tax (μs) | tax (%) |
|
||||
|---:|---:|---:|---:|---:|---:|
|
||||
| 1 | 2630 | 70 | 60 | -10 | -14.3 |
|
||||
| 2 | 4382 | 60 | 85 | +25 | +41.7 |
|
||||
| 3 | 6136 | 138 | 94 | -44 | -31.9 |
|
||||
| 4 | 7888 | 96 | 121 | +25 | +26.0 |
|
||||
| 5 | 9642 | 134 | 101 | -33 | -24.6 |
|
||||
| 6 | 11394 | 114 | 150 | +36 | +31.6 |
|
||||
| 7 | 13148 | 108 | 160 | +52 | +48.1 |
|
||||
| 8 | 14900 | 142 | 158 | +16 | +11.3 |
|
||||
| 9 | 16654 | 72 | 132 | +60 | +83.3 |
|
||||
|
||||
## DR-fix savings — mooncake_both vs mooncake_both_drfix
|
||||
|
||||
| bin | cache mid | baseline step p50 | target step p50 | tax (μs) | tax (%) |
|
||||
|---:|---:|---:|---:|---:|---:|
|
||||
| 0 | 876 | 60 | 655 | +595 | +991.7 |
|
||||
| 1 | 2629 | 85 | 809 | +724 | +851.8 |
|
||||
| 2 | 4382 | 94 | 1157 | +1063 | +1130.9 |
|
||||
| 3 | 6135 | 121 | 981 | +860 | +710.7 |
|
||||
| 4 | 7888 | 101 | 1052 | +951 | +941.6 |
|
||||
| 5 | 9640 | 150 | 1228 | +1078 | +718.7 |
|
||||
| 6 | 11393 | 160 | 1298 | +1138 | +711.2 |
|
||||
| 7 | 13146 | 158 | 1550 | +1392 | +881.0 |
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 88 KiB |
@@ -0,0 +1,624 @@
|
||||
# HELP python_gc_objects_collected_total Objects collected during gc
|
||||
# TYPE python_gc_objects_collected_total counter
|
||||
python_gc_objects_collected_total{generation="0"} 11855.0
|
||||
python_gc_objects_collected_total{generation="1"} 1664.0
|
||||
python_gc_objects_collected_total{generation="2"} 855.0
|
||||
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
|
||||
# TYPE python_gc_objects_uncollectable_total counter
|
||||
python_gc_objects_uncollectable_total{generation="0"} 0.0
|
||||
python_gc_objects_uncollectable_total{generation="1"} 0.0
|
||||
python_gc_objects_uncollectable_total{generation="2"} 0.0
|
||||
# HELP python_gc_collections_total Number of times this generation was collected
|
||||
# TYPE python_gc_collections_total counter
|
||||
python_gc_collections_total{generation="0"} 1337.0
|
||||
python_gc_collections_total{generation="1"} 122.0
|
||||
python_gc_collections_total{generation="2"} 9.0
|
||||
# HELP python_info Python platform information
|
||||
# TYPE python_info gauge
|
||||
python_info{implementation="CPython",major="3",minor="12",patchlevel="3",version="3.12.3"} 1.0
|
||||
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
|
||||
# TYPE process_virtual_memory_bytes gauge
|
||||
process_virtual_memory_bytes 4.106807296e+010
|
||||
# HELP process_resident_memory_bytes Resident memory size in bytes.
|
||||
# TYPE process_resident_memory_bytes gauge
|
||||
process_resident_memory_bytes 1.381404672e+09
|
||||
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
|
||||
# TYPE process_start_time_seconds gauge
|
||||
process_start_time_seconds 1.77981057777e+09
|
||||
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
|
||||
# TYPE process_cpu_seconds_total counter
|
||||
process_cpu_seconds_total 38.64
|
||||
# HELP process_open_fds Number of open file descriptors.
|
||||
# TYPE process_open_fds gauge
|
||||
process_open_fds 67.0
|
||||
# HELP process_max_fds Maximum number of open file descriptors.
|
||||
# TYPE process_max_fds gauge
|
||||
process_max_fds 1.048575e+06
|
||||
# HELP vllm:estimated_flops_per_gpu_total Estimated number of floating point operations per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_flops_per_gpu_total counter
|
||||
vllm:estimated_flops_per_gpu_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:estimated_flops_per_gpu_created Estimated number of floating point operations per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_flops_per_gpu_created gauge
|
||||
vllm:estimated_flops_per_gpu_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826448212e+09
|
||||
# HELP vllm:estimated_read_bytes_per_gpu_total Estimated number of bytes read from memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_read_bytes_per_gpu_total counter
|
||||
vllm:estimated_read_bytes_per_gpu_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:estimated_read_bytes_per_gpu_created Estimated number of bytes read from memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_read_bytes_per_gpu_created gauge
|
||||
vllm:estimated_read_bytes_per_gpu_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826448493e+09
|
||||
# HELP vllm:estimated_write_bytes_per_gpu_total Estimated number of bytes written to memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_write_bytes_per_gpu_total counter
|
||||
vllm:estimated_write_bytes_per_gpu_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:estimated_write_bytes_per_gpu_created Estimated number of bytes written to memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_write_bytes_per_gpu_created gauge
|
||||
vllm:estimated_write_bytes_per_gpu_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779810682644866e+09
|
||||
# HELP vllm:num_requests_running Number of requests in model execution batches.
|
||||
# TYPE vllm:num_requests_running gauge
|
||||
vllm:num_requests_running{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:num_requests_waiting Number of requests waiting to be processed.
|
||||
# TYPE vllm:num_requests_waiting gauge
|
||||
vllm:num_requests_waiting{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:engine_sleep_state Engine sleep state; awake = 0 means engine is sleeping; awake = 1 means engine is awake; weights_offloaded = 1 means sleep level 1; discard_all = 1 means sleep level 2.
|
||||
# TYPE vllm:engine_sleep_state gauge
|
||||
vllm:engine_sleep_state{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",sleep_state="awake"} 1.0
|
||||
vllm:engine_sleep_state{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",sleep_state="weights_offloaded"} 0.0
|
||||
vllm:engine_sleep_state{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",sleep_state="discard_all"} 0.0
|
||||
# HELP vllm:kv_cache_usage_perc KV-cache usage. 1 means 100 percent usage.
|
||||
# TYPE vllm:kv_cache_usage_perc gauge
|
||||
vllm:kv_cache_usage_perc{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prefix_cache_queries_total Prefix cache queries, in terms of number of queried tokens.
|
||||
# TYPE vllm:prefix_cache_queries_total counter
|
||||
vllm:prefix_cache_queries_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.455799e+06
|
||||
# HELP vllm:prefix_cache_queries_created Prefix cache queries, in terms of number of queried tokens.
|
||||
# TYPE vllm:prefix_cache_queries_created gauge
|
||||
vllm:prefix_cache_queries_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826450274e+09
|
||||
# HELP vllm:prefix_cache_hits_total Prefix cache hits, in terms of number of cached tokens.
|
||||
# TYPE vllm:prefix_cache_hits_total counter
|
||||
vllm:prefix_cache_hits_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prefix_cache_hits_created Prefix cache hits, in terms of number of cached tokens.
|
||||
# TYPE vllm:prefix_cache_hits_created gauge
|
||||
vllm:prefix_cache_hits_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826450443e+09
|
||||
# HELP vllm:external_prefix_cache_queries_total External prefix cache queries from KV connector cross-instance cache sharing, in terms of number of queried tokens.
|
||||
# TYPE vllm:external_prefix_cache_queries_total counter
|
||||
vllm:external_prefix_cache_queries_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.455799e+06
|
||||
# HELP vllm:external_prefix_cache_queries_created External prefix cache queries from KV connector cross-instance cache sharing, in terms of number of queried tokens.
|
||||
# TYPE vllm:external_prefix_cache_queries_created gauge
|
||||
vllm:external_prefix_cache_queries_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826450567e+09
|
||||
# HELP vllm:external_prefix_cache_hits_total External prefix cache hits from KV connector cross-instance cache sharing, in terms of number of cached tokens.
|
||||
# TYPE vllm:external_prefix_cache_hits_total counter
|
||||
vllm:external_prefix_cache_hits_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:external_prefix_cache_hits_created External prefix cache hits from KV connector cross-instance cache sharing, in terms of number of cached tokens.
|
||||
# TYPE vllm:external_prefix_cache_hits_created gauge
|
||||
vllm:external_prefix_cache_hits_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.77981068264507e+09
|
||||
# HELP vllm:mm_cache_queries_total Multi-modal cache queries, in terms of number of queried items.
|
||||
# TYPE vllm:mm_cache_queries_total counter
|
||||
vllm:mm_cache_queries_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:mm_cache_queries_created Multi-modal cache queries, in terms of number of queried items.
|
||||
# TYPE vllm:mm_cache_queries_created gauge
|
||||
vllm:mm_cache_queries_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826450815e+09
|
||||
# HELP vllm:mm_cache_hits_total Multi-modal cache hits, in terms of number of cached items.
|
||||
# TYPE vllm:mm_cache_hits_total counter
|
||||
vllm:mm_cache_hits_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:mm_cache_hits_created Multi-modal cache hits, in terms of number of cached items.
|
||||
# TYPE vllm:mm_cache_hits_created gauge
|
||||
vllm:mm_cache_hits_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779810682645094e+09
|
||||
# HELP vllm:num_preemptions_total Cumulative number of preemption from the engine.
|
||||
# TYPE vllm:num_preemptions_total counter
|
||||
vllm:num_preemptions_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:num_preemptions_created Cumulative number of preemption from the engine.
|
||||
# TYPE vllm:num_preemptions_created gauge
|
||||
vllm:num_preemptions_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826451044e+09
|
||||
# HELP vllm:prompt_tokens_total Number of prefill tokens processed.
|
||||
# TYPE vllm:prompt_tokens_total counter
|
||||
vllm:prompt_tokens_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.455799e+06
|
||||
# HELP vllm:prompt_tokens_created Number of prefill tokens processed.
|
||||
# TYPE vllm:prompt_tokens_created gauge
|
||||
vllm:prompt_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826451144e+09
|
||||
# HELP vllm:prompt_tokens_by_source_total Number of prompt tokens by source.
|
||||
# TYPE vllm:prompt_tokens_by_source_total counter
|
||||
vllm:prompt_tokens_by_source_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_compute"} 1.455799e+06
|
||||
vllm:prompt_tokens_by_source_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_cache_hit"} 0.0
|
||||
vllm:prompt_tokens_by_source_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="external_kv_transfer"} 0.0
|
||||
# HELP vllm:prompt_tokens_by_source_created Number of prompt tokens by source.
|
||||
# TYPE vllm:prompt_tokens_by_source_created gauge
|
||||
vllm:prompt_tokens_by_source_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_compute"} 1.7798106826451285e+09
|
||||
vllm:prompt_tokens_by_source_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_cache_hit"} 1.7798106826451337e+09
|
||||
vllm:prompt_tokens_by_source_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="external_kv_transfer"} 1.779810682645141e+09
|
||||
# HELP vllm:prompt_tokens_cached_total Number of cached prompt tokens (local + external).
|
||||
# TYPE vllm:prompt_tokens_cached_total counter
|
||||
vllm:prompt_tokens_cached_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prompt_tokens_cached_created Number of cached prompt tokens (local + external).
|
||||
# TYPE vllm:prompt_tokens_cached_created gauge
|
||||
vllm:prompt_tokens_cached_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826451623e+09
|
||||
# HELP vllm:prompt_tokens_recomputed_total Number of cached tokens recomputed for forward pass.
|
||||
# TYPE vllm:prompt_tokens_recomputed_total counter
|
||||
vllm:prompt_tokens_recomputed_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prompt_tokens_recomputed_created Number of cached tokens recomputed for forward pass.
|
||||
# TYPE vllm:prompt_tokens_recomputed_created gauge
|
||||
vllm:prompt_tokens_recomputed_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826451778e+09
|
||||
# HELP vllm:generation_tokens_total Number of generation tokens processed.
|
||||
# TYPE vllm:generation_tokens_total counter
|
||||
vllm:generation_tokens_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93440.0
|
||||
# HELP vllm:generation_tokens_created Number of generation tokens processed.
|
||||
# TYPE vllm:generation_tokens_created gauge
|
||||
vllm:generation_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826452005e+09
|
||||
# HELP vllm:request_success_total Count of successfully processed requests.
|
||||
# TYPE vllm:request_success_total counter
|
||||
vllm:request_success_total{engine="0",finished_reason="stop",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_success_total{engine="0",finished_reason="length",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_success_total{engine="0",finished_reason="abort",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_success_total{engine="0",finished_reason="error",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_success_total{engine="0",finished_reason="repetition",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:request_success_created Count of successfully processed requests.
|
||||
# TYPE vllm:request_success_created gauge
|
||||
vllm:request_success_created{engine="0",finished_reason="stop",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826452274e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="length",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826452372e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="abort",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826452427e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="error",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826452477e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="repetition",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826452534e+09
|
||||
# HELP vllm:request_prompt_tokens Number of prefill tokens processed.
|
||||
# TYPE vllm:request_prompt_tokens histogram
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prompt_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prompt_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.455799e+06
|
||||
# HELP vllm:request_prompt_tokens_created Number of prefill tokens processed.
|
||||
# TYPE vllm:request_prompt_tokens_created gauge
|
||||
vllm:request_prompt_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826452968e+09
|
||||
# HELP vllm:request_generation_tokens Number of generation tokens processed.
|
||||
# TYPE vllm:request_generation_tokens histogram
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_generation_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93440.0
|
||||
# HELP vllm:request_generation_tokens_created Number of generation tokens processed.
|
||||
# TYPE vllm:request_generation_tokens_created gauge
|
||||
vllm:request_generation_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826453574e+09
|
||||
# HELP vllm:iteration_tokens_total Histogram of number of tokens per engine_step.
|
||||
# TYPE vllm:iteration_tokens_total histogram
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 3061.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="8.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 12382.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="16.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15890.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="32.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16333.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="64.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16333.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="128.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16333.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="256.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16333.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="512.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16333.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="1024.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16333.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="2048.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16333.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="4096.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16646.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="8192.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16672.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="16384.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16672.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16672.0
|
||||
vllm:iteration_tokens_total_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 16672.0
|
||||
vllm:iteration_tokens_total_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.549239e+06
|
||||
# HELP vllm:iteration_tokens_total_created Histogram of number of tokens per engine_step.
|
||||
# TYPE vllm:iteration_tokens_total_created gauge
|
||||
vllm:iteration_tokens_total_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826453998e+09
|
||||
# HELP vllm:request_max_num_generation_tokens Histogram of maximum number of requested generation tokens.
|
||||
# TYPE vllm:request_max_num_generation_tokens histogram
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_max_num_generation_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93440.0
|
||||
# HELP vllm:request_max_num_generation_tokens_created Histogram of maximum number of requested generation tokens.
|
||||
# TYPE vllm:request_max_num_generation_tokens_created gauge
|
||||
vllm:request_max_num_generation_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779810682645445e+09
|
||||
# HELP vllm:request_params_n Histogram of the n request parameter.
|
||||
# TYPE vllm:request_params_n histogram
|
||||
vllm:request_params_n_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_n_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_n_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_n_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_n_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_n_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_n_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_n_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
# HELP vllm:request_params_n_created Histogram of the n request parameter.
|
||||
# TYPE vllm:request_params_n_created gauge
|
||||
vllm:request_params_n_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826454847e+09
|
||||
# HELP vllm:request_params_max_tokens Histogram of the max_tokens request parameter.
|
||||
# TYPE vllm:request_params_max_tokens histogram
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_params_max_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93440.0
|
||||
# HELP vllm:request_params_max_tokens_created Histogram of the max_tokens request parameter.
|
||||
# TYPE vllm:request_params_max_tokens_created gauge
|
||||
vllm:request_params_max_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826455128e+09
|
||||
# HELP vllm:time_to_first_token_seconds Histogram of time to first token in seconds.
|
||||
# TYPE vllm:time_to_first_token_seconds histogram
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.001",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.005",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.01",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.02",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.04",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.06",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.08",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.1",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.25",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 4.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 309.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.75",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 348.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 362.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="7.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="80.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="160.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="640.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="2560.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:time_to_first_token_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 130.10236072540283
|
||||
# HELP vllm:time_to_first_token_seconds_created Histogram of time to first token in seconds.
|
||||
# TYPE vllm:time_to_first_token_seconds_created gauge
|
||||
vllm:time_to_first_token_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826455622e+09
|
||||
# HELP vllm:inter_token_latency_seconds Histogram of inter-token latency in seconds.
|
||||
# TYPE vllm:inter_token_latency_seconds histogram
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.01",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 30048.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.025",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 88447.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.05",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 90523.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.075",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 90637.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.1",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 90637.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.15",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 90637.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.2",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 90960.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 92862.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.4",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 92862.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.75",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="7.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="80.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 93075.0
|
||||
vllm:inter_token_latency_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1695.5502396721859
|
||||
# HELP vllm:inter_token_latency_seconds_created Histogram of inter-token latency in seconds.
|
||||
# TYPE vllm:inter_token_latency_seconds_created gauge
|
||||
vllm:inter_token_latency_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826456032e+09
|
||||
# HELP vllm:request_time_per_output_token_seconds Histogram of time_per_output_token_seconds per request.
|
||||
# TYPE vllm:request_time_per_output_token_seconds histogram
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.01",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 42.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.025",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 305.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.05",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.075",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.1",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.15",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.2",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.4",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.75",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="7.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="80.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_time_per_output_token_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 6.649216626165432
|
||||
# HELP vllm:request_time_per_output_token_seconds_created Histogram of time_per_output_token_seconds per request.
|
||||
# TYPE vllm:request_time_per_output_token_seconds_created gauge
|
||||
vllm:request_time_per_output_token_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779810682645645e+09
|
||||
# HELP vllm:e2e_request_latency_seconds Histogram of e2e request latency in seconds.
|
||||
# TYPE vllm:e2e_request_latency_seconds histogram
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 2.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 11.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 29.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 182.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:e2e_request_latency_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1825.4403264522552
|
||||
# HELP vllm:e2e_request_latency_seconds_created Histogram of e2e request latency in seconds.
|
||||
# TYPE vllm:e2e_request_latency_seconds_created gauge
|
||||
vllm:e2e_request_latency_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826457312e+09
|
||||
# HELP vllm:request_queue_time_seconds Histogram of time spent in WAITING phase for request.
|
||||
# TYPE vllm:request_queue_time_seconds histogram
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_queue_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.04605136066675186
|
||||
# HELP vllm:request_queue_time_seconds_created Histogram of time spent in WAITING phase for request.
|
||||
# TYPE vllm:request_queue_time_seconds_created gauge
|
||||
vllm:request_queue_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826457753e+09
|
||||
# HELP vllm:request_inference_time_seconds Histogram of time spent in RUNNING phase for request.
|
||||
# TYPE vllm:request_inference_time_seconds histogram
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 3.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 11.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 30.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 188.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_inference_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1804.7606877679937
|
||||
# HELP vllm:request_inference_time_seconds_created Histogram of time spent in RUNNING phase for request.
|
||||
# TYPE vllm:request_inference_time_seconds_created gauge
|
||||
vllm:request_inference_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826458116e+09
|
||||
# HELP vllm:request_prefill_time_seconds Histogram of time spent in PREFILL phase for request.
|
||||
# TYPE vllm:request_prefill_time_seconds histogram
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 292.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 347.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 363.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 109.2104480958078
|
||||
# HELP vllm:request_prefill_time_seconds_created Histogram of time spent in PREFILL phase for request.
|
||||
# TYPE vllm:request_prefill_time_seconds_created gauge
|
||||
vllm:request_prefill_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826458645e+09
|
||||
# HELP vllm:request_decode_time_seconds Histogram of time spent in DECODE phase for request.
|
||||
# TYPE vllm:request_decode_time_seconds histogram
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 5.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 21.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 42.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 216.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_decode_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1695.5502396721859
|
||||
# HELP vllm:request_decode_time_seconds_created Histogram of time spent in DECODE phase for request.
|
||||
# TYPE vllm:request_decode_time_seconds_created gauge
|
||||
vllm:request_decode_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779810682645899e+09
|
||||
# HELP vllm:request_prefill_kv_computed_tokens Histogram of new KV tokens computed during prefill (excluding cached tokens).
|
||||
# TYPE vllm:request_prefill_kv_computed_tokens histogram
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_kv_computed_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 365.0
|
||||
vllm:request_prefill_kv_computed_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.455799e+06
|
||||
# HELP vllm:request_prefill_kv_computed_tokens_created Histogram of new KV tokens computed during prefill (excluding cached tokens).
|
||||
# TYPE vllm:request_prefill_kv_computed_tokens_created gauge
|
||||
vllm:request_prefill_kv_computed_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798106826459525e+09
|
||||
# HELP vllm:cache_config_info Information of the LLMEngine CacheConfig
|
||||
# TYPE vllm:cache_config_info gauge
|
||||
vllm:cache_config_info{_block_size_resolved="True",block_size="16",cache_dtype="auto",calculate_kv_scales="False",cpu_kvcache_space_bytes="None",enable_prefix_caching="True",engine="0",gpu_memory_utilization="0.9",is_attention_free="False",kv_cache_memory_bytes="None",kv_offloading_backend="native",kv_offloading_size="None",kv_sharing_fast_prefill="False",mamba_block_size="None",mamba_cache_dtype="auto",mamba_cache_mode="none",mamba_page_size_padded="None",mamba_ssm_cache_dtype="auto",num_cpu_blocks="None",num_gpu_blocks="17590",num_gpu_blocks_override="None",prefix_caching_hash_algo="sha256",sliding_window="None",user_specified_block_size="False"} 1.0
|
||||
# HELP http_requests_total Total number of requests by method, status and handler.
|
||||
# TYPE http_requests_total counter
|
||||
http_requests_total{handler="/v1/models",method="GET",status="2xx"} 1.0
|
||||
http_requests_total{handler="/v1/chat/completions",method="POST",status="2xx"} 365.0
|
||||
# HELP http_requests_created Total number of requests by method, status and handler.
|
||||
# TYPE http_requests_created gauge
|
||||
http_requests_created{handler="/v1/models",method="GET",status="2xx"} 1.7798106845094843e+09
|
||||
http_requests_created{handler="/v1/chat/completions",method="POST",status="2xx"} 1.7798106915180178e+09
|
||||
# HELP http_request_size_bytes Content length of incoming requests by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_request_size_bytes summary
|
||||
http_request_size_bytes_count{handler="/v1/models"} 1.0
|
||||
http_request_size_bytes_sum{handler="/v1/models"} 0.0
|
||||
http_request_size_bytes_count{handler="/v1/chat/completions"} 365.0
|
||||
http_request_size_bytes_sum{handler="/v1/chat/completions"} 1.92501e+06
|
||||
# HELP http_request_size_bytes_created Content length of incoming requests by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_request_size_bytes_created gauge
|
||||
http_request_size_bytes_created{handler="/v1/models"} 1.7798106845095081e+09
|
||||
http_request_size_bytes_created{handler="/v1/chat/completions"} 1.7798106915180402e+09
|
||||
# HELP http_response_size_bytes Content length of outgoing responses by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_response_size_bytes summary
|
||||
http_response_size_bytes_count{handler="/v1/models"} 1.0
|
||||
http_response_size_bytes_sum{handler="/v1/models"} 558.0
|
||||
http_response_size_bytes_count{handler="/v1/chat/completions"} 365.0
|
||||
http_response_size_bytes_sum{handler="/v1/chat/completions"} 0.0
|
||||
# HELP http_response_size_bytes_created Content length of outgoing responses by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_response_size_bytes_created gauge
|
||||
http_response_size_bytes_created{handler="/v1/models"} 1.7798106845095322e+09
|
||||
http_response_size_bytes_created{handler="/v1/chat/completions"} 1.779810691518067e+09
|
||||
# HELP http_request_duration_highr_seconds Latency with many buckets but no API specific labels. Made for more accurate percentile calculations.
|
||||
# TYPE http_request_duration_highr_seconds histogram
|
||||
http_request_duration_highr_seconds_bucket{le="0.01"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.025"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.05"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.075"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.1"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.25"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.5"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.75"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="1.0"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="1.5"} 3.0
|
||||
http_request_duration_highr_seconds_bucket{le="2.0"} 12.0
|
||||
http_request_duration_highr_seconds_bucket{le="2.5"} 30.0
|
||||
http_request_duration_highr_seconds_bucket{le="3.0"} 51.0
|
||||
http_request_duration_highr_seconds_bucket{le="3.5"} 89.0
|
||||
http_request_duration_highr_seconds_bucket{le="4.0"} 116.0
|
||||
http_request_duration_highr_seconds_bucket{le="4.5"} 152.0
|
||||
http_request_duration_highr_seconds_bucket{le="5.0"} 183.0
|
||||
http_request_duration_highr_seconds_bucket{le="7.5"} 328.0
|
||||
http_request_duration_highr_seconds_bucket{le="10.0"} 366.0
|
||||
http_request_duration_highr_seconds_bucket{le="30.0"} 366.0
|
||||
http_request_duration_highr_seconds_bucket{le="60.0"} 366.0
|
||||
http_request_duration_highr_seconds_bucket{le="+Inf"} 366.0
|
||||
http_request_duration_highr_seconds_count 366.0
|
||||
http_request_duration_highr_seconds_sum 1825.9695772863342
|
||||
# HELP http_request_duration_highr_seconds_created Latency with many buckets but no API specific labels. Made for more accurate percentile calculations.
|
||||
# TYPE http_request_duration_highr_seconds_created gauge
|
||||
http_request_duration_highr_seconds_created 1.7798106830941143e+09
|
||||
# HELP http_request_duration_seconds Latency with only few buckets by handler. Made to be only used if aggregation by handler is important.
|
||||
# TYPE http_request_duration_seconds histogram
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="0.1",method="GET"} 1.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="0.5",method="GET"} 1.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="1.0",method="GET"} 1.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="+Inf",method="GET"} 1.0
|
||||
http_request_duration_seconds_count{handler="/v1/models",method="GET"} 1.0
|
||||
http_request_duration_seconds_sum{handler="/v1/models",method="GET"} 0.0021685969550162554
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="0.1",method="POST"} 0.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="0.5",method="POST"} 0.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="1.0",method="POST"} 0.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="+Inf",method="POST"} 365.0
|
||||
http_request_duration_seconds_count{handler="/v1/chat/completions",method="POST"} 365.0
|
||||
http_request_duration_seconds_sum{handler="/v1/chat/completions",method="POST"} 1825.9674086893792
|
||||
# HELP http_request_duration_seconds_created Latency with only few buckets by handler. Made to be only used if aggregation by handler is important.
|
||||
# TYPE http_request_duration_seconds_created gauge
|
||||
http_request_duration_seconds_created{handler="/v1/models",method="GET"} 1.7798106845095782e+09
|
||||
http_request_duration_seconds_created{handler="/v1/chat/completions",method="POST"} 1.7798106915180964e+09
|
||||
@@ -0,0 +1,365 @@
|
||||
{"req_id": "7d2956d61ef64657", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379820362946705, "t_first_token_ns": 379821622826822, "t_last_token_ns": 379826504308788, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "90e6b3c27a094531", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379820772125236, "t_first_token_ns": 379821668646300, "t_last_token_ns": 379826517895756, "prompt_tokens": 4045, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "d84465cb74cb41eb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379820749218279, "t_first_token_ns": 379821668826800, "t_last_token_ns": 379826518226702, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "b103f08a81dc452b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379821302679299, "t_first_token_ns": 379822071202910, "t_last_token_ns": 379826529632152, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "0c75c3643d30425d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379821271167747, "t_first_token_ns": 379822071302396, "t_last_token_ns": 379826529851769, "prompt_tokens": 3950, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "666effe4f7c44858", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379822936329897, "t_first_token_ns": 379823202881627, "t_last_token_ns": 379827577125553, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "2a5ee38a84d94b05", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379823040294890, "t_first_token_ns": 379823449406908, "t_last_token_ns": 379827597217446, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "49bd80cf458d43b2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379824059159908, "t_first_token_ns": 379824334244061, "t_last_token_ns": 379828152294403, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "4c10c8677a0c45ad", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379824544667355, "t_first_token_ns": 379824826102939, "t_last_token_ns": 379828587359541, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "b9753319a1104b84", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379825290219156, "t_first_token_ns": 379825557357817, "t_last_token_ns": 379830412699838, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "7c8e318f27e44f15", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379825727450895, "t_first_token_ns": 379826006545657, "t_last_token_ns": 379830814842184, "prompt_tokens": 3955, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "e7a57635d2ae416c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379826011811197, "t_first_token_ns": 379826283501669, "t_last_token_ns": 379830860782242, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "804d5db8dde046f0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379828277418023, "t_first_token_ns": 379828538752877, "t_last_token_ns": 379835150283002, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "9a6cfa5922604b88", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379828676735697, "t_first_token_ns": 379828936122975, "t_last_token_ns": 379835643428723, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "b463d77016cb4ccc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379828732743157, "t_first_token_ns": 379829181358372, "t_last_token_ns": 379835675453955, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "53032180c92a4bf9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379829130436545, "t_first_token_ns": 379829424702350, "t_last_token_ns": 379835704133197, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "0d3496d027f347f0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379829597364035, "t_first_token_ns": 379829870081425, "t_last_token_ns": 379836914618061, "prompt_tokens": 4032, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "3e56afe643854728", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379829656241683, "t_first_token_ns": 379830108153488, "t_last_token_ns": 379837133667817, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "5b55707b34f143b7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379829996090948, "t_first_token_ns": 379830356154277, "t_last_token_ns": 379837160229145, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "e03bb0fdef344c75", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379830302540004, "t_first_token_ns": 379830611295006, "t_last_token_ns": 379837189016935, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "1b270898038a4339", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379831144241223, "t_first_token_ns": 379831421212502, "t_last_token_ns": 379837857245877, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "d9b0b664bdc94006", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379831266082254, "t_first_token_ns": 379831667170199, "t_last_token_ns": 379837885077785, "prompt_tokens": 3938, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "484572b86bd3478b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379831476166691, "t_first_token_ns": 379831916951355, "t_last_token_ns": 379837910864725, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "b4a54fdbcbd64d7c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379832476075783, "t_first_token_ns": 379832757424622, "t_last_token_ns": 379838769426034, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "061eb1f70aa544e1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379832599272549, "t_first_token_ns": 379833014644994, "t_last_token_ns": 379839031921773, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "9c4be770e2ba4ee1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379833546085769, "t_first_token_ns": 379833827141086, "t_last_token_ns": 379839790170968, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "85a0bc53dc02493d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379834453685979, "t_first_token_ns": 379834738212464, "t_last_token_ns": 379840861431283, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "325ad11fa55146e7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379835081930069, "t_first_token_ns": 379835367468841, "t_last_token_ns": 379841458846312, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "c249979c26934f77", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379835848174757, "t_first_token_ns": 379836130283853, "t_last_token_ns": 379841970114646, "prompt_tokens": 3951, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "f7f203e4fb524f7c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379836095724595, "t_first_token_ns": 379836386987139, "t_last_token_ns": 379841997488094, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "20dc7c5ec4a743a3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379836245057695, "t_first_token_ns": 379836861425298, "t_last_token_ns": 379842021799972, "prompt_tokens": 4033, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "2bca51aabf364101", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379836339869785, "t_first_token_ns": 379836861754602, "t_last_token_ns": 379842021942272, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "1fe79265cf374cff", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379836848747031, "t_first_token_ns": 379837133876728, "t_last_token_ns": 379842055358845, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "d2d6af278fe6410c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379838113616550, "t_first_token_ns": 379838389791219, "t_last_token_ns": 379842764907671, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "71a67cdaf25f4813", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379838708072829, "t_first_token_ns": 379838986877280, "t_last_token_ns": 379843020042630, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "d8b2c0fafef7439c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379838960949981, "t_first_token_ns": 379839245821450, "t_last_token_ns": 379843036689690, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "8537d5d326a3486e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379840171524882, "t_first_token_ns": 379840449373983, "t_last_token_ns": 379843538463295, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "0db83c918e4c477c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379840226092665, "t_first_token_ns": 379840689072220, "t_last_token_ns": 379843544245493, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "11f7d009b77f43e0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379840976850967, "t_first_token_ns": 379841258056534, "t_last_token_ns": 379844147602194, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "190ea1d119634b47", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379843562312240, "t_first_token_ns": 379843817626420, "t_last_token_ns": 379846975257635, "prompt_tokens": 3954, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "fcd44038bee74a70", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379843795407823, "t_first_token_ns": 379844057425528, "t_last_token_ns": 379846996579850, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "6c99e0a561dc48ca", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379844674239692, "t_first_token_ns": 379844985114709, "t_last_token_ns": 379849519067272, "prompt_tokens": 4019, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "b813a7e99a9542bf", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379844699172825, "t_first_token_ns": 379845394502595, "t_last_token_ns": 379849531022488, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "a1651313a59d4953", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379844715356267, "t_first_token_ns": 379845394688033, "t_last_token_ns": 379849531213116, "prompt_tokens": 3963, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "66921c0fef9448d1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379845972236760, "t_first_token_ns": 379846234512548, "t_last_token_ns": 379850739579424, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "276a5d2e1d824354", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379847124582169, "t_first_token_ns": 379847383903104, "t_last_token_ns": 379851903855731, "prompt_tokens": 3958, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "45db7a3bc3324603", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379847348339671, "t_first_token_ns": 379847628262593, "t_last_token_ns": 379851926509949, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "fb552f3db33e41e8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379847816334547, "t_first_token_ns": 379848086210421, "t_last_token_ns": 379852146354769, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "42807bb3de8e48b1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379848339784480, "t_first_token_ns": 379848609818811, "t_last_token_ns": 379852407502639, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "53b6cd29b63c463b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379848451640285, "t_first_token_ns": 379848858906897, "t_last_token_ns": 379852426436796, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "9ad056de047449d7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379848940062934, "t_first_token_ns": 379849213660423, "t_last_token_ns": 379852970564558, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "e19ba299ceeb48dd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379849547524858, "t_first_token_ns": 379849817307341, "t_last_token_ns": 379853937237677, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "3a1f094f2e184480", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379849710941426, "t_first_token_ns": 379850068187495, "t_last_token_ns": 379853955257029, "prompt_tokens": 4035, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "0dd71fadfded4c98", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379852412078080, "t_first_token_ns": 379852676916378, "t_last_token_ns": 379856269472489, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "c96bbc04639749ed", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379852689019503, "t_first_token_ns": 379852953891068, "t_last_token_ns": 379856331659040, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "add9279a7cf14032", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379852969025041, "t_first_token_ns": 379853233309208, "t_last_token_ns": 379856380372161, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "013d5b98d8b942f2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379853106503572, "t_first_token_ns": 379853478942229, "t_last_token_ns": 379856395350540, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "1881bf8751454b6d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379853501463777, "t_first_token_ns": 379853769348043, "t_last_token_ns": 379856436556781, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "eb7d378279944545", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379854829897074, "t_first_token_ns": 379855099780051, "t_last_token_ns": 379856967381009, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "55a48a4e22284d96", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379858376581097, "t_first_token_ns": 379858636456620, "t_last_token_ns": 379860856237263, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "5242befccdb94d0d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379858731552286, "t_first_token_ns": 379858990380967, "t_last_token_ns": 379861008308030, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "26456ad510854b98", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379858997454813, "t_first_token_ns": 379859262284703, "t_last_token_ns": 379861080564449, "prompt_tokens": 4034, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "485c374c25ba4d2c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379861017529168, "t_first_token_ns": 379861276036898, "t_last_token_ns": 379862494591840, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "8c356f04318345eb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379862797392337, "t_first_token_ns": 379863054352340, "t_last_token_ns": 379865613418051, "prompt_tokens": 3959, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "143cf23a39224677", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379862917124421, "t_first_token_ns": 379863292834758, "t_last_token_ns": 379865629176195, "prompt_tokens": 3960, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "7556823a019f4b26", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379863124396291, "t_first_token_ns": 379863533719190, "t_last_token_ns": 379865872588321, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "c8256ab634744394", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379864693813193, "t_first_token_ns": 379864954733602, "t_last_token_ns": 379869469889352, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "6705b89a74104371", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379865608243060, "t_first_token_ns": 379865872894838, "t_last_token_ns": 379870809276292, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "0cb78b5669ac41cb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379865705797024, "t_first_token_ns": 379866114087268, "t_last_token_ns": 379870832950324, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "3fa34ebdd1d0430d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379866079973429, "t_first_token_ns": 379866352456343, "t_last_token_ns": 379870855469281, "prompt_tokens": 3952, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "d904d17326514df1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379866498298201, "t_first_token_ns": 379866759312913, "t_last_token_ns": 379871134913751, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "0fe189a95fad484c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379866925693507, "t_first_token_ns": 379867198747907, "t_last_token_ns": 379871590991578, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "82223b8ba63f4fa7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379866986850563, "t_first_token_ns": 379867488771090, "t_last_token_ns": 379871609250033, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "ab69cd5357624ea8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379867220235983, "t_first_token_ns": 379867685378863, "t_last_token_ns": 379871618881641, "prompt_tokens": 4025, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "779151540e2d4a93", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379867460614877, "t_first_token_ns": 379867925962751, "t_last_token_ns": 379871626596282, "prompt_tokens": 4008, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "5fd4848542654d4c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379869072934639, "t_first_token_ns": 379869347245604, "t_last_token_ns": 379873520292456, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "ac6ff0fcff0c46a9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379870006255486, "t_first_token_ns": 379870287546215, "t_last_token_ns": 379874087282976, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "95fe0c87f8914765", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379871224682529, "t_first_token_ns": 379871488424401, "t_last_token_ns": 379875095260742, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "f2ca999bd3984ddd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379871720734510, "t_first_token_ns": 379871988150954, "t_last_token_ns": 379875403638345, "prompt_tokens": 4022, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "5cfb7b497945481d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379871815631006, "t_first_token_ns": 379872232013897, "t_last_token_ns": 379875419311938, "prompt_tokens": 4030, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "0787a72f2ee3496d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379872807120162, "t_first_token_ns": 379873077293871, "t_last_token_ns": 379875885342821, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "bfdfbab762594960", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379873005049018, "t_first_token_ns": 379873325758487, "t_last_token_ns": 379875936602149, "prompt_tokens": 4015, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "1e944328a70b4f3e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379874669745230, "t_first_token_ns": 379874942786735, "t_last_token_ns": 379878099049087, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "a87dd11d61f84954", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379875872042544, "t_first_token_ns": 379876142237396, "t_last_token_ns": 379879428406869, "prompt_tokens": 4045, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "ecf9c3191c0e4afc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379876207463349, "t_first_token_ns": 379876460960896, "t_last_token_ns": 379879581306622, "prompt_tokens": 3950, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "570ff52fa2c44d92", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379876777186547, "t_first_token_ns": 379877039700987, "t_last_token_ns": 379879961722716, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "9fa5b81f4c6a4d54", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379877483244100, "t_first_token_ns": 379877745329538, "t_last_token_ns": 379880404022093, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "708287f0644f44d2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379877576059398, "t_first_token_ns": 379877989480013, "t_last_token_ns": 379880416016835, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "73eb9883238a4941", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379878626078600, "t_first_token_ns": 379878893391535, "t_last_token_ns": 379880754879576, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "6a50cea1487c4af3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379881174923493, "t_first_token_ns": 379881432006950, "t_last_token_ns": 379882649195018, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "a16f9cec4e1d4ee7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379883271466528, "t_first_token_ns": 379883528070870, "t_last_token_ns": 379886447880937, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "8332da64241f4f61", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379883347473861, "t_first_token_ns": 379883769759290, "t_last_token_ns": 379886465686040, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "06261b32bab845e6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379884308573749, "t_first_token_ns": 379884569076873, "t_last_token_ns": 379887511697928, "prompt_tokens": 3971, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "12cf982912444acd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379884850905546, "t_first_token_ns": 379885121026059, "t_last_token_ns": 379887859538448, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "0814397869ea431e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379885836203234, "t_first_token_ns": 379886101735013, "t_last_token_ns": 379888567888063, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "a9a6ba57913448d5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379886115694736, "t_first_token_ns": 379886384907383, "t_last_token_ns": 379888603085693, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "0cbcc4b61bc04b53", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379886918438340, "t_first_token_ns": 379887177160598, "t_last_token_ns": 379890052139868, "prompt_tokens": 3943, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "8305045c8e804938", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379888637081061, "t_first_token_ns": 379888897753864, "t_last_token_ns": 379893194950967, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "ec152bcae34848e9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379888861642882, "t_first_token_ns": 379889351889088, "t_last_token_ns": 379893217207127, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "5c86adb831ff473d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379888874827918, "t_first_token_ns": 379889351787379, "t_last_token_ns": 379893217347077, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "59be743f49404400", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379889544089178, "t_first_token_ns": 379889805365498, "t_last_token_ns": 379894223524953, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "3d88b9745f924631", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379890515500606, "t_first_token_ns": 379890777235574, "t_last_token_ns": 379896459172627, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "1b423ea6f1c34d0e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379890918661217, "t_first_token_ns": 379891190620894, "t_last_token_ns": 379896724529398, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "bed4ffe43ac3423a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379891087137213, "t_first_token_ns": 379891435123076, "t_last_token_ns": 379896753454664, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "8c78745b8b9f4731", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379891205852807, "t_first_token_ns": 379891674313442, "t_last_token_ns": 379896765828065, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "45131e50d3734570", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379891540473429, "t_first_token_ns": 379891921427385, "t_last_token_ns": 379896789277889, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "c33262f161a24240", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379893288703048, "t_first_token_ns": 379893557425668, "t_last_token_ns": 379898414690536, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "f8fdfea33ed54fb7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379893444627950, "t_first_token_ns": 379893803600621, "t_last_token_ns": 379898437389852, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "3c787dec088d4f55", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379893902364180, "t_first_token_ns": 379894176757020, "t_last_token_ns": 379898590827105, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "b30144cf048041f3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379894222740410, "t_first_token_ns": 379894500243043, "t_last_token_ns": 379898674154513, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "13d3237d2b674110", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379894997311663, "t_first_token_ns": 379895268822996, "t_last_token_ns": 379899145680460, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "1af89108b97649d0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379895615498666, "t_first_token_ns": 379895897988313, "t_last_token_ns": 379899390396526, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "a57adb07225041f3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379895871231031, "t_first_token_ns": 379896153308092, "t_last_token_ns": 379899405776069, "prompt_tokens": 4017, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "a77efdcbbfc14a7b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379896095104691, "t_first_token_ns": 379896404346882, "t_last_token_ns": 379899416501619, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "46ebce3e106a4d78", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379897854209359, "t_first_token_ns": 379898131521655, "t_last_token_ns": 379900492431440, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "b81b324df79a44b7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379899929955475, "t_first_token_ns": 379900190349347, "t_last_token_ns": 379902292163118, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "464ce2cef99344ed", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379900207016934, "t_first_token_ns": 379900465507318, "t_last_token_ns": 379902338346743, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "e0f87f5239cb46bb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379901277139805, "t_first_token_ns": 379901537771506, "t_last_token_ns": 379903329175677, "prompt_tokens": 4021, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "ec1de604cbc3478f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379902642661089, "t_first_token_ns": 379902894859969, "t_last_token_ns": 379904626528579, "prompt_tokens": 3947, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "aebfecbd379e4c4a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379903353666034, "t_first_token_ns": 379903609690406, "t_last_token_ns": 379905029278711, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "aca54e3e0d944dfa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379905063933278, "t_first_token_ns": 379905320453164, "t_last_token_ns": 379907892846305, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "3ebad2ed017544c9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379905257730016, "t_first_token_ns": 379905776162055, "t_last_token_ns": 379907905966421, "prompt_tokens": 4019, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "3da1de8099914e03", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379905290549708, "t_first_token_ns": 379905776267861, "t_last_token_ns": 379907906119562, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "dba820c6aff74730", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379906435975344, "t_first_token_ns": 379906700195838, "t_last_token_ns": 379908386665853, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "ebdd936df90c4551", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379909103062133, "t_first_token_ns": 379909358450263, "t_last_token_ns": 379912818819094, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "1a1ca080675f44c4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379909563207927, "t_first_token_ns": 379909824469205, "t_last_token_ns": 379913289121939, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "d8024e51428b47ff", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379909767273193, "t_first_token_ns": 379910065311040, "t_last_token_ns": 379913308550475, "prompt_tokens": 4020, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "09add09a19db4956", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379910320335920, "t_first_token_ns": 379910584485693, "t_last_token_ns": 379913630591291, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "deff2a22eec94d7c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379910825783535, "t_first_token_ns": 379911092162045, "t_last_token_ns": 379913888255470, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "77971ae8acac4054", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379911523973124, "t_first_token_ns": 379911786946448, "t_last_token_ns": 379914446917223, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "c090762fbe3546c6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379912136655119, "t_first_token_ns": 379912401871424, "t_last_token_ns": 379915410378478, "prompt_tokens": 3954, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "c70dc731555c49a7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379913958854825, "t_first_token_ns": 379914220486755, "t_last_token_ns": 379918395709670, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "936b099d2f0a4005", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379914575835554, "t_first_token_ns": 379914840761129, "t_last_token_ns": 379920105364678, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "c6170dc0d050414b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379914787531085, "t_first_token_ns": 379915082703270, "t_last_token_ns": 379920133191685, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "b8b9c4f636e74d82", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379915048536747, "t_first_token_ns": 379915319597868, "t_last_token_ns": 379920160577518, "prompt_tokens": 3947, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "e9d0b23155024880", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379916259255405, "t_first_token_ns": 379916524671601, "t_last_token_ns": 379922130584590, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "2ff750dc588d4dca", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379916742319754, "t_first_token_ns": 379917011988424, "t_last_token_ns": 379922487918849, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "965fe49db23b4981", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379916860313188, "t_first_token_ns": 379917257323596, "t_last_token_ns": 379922512176973, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "1891c09f786541c6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379917354540842, "t_first_token_ns": 379917633486686, "t_last_token_ns": 379922660435420, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "f0f176b6c6494733", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379917400535130, "t_first_token_ns": 379917868233239, "t_last_token_ns": 379922671703580, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "75bbe2f819324c63", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379917976221046, "t_first_token_ns": 379918245431553, "t_last_token_ns": 379922794843262, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "3583f0275bf14391", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379918515937167, "t_first_token_ns": 379918784840099, "t_last_token_ns": 379923049316662, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "6fbb845fdf544706", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379918887086420, "t_first_token_ns": 379919163482717, "t_last_token_ns": 379923142037210, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "8c11f791a7f24f8c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379918902906770, "t_first_token_ns": 379919403616465, "t_last_token_ns": 379923149071335, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "2bb4502a3bf54f45", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379919240923074, "t_first_token_ns": 379919655307085, "t_last_token_ns": 379923161088238, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "9d51a57f7cdf432e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379920451464583, "t_first_token_ns": 379920726928202, "t_last_token_ns": 379923734840800, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "787275fee5b24202", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379923198217942, "t_first_token_ns": 379923455352386, "t_last_token_ns": 379926084205163, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "b3c0e7b9ea8f4794", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379924020480904, "t_first_token_ns": 379924275990496, "t_last_token_ns": 379927148990262, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "85838fd3f47748f1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379924975914110, "t_first_token_ns": 379925241988318, "t_last_token_ns": 379928196279257, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "d5cde1bf13604f82", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379925020243267, "t_first_token_ns": 379925477754390, "t_last_token_ns": 379928202766296, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "3b55c390145b4fef", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379925535177539, "t_first_token_ns": 379925796528076, "t_last_token_ns": 379928273418323, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "d13e1944ff804ed5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379925675726136, "t_first_token_ns": 379926041145260, "t_last_token_ns": 379928281979538, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "3482f197cb634342", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379929283625855, "t_first_token_ns": 379929541415388, "t_last_token_ns": 379932503623043, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "b26b6babb1044c68", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379929585756880, "t_first_token_ns": 379929841942394, "t_last_token_ns": 379932640083502, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "ef7ba887fceb451b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379930975914748, "t_first_token_ns": 379931236047067, "t_last_token_ns": 379939649302818, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "e8b20b43fb7f4758", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379931143468331, "t_first_token_ns": 379931690671327, "t_last_token_ns": 379939717125041, "prompt_tokens": 4010, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "2c7f920554dd454e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379931040231350, "t_first_token_ns": 379931690772044, "t_last_token_ns": 379939717864123, "prompt_tokens": 3971, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "642fe2ede1fd48fd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379931434329020, "t_first_token_ns": 379931932396577, "t_last_token_ns": 379939733897195, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "ecdb6da906734c64", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379932216516612, "t_first_token_ns": 379932483478878, "t_last_token_ns": 379940898703335, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "825b3fbb0f0c49c7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379932632268088, "t_first_token_ns": 379932894241451, "t_last_token_ns": 379941318551566, "prompt_tokens": 3961, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "1074f653d1fd49dd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379933327073511, "t_first_token_ns": 379933591697224, "t_last_token_ns": 379942203441297, "prompt_tokens": 4021, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "edd1ba81d9694644", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379933555174604, "t_first_token_ns": 379933838161470, "t_last_token_ns": 379942238969455, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "aab17454b35644e2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379933597305917, "t_first_token_ns": 379934077873667, "t_last_token_ns": 379942256230420, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "aa4502c3e9bb4458", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379933839324832, "t_first_token_ns": 379934327168924, "t_last_token_ns": 379942285443044, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "36b70c6e308b4f68", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379934289638134, "t_first_token_ns": 379934576456547, "t_last_token_ns": 379942315324992, "prompt_tokens": 3934, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "9ff845d29c8a428f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379934323490166, "t_first_token_ns": 379934811046822, "t_last_token_ns": 379942330928377, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "0f1033e087514f7f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379934870164299, "t_first_token_ns": 379935142474860, "t_last_token_ns": 379942440406671, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "b23a36d2525b4bba", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379935271937068, "t_first_token_ns": 379935549263556, "t_last_token_ns": 379942625981450, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "038fcbde43f74e01", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379935615411189, "t_first_token_ns": 379935893387259, "t_last_token_ns": 379943412111289, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "3eff78df55234372", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379935629162482, "t_first_token_ns": 379936133904511, "t_last_token_ns": 379943865199504, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "73d28df082174b5e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379936226322963, "t_first_token_ns": 379936553881691, "t_last_token_ns": 379944022795012, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "1c737ea8ebd84497", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379936328903566, "t_first_token_ns": 379936760993957, "t_last_token_ns": 379944038345209, "prompt_tokens": 4008, "completion_tokens": 256, "inflight_at_send": 18, "error": null}
|
||||
{"req_id": "c5b93514ae0440cb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379937155992130, "t_first_token_ns": 379937442061611, "t_last_token_ns": 379944871724529, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "ef70043206a04798", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379937704751832, "t_first_token_ns": 379937988723438, "t_last_token_ns": 379945381900852, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 20, "error": null}
|
||||
{"req_id": "f74d45a88b2d47e2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379938086705710, "t_first_token_ns": 379938375455071, "t_last_token_ns": 379945495104184, "prompt_tokens": 3949, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "a1ddc95f7bd94907", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379938606601930, "t_first_token_ns": 379938903262820, "t_last_token_ns": 379945935422333, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 22, "error": null}
|
||||
{"req_id": "f120b57c95494215", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379938780862857, "t_first_token_ns": 379939164407921, "t_last_token_ns": 379945963238256, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 23, "error": null}
|
||||
{"req_id": "ceec6337ff7a4f31", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379939827232217, "t_first_token_ns": 379940115158574, "t_last_token_ns": 379946705600956, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 20, "error": null}
|
||||
{"req_id": "7fb062b274f9490c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379940000420025, "t_first_token_ns": 379940377413434, "t_last_token_ns": 379946956327039, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "fd99cae164094c02", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379942640376980, "t_first_token_ns": 379942920962538, "t_last_token_ns": 379951752249864, "prompt_tokens": 3947, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "137d0f2a4a5942ac", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379942893228081, "t_first_token_ns": 379943174697457, "t_last_token_ns": 379951994062998, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "d8d832aaa0d34b5e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379942949421367, "t_first_token_ns": 379943411068940, "t_last_token_ns": 379952013270311, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "c03d757c1dfb492c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379943006332498, "t_first_token_ns": 379943864990617, "t_last_token_ns": 379952032812954, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "a2b7fb4ee3174c93", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379942981582547, "t_first_token_ns": 379943864094122, "t_last_token_ns": 379952033039709, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "4352019a50b14c4b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379944046939244, "t_first_token_ns": 379944321057199, "t_last_token_ns": 379952373114121, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "14f0f19e268f4ba0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379944505512124, "t_first_token_ns": 379944786205196, "t_last_token_ns": 379952924260054, "prompt_tokens": 4015, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "d7f02134d29546de", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379944866167464, "t_first_token_ns": 379945143149566, "t_last_token_ns": 379953075630383, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "d3ef8d55734f4bb1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379945645939870, "t_first_token_ns": 379945921440392, "t_last_token_ns": 379954210302304, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "f67944056cb2454b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379946078837993, "t_first_token_ns": 379946358474530, "t_last_token_ns": 379954691160870, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "faeb815ad9924c6b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379946680786215, "t_first_token_ns": 379946957219644, "t_last_token_ns": 379955146651367, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "97030bca5a6a49d7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379948395943789, "t_first_token_ns": 379948680303141, "t_last_token_ns": 379957196742620, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "2f167bba4b4e43b1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379948594796034, "t_first_token_ns": 379949148527637, "t_last_token_ns": 379957230691090, "prompt_tokens": 4010, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "33fe6f6a83b94371", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379948631652112, "t_first_token_ns": 379949149256744, "t_last_token_ns": 379957230975097, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "51606e7af02c4126", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379948902853580, "t_first_token_ns": 379949666692722, "t_last_token_ns": 379957258058532, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "d9f230a3b2b142a9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379948770054101, "t_first_token_ns": 379949666515848, "t_last_token_ns": 379957258258726, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "5c153b98d24d4db7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379949084950597, "t_first_token_ns": 379950070761932, "t_last_token_ns": 379957272164751, "prompt_tokens": 3956, "completion_tokens": 256, "inflight_at_send": 18, "error": null}
|
||||
{"req_id": "f9884d2f0d54459d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379949011485898, "t_first_token_ns": 379950070989177, "t_last_token_ns": 379957272285493, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "c427a2b5d19c40a2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379949252890335, "t_first_token_ns": 379950309860672, "t_last_token_ns": 379957285753663, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "a9c2766942054cb4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379949902399344, "t_first_token_ns": 379950766626031, "t_last_token_ns": 379957299507474, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 20, "error": null}
|
||||
{"req_id": "07e0b70bdb8448fd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379949903348040, "t_first_token_ns": 379950765812463, "t_last_token_ns": 379957299711417, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "9411243b745040be", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379950505103902, "t_first_token_ns": 379951239320047, "t_last_token_ns": 379957324223034, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 23, "error": null}
|
||||
{"req_id": "df4e833ae3454c39", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379950364738177, "t_first_token_ns": 379951240048358, "t_last_token_ns": 379957324433350, "prompt_tokens": 3961, "completion_tokens": 256, "inflight_at_send": 22, "error": null}
|
||||
{"req_id": "88edc74e28f143bb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379950993420396, "t_first_token_ns": 379951501111920, "t_last_token_ns": 379957346164496, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 24, "error": null}
|
||||
{"req_id": "b9744fc1f22342eb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379951686759427, "t_first_token_ns": 379951977620422, "t_last_token_ns": 379957506705584, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 25, "error": null}
|
||||
{"req_id": "bb02c6e2234b4051", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379952293844444, "t_first_token_ns": 379952594181525, "t_last_token_ns": 379958121379070, "prompt_tokens": 3951, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "608c2ac8b39e4d74", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379953236580598, "t_first_token_ns": 379953518310373, "t_last_token_ns": 379959909774400, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "e5f0714ecc9d4b78", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379953650052392, "t_first_token_ns": 379953927358654, "t_last_token_ns": 379960055274962, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 20, "error": null}
|
||||
{"req_id": "dfd25f3d8dcf48ac", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379954365804300, "t_first_token_ns": 379954652320544, "t_last_token_ns": 379960420377378, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 20, "error": null}
|
||||
{"req_id": "0bdf0018c1974daf", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379956202593403, "t_first_token_ns": 379956483970080, "t_last_token_ns": 379961803138347, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "633087163bf94679", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379957445298145, "t_first_token_ns": 379957716899267, "t_last_token_ns": 379962604726226, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "cc88307e9c3540b4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379957829256836, "t_first_token_ns": 379958092049905, "t_last_token_ns": 379962794908165, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "077b2207ef774a3a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379958112650830, "t_first_token_ns": 379958378114619, "t_last_token_ns": 379962867741022, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "33270df6f6f34bbb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379958441558754, "t_first_token_ns": 379958712205726, "t_last_token_ns": 379962965163946, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "8cbd3b79749f4a92", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379958779353059, "t_first_token_ns": 379959048225267, "t_last_token_ns": 379963065822793, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "298ca6f0fa0d4ee4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379958819108294, "t_first_token_ns": 379959288103964, "t_last_token_ns": 379963073691180, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "1812ce266dea402c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379959163635098, "t_first_token_ns": 379959753557760, "t_last_token_ns": 379963087649525, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "2402c050fa9e4050", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379959186465368, "t_first_token_ns": 379959753382699, "t_last_token_ns": 379963087764419, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "0e42d986f3844586", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379961021869570, "t_first_token_ns": 379961297723399, "t_last_token_ns": 379963592898690, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "a46da57ae0a3480c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379964239122989, "t_first_token_ns": 379964491939495, "t_last_token_ns": 379966179922856, "prompt_tokens": 3949, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "f2e361c68a3b4d14", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379965651558442, "t_first_token_ns": 379965908291314, "t_last_token_ns": 379967921199021, "prompt_tokens": 3961, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "6a306c96f42b4852", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379965850716788, "t_first_token_ns": 379966146259141, "t_last_token_ns": 379967933146587, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "904d84eb9eb641f8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379967405077117, "t_first_token_ns": 379967662577013, "t_last_token_ns": 379969206686104, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "7ceb3d68b9ef48c3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379968946046630, "t_first_token_ns": 379969203128509, "t_last_token_ns": 379973102222082, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "399b012616104a0c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379969248225827, "t_first_token_ns": 379969504490835, "t_last_token_ns": 379973243010415, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "85c209f309c04985", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379969406991384, "t_first_token_ns": 379969744707585, "t_last_token_ns": 379973262918981, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "7265acf17fea49c7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379969690693026, "t_first_token_ns": 379969984868724, "t_last_token_ns": 379973280176624, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "c9f9a5e574b94b12", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379969997814452, "t_first_token_ns": 379970256192531, "t_last_token_ns": 379973325577970, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "dad9c579e412421f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379970206424257, "t_first_token_ns": 379970501509702, "t_last_token_ns": 379973336931493, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "0d63d66e76014ce0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379971708749738, "t_first_token_ns": 379971972802585, "t_last_token_ns": 379974197580563, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "3fa1e9449e9746e3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379973894341558, "t_first_token_ns": 379974147365431, "t_last_token_ns": 379976080821609, "prompt_tokens": 3943, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "f9550d6169364042", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379974695731753, "t_first_token_ns": 379974954263374, "t_last_token_ns": 379976778319950, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "ac343c252cb94b40", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379975394206684, "t_first_token_ns": 379975663255983, "t_last_token_ns": 379977160283082, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "898e92febc224355", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379977475149962, "t_first_token_ns": 379977734397593, "t_last_token_ns": 379981068570551, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "9df9f912c707492b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379977545159966, "t_first_token_ns": 379978188159945, "t_last_token_ns": 379981087690131, "prompt_tokens": 4026, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "5ff51d1d71aa4175", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379977658302671, "t_first_token_ns": 379978188045516, "t_last_token_ns": 379981087899474, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "f6c02b2d39df4eb0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379978459736923, "t_first_token_ns": 379978717120970, "t_last_token_ns": 379981432898849, "prompt_tokens": 3943, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "559e45a8968b4720", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379978719396348, "t_first_token_ns": 379978982655236, "t_last_token_ns": 379981462357312, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "8db22411bc544046", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379979457809362, "t_first_token_ns": 379979726569116, "t_last_token_ns": 379982463974934, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "ba1d79c6e50c4e06", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379981578528090, "t_first_token_ns": 379981839444059, "t_last_token_ns": 379986041109484, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "92fa3b5376f842fb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379981784817853, "t_first_token_ns": 379982292564794, "t_last_token_ns": 379986063767941, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "f767e4f5a33947da", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379981778394519, "t_first_token_ns": 379982292262959, "t_last_token_ns": 379986064232565, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "ecf431e6f7da4f26", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379982998737736, "t_first_token_ns": 379983264050342, "t_last_token_ns": 379989181659716, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "16eca60c380745cc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379983080278900, "t_first_token_ns": 379983719400193, "t_last_token_ns": 379989209202656, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "d3257d0befbe48e3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379983187370649, "t_first_token_ns": 379983719673027, "t_last_token_ns": 379989210008103, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "9efb77c7cf30455e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379983492580261, "t_first_token_ns": 379983962460483, "t_last_token_ns": 379989235213833, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "317439e035004c07", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379983804315610, "t_first_token_ns": 379984208817646, "t_last_token_ns": 379989262597032, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "54d120b753f44cac", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379985573812336, "t_first_token_ns": 379985840317189, "t_last_token_ns": 379991576593960, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "777f1504e5d2417b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379986290555228, "t_first_token_ns": 379986560283517, "t_last_token_ns": 379992212996103, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "6d4de337a47d467f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379986358681543, "t_first_token_ns": 379987070915256, "t_last_token_ns": 379992241043527, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "df2d79e4878f48d0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379986528546519, "t_first_token_ns": 379987071504554, "t_last_token_ns": 379992241733802, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "fc7f65bbd39e4cca", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379986541507994, "t_first_token_ns": 379987258889371, "t_last_token_ns": 379992252693875, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "8897ab8520b94c70", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379986649872567, "t_first_token_ns": 379987495806088, "t_last_token_ns": 379992265079363, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "fa36ed9c079240e9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379987586200712, "t_first_token_ns": 379987867676713, "t_last_token_ns": 379993292559755, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "6f71943be5b04c80", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379988137567800, "t_first_token_ns": 379988417915564, "t_last_token_ns": 379993803245754, "prompt_tokens": 4023, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "c7b00e2644244605", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379988858986586, "t_first_token_ns": 379989138961549, "t_last_token_ns": 379994185875377, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "7fea04403533475b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379989820266612, "t_first_token_ns": 379990096142639, "t_last_token_ns": 379995861520233, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "b3d96c2cee9e4456", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379990397565916, "t_first_token_ns": 379990668719167, "t_last_token_ns": 379996675895176, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "a59f3b9d44394037", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379991140144994, "t_first_token_ns": 379991423989645, "t_last_token_ns": 379997234885250, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "c2b1fab39cc645f6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379992264529630, "t_first_token_ns": 379992533116547, "t_last_token_ns": 379998368592369, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "93cdb9ef3cdb47d2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379992462699125, "t_first_token_ns": 379992779990651, "t_last_token_ns": 379998396654460, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "a1ac5e68ca6c4fee", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379992529479916, "t_first_token_ns": 379993016017674, "t_last_token_ns": 379998410841386, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "ecd93ba2b25c482c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379992712832346, "t_first_token_ns": 379993255575213, "t_last_token_ns": 379998422816663, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "08a80ca6a6484099", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379993364946585, "t_first_token_ns": 379993634922310, "t_last_token_ns": 379999043168959, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "aee9c99a98f14991", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379994385337865, "t_first_token_ns": 379994663270764, "t_last_token_ns": 380001569032092, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "856ca533f1a04b2f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379994871029631, "t_first_token_ns": 379995142186161, "t_last_token_ns": 380001883721587, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "6876ecc9a8fb43ef", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379995212201800, "t_first_token_ns": 379995489321146, "t_last_token_ns": 380002016993749, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "97676de52dd946a2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379995567237894, "t_first_token_ns": 379995849307727, "t_last_token_ns": 380002157417220, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "d407671f7be74526", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379995890227851, "t_first_token_ns": 379996159058412, "t_last_token_ns": 380002240586664, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "e36987422dd8475e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379996391966107, "t_first_token_ns": 379996675822712, "t_last_token_ns": 380002517912209, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "a271d2f136b144ce", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379997602172241, "t_first_token_ns": 379997877659000, "t_last_token_ns": 380003681464891, "prompt_tokens": 3963, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "eabc46dca5a6499f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379998412782931, "t_first_token_ns": 379998687539544, "t_last_token_ns": 380004704747877, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "3483fccfe6224332", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379998633647282, "t_first_token_ns": 379998935693092, "t_last_token_ns": 380004731763884, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "97f29905fd9d42c2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379999059157318, "t_first_token_ns": 379999328335837, "t_last_token_ns": 380004905986154, "prompt_tokens": 3971, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "b516e452408144ae", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379999124823848, "t_first_token_ns": 379999577467973, "t_last_token_ns": 380004927028713, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "e020111200134b23", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379999744453936, "t_first_token_ns": 380000020527368, "t_last_token_ns": 380005111591383, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "8258793a15fc4dea", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380000089153546, "t_first_token_ns": 380000372737426, "t_last_token_ns": 380005219537889, "prompt_tokens": 4030, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "5a482e9824ef4059", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380000178087482, "t_first_token_ns": 380000842328497, "t_last_token_ns": 380005238674096, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "e114d43b06c14189", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380000328940927, "t_first_token_ns": 380000842718735, "t_last_token_ns": 380005238925121, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "ee858d5da1c940b6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380001209582743, "t_first_token_ns": 380001482795709, "t_last_token_ns": 380005455403576, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "1f3fc35a6fea433d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380002880522980, "t_first_token_ns": 380003152456791, "t_last_token_ns": 380006445282007, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "aa7af6000a2047b0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380003941414695, "t_first_token_ns": 380004223081806, "t_last_token_ns": 380007156818143, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "2494688292114a6d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380004126623617, "t_first_token_ns": 380004469663430, "t_last_token_ns": 380007172562782, "prompt_tokens": 3935, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "4d4e26b26a234b5f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380005806561803, "t_first_token_ns": 380006072808876, "t_last_token_ns": 380008241944746, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "6b3972bf77414d48", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380006537928679, "t_first_token_ns": 380006800335816, "t_last_token_ns": 380008627053946, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "f23420c10c284609", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380007963344978, "t_first_token_ns": 380008221974125, "t_last_token_ns": 380009767711539, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "19820874e1ed4b30", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380009454184840, "t_first_token_ns": 380009711238403, "t_last_token_ns": 380011212522045, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "c75f5d74ffe14978", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380010737226928, "t_first_token_ns": 380010997752034, "t_last_token_ns": 380012827673806, "prompt_tokens": 4017, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "e16f36d3bca44229", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380011992279348, "t_first_token_ns": 380012252676702, "t_last_token_ns": 380014648794804, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "362612433e66441f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380012023660428, "t_first_token_ns": 380012486528766, "t_last_token_ns": 380014656718864, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "021188a806104eaa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380013382886010, "t_first_token_ns": 380013644226617, "t_last_token_ns": 380016693951072, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "8221dfcd5000452a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380014096672318, "t_first_token_ns": 380014357477501, "t_last_token_ns": 380017575509005, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "47c1d4620ca948dc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380014892222803, "t_first_token_ns": 380015154741553, "t_last_token_ns": 380018893709912, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "f569bc06ce1b4526", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380015287710796, "t_first_token_ns": 380015546049240, "t_last_token_ns": 380019587237855, "prompt_tokens": 3955, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "c78cf5b74f424793", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380015865979454, "t_first_token_ns": 380016135278957, "t_last_token_ns": 380020326686638, "prompt_tokens": 4021, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "be36c82ed8d54aac", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380016380982108, "t_first_token_ns": 380016646548209, "t_last_token_ns": 380020665205444, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "14def4c922684535", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380017225511373, "t_first_token_ns": 380017493949593, "t_last_token_ns": 380021259765543, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "5b7a4bd0e5a64281", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380018033833354, "t_first_token_ns": 380018302579707, "t_last_token_ns": 380023274782671, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "db68e35ac1474eea", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380018370185755, "t_first_token_ns": 380018642774849, "t_last_token_ns": 380023443073893, "prompt_tokens": 4036, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "6f4c86bdbaa848a8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380018900386457, "t_first_token_ns": 380019173161748, "t_last_token_ns": 380023758426148, "prompt_tokens": 4010, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "0a0fb3afc61f4fc8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380019063436838, "t_first_token_ns": 380019423291958, "t_last_token_ns": 380023779450454, "prompt_tokens": 4035, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "ee83fd025f85460b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380019745068388, "t_first_token_ns": 380020010673622, "t_last_token_ns": 380024823415287, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "420dcfe4c318462b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380021459806383, "t_first_token_ns": 380021729431005, "t_last_token_ns": 380027322838111, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "ebcfceca6a7344ef", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380021642987959, "t_first_token_ns": 380021976247431, "t_last_token_ns": 380027349081127, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "ba9ce8fae9dc4607", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380021995409957, "t_first_token_ns": 380022269389738, "t_last_token_ns": 380027417153905, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "e713a978066d411f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380022384905005, "t_first_token_ns": 380022664717137, "t_last_token_ns": 380027584727293, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "3128a3946cdb49db", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380022497271151, "t_first_token_ns": 380023128520923, "t_last_token_ns": 380027606922528, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "a52714a687124909", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380022455362707, "t_first_token_ns": 380023128772404, "t_last_token_ns": 380027607212670, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "13322d6e58414c0b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380023839842432, "t_first_token_ns": 380024111601341, "t_last_token_ns": 380028710470209, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "9b73d02fbfb447df", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380024294583532, "t_first_token_ns": 380024561040669, "t_last_token_ns": 380028907733084, "prompt_tokens": 3955, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "7dde05f1ab1a43ec", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380024489144763, "t_first_token_ns": 380024811373629, "t_last_token_ns": 380028928223479, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "ffab0880df1f4f2e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380024799121026, "t_first_token_ns": 380025073121421, "t_last_token_ns": 380028949963230, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "3d42d9d7b38b493e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380026554747085, "t_first_token_ns": 380026833575446, "t_last_token_ns": 380031869893409, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "4c75ed202202418d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380027733398726, "t_first_token_ns": 380028001641094, "t_last_token_ns": 380033945571817, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "38efe24afff54b9a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380028034009576, "t_first_token_ns": 380028302719200, "t_last_token_ns": 380034061439810, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "820569e407174daa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380028997857926, "t_first_token_ns": 380029261042811, "t_last_token_ns": 380035297478840, "prompt_tokens": 4008, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "b1e88bdbf24647d7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380029273176669, "t_first_token_ns": 380029537927010, "t_last_token_ns": 380035377951844, "prompt_tokens": 4027, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "fa6d7121d3124c09", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380029484077987, "t_first_token_ns": 380030003534608, "t_last_token_ns": 380035403594127, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "8f80e3c1ed84442c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380029348114250, "t_first_token_ns": 380030003657526, "t_last_token_ns": 380035403899373, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "c1014d1535de4fbc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380029869576197, "t_first_token_ns": 380030248629958, "t_last_token_ns": 380035426708711, "prompt_tokens": 4022, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "652ecc45c3a74d9e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380030110182789, "t_first_token_ns": 380030709714344, "t_last_token_ns": 380035448898329, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "4236e64edfa444e6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380030071011821, "t_first_token_ns": 380030709884258, "t_last_token_ns": 380035449276480, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "a411e6eb24ce4cc3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380032501837550, "t_first_token_ns": 380032777466786, "t_last_token_ns": 380038272397698, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "d1e2c776e7244c89", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380032624069742, "t_first_token_ns": 380033240982889, "t_last_token_ns": 380038300570628, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "3dbc3d9bb90d491e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380032683092099, "t_first_token_ns": 380033241166971, "t_last_token_ns": 380038301055190, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "fabee6d113d84933", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380033044765102, "t_first_token_ns": 380033493362816, "t_last_token_ns": 380038322651234, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "f17ef92c8cfa4f8e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380033614400997, "t_first_token_ns": 380033887315375, "t_last_token_ns": 380038916573460, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "882210807b1d4e90", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380034553394468, "t_first_token_ns": 380034835062444, "t_last_token_ns": 380040725866782, "prompt_tokens": 3954, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "fa07b1b590aa4c04", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380035864011842, "t_first_token_ns": 380036133298789, "t_last_token_ns": 380042684860436, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "f762a616f72949e7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380036689135853, "t_first_token_ns": 380036962681111, "t_last_token_ns": 380043452632854, "prompt_tokens": 3971, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "652325e94a5e490f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380036758884890, "t_first_token_ns": 380037424809919, "t_last_token_ns": 380043482673697, "prompt_tokens": 4015, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "f8e7df30d9ff4d60", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380036829918954, "t_first_token_ns": 380037425550149, "t_last_token_ns": 380043483499471, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "dffe72e0a66445d8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380037777075722, "t_first_token_ns": 380038057233220, "t_last_token_ns": 380043899763653, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "ef56e4297ec84100", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380038399357599, "t_first_token_ns": 380038669191022, "t_last_token_ns": 380044564242739, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "732ea914fb3e42fa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380038475541493, "t_first_token_ns": 380038916972111, "t_last_token_ns": 380044593776782, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "d84224666945447f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380038790363795, "t_first_token_ns": 380039163637490, "t_last_token_ns": 380044617112686, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "b8ce0628eade4d18", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380039110396090, "t_first_token_ns": 380039412464442, "t_last_token_ns": 380044642286764, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "27e8aa9b8a814812", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380039534546735, "t_first_token_ns": 380039809921839, "t_last_token_ns": 380044797181428, "prompt_tokens": 4010, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "0683fe9c7619423a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380039593180251, "t_first_token_ns": 380040275840664, "t_last_token_ns": 380044817580511, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "90f2344b43a243d6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380039758564385, "t_first_token_ns": 380040276175656, "t_last_token_ns": 380044817764976, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "829591344d9d4daf", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380040728515067, "t_first_token_ns": 380041005743750, "t_last_token_ns": 380045114934364, "prompt_tokens": 3952, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "7b93c2e1501340fa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380041630442000, "t_first_token_ns": 380041905173188, "t_last_token_ns": 380046965411913, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "f86d0fed800d4a49", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380041889144439, "t_first_token_ns": 380042176912315, "t_last_token_ns": 380046997705960, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "228450c015874403", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380043925048856, "t_first_token_ns": 380044195307622, "t_last_token_ns": 380050118463225, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "0051b28f5e2046cc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380045128946010, "t_first_token_ns": 380045390578725, "t_last_token_ns": 380051428712750, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "71d56fe4d51f4361", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380045217423645, "t_first_token_ns": 380045854553717, "t_last_token_ns": 380051458103011, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "6528f9824e09423a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380045201983866, "t_first_token_ns": 380045855096618, "t_last_token_ns": 380051458717122, "prompt_tokens": 3955, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "4c2cdad83f63475d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380045266705565, "t_first_token_ns": 380046078785721, "t_last_token_ns": 380051471683501, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "09ce43b6600a49bb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380046355664635, "t_first_token_ns": 380046631271990, "t_last_token_ns": 380051823975276, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "be98d9365ad643b5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380046663660809, "t_first_token_ns": 380046934016518, "t_last_token_ns": 380051904757786, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "cbfb9a0615f54565", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380047443516639, "t_first_token_ns": 380047716029605, "t_last_token_ns": 380052481531005, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "fb88932312214fe4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380047737817439, "t_first_token_ns": 380048009117656, "t_last_token_ns": 380052542514252, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "90ef588fadca4ef3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380048127332510, "t_first_token_ns": 380048394721445, "t_last_token_ns": 380052686268652, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "7c0a819844a74f8f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380048725487890, "t_first_token_ns": 380048999662066, "t_last_token_ns": 380052922583409, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "13073bbe05cb4a4e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380048837487613, "t_first_token_ns": 380049248589544, "t_last_token_ns": 380052937971909, "prompt_tokens": 3956, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "68d551b9ab844bce", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380049257688704, "t_first_token_ns": 380049530214951, "t_last_token_ns": 380052960935925, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "0967a20506804d58", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380049344593070, "t_first_token_ns": 380049785171020, "t_last_token_ns": 380052969399107, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "b1e2f47f8bc34188", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380053348271817, "t_first_token_ns": 380053602256514, "t_last_token_ns": 380054864555602, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "9b5088dc35434b28", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380054804650417, "t_first_token_ns": 380055059093236, "t_last_token_ns": 380058456807978, "prompt_tokens": 3956, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "b271dad9cc5640ba", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380054964215995, "t_first_token_ns": 380055299355396, "t_last_token_ns": 380058477149150, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "d9994969d8204185", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380055361387865, "t_first_token_ns": 380055621146012, "t_last_token_ns": 380058639098308, "prompt_tokens": 4026, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "be18c90d07a4424b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380056606445809, "t_first_token_ns": 380056873471379, "t_last_token_ns": 380060239848719, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "2211516d1f6949f6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380057228368624, "t_first_token_ns": 380057493298090, "t_last_token_ns": 380060681582892, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "b8ab3841cbc8490a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380057392828606, "t_first_token_ns": 380057739708281, "t_last_token_ns": 380060696419226, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "132c139518774b1b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380057702870257, "t_first_token_ns": 380057984838867, "t_last_token_ns": 380060708322844, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "f6e58874e86540a6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380058717313705, "t_first_token_ns": 380058978566642, "t_last_token_ns": 380061056273561, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"rate": 1.5,
|
||||
"input_tokens": 4096,
|
||||
"output_tokens": 256,
|
||||
"duration_target_s": 240.0,
|
||||
"duration_actual_s": 241.02416862000246,
|
||||
"n_requests": 365
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,624 @@
|
||||
# HELP python_gc_objects_collected_total Objects collected during gc
|
||||
# TYPE python_gc_objects_collected_total counter
|
||||
python_gc_objects_collected_total{generation="0"} 11970.0
|
||||
python_gc_objects_collected_total{generation="1"} 1549.0
|
||||
python_gc_objects_collected_total{generation="2"} 855.0
|
||||
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
|
||||
# TYPE python_gc_objects_uncollectable_total counter
|
||||
python_gc_objects_uncollectable_total{generation="0"} 0.0
|
||||
python_gc_objects_uncollectable_total{generation="1"} 0.0
|
||||
python_gc_objects_uncollectable_total{generation="2"} 0.0
|
||||
# HELP python_gc_collections_total Number of times this generation was collected
|
||||
# TYPE python_gc_collections_total counter
|
||||
python_gc_collections_total{generation="0"} 1351.0
|
||||
python_gc_collections_total{generation="1"} 123.0
|
||||
python_gc_collections_total{generation="2"} 9.0
|
||||
# HELP python_info Python platform information
|
||||
# TYPE python_info gauge
|
||||
python_info{implementation="CPython",major="3",minor="12",patchlevel="3",version="3.12.3"} 1.0
|
||||
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
|
||||
# TYPE process_virtual_memory_bytes gauge
|
||||
process_virtual_memory_bytes 4.106817536e+010
|
||||
# HELP process_resident_memory_bytes Resident memory size in bytes.
|
||||
# TYPE process_resident_memory_bytes gauge
|
||||
process_resident_memory_bytes 1.381801984e+09
|
||||
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
|
||||
# TYPE process_start_time_seconds gauge
|
||||
process_start_time_seconds 1.77981092963e+09
|
||||
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
|
||||
# TYPE process_cpu_seconds_total counter
|
||||
process_cpu_seconds_total 39.55
|
||||
# HELP process_open_fds Number of open file descriptors.
|
||||
# TYPE process_open_fds gauge
|
||||
process_open_fds 67.0
|
||||
# HELP process_max_fds Maximum number of open file descriptors.
|
||||
# TYPE process_max_fds gauge
|
||||
process_max_fds 1.048575e+06
|
||||
# HELP vllm:estimated_flops_per_gpu_total Estimated number of floating point operations per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_flops_per_gpu_total counter
|
||||
vllm:estimated_flops_per_gpu_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:estimated_flops_per_gpu_created Estimated number of floating point operations per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_flops_per_gpu_created gauge
|
||||
vllm:estimated_flops_per_gpu_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344604213e+09
|
||||
# HELP vllm:estimated_read_bytes_per_gpu_total Estimated number of bytes read from memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_read_bytes_per_gpu_total counter
|
||||
vllm:estimated_read_bytes_per_gpu_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:estimated_read_bytes_per_gpu_created Estimated number of bytes read from memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_read_bytes_per_gpu_created gauge
|
||||
vllm:estimated_read_bytes_per_gpu_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344604573e+09
|
||||
# HELP vllm:estimated_write_bytes_per_gpu_total Estimated number of bytes written to memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_write_bytes_per_gpu_total counter
|
||||
vllm:estimated_write_bytes_per_gpu_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:estimated_write_bytes_per_gpu_created Estimated number of bytes written to memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_write_bytes_per_gpu_created gauge
|
||||
vllm:estimated_write_bytes_per_gpu_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344604774e+09
|
||||
# HELP vllm:num_requests_running Number of requests in model execution batches.
|
||||
# TYPE vllm:num_requests_running gauge
|
||||
vllm:num_requests_running{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:num_requests_waiting Number of requests waiting to be processed.
|
||||
# TYPE vllm:num_requests_waiting gauge
|
||||
vllm:num_requests_waiting{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:engine_sleep_state Engine sleep state; awake = 0 means engine is sleeping; awake = 1 means engine is awake; weights_offloaded = 1 means sleep level 1; discard_all = 1 means sleep level 2.
|
||||
# TYPE vllm:engine_sleep_state gauge
|
||||
vllm:engine_sleep_state{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",sleep_state="awake"} 1.0
|
||||
vllm:engine_sleep_state{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",sleep_state="weights_offloaded"} 0.0
|
||||
vllm:engine_sleep_state{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",sleep_state="discard_all"} 0.0
|
||||
# HELP vllm:kv_cache_usage_perc KV-cache usage. 1 means 100 percent usage.
|
||||
# TYPE vllm:kv_cache_usage_perc gauge
|
||||
vllm:kv_cache_usage_perc{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prefix_cache_queries_total Prefix cache queries, in terms of number of queried tokens.
|
||||
# TYPE vllm:prefix_cache_queries_total counter
|
||||
vllm:prefix_cache_queries_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.532108e+06
|
||||
# HELP vllm:prefix_cache_queries_created Prefix cache queries, in terms of number of queried tokens.
|
||||
# TYPE vllm:prefix_cache_queries_created gauge
|
||||
vllm:prefix_cache_queries_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344606564e+09
|
||||
# HELP vllm:prefix_cache_hits_total Prefix cache hits, in terms of number of cached tokens.
|
||||
# TYPE vllm:prefix_cache_hits_total counter
|
||||
vllm:prefix_cache_hits_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prefix_cache_hits_created Prefix cache hits, in terms of number of cached tokens.
|
||||
# TYPE vllm:prefix_cache_hits_created gauge
|
||||
vllm:prefix_cache_hits_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344606721e+09
|
||||
# HELP vllm:external_prefix_cache_queries_total External prefix cache queries from KV connector cross-instance cache sharing, in terms of number of queried tokens.
|
||||
# TYPE vllm:external_prefix_cache_queries_total counter
|
||||
vllm:external_prefix_cache_queries_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.532108e+06
|
||||
# HELP vllm:external_prefix_cache_queries_created External prefix cache queries from KV connector cross-instance cache sharing, in terms of number of queried tokens.
|
||||
# TYPE vllm:external_prefix_cache_queries_created gauge
|
||||
vllm:external_prefix_cache_queries_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034460689e+09
|
||||
# HELP vllm:external_prefix_cache_hits_total External prefix cache hits from KV connector cross-instance cache sharing, in terms of number of cached tokens.
|
||||
# TYPE vllm:external_prefix_cache_hits_total counter
|
||||
vllm:external_prefix_cache_hits_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:external_prefix_cache_hits_created External prefix cache hits from KV connector cross-instance cache sharing, in terms of number of cached tokens.
|
||||
# TYPE vllm:external_prefix_cache_hits_created gauge
|
||||
vllm:external_prefix_cache_hits_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344607024e+09
|
||||
# HELP vllm:mm_cache_queries_total Multi-modal cache queries, in terms of number of queried items.
|
||||
# TYPE vllm:mm_cache_queries_total counter
|
||||
vllm:mm_cache_queries_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:mm_cache_queries_created Multi-modal cache queries, in terms of number of queried items.
|
||||
# TYPE vllm:mm_cache_queries_created gauge
|
||||
vllm:mm_cache_queries_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034460714e+09
|
||||
# HELP vllm:mm_cache_hits_total Multi-modal cache hits, in terms of number of cached items.
|
||||
# TYPE vllm:mm_cache_hits_total counter
|
||||
vllm:mm_cache_hits_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:mm_cache_hits_created Multi-modal cache hits, in terms of number of cached items.
|
||||
# TYPE vllm:mm_cache_hits_created gauge
|
||||
vllm:mm_cache_hits_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344607253e+09
|
||||
# HELP vllm:num_preemptions_total Cumulative number of preemption from the engine.
|
||||
# TYPE vllm:num_preemptions_total counter
|
||||
vllm:num_preemptions_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:num_preemptions_created Cumulative number of preemption from the engine.
|
||||
# TYPE vllm:num_preemptions_created gauge
|
||||
vllm:num_preemptions_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344607375e+09
|
||||
# HELP vllm:prompt_tokens_total Number of prefill tokens processed.
|
||||
# TYPE vllm:prompt_tokens_total counter
|
||||
vllm:prompt_tokens_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.532108e+06
|
||||
# HELP vllm:prompt_tokens_created Number of prefill tokens processed.
|
||||
# TYPE vllm:prompt_tokens_created gauge
|
||||
vllm:prompt_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344607503e+09
|
||||
# HELP vllm:prompt_tokens_by_source_total Number of prompt tokens by source.
|
||||
# TYPE vllm:prompt_tokens_by_source_total counter
|
||||
vllm:prompt_tokens_by_source_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_compute"} 1.532108e+06
|
||||
vllm:prompt_tokens_by_source_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_cache_hit"} 0.0
|
||||
vllm:prompt_tokens_by_source_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="external_kv_transfer"} 0.0
|
||||
# HELP vllm:prompt_tokens_by_source_created Number of prompt tokens by source.
|
||||
# TYPE vllm:prompt_tokens_by_source_created gauge
|
||||
vllm:prompt_tokens_by_source_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_compute"} 1.7798110344607658e+09
|
||||
vllm:prompt_tokens_by_source_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_cache_hit"} 1.7798110344607716e+09
|
||||
vllm:prompt_tokens_by_source_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="external_kv_transfer"} 1.779811034460776e+09
|
||||
# HELP vllm:prompt_tokens_cached_total Number of cached prompt tokens (local + external).
|
||||
# TYPE vllm:prompt_tokens_cached_total counter
|
||||
vllm:prompt_tokens_cached_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prompt_tokens_cached_created Number of cached prompt tokens (local + external).
|
||||
# TYPE vllm:prompt_tokens_cached_created gauge
|
||||
vllm:prompt_tokens_cached_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344607887e+09
|
||||
# HELP vllm:prompt_tokens_recomputed_total Number of cached tokens recomputed for forward pass.
|
||||
# TYPE vllm:prompt_tokens_recomputed_total counter
|
||||
vllm:prompt_tokens_recomputed_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prompt_tokens_recomputed_created Number of cached tokens recomputed for forward pass.
|
||||
# TYPE vllm:prompt_tokens_recomputed_created gauge
|
||||
vllm:prompt_tokens_recomputed_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344608095e+09
|
||||
# HELP vllm:generation_tokens_total Number of generation tokens processed.
|
||||
# TYPE vllm:generation_tokens_total counter
|
||||
vllm:generation_tokens_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 98304.0
|
||||
# HELP vllm:generation_tokens_created Number of generation tokens processed.
|
||||
# TYPE vllm:generation_tokens_created gauge
|
||||
vllm:generation_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034460824e+09
|
||||
# HELP vllm:request_success_total Count of successfully processed requests.
|
||||
# TYPE vllm:request_success_total counter
|
||||
vllm:request_success_total{engine="0",finished_reason="stop",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_success_total{engine="0",finished_reason="length",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_success_total{engine="0",finished_reason="abort",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_success_total{engine="0",finished_reason="error",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_success_total{engine="0",finished_reason="repetition",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:request_success_created Count of successfully processed requests.
|
||||
# TYPE vllm:request_success_created gauge
|
||||
vllm:request_success_created{engine="0",finished_reason="stop",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344608517e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="length",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344608617e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="abort",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344608693e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="error",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344608765e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="repetition",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344608822e+09
|
||||
# HELP vllm:request_prompt_tokens Number of prefill tokens processed.
|
||||
# TYPE vllm:request_prompt_tokens histogram
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prompt_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prompt_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.532108e+06
|
||||
# HELP vllm:request_prompt_tokens_created Number of prefill tokens processed.
|
||||
# TYPE vllm:request_prompt_tokens_created gauge
|
||||
vllm:request_prompt_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034460934e+09
|
||||
# HELP vllm:request_generation_tokens Number of generation tokens processed.
|
||||
# TYPE vllm:request_generation_tokens histogram
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_generation_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 98304.0
|
||||
# HELP vllm:request_generation_tokens_created Number of generation tokens processed.
|
||||
# TYPE vllm:request_generation_tokens_created gauge
|
||||
vllm:request_generation_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344610207e+09
|
||||
# HELP vllm:iteration_tokens_total Histogram of number of tokens per engine_step.
|
||||
# TYPE vllm:iteration_tokens_total histogram
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1487.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="8.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 10979.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="16.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 14922.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="32.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15102.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="64.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15102.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="128.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15102.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="256.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15102.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="512.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15102.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="1024.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15102.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="2048.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15102.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="4096.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15450.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="8192.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15468.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="16384.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15468.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15468.0
|
||||
vllm:iteration_tokens_total_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15468.0
|
||||
vllm:iteration_tokens_total_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.630412e+06
|
||||
# HELP vllm:iteration_tokens_total_created Histogram of number of tokens per engine_step.
|
||||
# TYPE vllm:iteration_tokens_total_created gauge
|
||||
vllm:iteration_tokens_total_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344610624e+09
|
||||
# HELP vllm:request_max_num_generation_tokens Histogram of maximum number of requested generation tokens.
|
||||
# TYPE vllm:request_max_num_generation_tokens histogram
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_max_num_generation_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 98304.0
|
||||
# HELP vllm:request_max_num_generation_tokens_created Histogram of maximum number of requested generation tokens.
|
||||
# TYPE vllm:request_max_num_generation_tokens_created gauge
|
||||
vllm:request_max_num_generation_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344610982e+09
|
||||
# HELP vllm:request_params_n Histogram of the n request parameter.
|
||||
# TYPE vllm:request_params_n histogram
|
||||
vllm:request_params_n_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_n_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_n_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_n_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_n_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_n_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_n_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_n_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
# HELP vllm:request_params_n_created Histogram of the n request parameter.
|
||||
# TYPE vllm:request_params_n_created gauge
|
||||
vllm:request_params_n_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034461132e+09
|
||||
# HELP vllm:request_params_max_tokens Histogram of the max_tokens request parameter.
|
||||
# TYPE vllm:request_params_max_tokens histogram
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_params_max_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 98304.0
|
||||
# HELP vllm:request_params_max_tokens_created Histogram of the max_tokens request parameter.
|
||||
# TYPE vllm:request_params_max_tokens_created gauge
|
||||
vllm:request_params_max_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344611626e+09
|
||||
# HELP vllm:time_to_first_token_seconds Histogram of time to first token in seconds.
|
||||
# TYPE vllm:time_to_first_token_seconds histogram
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.001",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.005",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.01",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.02",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.04",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.06",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.08",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.1",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.25",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 8.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 341.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.75",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 378.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 383.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="7.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="80.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="160.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="640.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="2560.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:time_to_first_token_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 130.13149738311768
|
||||
# HELP vllm:time_to_first_token_seconds_created Histogram of time to first token in seconds.
|
||||
# TYPE vllm:time_to_first_token_seconds_created gauge
|
||||
vllm:time_to_first_token_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034461198e+09
|
||||
# HELP vllm:inter_token_latency_seconds Histogram of inter-token latency in seconds.
|
||||
# TYPE vllm:inter_token_latency_seconds histogram
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.01",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 26548.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.025",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 92857.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.05",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 95181.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.075",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 95239.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.1",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 95239.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.15",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 95239.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.2",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 95356.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97766.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.4",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97766.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.75",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="7.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="80.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 97920.0
|
||||
vllm:inter_token_latency_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1796.117674704059
|
||||
# HELP vllm:inter_token_latency_seconds_created Histogram of inter-token latency in seconds.
|
||||
# TYPE vllm:inter_token_latency_seconds_created gauge
|
||||
vllm:inter_token_latency_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034461242e+09
|
||||
# HELP vllm:request_time_per_output_token_seconds Histogram of time_per_output_token_seconds per request.
|
||||
# TYPE vllm:request_time_per_output_token_seconds histogram
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.01",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 23.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.025",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 326.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.05",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.075",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.1",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.15",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.2",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.4",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.75",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="7.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="80.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_time_per_output_token_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 7.043598724329641
|
||||
# HELP vllm:request_time_per_output_token_seconds_created Histogram of time_per_output_token_seconds per request.
|
||||
# TYPE vllm:request_time_per_output_token_seconds_created gauge
|
||||
vllm:request_time_per_output_token_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344612813e+09
|
||||
# HELP vllm:e2e_request_latency_seconds Histogram of e2e request latency in seconds.
|
||||
# TYPE vllm:e2e_request_latency_seconds histogram
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 7.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 13.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 198.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:e2e_request_latency_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1926.119155883789
|
||||
# HELP vllm:e2e_request_latency_seconds_created Histogram of e2e request latency in seconds.
|
||||
# TYPE vllm:e2e_request_latency_seconds_created gauge
|
||||
vllm:e2e_request_latency_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034461369e+09
|
||||
# HELP vllm:request_queue_time_seconds Histogram of time spent in WAITING phase for request.
|
||||
# TYPE vllm:request_queue_time_seconds histogram
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_queue_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0028938307659700513
|
||||
# HELP vllm:request_queue_time_seconds_created Histogram of time spent in WAITING phase for request.
|
||||
# TYPE vllm:request_queue_time_seconds_created gauge
|
||||
vllm:request_queue_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344614108e+09
|
||||
# HELP vllm:request_inference_time_seconds Histogram of time spent in RUNNING phase for request.
|
||||
# TYPE vllm:request_inference_time_seconds histogram
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 7.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 13.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 205.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_inference_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1906.919375016354
|
||||
# HELP vllm:request_inference_time_seconds_created Histogram of time spent in RUNNING phase for request.
|
||||
# TYPE vllm:request_inference_time_seconds_created gauge
|
||||
vllm:request_inference_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798110344614592e+09
|
||||
# HELP vllm:request_prefill_time_seconds Histogram of time spent in PREFILL phase for request.
|
||||
# TYPE vllm:request_prefill_time_seconds histogram
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 317.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 376.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 110.801700312295
|
||||
# HELP vllm:request_prefill_time_seconds_created Histogram of time spent in PREFILL phase for request.
|
||||
# TYPE vllm:request_prefill_time_seconds_created gauge
|
||||
vllm:request_prefill_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034461503e+09
|
||||
# HELP vllm:request_decode_time_seconds Histogram of time spent in DECODE phase for request.
|
||||
# TYPE vllm:request_decode_time_seconds histogram
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 10.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 22.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 230.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_decode_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1796.117674704059
|
||||
# HELP vllm:request_decode_time_seconds_created Histogram of time spent in DECODE phase for request.
|
||||
# TYPE vllm:request_decode_time_seconds_created gauge
|
||||
vllm:request_decode_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034461556e+09
|
||||
# HELP vllm:request_prefill_kv_computed_tokens Histogram of new KV tokens computed during prefill (excluding cached tokens).
|
||||
# TYPE vllm:request_prefill_kv_computed_tokens histogram
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_kv_computed_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 384.0
|
||||
vllm:request_prefill_kv_computed_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.532108e+06
|
||||
# HELP vllm:request_prefill_kv_computed_tokens_created Histogram of new KV tokens computed during prefill (excluding cached tokens).
|
||||
# TYPE vllm:request_prefill_kv_computed_tokens_created gauge
|
||||
vllm:request_prefill_kv_computed_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779811034461603e+09
|
||||
# HELP vllm:cache_config_info Information of the LLMEngine CacheConfig
|
||||
# TYPE vllm:cache_config_info gauge
|
||||
vllm:cache_config_info{_block_size_resolved="True",block_size="16",cache_dtype="auto",calculate_kv_scales="False",cpu_kvcache_space_bytes="None",enable_prefix_caching="True",engine="0",gpu_memory_utilization="0.9",is_attention_free="False",kv_cache_memory_bytes="None",kv_offloading_backend="native",kv_offloading_size="None",kv_sharing_fast_prefill="False",mamba_block_size="None",mamba_cache_dtype="auto",mamba_cache_mode="none",mamba_page_size_padded="None",mamba_ssm_cache_dtype="auto",num_cpu_blocks="None",num_gpu_blocks="17590",num_gpu_blocks_override="None",prefix_caching_hash_algo="sha256",sliding_window="None",user_specified_block_size="False"} 1.0
|
||||
# HELP http_requests_total Total number of requests by method, status and handler.
|
||||
# TYPE http_requests_total counter
|
||||
http_requests_total{handler="/v1/models",method="GET",status="2xx"} 1.0
|
||||
http_requests_total{handler="/v1/chat/completions",method="POST",status="2xx"} 384.0
|
||||
# HELP http_requests_created Total number of requests by method, status and handler.
|
||||
# TYPE http_requests_created gauge
|
||||
http_requests_created{handler="/v1/models",method="GET",status="2xx"} 1.7798110363671439e+09
|
||||
http_requests_created{handler="/v1/chat/completions",method="POST",status="2xx"} 1.7798110411735728e+09
|
||||
# HELP http_request_size_bytes Content length of incoming requests by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_request_size_bytes summary
|
||||
http_request_size_bytes_count{handler="/v1/models"} 1.0
|
||||
http_request_size_bytes_sum{handler="/v1/models"} 0.0
|
||||
http_request_size_bytes_count{handler="/v1/chat/completions"} 384.0
|
||||
http_request_size_bytes_sum{handler="/v1/chat/completions"} 2.025216e+06
|
||||
# HELP http_request_size_bytes_created Content length of incoming requests by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_request_size_bytes_created gauge
|
||||
http_request_size_bytes_created{handler="/v1/models"} 1.779811036367173e+09
|
||||
http_request_size_bytes_created{handler="/v1/chat/completions"} 1.7798110411735935e+09
|
||||
# HELP http_response_size_bytes Content length of outgoing responses by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_response_size_bytes summary
|
||||
http_response_size_bytes_count{handler="/v1/models"} 1.0
|
||||
http_response_size_bytes_sum{handler="/v1/models"} 558.0
|
||||
http_response_size_bytes_count{handler="/v1/chat/completions"} 384.0
|
||||
http_response_size_bytes_sum{handler="/v1/chat/completions"} 0.0
|
||||
# HELP http_response_size_bytes_created Content length of outgoing responses by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_response_size_bytes_created gauge
|
||||
http_response_size_bytes_created{handler="/v1/models"} 1.7798110363672004e+09
|
||||
http_response_size_bytes_created{handler="/v1/chat/completions"} 1.7798110411736193e+09
|
||||
# HELP http_request_duration_highr_seconds Latency with many buckets but no API specific labels. Made for more accurate percentile calculations.
|
||||
# TYPE http_request_duration_highr_seconds histogram
|
||||
http_request_duration_highr_seconds_bucket{le="0.01"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.025"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.05"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.075"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.1"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.25"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.5"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.75"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="1.0"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="1.5"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="2.0"} 8.0
|
||||
http_request_duration_highr_seconds_bucket{le="2.5"} 14.0
|
||||
http_request_duration_highr_seconds_bucket{le="3.0"} 33.0
|
||||
http_request_duration_highr_seconds_bucket{le="3.5"} 64.0
|
||||
http_request_duration_highr_seconds_bucket{le="4.0"} 99.0
|
||||
http_request_duration_highr_seconds_bucket{le="4.5"} 138.0
|
||||
http_request_duration_highr_seconds_bucket{le="5.0"} 198.0
|
||||
http_request_duration_highr_seconds_bucket{le="7.5"} 367.0
|
||||
http_request_duration_highr_seconds_bucket{le="10.0"} 385.0
|
||||
http_request_duration_highr_seconds_bucket{le="30.0"} 385.0
|
||||
http_request_duration_highr_seconds_bucket{le="60.0"} 385.0
|
||||
http_request_duration_highr_seconds_bucket{le="+Inf"} 385.0
|
||||
http_request_duration_highr_seconds_count 385.0
|
||||
http_request_duration_highr_seconds_sum 1926.6731046375935
|
||||
# HELP http_request_duration_highr_seconds_created Latency with many buckets but no API specific labels. Made for more accurate percentile calculations.
|
||||
# TYPE http_request_duration_highr_seconds_created gauge
|
||||
http_request_duration_highr_seconds_created 1.7798110349373233e+09
|
||||
# HELP http_request_duration_seconds Latency with only few buckets by handler. Made to be only used if aggregation by handler is important.
|
||||
# TYPE http_request_duration_seconds histogram
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="0.1",method="GET"} 1.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="0.5",method="GET"} 1.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="1.0",method="GET"} 1.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="+Inf",method="GET"} 1.0
|
||||
http_request_duration_seconds_count{handler="/v1/models",method="GET"} 1.0
|
||||
http_request_duration_seconds_sum{handler="/v1/models",method="GET"} 0.002213339030276984
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="0.1",method="POST"} 0.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="0.5",method="POST"} 0.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="1.0",method="POST"} 0.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="+Inf",method="POST"} 384.0
|
||||
http_request_duration_seconds_count{handler="/v1/chat/completions",method="POST"} 384.0
|
||||
http_request_duration_seconds_sum{handler="/v1/chat/completions",method="POST"} 1926.6708912985632
|
||||
# HELP http_request_duration_seconds_created Latency with only few buckets by handler. Made to be only used if aggregation by handler is important.
|
||||
# TYPE http_request_duration_seconds_created gauge
|
||||
http_request_duration_seconds_created{handler="/v1/models",method="GET"} 1.779811036367241e+09
|
||||
http_request_duration_seconds_created{handler="/v1/chat/completions",method="POST"} 1.7798110411736486e+09
|
||||
@@ -0,0 +1,384 @@
|
||||
{"req_id": "a91a3b23a5214312", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380172223962306, "t_first_token_ns": 380173264034343, "t_last_token_ns": 380176159861121, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "4912669dee0341cd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380172747157412, "t_first_token_ns": 380173267977576, "t_last_token_ns": 380176164227657, "prompt_tokens": 4030, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "d5936ef818b54591", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380173411960596, "t_first_token_ns": 380173670902246, "t_last_token_ns": 380176620448112, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "d2f711899e7845a4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380173438274074, "t_first_token_ns": 380173905285089, "t_last_token_ns": 380176631769529, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "70ccdf3d69824d77", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380174903914522, "t_first_token_ns": 380175163826134, "t_last_token_ns": 380177420837142, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "85071cfa8f3a4176", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380176169789244, "t_first_token_ns": 380176431622927, "t_last_token_ns": 380177949033841, "prompt_tokens": 4029, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "4e2f734742004c92", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380178022800553, "t_first_token_ns": 380178274819867, "t_last_token_ns": 380181724866520, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "89a06802caf34c34", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380178171635144, "t_first_token_ns": 380178513970714, "t_last_token_ns": 380181745526039, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "33d0c63b7b2d4028", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380178441712587, "t_first_token_ns": 380178757415619, "t_last_token_ns": 380181762882651, "prompt_tokens": 4030, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "e2a39f4dc0064282", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380178644840667, "t_first_token_ns": 380179000872322, "t_last_token_ns": 380181778105042, "prompt_tokens": 4029, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "3150bb16fcc04e85", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380179805701235, "t_first_token_ns": 380180067889454, "t_last_token_ns": 380184214458009, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "40e13c904f254220", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380179853822023, "t_first_token_ns": 380180303026404, "t_last_token_ns": 380184225734895, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "caec399f87d24bfe", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380181936415866, "t_first_token_ns": 380182190734758, "t_last_token_ns": 380187610273050, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "cd55e602fd8f4f31", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380182255432041, "t_first_token_ns": 380182520793767, "t_last_token_ns": 380187799968396, "prompt_tokens": 4019, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "cf66c4d4e4aa44b2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380182806935220, "t_first_token_ns": 380183066249574, "t_last_token_ns": 380188552246091, "prompt_tokens": 3936, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "cfa8c02b361b47c1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380183002841318, "t_first_token_ns": 380183311890620, "t_last_token_ns": 380188577172460, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "c43f95724ae84a10", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380183277140299, "t_first_token_ns": 380183558168895, "t_last_token_ns": 380188601353082, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "c138de0ebbad4307", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380183562348759, "t_first_token_ns": 380183829978705, "t_last_token_ns": 380188652966039, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "739dc93c4f174805", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380183713051861, "t_first_token_ns": 380184079199958, "t_last_token_ns": 380188710718735, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "055a02ee8aaa4d59", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380184551735653, "t_first_token_ns": 380184827134866, "t_last_token_ns": 380189361220870, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "6c8427349b2641c7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380184635313467, "t_first_token_ns": 380185077513057, "t_last_token_ns": 380189380674592, "prompt_tokens": 4008, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "b0d22336e44c4f99", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380184847664662, "t_first_token_ns": 380185313774699, "t_last_token_ns": 380189388687624, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "2078cb13a8f9495e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380187001268959, "t_first_token_ns": 380187282522021, "t_last_token_ns": 380190965110139, "prompt_tokens": 4025, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "216d6686edff4b6d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380188058502175, "t_first_token_ns": 380188336936891, "t_last_token_ns": 380193065371753, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "a044cd04fc8e48e6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380188639625219, "t_first_token_ns": 380188918388549, "t_last_token_ns": 380194051798088, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "31669aa3067c410a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380189513162575, "t_first_token_ns": 380189777943826, "t_last_token_ns": 380195949793353, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "6c9767b53a2d42a5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380189952458289, "t_first_token_ns": 380190217854989, "t_last_token_ns": 380196347553348, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "3c80cabd577648ae", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380190959703449, "t_first_token_ns": 380191231228571, "t_last_token_ns": 380198194568101, "prompt_tokens": 4048, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "70b238680a5e41e7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380191285254161, "t_first_token_ns": 380191550944234, "t_last_token_ns": 380198334417067, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "a298e1c9fa0c422f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380191553206726, "t_first_token_ns": 380191817129073, "t_last_token_ns": 380198625373108, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "eb04d789fcb44f81", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380191862087280, "t_first_token_ns": 380192354376376, "t_last_token_ns": 380198973751884, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "bd0c9c6f2ce74f83", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380191871532851, "t_first_token_ns": 380192354919066, "t_last_token_ns": 380198974680257, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "a1f99f9438de483b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380192180987373, "t_first_token_ns": 380192602600955, "t_last_token_ns": 380199029187836, "prompt_tokens": 3971, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "220a2878af694831", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380193292842938, "t_first_token_ns": 380193569532321, "t_last_token_ns": 380201453416619, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "4ff2c6ef2ec54eae", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380193529914360, "t_first_token_ns": 380194042692745, "t_last_token_ns": 380201486044468, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "11150ac555124834", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380193532702061, "t_first_token_ns": 380194042923085, "t_last_token_ns": 380201486475037, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "176f737441a8454b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380194246774685, "t_first_token_ns": 380194521334319, "t_last_token_ns": 380201755129224, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "02dbce05fa7e41c7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380194414505509, "t_first_token_ns": 380194775940446, "t_last_token_ns": 380201784594138, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "842225615f164d82", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380195035388829, "t_first_token_ns": 380195309768288, "t_last_token_ns": 380202555205854, "prompt_tokens": 3963, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "ecde76de9a8c436d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380195246743695, "t_first_token_ns": 380195567226802, "t_last_token_ns": 380202585076592, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "39d8b07631b04f30", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380196346131112, "t_first_token_ns": 380196631853834, "t_last_token_ns": 380203449884594, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "df4fb56bb1d044bf", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380196680974524, "t_first_token_ns": 380196966149767, "t_last_token_ns": 380203548815334, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "86caac68547749bd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380197582929363, "t_first_token_ns": 380197860798592, "t_last_token_ns": 380204211790040, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "a8cc3a02c04b45ec", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380198338189591, "t_first_token_ns": 380198625931193, "t_last_token_ns": 380204737488557, "prompt_tokens": 4037, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "41dc9554246a4d6f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380198392776331, "t_first_token_ns": 380198870654298, "t_last_token_ns": 380204751522524, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "eaa0aa9288bb4989", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380198960046669, "t_first_token_ns": 380199248416799, "t_last_token_ns": 380204873598685, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "3f42e18d347b42eb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380199417881493, "t_first_token_ns": 380199694839651, "t_last_token_ns": 380205279462105, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "b7158f7198ce4292", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380199931064431, "t_first_token_ns": 380200217022670, "t_last_token_ns": 380205495064764, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "3d95b7381fe143c9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380199982074734, "t_first_token_ns": 380200459241306, "t_last_token_ns": 380205506562498, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "f706903e1c814ab4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380200664975083, "t_first_token_ns": 380200986854146, "t_last_token_ns": 380205672992448, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "dd41db56e60742ed", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380200915958913, "t_first_token_ns": 380201193428492, "t_last_token_ns": 380205683221355, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 18, "error": null}
|
||||
{"req_id": "c84632ac963a4d71", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380201167109643, "t_first_token_ns": 380201452762278, "t_last_token_ns": 380205698861511, "prompt_tokens": 3963, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "c9b99f4f41294ec4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380201952203341, "t_first_token_ns": 380202230379414, "t_last_token_ns": 380207362983947, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "78a9b1aba2c74ffe", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380202018177826, "t_first_token_ns": 380202485910421, "t_last_token_ns": 380207384742424, "prompt_tokens": 3963, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "b0f8cf67df3644d0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380205004291629, "t_first_token_ns": 380205279695863, "t_last_token_ns": 380212214985961, "prompt_tokens": 3946, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "6450b72d859146d4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380205849417671, "t_first_token_ns": 380206108065600, "t_last_token_ns": 380213523083501, "prompt_tokens": 4008, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "2b772bd33a274082", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380205909859201, "t_first_token_ns": 380206564639629, "t_last_token_ns": 380213559544496, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "d9f5bc075bd24419", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380206008169725, "t_first_token_ns": 380206564981494, "t_last_token_ns": 380213561097333, "prompt_tokens": 3958, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "0ec59da06fdd438a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380206105654726, "t_first_token_ns": 380206802956669, "t_last_token_ns": 380213575378534, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "76e80a28624448de", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380206461553596, "t_first_token_ns": 380207039479792, "t_last_token_ns": 380213590183138, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "4dd78135df8d431b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380206787503432, "t_first_token_ns": 380207278071262, "t_last_token_ns": 380213604337149, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "4664ff35bef04c7e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380207909459036, "t_first_token_ns": 380208183336390, "t_last_token_ns": 380215335770449, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "8868a192ffe24b2e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380208067595798, "t_first_token_ns": 380208431523829, "t_last_token_ns": 380215364100675, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "fe3ca70a4a5d403f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380208594663720, "t_first_token_ns": 380208867365956, "t_last_token_ns": 380215630151250, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "ebd49e96c0434ca9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380208925985796, "t_first_token_ns": 380209204629611, "t_last_token_ns": 380215743585062, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "3221d1317b4a4971", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380209044316610, "t_first_token_ns": 380209458725005, "t_last_token_ns": 380215772581981, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "03723a297de6446b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380210653441896, "t_first_token_ns": 380210939337133, "t_last_token_ns": 380217449341478, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "b38306f5aac04aab", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380210677065514, "t_first_token_ns": 380211180876039, "t_last_token_ns": 380217463171717, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "fdbb63cc50b34cdd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380210937149031, "t_first_token_ns": 380211437382944, "t_last_token_ns": 380217489185928, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "2ea8fe348ac24810", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380211247336852, "t_first_token_ns": 380211694023129, "t_last_token_ns": 380217516165787, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "138d30ec46164465", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380211533079208, "t_first_token_ns": 380211953051514, "t_last_token_ns": 380217539266902, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "07a9b3a3c1ec4d1c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380211772239539, "t_first_token_ns": 380212214416881, "t_last_token_ns": 380217560889953, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 18, "error": null}
|
||||
{"req_id": "811ce052a8a447e6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380212453265736, "t_first_token_ns": 380212746193929, "t_last_token_ns": 380217757900655, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 18, "error": null}
|
||||
{"req_id": "d679bc7ef7424267", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380214141320010, "t_first_token_ns": 380214423524009, "t_last_token_ns": 380219000272135, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "ea526a2a93fe4d8e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380214370059657, "t_first_token_ns": 380214891433746, "t_last_token_ns": 380219018648799, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "83d6a66085474059", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380214367640131, "t_first_token_ns": 380214892530391, "t_last_token_ns": 380219019178065, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "edc38948698a4427", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380214755143219, "t_first_token_ns": 380215145044971, "t_last_token_ns": 380219067742442, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "44c9f67fe8d44350", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380216417158598, "t_first_token_ns": 380216697065660, "t_last_token_ns": 380221357231345, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "7930229ff23f463e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380216956662142, "t_first_token_ns": 380217235971338, "t_last_token_ns": 380221598173369, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "521a066393fd4f64", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380218063807204, "t_first_token_ns": 380218329975183, "t_last_token_ns": 380222707245537, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "f9c73c65ba094795", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380219006913662, "t_first_token_ns": 380219276021895, "t_last_token_ns": 380223646842893, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "82de17b1e0594a0c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380219159545529, "t_first_token_ns": 380219518941608, "t_last_token_ns": 380223668775469, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "c6e34e86ea6d4bc8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380219715059145, "t_first_token_ns": 380219984090372, "t_last_token_ns": 380223900022637, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "bb9b88e71a824ffa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380220270079178, "t_first_token_ns": 380220539586415, "t_last_token_ns": 380224237272276, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "3ba291a222b94903", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380220506822148, "t_first_token_ns": 380220790464864, "t_last_token_ns": 380224253151439, "prompt_tokens": 4015, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "235cc798d5744b93", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380220758546758, "t_first_token_ns": 380221039766620, "t_last_token_ns": 380224267970777, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "bb1412bc5f9f409f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380222131246473, "t_first_token_ns": 380222402827863, "t_last_token_ns": 380225178721627, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "3ba98f19a4614cd5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380223217341714, "t_first_token_ns": 380223483764575, "t_last_token_ns": 380226142961646, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "a7b5558785904643", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380224451819210, "t_first_token_ns": 380224709350501, "t_last_token_ns": 380227537623343, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "b7361bbb399b492c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380225366880463, "t_first_token_ns": 380225621278668, "t_last_token_ns": 380228578389164, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "9336e9a9899f4640", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380225834090001, "t_first_token_ns": 380226098290251, "t_last_token_ns": 380228847324275, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "111c9031375e4a7f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380226410519687, "t_first_token_ns": 380226671408812, "t_last_token_ns": 380229956778425, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "df97f3a6d9f445bc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380227031440690, "t_first_token_ns": 380227299353376, "t_last_token_ns": 380230450283244, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "6e33faf38f474c5c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380228057776034, "t_first_token_ns": 380228321699546, "t_last_token_ns": 380231222346132, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "064e5cd9e74d43b4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380228868651608, "t_first_token_ns": 380229128382817, "t_last_token_ns": 380232944347045, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "89f707860ab34b9f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380229096360851, "t_first_token_ns": 380229372056864, "t_last_token_ns": 380233195097107, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "a081a2eb34364061", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380229653079276, "t_first_token_ns": 380229920850304, "t_last_token_ns": 380234019660891, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "eb70b05923f942a9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380231326849144, "t_first_token_ns": 380231587882860, "t_last_token_ns": 380238905453347, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "18892c50d4644638", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380231474897272, "t_first_token_ns": 380231826378697, "t_last_token_ns": 380238938596258, "prompt_tokens": 3946, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "65b4a62eb7dc4520", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380232108659487, "t_first_token_ns": 380232373815052, "t_last_token_ns": 380239570635328, "prompt_tokens": 3954, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "53c5400c2ab54633", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380232204145707, "t_first_token_ns": 380232829771208, "t_last_token_ns": 380239599434770, "prompt_tokens": 3960, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "dc94ead145884ec8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380232240400009, "t_first_token_ns": 380232830147033, "t_last_token_ns": 380239600310493, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "8078de01dc534538", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380232926076385, "t_first_token_ns": 380233194994263, "t_last_token_ns": 380239798835895, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "d6b0bb999cc94b78", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380233122954932, "t_first_token_ns": 380233443163434, "t_last_token_ns": 380239828850991, "prompt_tokens": 4008, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "ae1b868a8aaa4a01", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380233628080594, "t_first_token_ns": 380233895253551, "t_last_token_ns": 380240133786068, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "e47d530e494c415e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380234349079602, "t_first_token_ns": 380234615890063, "t_last_token_ns": 380241400511144, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "3ed371b0b4fe4965", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380234357759497, "t_first_token_ns": 380234853877479, "t_last_token_ns": 380241414431165, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "059f633c7970490f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380234940146975, "t_first_token_ns": 380235216941692, "t_last_token_ns": 380241558726243, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "c5c78ab818f74da1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380235459385204, "t_first_token_ns": 380235737799374, "t_last_token_ns": 380242325585489, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "f2a517d7a73242b9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380236307045456, "t_first_token_ns": 380236579793637, "t_last_token_ns": 380243404222815, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "7c7c0ec253734473", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380236491072773, "t_first_token_ns": 380236832337168, "t_last_token_ns": 380243431437536, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "ec65c3fc1e9e41c1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380236577867200, "t_first_token_ns": 380237071786709, "t_last_token_ns": 380243445046058, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "741163930fd34e0f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380237147807172, "t_first_token_ns": 380237430311495, "t_last_token_ns": 380243568091500, "prompt_tokens": 3956, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "c8835e0d9a2246ca", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380237375240835, "t_first_token_ns": 380237895814190, "t_last_token_ns": 380243594217006, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 18, "error": null}
|
||||
{"req_id": "78a93090cf634a10", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380237291628936, "t_first_token_ns": 380237896832354, "t_last_token_ns": 380243594978428, "prompt_tokens": 3943, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "cdbc603e1ddf46c3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380237625349832, "t_first_token_ns": 380238151778769, "t_last_token_ns": 380243614761688, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "ba2d1f760b134b65", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380240630094554, "t_first_token_ns": 380240909566645, "t_last_token_ns": 380247225095955, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "92af515606fa4a00", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380240685048261, "t_first_token_ns": 380241372218633, "t_last_token_ns": 380247252735647, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "87d9a7d2ffeb4fa6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380240887932476, "t_first_token_ns": 380241372695747, "t_last_token_ns": 380247253359111, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "846f0444dc8a4537", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380241602686293, "t_first_token_ns": 380241873141790, "t_last_token_ns": 380247516938595, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "441b2285d1774f11", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380241706152612, "t_first_token_ns": 380242124824760, "t_last_token_ns": 380247537967758, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "56e8efed38804a50", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380242602333556, "t_first_token_ns": 380242880341840, "t_last_token_ns": 380247942230815, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "cbb2184b9f684743", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380242868784916, "t_first_token_ns": 380243149099838, "t_last_token_ns": 380247977345211, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "315957c96b224b9e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380243626151780, "t_first_token_ns": 380243898777156, "t_last_token_ns": 380249013811967, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "330030cc76b34191", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380243859637229, "t_first_token_ns": 380244147722928, "t_last_token_ns": 380249263380559, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "1fc7cc35700c4633", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380244015626572, "t_first_token_ns": 380244395067427, "t_last_token_ns": 380249282376824, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "f4a833e420c74cd5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380244496077704, "t_first_token_ns": 380244767635612, "t_last_token_ns": 380249858712362, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "587eca1ff648419d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380245147370587, "t_first_token_ns": 380245417335044, "t_last_token_ns": 380250179877714, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "d5a77d109bf84620", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380246636211621, "t_first_token_ns": 380246916960572, "t_last_token_ns": 380251638598543, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "189560bec6ce428e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380248242047837, "t_first_token_ns": 380248508998317, "t_last_token_ns": 380252957746914, "prompt_tokens": 4010, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "31d60eb1dbe14cb0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380248409026107, "t_first_token_ns": 380248971740203, "t_last_token_ns": 380252980560243, "prompt_tokens": 4015, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "bb1fcfe38b604f29", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380248359051546, "t_first_token_ns": 380248971853698, "t_last_token_ns": 380252980824853, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "c2793c2a50244305", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380248993232543, "t_first_token_ns": 380249263569954, "t_last_token_ns": 380253038785785, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "8d3e70e9fe854462", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380249339528947, "t_first_token_ns": 380249610677187, "t_last_token_ns": 380253129712285, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "bf5ec12cb1e342f2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380249488685752, "t_first_token_ns": 380249858952996, "t_last_token_ns": 380253142861544, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "e11326160b404708", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380250328363830, "t_first_token_ns": 380250604018794, "t_last_token_ns": 380253417298580, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "87e66c509dc84432", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380250726205363, "t_first_token_ns": 380251001028418, "t_last_token_ns": 380253487949652, "prompt_tokens": 4045, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "60c66d13ba5e4214", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380254160268928, "t_first_token_ns": 380254412563109, "t_last_token_ns": 380257743238556, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "366461f2b6544cbe", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380255319075248, "t_first_token_ns": 380255575150944, "t_last_token_ns": 380261187935472, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "22802de9a3dd4bcb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380255571899239, "t_first_token_ns": 380255831935650, "t_last_token_ns": 380261254531756, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "88da7dfabd3d431d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380255685165955, "t_first_token_ns": 380256079183338, "t_last_token_ns": 380261282936273, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "4ce10a6701574b67", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380256019870919, "t_first_token_ns": 380256319536169, "t_last_token_ns": 380261309707860, "prompt_tokens": 4022, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "19699bd4c1604943", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380256293694419, "t_first_token_ns": 380256564539400, "t_last_token_ns": 380261333047483, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "50af0f5c5dae422f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380256655072499, "t_first_token_ns": 380256914756986, "t_last_token_ns": 380261487023660, "prompt_tokens": 3954, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "e13eb7102ac741cf", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380256930577743, "t_first_token_ns": 380257195545465, "t_last_token_ns": 380261544647144, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "dd991affed5c4b39", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380257419368899, "t_first_token_ns": 380257686692208, "t_last_token_ns": 380261774925130, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "d22bd07b60324e61", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380259003975962, "t_first_token_ns": 380259276214921, "t_last_token_ns": 380263153786551, "prompt_tokens": 3958, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "1f2f029951f3413a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380259669764585, "t_first_token_ns": 380259940154538, "t_last_token_ns": 380263512720269, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "c6b3460e06254826", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380260287469807, "t_first_token_ns": 380260556645415, "t_last_token_ns": 380263742701966, "prompt_tokens": 3963, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "417abeb4cbcd4193", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380260615928466, "t_first_token_ns": 380260893976047, "t_last_token_ns": 380263801385124, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "9b22fb6a6ced481a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380260822339361, "t_first_token_ns": 380261147355946, "t_last_token_ns": 380263813017859, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "bcf746418949459c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380262045426204, "t_first_token_ns": 380262305464263, "t_last_token_ns": 380266415588149, "prompt_tokens": 3971, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "597475c888bd4710", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380263805928463, "t_first_token_ns": 380264063307833, "t_last_token_ns": 380269287888496, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "ba57dc5f1b514156", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380264059099016, "t_first_token_ns": 380264320202077, "t_last_token_ns": 380269356380736, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "0d5a632ed2fb467e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380264338849549, "t_first_token_ns": 380264592296899, "t_last_token_ns": 380269436952824, "prompt_tokens": 3949, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "440f90d387324405", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380264623676061, "t_first_token_ns": 380264879883750, "t_last_token_ns": 380269527843669, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "da21de6fd72f4ca7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380264973402730, "t_first_token_ns": 380265238843573, "t_last_token_ns": 380269691877944, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "7e3246d999a04b65", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380265583834643, "t_first_token_ns": 380265845386460, "t_last_token_ns": 380270057067303, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "fe1c1ccb19494c36", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380265638948149, "t_first_token_ns": 380266094447911, "t_last_token_ns": 380270076310293, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "86c38816a9724d64", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380266128971326, "t_first_token_ns": 380266394619659, "t_last_token_ns": 380270134191826, "prompt_tokens": 3951, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "cc000ef625494ea0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380267058289529, "t_first_token_ns": 380267331236129, "t_last_token_ns": 380270826204753, "prompt_tokens": 4023, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "b575bfa538ff46f5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380267236531246, "t_first_token_ns": 380267796458222, "t_last_token_ns": 380270840453019, "prompt_tokens": 4017, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "238f303b7cb944e3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380267159998109, "t_first_token_ns": 380267795757121, "t_last_token_ns": 380270840591994, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "b8a3471f287a406d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380270409342066, "t_first_token_ns": 380270668130628, "t_last_token_ns": 380273178230314, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "e770109a78ff4383", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380270885720828, "t_first_token_ns": 380271141574414, "t_last_token_ns": 380273460886735, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "e34e90f515d540c9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380271316676982, "t_first_token_ns": 380271574369876, "t_last_token_ns": 380274419845459, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "3242c26d92064a03", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380272588682982, "t_first_token_ns": 380272850227109, "t_last_token_ns": 380277131162878, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "97ef2a6f5eb94974", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380273478363228, "t_first_token_ns": 380273740325238, "t_last_token_ns": 380279524374127, "prompt_tokens": 4017, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "c5239ec78d86418f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380273559839028, "t_first_token_ns": 380274192531393, "t_last_token_ns": 380279554642739, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "9a8988b09d644c80", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380273589345997, "t_first_token_ns": 380274192924880, "t_last_token_ns": 380279555197363, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "ed6c1615c8144d65", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380274765448697, "t_first_token_ns": 380275029342555, "t_last_token_ns": 380280640020159, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "8dd056b368b642e5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380274832638118, "t_first_token_ns": 380275271522940, "t_last_token_ns": 380280668473555, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "ff14ea8ef4c64951", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380275429811447, "t_first_token_ns": 380275696993690, "t_last_token_ns": 380281407293618, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "b95d1287695645e9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380275966724651, "t_first_token_ns": 380276240575455, "t_last_token_ns": 380281786967718, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "f8e15ecefaec41db", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380276572906837, "t_first_token_ns": 380276844422534, "t_last_token_ns": 380282246408882, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "29bdb807221c4a6d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380276637827469, "t_first_token_ns": 380277092269944, "t_last_token_ns": 380282268914994, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "467f52722d874271", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380277831389851, "t_first_token_ns": 380278103573542, "t_last_token_ns": 380283021720871, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "77a2201e906748e6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380277878657777, "t_first_token_ns": 380278342113080, "t_last_token_ns": 380283032087956, "prompt_tokens": 3971, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "d1ef071be0b64ca1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380278024145236, "t_first_token_ns": 380278578794467, "t_last_token_ns": 380283040288545, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "d50be0eefdd24b78", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380278257152650, "t_first_token_ns": 380279029793178, "t_last_token_ns": 380283049756612, "prompt_tokens": 3952, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "f7a1e0c704244115", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380278176667107, "t_first_token_ns": 380279030389843, "t_last_token_ns": 380283049921973, "prompt_tokens": 4008, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "be1e72d99e9645f3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380279001961599, "t_first_token_ns": 380279280223001, "t_last_token_ns": 380283063867531, "prompt_tokens": 3959, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "ea5e23c4b0a64733", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380280688073292, "t_first_token_ns": 380280966477775, "t_last_token_ns": 380284426663154, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "7411d28ce81f4bdc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380280980895080, "t_first_token_ns": 380281258930917, "t_last_token_ns": 380284464717731, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "d7c882e14cd54780", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380283455237635, "t_first_token_ns": 380283713110651, "t_last_token_ns": 380286293411952, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "b7dda7c35ba64016", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380283512045191, "t_first_token_ns": 380283954489406, "t_last_token_ns": 380286308045920, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "7b92cf5fd0e941bd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380283990355773, "t_first_token_ns": 380284248247746, "t_last_token_ns": 380286360862145, "prompt_tokens": 3960, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "fbbf2a473a6f441b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380285797521295, "t_first_token_ns": 380286059983973, "t_last_token_ns": 380288189672380, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "28b97c10515048fb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380286544121140, "t_first_token_ns": 380286801342118, "t_last_token_ns": 380288688205471, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "ce367beabf0c4d49", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380287128036237, "t_first_token_ns": 380287381129894, "t_last_token_ns": 380288969322284, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "7f24fbf23553445d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380289859886037, "t_first_token_ns": 380290115271159, "t_last_token_ns": 380293660641912, "prompt_tokens": 4010, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "56ecb21dc44d425f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380290282764533, "t_first_token_ns": 380290538231417, "t_last_token_ns": 380294135246742, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "ec4e1436f3d64f4e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380290739990016, "t_first_token_ns": 380290996991735, "t_last_token_ns": 380295236424610, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "34df7f5ffbd84354", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380291480913367, "t_first_token_ns": 380291739459528, "t_last_token_ns": 380296295405717, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "8b5bafb4d68d4011", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380291704424367, "t_first_token_ns": 380291981773757, "t_last_token_ns": 380296319593797, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "d1b3106779b34d2f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380292712905656, "t_first_token_ns": 380292977254009, "t_last_token_ns": 380299262482402, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "72c0d6f3192e4fd2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380292915377569, "t_first_token_ns": 380293222240356, "t_last_token_ns": 380299292031117, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "7619ce5a10cc4a5c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380293054854072, "t_first_token_ns": 380293470873338, "t_last_token_ns": 380299321031811, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "13a7efb2515247b5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380294167600379, "t_first_token_ns": 380294424740285, "t_last_token_ns": 380301163811193, "prompt_tokens": 3942, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "1f0632e690da42cd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380294400149369, "t_first_token_ns": 380294673188093, "t_last_token_ns": 380301189739430, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "e412a7b403eb4d5c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380294508050787, "t_first_token_ns": 380294918951088, "t_last_token_ns": 380301218299456, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "f5852fec313247d5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380295268366021, "t_first_token_ns": 380295537352457, "t_last_token_ns": 380301952378654, "prompt_tokens": 3951, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "d87324bfb66b4f4c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380296313333147, "t_first_token_ns": 380296581322223, "t_last_token_ns": 380304638441974, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "0e94e18b7c2b40d1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380296317727039, "t_first_token_ns": 380296819300102, "t_last_token_ns": 380304654447180, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "7150fde11bcc46ac", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380296634766518, "t_first_token_ns": 380297068693255, "t_last_token_ns": 380304690267858, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "22ebd3d994084153", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380296920524577, "t_first_token_ns": 380297321153607, "t_last_token_ns": 380304953243150, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "c0294539ec6d4e3a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380297276793812, "t_first_token_ns": 380297574878845, "t_last_token_ns": 380305212460684, "prompt_tokens": 4023, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "e880b1f4c44e432d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380297659936895, "t_first_token_ns": 380297936909071, "t_last_token_ns": 380305386472915, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "da0b5958e60c4266", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380297662282630, "t_first_token_ns": 380298175566686, "t_last_token_ns": 380305400453329, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "8b80816f775b4972", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380297977394713, "t_first_token_ns": 380298430869933, "t_last_token_ns": 380305428416009, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "f759975cb3714eb1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380299387750770, "t_first_token_ns": 380299662308239, "t_last_token_ns": 380306895241806, "prompt_tokens": 4026, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "f068d9b8a89b406e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380300383715437, "t_first_token_ns": 380300665313231, "t_last_token_ns": 380307926791109, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "45983dbfe045446b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380300627319716, "t_first_token_ns": 380300919936534, "t_last_token_ns": 380307956233616, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "66112581276748bc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380300681416854, "t_first_token_ns": 380301163390027, "t_last_token_ns": 380307970975251, "prompt_tokens": 4020, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "b3b4e8e70ea84128", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380301257020720, "t_first_token_ns": 380301543414411, "t_last_token_ns": 380308124785914, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "008a9aed4a9b4ecf", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380301939522818, "t_first_token_ns": 380302220465126, "t_last_token_ns": 380308786652857, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "2b4a4c7c9dd24124", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380302822945298, "t_first_token_ns": 380303097942398, "t_last_token_ns": 380309394192553, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "37304a8b4a2348e7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380303105792740, "t_first_token_ns": 380303383480264, "t_last_token_ns": 380309451163094, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "cfb1d3327d01461e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380303273822779, "t_first_token_ns": 380303640059263, "t_last_token_ns": 380309478694187, "prompt_tokens": 4029, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "b9643d44654a4c7c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380303446961408, "t_first_token_ns": 380303899940362, "t_last_token_ns": 380309502019102, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 18, "error": null}
|
||||
{"req_id": "f3d7db8ccfa94fa9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380303854937867, "t_first_token_ns": 380304373552336, "t_last_token_ns": 380309524168408, "prompt_tokens": 4019, "completion_tokens": 256, "inflight_at_send": 20, "error": null}
|
||||
{"req_id": "2c3230964bf44d5d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380303813919377, "t_first_token_ns": 380304373076601, "t_last_token_ns": 380309524408410, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "eeec19db344b48cc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380304656126701, "t_first_token_ns": 380304953862568, "t_last_token_ns": 380309698302922, "prompt_tokens": 4010, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "4bc2338680454abe", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380304787935182, "t_first_token_ns": 380305212237920, "t_last_token_ns": 380309718446723, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "62f0137bd18142d8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380305802318463, "t_first_token_ns": 380306078316779, "t_last_token_ns": 380310287949323, "prompt_tokens": 4034, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "f69a8b421d8e4ba9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380306364445661, "t_first_token_ns": 380306641472987, "t_last_token_ns": 380310464535084, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "316d5f1d76ef4fb2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380307487891677, "t_first_token_ns": 380307770717899, "t_last_token_ns": 380310912670814, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "8c72b2e22f49478e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380308127350730, "t_first_token_ns": 380308406440111, "t_last_token_ns": 380311083513353, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "6d1425de3bd24f2f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380309976079655, "t_first_token_ns": 380310240666756, "t_last_token_ns": 380312501457276, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "3bd82657255147e5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380311103174703, "t_first_token_ns": 380311354628968, "t_last_token_ns": 380315112408857, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "cb50d465e5974e12", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380311150962796, "t_first_token_ns": 380311594808233, "t_last_token_ns": 380315134248344, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "5b663edb823a45ab", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380312781842316, "t_first_token_ns": 380313040137146, "t_last_token_ns": 380319574935004, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "d669a3f276c44282", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380313076821340, "t_first_token_ns": 380313334361459, "t_last_token_ns": 380319710077774, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "bc61967b14bf4d56", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380313498552872, "t_first_token_ns": 380313762110928, "t_last_token_ns": 380320103997262, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "526ed27937eb4aac", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380313585047927, "t_first_token_ns": 380314002186836, "t_last_token_ns": 380320133166951, "prompt_tokens": 3947, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "db34ca33f3e147eb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380314117945845, "t_first_token_ns": 380314384632859, "t_last_token_ns": 380320355553262, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "3231aedaf733446e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380314437288989, "t_first_token_ns": 380314704713734, "t_last_token_ns": 380320463332135, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "9e31372385a14c71", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380314519267904, "t_first_token_ns": 380314952575923, "t_last_token_ns": 380320490043445, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "bc65bfb893a94829", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380315335483628, "t_first_token_ns": 380315603393342, "t_last_token_ns": 380320909157238, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "6f9a8d07224249bb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380315459664362, "t_first_token_ns": 380316064791089, "t_last_token_ns": 380320930210193, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "6938975836874a78", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380315420200959, "t_first_token_ns": 380316065324482, "t_last_token_ns": 380320930469338, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "3020681d97294607", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380315812638886, "t_first_token_ns": 380316312148829, "t_last_token_ns": 380320948152303, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "8081ba5bb55d4141", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380316469123766, "t_first_token_ns": 380316752987551, "t_last_token_ns": 380321101826651, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "a8dc3cfaf5084266", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380316836077913, "t_first_token_ns": 380317112593445, "t_last_token_ns": 380321184447733, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "868ecf463c144507", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380318186761265, "t_first_token_ns": 380318469302980, "t_last_token_ns": 380322307954195, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "1340fc154c924f42", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380318699890433, "t_first_token_ns": 380318984045571, "t_last_token_ns": 380322458228518, "prompt_tokens": 4048, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "52626ce7a5454e4a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380318709700185, "t_first_token_ns": 380319224116046, "t_last_token_ns": 380322465678604, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "e415754d0d934327", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380321278985241, "t_first_token_ns": 380321542366716, "t_last_token_ns": 380324741490146, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "3404e5b694e34301", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380321749754073, "t_first_token_ns": 380322011647804, "t_last_token_ns": 380325257814470, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "beaba398d1d242da", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380322806408410, "t_first_token_ns": 380323062505914, "t_last_token_ns": 380327946450736, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "e2491b25dd0b4fe6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380323084490250, "t_first_token_ns": 380323344375944, "t_last_token_ns": 380328042245705, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "29c14d45a2224a7e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380323328279702, "t_first_token_ns": 380323801424438, "t_last_token_ns": 380328066120193, "prompt_tokens": 4023, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "b74c349330504260", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380323258856479, "t_first_token_ns": 380323801634582, "t_last_token_ns": 380328066319061, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "01f8a78b9b11483a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380324731549559, "t_first_token_ns": 380324999263204, "t_last_token_ns": 380330659673774, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "bd99afed1a304615", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380325485696217, "t_first_token_ns": 380325747895124, "t_last_token_ns": 380331678426434, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "7ae6ce22597b4c5b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380325655533422, "t_first_token_ns": 380325993253919, "t_last_token_ns": 380331708471973, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "f4fce0fdee89401e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380325749327140, "t_first_token_ns": 380326230107289, "t_last_token_ns": 380331723287571, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "2d2763847ba94ce2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380325930920003, "t_first_token_ns": 380326468634915, "t_last_token_ns": 380331737426239, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "2b28439a7c3446e1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380326884180680, "t_first_token_ns": 380327160368979, "t_last_token_ns": 380332455413162, "prompt_tokens": 4010, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "09608580b0d24670", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380327456260997, "t_first_token_ns": 380327727533219, "t_last_token_ns": 380332738406426, "prompt_tokens": 4017, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "5f3660cbd99c4b5b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380328460429988, "t_first_token_ns": 380328730707554, "t_last_token_ns": 380333716502474, "prompt_tokens": 4008, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "58164246f65247f9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380329052245929, "t_first_token_ns": 380329319841064, "t_last_token_ns": 380334101018804, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "2ad5211187984515", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380329191378365, "t_first_token_ns": 380329572634897, "t_last_token_ns": 380334125106745, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "8d8baddbe02f4e89", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380329628329462, "t_first_token_ns": 380329907786294, "t_last_token_ns": 380334205793839, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "588b0c2f0a6c4c52", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380329957617619, "t_first_token_ns": 380330258052137, "t_last_token_ns": 380334292949603, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "a948d520588a4094", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380330254779106, "t_first_token_ns": 380330523242730, "t_last_token_ns": 380334316208875, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "4b3be2ebb2454e8f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380330652290673, "t_first_token_ns": 380330929948765, "t_last_token_ns": 380334411445224, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "038aee7d32c245bd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380332180825586, "t_first_token_ns": 380332456308430, "t_last_token_ns": 380335272630250, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "a3e064ee509f4206", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380333230957628, "t_first_token_ns": 380333503143912, "t_last_token_ns": 380335970465746, "prompt_tokens": 4017, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "ca8f672efc2a46ea", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380334689869290, "t_first_token_ns": 380334948668984, "t_last_token_ns": 380339048522551, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "d7cbb48c8d7646d2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380335476291989, "t_first_token_ns": 380335731285791, "t_last_token_ns": 380340806913751, "prompt_tokens": 3956, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "c48f36097a2c455d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380335989251903, "t_first_token_ns": 380336249924131, "t_last_token_ns": 380341379253755, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "62383a6809e84ab5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380336115654779, "t_first_token_ns": 380336492161346, "t_last_token_ns": 380341401784503, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "7e34518667bc4621", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380336260088484, "t_first_token_ns": 380336723383652, "t_last_token_ns": 380341412425915, "prompt_tokens": 3945, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "6b235c91b4ae4795", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380336341461634, "t_first_token_ns": 380336960520760, "t_last_token_ns": 380341424094909, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "0ffb79f2dbe74715", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380336692703589, "t_first_token_ns": 380337196530499, "t_last_token_ns": 380341433416760, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "574e0c6093694eec", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380336752884851, "t_first_token_ns": 380337432420731, "t_last_token_ns": 380341443064069, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "153011fee0a64793", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380337588054576, "t_first_token_ns": 380337856682299, "t_last_token_ns": 380341831986150, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "b39f5a5224ef4c25", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380339156258259, "t_first_token_ns": 380339430107329, "t_last_token_ns": 380344387110278, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "d838b670d1894c67", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380339712397423, "t_first_token_ns": 380339990114199, "t_last_token_ns": 380344973758217, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "e729a20730724ec5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380340304676792, "t_first_token_ns": 380340582101193, "t_last_token_ns": 380345996262635, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "7ff1158851fd4746", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380341501082435, "t_first_token_ns": 380341765461743, "t_last_token_ns": 380347054380477, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "427a805bbd894adf", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380342092903581, "t_first_token_ns": 380342353717811, "t_last_token_ns": 380348340238243, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "0df73a5c28be4d37", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380342670380366, "t_first_token_ns": 380342935160705, "t_last_token_ns": 380348876675223, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "c58e5501154a4c9c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380342785927622, "t_first_token_ns": 380343390689054, "t_last_token_ns": 380348905863355, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "304805f07d834123", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380342917419254, "t_first_token_ns": 380343390856983, "t_last_token_ns": 380348906278481, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "c1f058cb42764084", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380343384993103, "t_first_token_ns": 380343658121399, "t_last_token_ns": 380348954245188, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "f2e868b3e31d48dc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380343526587001, "t_first_token_ns": 380343906237321, "t_last_token_ns": 380348976929702, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "6e6a4a4d70704303", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380344528083831, "t_first_token_ns": 380344792040412, "t_last_token_ns": 380349579264108, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "58660ff8f2604a02", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380345043169805, "t_first_token_ns": 380345309317107, "t_last_token_ns": 380349785398203, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "2571ce26a46b4d09", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380345214061654, "t_first_token_ns": 380345559271807, "t_last_token_ns": 380349803976982, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "ca89351cd356405e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380345410817792, "t_first_token_ns": 380345811672330, "t_last_token_ns": 380349821046366, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "3367ad593f4b4e1a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380346990268030, "t_first_token_ns": 380347272813400, "t_last_token_ns": 380350725737650, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "4173873953c24e0d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380347489856091, "t_first_token_ns": 380347984109965, "t_last_token_ns": 380350860701792, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "8709fa0ce74d4ea9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380347499218503, "t_first_token_ns": 380347984582436, "t_last_token_ns": 380350861023092, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "1d0356419f3146a8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380350166616428, "t_first_token_ns": 380350429090968, "t_last_token_ns": 380352353651686, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "20dd29a27ce846a1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380351525151528, "t_first_token_ns": 380351776614117, "t_last_token_ns": 380355375696558, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "223fcabddf5b4343", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380351915807590, "t_first_token_ns": 380352173904954, "t_last_token_ns": 380355709692090, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "c60aa7b9cee947af", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380352793473509, "t_first_token_ns": 380353051717530, "t_last_token_ns": 380357182697939, "prompt_tokens": 3950, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "afeb1e15b1214e6c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380353065693105, "t_first_token_ns": 380353326072649, "t_last_token_ns": 380357262207255, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "153c237a20ba4580", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380353517281008, "t_first_token_ns": 380353775439813, "t_last_token_ns": 380357532797081, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "ef979367a85049f7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380353802291891, "t_first_token_ns": 380354068323997, "t_last_token_ns": 380357601989522, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "66871966c59d4f92", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380354230982639, "t_first_token_ns": 380354501219046, "t_last_token_ns": 380357773976290, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "b7c00157de2142ae", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380354739058312, "t_first_token_ns": 380355014697832, "t_last_token_ns": 380357950855598, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "5fa7483d6f0748b0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380356413816337, "t_first_token_ns": 380356679182348, "t_last_token_ns": 380359779078142, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "79bae44205ed4195", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380356521453949, "t_first_token_ns": 380356927753710, "t_last_token_ns": 380359794358173, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "690c3b0275f541f1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380358180291048, "t_first_token_ns": 380358439161701, "t_last_token_ns": 380363321580712, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "5b8f84d673aa4683", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380358444271536, "t_first_token_ns": 380358702177464, "t_last_token_ns": 380363389155559, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "2ddda6db85f84c05", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380358753663059, "t_first_token_ns": 380359010223818, "t_last_token_ns": 380363515336413, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "c6ea317d4b474b14", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380359811869338, "t_first_token_ns": 380360073265938, "t_last_token_ns": 380364563804284, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "e216ec3ef3294494", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380360454697847, "t_first_token_ns": 380360708591018, "t_last_token_ns": 380365407870834, "prompt_tokens": 3934, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "69003f31add64c08", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380360489664046, "t_first_token_ns": 380360945255255, "t_last_token_ns": 380365419617321, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "92228b44508f49d3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380360667728782, "t_first_token_ns": 380361393823803, "t_last_token_ns": 380365428408540, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "faa4e7d503cb4493", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380360528696461, "t_first_token_ns": 380361394065389, "t_last_token_ns": 380365428800638, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "cfdaa910137642f9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380360867261687, "t_first_token_ns": 380361629258064, "t_last_token_ns": 380365435559042, "prompt_tokens": 3971, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "57d788bf62da4320", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380361548416829, "t_first_token_ns": 380361876436694, "t_last_token_ns": 380365449410658, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "0fd87b7c256a4939", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380361806628372, "t_first_token_ns": 380362125519400, "t_last_token_ns": 380365461094025, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "ff172d7e4d4e40c9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380365046744710, "t_first_token_ns": 380365321545281, "t_last_token_ns": 380366917546545, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "b397de9d22d84d08", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380366269306074, "t_first_token_ns": 380366529367995, "t_last_token_ns": 380370141812554, "prompt_tokens": 4042, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "c28ea223237e49e1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380367178528379, "t_first_token_ns": 380367435331640, "t_last_token_ns": 380373182399062, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "0ea33848e9514876", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380367354711914, "t_first_token_ns": 380367675176163, "t_last_token_ns": 380373211459247, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "d9e100dcd4784d65", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380367604347271, "t_first_token_ns": 380367918427436, "t_last_token_ns": 380373239835196, "prompt_tokens": 4025, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "ecc677a8797c40db", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380367827840285, "t_first_token_ns": 380368160479527, "t_last_token_ns": 380373268069376, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "70c2f20559964b6c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380368189184298, "t_first_token_ns": 380368452178246, "t_last_token_ns": 380373348757154, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "73f1f90e5a3849bb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380368600106036, "t_first_token_ns": 380368871306534, "t_last_token_ns": 380373591364641, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "be977b92c40b4821", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380368719198049, "t_first_token_ns": 380369117758629, "t_last_token_ns": 380373615331609, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "74b641ef6ce04fe1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380370379893226, "t_first_token_ns": 380370652464577, "t_last_token_ns": 380375362233580, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "d55a8c9a629b4235", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380370517961608, "t_first_token_ns": 380370902586636, "t_last_token_ns": 380375608977921, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "44b467c2223a4119", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380370811821552, "t_first_token_ns": 380371152164287, "t_last_token_ns": 380375629927355, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "30780bdfe36944b5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380371365975944, "t_first_token_ns": 380371640297365, "t_last_token_ns": 380375825983830, "prompt_tokens": 4015, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "69eb34f3b9cf4fe9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380372018600686, "t_first_token_ns": 380372292069259, "t_last_token_ns": 380376129858550, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "7f74119268ae4cf9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380372559589900, "t_first_token_ns": 380372844333749, "t_last_token_ns": 380376770836765, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "d2a48676a9c842d2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380374004818716, "t_first_token_ns": 380374271544307, "t_last_token_ns": 380378266969246, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "04d9a8e7b6664224", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380375087649316, "t_first_token_ns": 380375362756618, "t_last_token_ns": 380378970902810, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "a39d7a3c9bf54796", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380375179689958, "t_first_token_ns": 380375608835757, "t_last_token_ns": 380378991625388, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "308e535673304e7e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380376222380861, "t_first_token_ns": 380376483214805, "t_last_token_ns": 380379576985747, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "a910cc30e2ac4e86", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380376417534455, "t_first_token_ns": 380376727986652, "t_last_token_ns": 380379592226336, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "3ce264c06a554023", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380377613020007, "t_first_token_ns": 380377883701219, "t_last_token_ns": 380380455994676, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "37c98a1ec59f4148", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380377833975645, "t_first_token_ns": 380378128809016, "t_last_token_ns": 380380468178953, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "57950c303ecb4218", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380379849108501, "t_first_token_ns": 380380106653634, "t_last_token_ns": 380381710894489, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "dc8b88c60082482d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380381288573886, "t_first_token_ns": 380381543426083, "t_last_token_ns": 380382797039813, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "35ba79dddd91405f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380382983842253, "t_first_token_ns": 380383236656740, "t_last_token_ns": 380384806206283, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "72a1205ca44b4fc9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380383892082655, "t_first_token_ns": 380384149596967, "t_last_token_ns": 380387979412084, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "d70f3a42a8f94780", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380384795679186, "t_first_token_ns": 380385056919957, "t_last_token_ns": 380389741388718, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "5214e4149d2946c4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380384872459259, "t_first_token_ns": 380385510536906, "t_last_token_ns": 380389764757966, "prompt_tokens": 4022, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "2ab826079396407d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380384893543319, "t_first_token_ns": 380385510298913, "t_last_token_ns": 380389765156838, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "71648eb5d82f4387", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380385076252283, "t_first_token_ns": 380385742314240, "t_last_token_ns": 380389775424043, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "c9b44814c74f41a6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380385148881610, "t_first_token_ns": 380385976922661, "t_last_token_ns": 380389785059064, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "b950b353610840b0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380386829053188, "t_first_token_ns": 380387099614042, "t_last_token_ns": 380390738758904, "prompt_tokens": 4021, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "1bd339e781654bde", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380387254393763, "t_first_token_ns": 380387522211609, "t_last_token_ns": 380390880856645, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "71363afa03df42e1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380387427715187, "t_first_token_ns": 380387769044392, "t_last_token_ns": 380390894515181, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "c3f3282d268543aa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380388941754874, "t_first_token_ns": 380389211835394, "t_last_token_ns": 380392078296122, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "7baa38bea13b4641", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380389262066233, "t_first_token_ns": 380389527044396, "t_last_token_ns": 380392137520177, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "d6bb25aaab3b4c48", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380391185570546, "t_first_token_ns": 380391441977828, "t_last_token_ns": 380393285265172, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "a9cbbcf488674396", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380391687160439, "t_first_token_ns": 380391946741693, "t_last_token_ns": 380393466050397, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "9d389e46c4f449bb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380393847011359, "t_first_token_ns": 380394100757871, "t_last_token_ns": 380396775112890, "prompt_tokens": 4026, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "b35d7c7ab903490c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380394269504570, "t_first_token_ns": 380394523885284, "t_last_token_ns": 380397104328156, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "0fbc675851e84ca0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380394385901058, "t_first_token_ns": 380394759598667, "t_last_token_ns": 380397150987379, "prompt_tokens": 3948, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "cd70ca93fd404791", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380395997198764, "t_first_token_ns": 380396262327816, "t_last_token_ns": 380400413848017, "prompt_tokens": 4020, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "bb220bb3b19b4c66", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380396347458956, "t_first_token_ns": 380396620993907, "t_last_token_ns": 380400609798035, "prompt_tokens": 4054, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "82c20c5cdb104c13", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380397097359344, "t_first_token_ns": 380397353882610, "t_last_token_ns": 380402297372373, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "1f44cc705da2465c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380397814201739, "t_first_token_ns": 380398078584382, "t_last_token_ns": 380403460645793, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "4ef0db74ab304185", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380397895083830, "t_first_token_ns": 380398321944203, "t_last_token_ns": 380403489593924, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "f120e75a3c30455b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380398354639647, "t_first_token_ns": 380398623692353, "t_last_token_ns": 380403585042389, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "32c20a6a07d54947", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380398610729370, "t_first_token_ns": 380398877659387, "t_last_token_ns": 380403619052186, "prompt_tokens": 3971, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "2f4e014cb74048a4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380398885884455, "t_first_token_ns": 380399157818007, "t_last_token_ns": 380403675855510, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "889b510bdd144e56", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380400094550441, "t_first_token_ns": 380400365943507, "t_last_token_ns": 380405969539984, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "aa34529bcd3c4e69", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380400714099177, "t_first_token_ns": 380400989158009, "t_last_token_ns": 380406433740552, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "60c447da3a4c4cd2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380401002023532, "t_first_token_ns": 380401275213079, "t_last_token_ns": 380406503232585, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "9003386ceebd4992", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380401231113883, "t_first_token_ns": 380401524944986, "t_last_token_ns": 380406525770782, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "3840989c978648c5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380401852689273, "t_first_token_ns": 380402132731360, "t_last_token_ns": 380406836266016, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "1a71835a7bd040eb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380402641707195, "t_first_token_ns": 380402911953257, "t_last_token_ns": 380407534795417, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "c08c8eb5f39d4582", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380404229176433, "t_first_token_ns": 380404497299423, "t_last_token_ns": 380409184368120, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "c55f074bc77d46a0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380404267577336, "t_first_token_ns": 380404734724618, "t_last_token_ns": 380409196020552, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "f6ab0ff0edd2427f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380404541509666, "t_first_token_ns": 380404983368080, "t_last_token_ns": 380409214914590, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "7e2a010c871842d8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380404754237717, "t_first_token_ns": 380405221803596, "t_last_token_ns": 380409224956573, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "59dad6885e6942ac", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380405103765493, "t_first_token_ns": 380405471470368, "t_last_token_ns": 380409239223526, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "3bb193406d89483e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380405239751499, "t_first_token_ns": 380405711085593, "t_last_token_ns": 380409246841523, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "d10635945a724ab5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380406863040419, "t_first_token_ns": 380407133704858, "t_last_token_ns": 380410320291806, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "09c3d7b7822d4d91", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380408668692265, "t_first_token_ns": 380408938170488, "t_last_token_ns": 380412008872901, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "f4d61cdcc7d9467d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380409536455141, "t_first_token_ns": 380409794434995, "t_last_token_ns": 380412654380967, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "305568d1cdc04446", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380409967588664, "t_first_token_ns": 380410229897912, "t_last_token_ns": 380412858676841, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "7945c39fe1d84900", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380410430247397, "t_first_token_ns": 380410695250931, "t_last_token_ns": 380413044577166, "prompt_tokens": 4021, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "83bebc385438466a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 380410718815340, "t_first_token_ns": 380410974969658, "t_last_token_ns": 380413077177160, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"rate": 1.5,
|
||||
"input_tokens": 4096,
|
||||
"output_tokens": 256,
|
||||
"duration_target_s": 240.0,
|
||||
"duration_actual_s": 241.17603391996818,
|
||||
"n_requests": 384
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,346 @@
|
||||
{
|
||||
"mooncake_both": {
|
||||
"config": "mooncake_both",
|
||||
"n_steps_total": 16694,
|
||||
"n_steps_after_warmup": 16194,
|
||||
"n_steps_decode_only": 15851,
|
||||
"decode_share": 0.9788193157959738,
|
||||
"rows_used_for_fit": "decode_only",
|
||||
"cache_size_max": 17528,
|
||||
"per_bin": [
|
||||
{
|
||||
"bin_id": 2,
|
||||
"cache_size_mid": 4382.0,
|
||||
"n": 23,
|
||||
"cache_size_p50": 5193,
|
||||
"step_duration_us_p50": 655,
|
||||
"step_duration_us_p90": 955,
|
||||
"build_meta_us_p50": 541,
|
||||
"build_meta_us_p90": 810
|
||||
},
|
||||
{
|
||||
"bin_id": 3,
|
||||
"cache_size_mid": 6134.8,
|
||||
"n": 127,
|
||||
"cache_size_p50": 6488,
|
||||
"step_duration_us_p50": 809,
|
||||
"step_duration_us_p90": 1141,
|
||||
"build_meta_us_p50": 673,
|
||||
"build_meta_us_p90": 963
|
||||
},
|
||||
{
|
||||
"bin_id": 4,
|
||||
"cache_size_mid": 7887.599999999999,
|
||||
"n": 138,
|
||||
"cache_size_p50": 8601,
|
||||
"step_duration_us_p50": 1157,
|
||||
"step_duration_us_p90": 1284,
|
||||
"build_meta_us_p50": 976,
|
||||
"build_meta_us_p90": 1112
|
||||
},
|
||||
{
|
||||
"bin_id": 5,
|
||||
"cache_size_mid": 9640.4,
|
||||
"n": 357,
|
||||
"cache_size_p50": 10274,
|
||||
"step_duration_us_p50": 981,
|
||||
"step_duration_us_p90": 1423,
|
||||
"build_meta_us_p50": 873,
|
||||
"build_meta_us_p90": 1272
|
||||
},
|
||||
{
|
||||
"bin_id": 6,
|
||||
"cache_size_mid": 11393.199999999999,
|
||||
"n": 273,
|
||||
"cache_size_p50": 11588,
|
||||
"step_duration_us_p50": 1052,
|
||||
"step_duration_us_p90": 1500,
|
||||
"build_meta_us_p50": 958,
|
||||
"build_meta_us_p90": 1384
|
||||
},
|
||||
{
|
||||
"bin_id": 7,
|
||||
"cache_size_mid": 13146.0,
|
||||
"n": 298,
|
||||
"cache_size_p50": 13956,
|
||||
"step_duration_us_p50": 1228,
|
||||
"step_duration_us_p90": 1707,
|
||||
"build_meta_us_p50": 1117,
|
||||
"build_meta_us_p90": 1574
|
||||
},
|
||||
{
|
||||
"bin_id": 8,
|
||||
"cache_size_mid": 14898.8,
|
||||
"n": 388,
|
||||
"cache_size_p50": 15585,
|
||||
"step_duration_us_p50": 1298,
|
||||
"step_duration_us_p90": 1806,
|
||||
"build_meta_us_p50": 1210,
|
||||
"build_meta_us_p90": 1686
|
||||
},
|
||||
{
|
||||
"bin_id": 9,
|
||||
"cache_size_mid": 16651.6,
|
||||
"n": 14247,
|
||||
"cache_size_p50": 17525,
|
||||
"step_duration_us_p50": 1550,
|
||||
"step_duration_us_p90": 2245,
|
||||
"build_meta_us_p50": 1459,
|
||||
"build_meta_us_p90": 2094
|
||||
}
|
||||
],
|
||||
"fit_step_duration": {
|
||||
"slope_us_per_block": 0.08095713405574696,
|
||||
"intercept_us": 310.32910374055535
|
||||
},
|
||||
"fit_build_meta": {
|
||||
"slope_us_per_block": 0.0842904575522931,
|
||||
"intercept_us": 151.76744781193196
|
||||
},
|
||||
"worker_summary": {
|
||||
"n": 16694,
|
||||
"get_finished_us_p50": 178,
|
||||
"get_finished_us_p90": 251,
|
||||
"get_finished_us_p99": 346,
|
||||
"start_load_kv_us_p50": 2,
|
||||
"start_load_kv_us_p90": 6
|
||||
}
|
||||
},
|
||||
"mooncake_both_drfix": {
|
||||
"config": "mooncake_both_drfix",
|
||||
"n_steps_total": 15480,
|
||||
"n_steps_after_warmup": 14980,
|
||||
"n_steps_decode_only": 14608,
|
||||
"decode_share": 0.9751668891855808,
|
||||
"rows_used_for_fit": "decode_only",
|
||||
"cache_size_max": 17528,
|
||||
"per_bin": [
|
||||
{
|
||||
"bin_id": 0,
|
||||
"cache_size_mid": 876.4,
|
||||
"n": 17,
|
||||
"cache_size_p50": 1594,
|
||||
"step_duration_us_p50": 60,
|
||||
"step_duration_us_p90": 73,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 6
|
||||
},
|
||||
{
|
||||
"bin_id": 1,
|
||||
"cache_size_mid": 2629.2,
|
||||
"n": 297,
|
||||
"cache_size_p50": 3136,
|
||||
"step_duration_us_p50": 85,
|
||||
"step_duration_us_p90": 105,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 7
|
||||
},
|
||||
{
|
||||
"bin_id": 2,
|
||||
"cache_size_mid": 4382.0,
|
||||
"n": 90,
|
||||
"cache_size_p50": 4683,
|
||||
"step_duration_us_p50": 94,
|
||||
"step_duration_us_p90": 129,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 8
|
||||
},
|
||||
{
|
||||
"bin_id": 3,
|
||||
"cache_size_mid": 6134.8,
|
||||
"n": 297,
|
||||
"cache_size_p50": 6044,
|
||||
"step_duration_us_p50": 121,
|
||||
"step_duration_us_p90": 156,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 7
|
||||
},
|
||||
{
|
||||
"bin_id": 4,
|
||||
"cache_size_mid": 7887.599999999999,
|
||||
"n": 152,
|
||||
"cache_size_p50": 7126,
|
||||
"step_duration_us_p50": 101,
|
||||
"step_duration_us_p90": 131,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 7
|
||||
},
|
||||
{
|
||||
"bin_id": 5,
|
||||
"cache_size_mid": 9640.4,
|
||||
"n": 95,
|
||||
"cache_size_p50": 10446,
|
||||
"step_duration_us_p50": 150,
|
||||
"step_duration_us_p90": 173,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 7
|
||||
},
|
||||
{
|
||||
"bin_id": 6,
|
||||
"cache_size_mid": 11393.199999999999,
|
||||
"n": 108,
|
||||
"cache_size_p50": 11291,
|
||||
"step_duration_us_p50": 160,
|
||||
"step_duration_us_p90": 184,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 7
|
||||
},
|
||||
{
|
||||
"bin_id": 7,
|
||||
"cache_size_mid": 13146.0,
|
||||
"n": 71,
|
||||
"cache_size_p50": 13621,
|
||||
"step_duration_us_p50": 158,
|
||||
"step_duration_us_p90": 188,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 7
|
||||
},
|
||||
{
|
||||
"bin_id": 8,
|
||||
"cache_size_mid": 14898.8,
|
||||
"n": 239,
|
||||
"cache_size_p50": 14251,
|
||||
"step_duration_us_p50": 132,
|
||||
"step_duration_us_p90": 165,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 7
|
||||
},
|
||||
{
|
||||
"bin_id": 9,
|
||||
"cache_size_mid": 16651.6,
|
||||
"n": 13242,
|
||||
"cache_size_p50": 17526,
|
||||
"step_duration_us_p50": 95,
|
||||
"step_duration_us_p90": 145,
|
||||
"build_meta_us_p50": 6,
|
||||
"build_meta_us_p90": 7
|
||||
}
|
||||
],
|
||||
"fit_step_duration": {
|
||||
"slope_us_per_block": -0.0006801218597615626,
|
||||
"intercept_us": 113.40368450402404
|
||||
},
|
||||
"fit_build_meta": {
|
||||
"slope_us_per_block": -5.060283461904103e-06,
|
||||
"intercept_us": 6.367372709753399
|
||||
},
|
||||
"worker_summary": {
|
||||
"n": 15480,
|
||||
"get_finished_us_p50": 183,
|
||||
"get_finished_us_p90": 262,
|
||||
"get_finished_us_p99": 352,
|
||||
"start_load_kv_us_p50": 2,
|
||||
"start_load_kv_us_p90": 2
|
||||
}
|
||||
},
|
||||
"plain": {
|
||||
"config": "plain",
|
||||
"n_steps_total": 18326,
|
||||
"n_steps_after_warmup": 17826,
|
||||
"n_steps_decode_only": 17500,
|
||||
"decode_share": 0.9817121059127117,
|
||||
"rows_used_for_fit": "decode_only",
|
||||
"cache_size_max": 17530,
|
||||
"per_bin": [
|
||||
{
|
||||
"bin_id": 1,
|
||||
"cache_size_mid": 2629.5,
|
||||
"n": 51,
|
||||
"cache_size_p50": 3423,
|
||||
"step_duration_us_p50": 70,
|
||||
"step_duration_us_p90": 93,
|
||||
"build_meta_us_p50": 0,
|
||||
"build_meta_us_p90": 0
|
||||
},
|
||||
{
|
||||
"bin_id": 2,
|
||||
"cache_size_mid": 4382.5,
|
||||
"n": 314,
|
||||
"cache_size_p50": 3694,
|
||||
"step_duration_us_p50": 60,
|
||||
"step_duration_us_p90": 99,
|
||||
"build_meta_us_p50": 0,
|
||||
"build_meta_us_p90": 0
|
||||
},
|
||||
{
|
||||
"bin_id": 3,
|
||||
"cache_size_mid": 6135.5,
|
||||
"n": 147,
|
||||
"cache_size_p50": 6774,
|
||||
"step_duration_us_p50": 138,
|
||||
"step_duration_us_p90": 163,
|
||||
"build_meta_us_p50": 0,
|
||||
"build_meta_us_p90": 0
|
||||
},
|
||||
{
|
||||
"bin_id": 4,
|
||||
"cache_size_mid": 7888.5,
|
||||
"n": 291,
|
||||
"cache_size_p50": 7146,
|
||||
"step_duration_us_p50": 96,
|
||||
"step_duration_us_p90": 129,
|
||||
"build_meta_us_p50": 0,
|
||||
"build_meta_us_p90": 0
|
||||
},
|
||||
{
|
||||
"bin_id": 5,
|
||||
"cache_size_mid": 9641.5,
|
||||
"n": 124,
|
||||
"cache_size_p50": 9975,
|
||||
"step_duration_us_p50": 134,
|
||||
"step_duration_us_p90": 168,
|
||||
"build_meta_us_p50": 0,
|
||||
"build_meta_us_p90": 0
|
||||
},
|
||||
{
|
||||
"bin_id": 6,
|
||||
"cache_size_mid": 11394.5,
|
||||
"n": 213,
|
||||
"cache_size_p50": 11592,
|
||||
"step_duration_us_p50": 114,
|
||||
"step_duration_us_p90": 142,
|
||||
"build_meta_us_p50": 0,
|
||||
"build_meta_us_p90": 0
|
||||
},
|
||||
{
|
||||
"bin_id": 7,
|
||||
"cache_size_mid": 13147.5,
|
||||
"n": 67,
|
||||
"cache_size_p50": 12913,
|
||||
"step_duration_us_p50": 108,
|
||||
"step_duration_us_p90": 130,
|
||||
"build_meta_us_p50": 0,
|
||||
"build_meta_us_p90": 0
|
||||
},
|
||||
{
|
||||
"bin_id": 8,
|
||||
"cache_size_mid": 14900.5,
|
||||
"n": 34,
|
||||
"cache_size_p50": 14691,
|
||||
"step_duration_us_p50": 142,
|
||||
"step_duration_us_p90": 166,
|
||||
"build_meta_us_p50": 0,
|
||||
"build_meta_us_p90": 0
|
||||
},
|
||||
{
|
||||
"bin_id": 9,
|
||||
"cache_size_mid": 16653.5,
|
||||
"n": 16259,
|
||||
"cache_size_p50": 17527,
|
||||
"step_duration_us_p50": 72,
|
||||
"step_duration_us_p90": 115,
|
||||
"build_meta_us_p50": 0,
|
||||
"build_meta_us_p90": 0
|
||||
}
|
||||
],
|
||||
"fit_step_duration": {
|
||||
"slope_us_per_block": -0.0018163641275035448,
|
||||
"intercept_us": 112.88251682390244
|
||||
},
|
||||
"fit_build_meta": {
|
||||
"slope_us_per_block": 0.0,
|
||||
"intercept_us": 0.0
|
||||
},
|
||||
"worker_summary": null
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,624 @@
|
||||
# HELP python_gc_objects_collected_total Objects collected during gc
|
||||
# TYPE python_gc_objects_collected_total counter
|
||||
python_gc_objects_collected_total{generation="0"} 11967.0
|
||||
python_gc_objects_collected_total{generation="1"} 1552.0
|
||||
python_gc_objects_collected_total{generation="2"} 855.0
|
||||
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
|
||||
# TYPE python_gc_objects_uncollectable_total counter
|
||||
python_gc_objects_uncollectable_total{generation="0"} 0.0
|
||||
python_gc_objects_uncollectable_total{generation="1"} 0.0
|
||||
python_gc_objects_uncollectable_total{generation="2"} 0.0
|
||||
# HELP python_gc_collections_total Number of times this generation was collected
|
||||
# TYPE python_gc_collections_total counter
|
||||
python_gc_collections_total{generation="0"} 1326.0
|
||||
python_gc_collections_total{generation="1"} 120.0
|
||||
python_gc_collections_total{generation="2"} 9.0
|
||||
# HELP python_info Python platform information
|
||||
# TYPE python_info gauge
|
||||
python_info{implementation="CPython",major="3",minor="12",patchlevel="3",version="3.12.3"} 1.0
|
||||
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
|
||||
# TYPE process_virtual_memory_bytes gauge
|
||||
process_virtual_memory_bytes 3.8945837056e+010
|
||||
# HELP process_resident_memory_bytes Resident memory size in bytes.
|
||||
# TYPE process_resident_memory_bytes gauge
|
||||
process_resident_memory_bytes 1.349496832e+09
|
||||
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
|
||||
# TYPE process_start_time_seconds gauge
|
||||
process_start_time_seconds 1.77981022391e+09
|
||||
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
|
||||
# TYPE process_cpu_seconds_total counter
|
||||
process_cpu_seconds_total 37.370000000000005
|
||||
# HELP process_open_fds Number of open file descriptors.
|
||||
# TYPE process_open_fds gauge
|
||||
process_open_fds 64.0
|
||||
# HELP process_max_fds Maximum number of open file descriptors.
|
||||
# TYPE process_max_fds gauge
|
||||
process_max_fds 65535.0
|
||||
# HELP vllm:estimated_flops_per_gpu_total Estimated number of floating point operations per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_flops_per_gpu_total counter
|
||||
vllm:estimated_flops_per_gpu_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:estimated_flops_per_gpu_created Estimated number of floating point operations per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_flops_per_gpu_created gauge
|
||||
vllm:estimated_flops_per_gpu_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305749674e+09
|
||||
# HELP vllm:estimated_read_bytes_per_gpu_total Estimated number of bytes read from memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_read_bytes_per_gpu_total counter
|
||||
vllm:estimated_read_bytes_per_gpu_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:estimated_read_bytes_per_gpu_created Estimated number of bytes read from memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_read_bytes_per_gpu_created gauge
|
||||
vllm:estimated_read_bytes_per_gpu_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305750008e+09
|
||||
# HELP vllm:estimated_write_bytes_per_gpu_total Estimated number of bytes written to memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_write_bytes_per_gpu_total counter
|
||||
vllm:estimated_write_bytes_per_gpu_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:estimated_write_bytes_per_gpu_created Estimated number of bytes written to memory per GPU (for Model Flops Utilization calculations).
|
||||
# TYPE vllm:estimated_write_bytes_per_gpu_created gauge
|
||||
vllm:estimated_write_bytes_per_gpu_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305750215e+09
|
||||
# HELP vllm:num_requests_running Number of requests in model execution batches.
|
||||
# TYPE vllm:num_requests_running gauge
|
||||
vllm:num_requests_running{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:num_requests_waiting Number of requests waiting to be processed.
|
||||
# TYPE vllm:num_requests_waiting gauge
|
||||
vllm:num_requests_waiting{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:engine_sleep_state Engine sleep state; awake = 0 means engine is sleeping; awake = 1 means engine is awake; weights_offloaded = 1 means sleep level 1; discard_all = 1 means sleep level 2.
|
||||
# TYPE vllm:engine_sleep_state gauge
|
||||
vllm:engine_sleep_state{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",sleep_state="awake"} 1.0
|
||||
vllm:engine_sleep_state{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",sleep_state="weights_offloaded"} 0.0
|
||||
vllm:engine_sleep_state{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",sleep_state="discard_all"} 0.0
|
||||
# HELP vllm:kv_cache_usage_perc KV-cache usage. 1 means 100 percent usage.
|
||||
# TYPE vllm:kv_cache_usage_perc gauge
|
||||
vllm:kv_cache_usage_perc{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prefix_cache_queries_total Prefix cache queries, in terms of number of queried tokens.
|
||||
# TYPE vllm:prefix_cache_queries_total counter
|
||||
vllm:prefix_cache_queries_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.28393e+06
|
||||
# HELP vllm:prefix_cache_queries_created Prefix cache queries, in terms of number of queried tokens.
|
||||
# TYPE vllm:prefix_cache_queries_created gauge
|
||||
vllm:prefix_cache_queries_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305751915e+09
|
||||
# HELP vllm:prefix_cache_hits_total Prefix cache hits, in terms of number of cached tokens.
|
||||
# TYPE vllm:prefix_cache_hits_total counter
|
||||
vllm:prefix_cache_hits_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prefix_cache_hits_created Prefix cache hits, in terms of number of cached tokens.
|
||||
# TYPE vllm:prefix_cache_hits_created gauge
|
||||
vllm:prefix_cache_hits_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305752053e+09
|
||||
# HELP vllm:external_prefix_cache_queries_total External prefix cache queries from KV connector cross-instance cache sharing, in terms of number of queried tokens.
|
||||
# TYPE vllm:external_prefix_cache_queries_total counter
|
||||
vllm:external_prefix_cache_queries_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:external_prefix_cache_queries_created External prefix cache queries from KV connector cross-instance cache sharing, in terms of number of queried tokens.
|
||||
# TYPE vllm:external_prefix_cache_queries_created gauge
|
||||
vllm:external_prefix_cache_queries_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305752184e+09
|
||||
# HELP vllm:external_prefix_cache_hits_total External prefix cache hits from KV connector cross-instance cache sharing, in terms of number of cached tokens.
|
||||
# TYPE vllm:external_prefix_cache_hits_total counter
|
||||
vllm:external_prefix_cache_hits_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:external_prefix_cache_hits_created External prefix cache hits from KV connector cross-instance cache sharing, in terms of number of cached tokens.
|
||||
# TYPE vllm:external_prefix_cache_hits_created gauge
|
||||
vllm:external_prefix_cache_hits_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305752301e+09
|
||||
# HELP vllm:mm_cache_queries_total Multi-modal cache queries, in terms of number of queried items.
|
||||
# TYPE vllm:mm_cache_queries_total counter
|
||||
vllm:mm_cache_queries_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:mm_cache_queries_created Multi-modal cache queries, in terms of number of queried items.
|
||||
# TYPE vllm:mm_cache_queries_created gauge
|
||||
vllm:mm_cache_queries_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305752425e+09
|
||||
# HELP vllm:mm_cache_hits_total Multi-modal cache hits, in terms of number of cached items.
|
||||
# TYPE vllm:mm_cache_hits_total counter
|
||||
vllm:mm_cache_hits_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:mm_cache_hits_created Multi-modal cache hits, in terms of number of cached items.
|
||||
# TYPE vllm:mm_cache_hits_created gauge
|
||||
vllm:mm_cache_hits_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305752535e+09
|
||||
# HELP vllm:num_preemptions_total Cumulative number of preemption from the engine.
|
||||
# TYPE vllm:num_preemptions_total counter
|
||||
vllm:num_preemptions_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:num_preemptions_created Cumulative number of preemption from the engine.
|
||||
# TYPE vllm:num_preemptions_created gauge
|
||||
vllm:num_preemptions_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305752664e+09
|
||||
# HELP vllm:prompt_tokens_total Number of prefill tokens processed.
|
||||
# TYPE vllm:prompt_tokens_total counter
|
||||
vllm:prompt_tokens_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.28393e+06
|
||||
# HELP vllm:prompt_tokens_created Number of prefill tokens processed.
|
||||
# TYPE vllm:prompt_tokens_created gauge
|
||||
vllm:prompt_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305752764e+09
|
||||
# HELP vllm:prompt_tokens_by_source_total Number of prompt tokens by source.
|
||||
# TYPE vllm:prompt_tokens_by_source_total counter
|
||||
vllm:prompt_tokens_by_source_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_compute"} 1.28393e+06
|
||||
vllm:prompt_tokens_by_source_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_cache_hit"} 0.0
|
||||
vllm:prompt_tokens_by_source_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="external_kv_transfer"} 0.0
|
||||
# HELP vllm:prompt_tokens_by_source_created Number of prompt tokens by source.
|
||||
# TYPE vllm:prompt_tokens_by_source_created gauge
|
||||
vllm:prompt_tokens_by_source_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_compute"} 1.779810330575295e+09
|
||||
vllm:prompt_tokens_by_source_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="local_cache_hit"} 1.7798103305753e+09
|
||||
vllm:prompt_tokens_by_source_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",source="external_kv_transfer"} 1.7798103305753045e+09
|
||||
# HELP vllm:prompt_tokens_cached_total Number of cached prompt tokens (local + external).
|
||||
# TYPE vllm:prompt_tokens_cached_total counter
|
||||
vllm:prompt_tokens_cached_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prompt_tokens_cached_created Number of cached prompt tokens (local + external).
|
||||
# TYPE vllm:prompt_tokens_cached_created gauge
|
||||
vllm:prompt_tokens_cached_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779810330575316e+09
|
||||
# HELP vllm:prompt_tokens_recomputed_total Number of cached tokens recomputed for forward pass.
|
||||
# TYPE vllm:prompt_tokens_recomputed_total counter
|
||||
vllm:prompt_tokens_recomputed_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:prompt_tokens_recomputed_created Number of cached tokens recomputed for forward pass.
|
||||
# TYPE vllm:prompt_tokens_recomputed_created gauge
|
||||
vllm:prompt_tokens_recomputed_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305753295e+09
|
||||
# HELP vllm:generation_tokens_total Number of generation tokens processed.
|
||||
# TYPE vllm:generation_tokens_total counter
|
||||
vllm:generation_tokens_total{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82432.0
|
||||
# HELP vllm:generation_tokens_created Number of generation tokens processed.
|
||||
# TYPE vllm:generation_tokens_created gauge
|
||||
vllm:generation_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305753443e+09
|
||||
# HELP vllm:request_success_total Count of successfully processed requests.
|
||||
# TYPE vllm:request_success_total counter
|
||||
vllm:request_success_total{engine="0",finished_reason="stop",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_success_total{engine="0",finished_reason="length",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_success_total{engine="0",finished_reason="abort",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_success_total{engine="0",finished_reason="error",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_success_total{engine="0",finished_reason="repetition",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
# HELP vllm:request_success_created Count of successfully processed requests.
|
||||
# TYPE vllm:request_success_created gauge
|
||||
vllm:request_success_created{engine="0",finished_reason="stop",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305753694e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="length",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305753772e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="abort",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779810330575384e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="error",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305753891e+09
|
||||
vllm:request_success_created{engine="0",finished_reason="repetition",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305753949e+09
|
||||
# HELP vllm:request_prompt_tokens Number of prefill tokens processed.
|
||||
# TYPE vllm:request_prompt_tokens histogram
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prompt_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prompt_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prompt_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.28393e+06
|
||||
# HELP vllm:request_prompt_tokens_created Number of prefill tokens processed.
|
||||
# TYPE vllm:request_prompt_tokens_created gauge
|
||||
vllm:request_prompt_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305754614e+09
|
||||
# HELP vllm:request_generation_tokens Number of generation tokens processed.
|
||||
# TYPE vllm:request_generation_tokens histogram
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_generation_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82432.0
|
||||
# HELP vllm:request_generation_tokens_created Number of generation tokens processed.
|
||||
# TYPE vllm:request_generation_tokens_created gauge
|
||||
vllm:request_generation_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305755272e+09
|
||||
# HELP vllm:iteration_tokens_total Histogram of number of tokens per engine_step.
|
||||
# TYPE vllm:iteration_tokens_total histogram
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 3260.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="8.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15983.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="16.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 17757.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="32.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 17987.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="64.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 17987.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="128.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 17987.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="256.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 17987.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="512.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 17987.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="1024.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 17987.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="2048.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 17987.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="4096.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 18279.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="8192.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 18294.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="16384.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 18294.0
|
||||
vllm:iteration_tokens_total_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 18294.0
|
||||
vllm:iteration_tokens_total_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 18294.0
|
||||
vllm:iteration_tokens_total_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.366362e+06
|
||||
# HELP vllm:iteration_tokens_total_created Histogram of number of tokens per engine_step.
|
||||
# TYPE vllm:iteration_tokens_total_created gauge
|
||||
vllm:iteration_tokens_total_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305755823e+09
|
||||
# HELP vllm:request_max_num_generation_tokens Histogram of maximum number of requested generation tokens.
|
||||
# TYPE vllm:request_max_num_generation_tokens histogram
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_max_num_generation_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82432.0
|
||||
# HELP vllm:request_max_num_generation_tokens_created Histogram of maximum number of requested generation tokens.
|
||||
# TYPE vllm:request_max_num_generation_tokens_created gauge
|
||||
vllm:request_max_num_generation_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305756216e+09
|
||||
# HELP vllm:request_params_n Histogram of the n request parameter.
|
||||
# TYPE vllm:request_params_n histogram
|
||||
vllm:request_params_n_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_n_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_n_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_n_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_n_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_n_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_n_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_n_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
# HELP vllm:request_params_n_created Histogram of the n request parameter.
|
||||
# TYPE vllm:request_params_n_created gauge
|
||||
vllm:request_params_n_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305756617e+09
|
||||
# HELP vllm:request_params_max_tokens Histogram of the max_tokens request parameter.
|
||||
# TYPE vllm:request_params_max_tokens histogram
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_params_max_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82432.0
|
||||
# HELP vllm:request_params_max_tokens_created Histogram of the max_tokens request parameter.
|
||||
# TYPE vllm:request_params_max_tokens_created gauge
|
||||
vllm:request_params_max_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305756886e+09
|
||||
# HELP vllm:time_to_first_token_seconds Histogram of time to first token in seconds.
|
||||
# TYPE vllm:time_to_first_token_seconds histogram
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.001",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.005",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.01",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.02",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.04",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.06",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.08",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.1",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.25",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 15.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 288.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="0.75",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 317.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 321.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="7.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="80.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="160.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="640.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="2560.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:time_to_first_token_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 107.43498420715332
|
||||
# HELP vllm:time_to_first_token_seconds_created Histogram of time to first token in seconds.
|
||||
# TYPE vllm:time_to_first_token_seconds_created gauge
|
||||
vllm:time_to_first_token_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305757225e+09
|
||||
# HELP vllm:inter_token_latency_seconds Histogram of inter-token latency in seconds.
|
||||
# TYPE vllm:inter_token_latency_seconds histogram
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.01",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 40478.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.025",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 78719.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.05",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 80264.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.075",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 80338.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.1",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 80338.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.15",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 80338.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.2",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 80574.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82001.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.4",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82001.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="0.75",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="7.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="80.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 82110.0
|
||||
vllm:inter_token_latency_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1312.5351079615648
|
||||
# HELP vllm:inter_token_latency_seconds_created Histogram of inter-token latency in seconds.
|
||||
# TYPE vllm:inter_token_latency_seconds_created gauge
|
||||
vllm:inter_token_latency_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305757735e+09
|
||||
# HELP vllm:request_time_per_output_token_seconds Histogram of time_per_output_token_seconds per request.
|
||||
# TYPE vllm:request_time_per_output_token_seconds histogram
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.01",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 58.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.025",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 285.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.05",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.075",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.1",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.15",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.2",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.4",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="0.75",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="7.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="80.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_time_per_output_token_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 5.1471965018100585
|
||||
# HELP vllm:request_time_per_output_token_seconds_created Histogram of time_per_output_token_seconds per request.
|
||||
# TYPE vllm:request_time_per_output_token_seconds_created gauge
|
||||
vllm:request_time_per_output_token_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305758116e+09
|
||||
# HELP vllm:e2e_request_latency_seconds Histogram of e2e request latency in seconds.
|
||||
# TYPE vllm:e2e_request_latency_seconds histogram
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 11.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 33.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 222.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 319.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:e2e_request_latency_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1419.8564944267273
|
||||
# HELP vllm:e2e_request_latency_seconds_created Histogram of e2e request latency in seconds.
|
||||
# TYPE vllm:e2e_request_latency_seconds_created gauge
|
||||
vllm:e2e_request_latency_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305772622e+09
|
||||
# HELP vllm:request_queue_time_seconds Histogram of time spent in WAITING phase for request.
|
||||
# TYPE vllm:request_queue_time_seconds histogram
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_queue_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0025250064209103584
|
||||
# HELP vllm:request_queue_time_seconds_created Histogram of time spent in WAITING phase for request.
|
||||
# TYPE vllm:request_queue_time_seconds_created gauge
|
||||
vllm:request_queue_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305773258e+09
|
||||
# HELP vllm:request_inference_time_seconds Histogram of time spent in RUNNING phase for request.
|
||||
# TYPE vllm:request_inference_time_seconds histogram
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 13.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 34.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 224.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 321.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_inference_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1404.3917311558616
|
||||
# HELP vllm:request_inference_time_seconds_created Histogram of time spent in RUNNING phase for request.
|
||||
# TYPE vllm:request_inference_time_seconds_created gauge
|
||||
vllm:request_inference_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305773704e+09
|
||||
# HELP vllm:request_prefill_time_seconds Histogram of time spent in PREFILL phase for request.
|
||||
# TYPE vllm:request_prefill_time_seconds histogram
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 267.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 313.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 91.85662319429684
|
||||
# HELP vllm:request_prefill_time_seconds_created Histogram of time spent in PREFILL phase for request.
|
||||
# TYPE vllm:request_prefill_time_seconds_created gauge
|
||||
vllm:request_prefill_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.779810330577415e+09
|
||||
# HELP vllm:request_decode_time_seconds Histogram of time spent in DECODE phase for request.
|
||||
# TYPE vllm:request_decode_time_seconds histogram
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="0.3",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="0.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="0.8",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="1.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 4.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 27.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="2.5",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 57.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 236.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="15.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="30.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="40.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="60.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="120.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="240.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="480.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="960.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="1920.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="7680.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_decode_time_seconds_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1312.5351079615648
|
||||
# HELP vllm:request_decode_time_seconds_created Histogram of time spent in DECODE phase for request.
|
||||
# TYPE vllm:request_decode_time_seconds_created gauge
|
||||
vllm:request_decode_time_seconds_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.7798103305774732e+09
|
||||
# HELP vllm:request_prefill_kv_computed_tokens Histogram of new KV tokens computed during prefill (excluding cached tokens).
|
||||
# TYPE vllm:request_prefill_kv_computed_tokens histogram
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="1.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="2.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="5.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="10.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="20.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="50.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="100.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="200.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="500.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="1000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="2000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 0.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="5000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="10000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="20000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="50000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="100000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="200000.0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_kv_computed_tokens_bucket{engine="0",le="+Inf",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_kv_computed_tokens_count{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 322.0
|
||||
vllm:request_prefill_kv_computed_tokens_sum{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.28393e+06
|
||||
# HELP vllm:request_prefill_kv_computed_tokens_created Histogram of new KV tokens computed during prefill (excluding cached tokens).
|
||||
# TYPE vllm:request_prefill_kv_computed_tokens_created gauge
|
||||
vllm:request_prefill_kv_computed_tokens_created{engine="0",model_name="/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct"} 1.77981033057752e+09
|
||||
# HELP vllm:cache_config_info Information of the LLMEngine CacheConfig
|
||||
# TYPE vllm:cache_config_info gauge
|
||||
vllm:cache_config_info{_block_size_resolved="True",block_size="16",cache_dtype="auto",calculate_kv_scales="False",cpu_kvcache_space_bytes="None",enable_prefix_caching="True",engine="0",gpu_memory_utilization="0.9",is_attention_free="False",kv_cache_memory_bytes="None",kv_offloading_backend="native",kv_offloading_size="None",kv_sharing_fast_prefill="False",mamba_block_size="None",mamba_cache_dtype="auto",mamba_cache_mode="none",mamba_page_size_padded="None",mamba_ssm_cache_dtype="auto",num_cpu_blocks="None",num_gpu_blocks="17590",num_gpu_blocks_override="None",prefix_caching_hash_algo="sha256",sliding_window="None",user_specified_block_size="False"} 1.0
|
||||
# HELP http_requests_total Total number of requests by method, status and handler.
|
||||
# TYPE http_requests_total counter
|
||||
http_requests_total{handler="/v1/models",method="GET",status="2xx"} 1.0
|
||||
http_requests_total{handler="/v1/chat/completions",method="POST",status="2xx"} 322.0
|
||||
# HELP http_requests_created Total number of requests by method, status and handler.
|
||||
# TYPE http_requests_created gauge
|
||||
http_requests_created{handler="/v1/models",method="GET",status="2xx"} 1.7798103326640093e+09
|
||||
http_requests_created{handler="/v1/chat/completions",method="POST",status="2xx"} 1.779810338509727e+09
|
||||
# HELP http_request_size_bytes Content length of incoming requests by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_request_size_bytes summary
|
||||
http_request_size_bytes_count{handler="/v1/models"} 1.0
|
||||
http_request_size_bytes_sum{handler="/v1/models"} 0.0
|
||||
http_request_size_bytes_count{handler="/v1/chat/completions"} 322.0
|
||||
http_request_size_bytes_sum{handler="/v1/chat/completions"} 1.698228e+06
|
||||
# HELP http_request_size_bytes_created Content length of incoming requests by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_request_size_bytes_created gauge
|
||||
http_request_size_bytes_created{handler="/v1/models"} 1.7798103326640337e+09
|
||||
http_request_size_bytes_created{handler="/v1/chat/completions"} 1.7798103385097458e+09
|
||||
# HELP http_response_size_bytes Content length of outgoing responses by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_response_size_bytes summary
|
||||
http_response_size_bytes_count{handler="/v1/models"} 1.0
|
||||
http_response_size_bytes_sum{handler="/v1/models"} 558.0
|
||||
http_response_size_bytes_count{handler="/v1/chat/completions"} 322.0
|
||||
http_response_size_bytes_sum{handler="/v1/chat/completions"} 0.0
|
||||
# HELP http_response_size_bytes_created Content length of outgoing responses by handler. Only value of header is respected. Otherwise ignored. No percentile calculated.
|
||||
# TYPE http_response_size_bytes_created gauge
|
||||
http_response_size_bytes_created{handler="/v1/models"} 1.7798103326640582e+09
|
||||
http_response_size_bytes_created{handler="/v1/chat/completions"} 1.7798103385097663e+09
|
||||
# HELP http_request_duration_highr_seconds Latency with many buckets but no API specific labels. Made for more accurate percentile calculations.
|
||||
# TYPE http_request_duration_highr_seconds histogram
|
||||
http_request_duration_highr_seconds_bucket{le="0.01"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.025"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.05"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.075"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.1"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.25"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.5"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="0.75"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="1.0"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="1.5"} 1.0
|
||||
http_request_duration_highr_seconds_bucket{le="2.0"} 12.0
|
||||
http_request_duration_highr_seconds_bucket{le="2.5"} 34.0
|
||||
http_request_duration_highr_seconds_bucket{le="3.0"} 70.0
|
||||
http_request_duration_highr_seconds_bucket{le="3.5"} 125.0
|
||||
http_request_duration_highr_seconds_bucket{le="4.0"} 170.0
|
||||
http_request_duration_highr_seconds_bucket{le="4.5"} 197.0
|
||||
http_request_duration_highr_seconds_bucket{le="5.0"} 223.0
|
||||
http_request_duration_highr_seconds_bucket{le="7.5"} 300.0
|
||||
http_request_duration_highr_seconds_bucket{le="10.0"} 320.0
|
||||
http_request_duration_highr_seconds_bucket{le="30.0"} 323.0
|
||||
http_request_duration_highr_seconds_bucket{le="60.0"} 323.0
|
||||
http_request_duration_highr_seconds_bucket{le="+Inf"} 323.0
|
||||
http_request_duration_highr_seconds_count 323.0
|
||||
http_request_duration_highr_seconds_sum 1420.29089475004
|
||||
# HELP http_request_duration_highr_seconds_created Latency with many buckets but no API specific labels. Made for more accurate percentile calculations.
|
||||
# TYPE http_request_duration_highr_seconds_created gauge
|
||||
http_request_duration_highr_seconds_created 1.7798103311036901e+09
|
||||
# HELP http_request_duration_seconds Latency with only few buckets by handler. Made to be only used if aggregation by handler is important.
|
||||
# TYPE http_request_duration_seconds histogram
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="0.1",method="GET"} 1.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="0.5",method="GET"} 1.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="1.0",method="GET"} 1.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/models",le="+Inf",method="GET"} 1.0
|
||||
http_request_duration_seconds_count{handler="/v1/models",method="GET"} 1.0
|
||||
http_request_duration_seconds_sum{handler="/v1/models",method="GET"} 0.003153438970912248
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="0.1",method="POST"} 0.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="0.5",method="POST"} 0.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="1.0",method="POST"} 0.0
|
||||
http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="+Inf",method="POST"} 322.0
|
||||
http_request_duration_seconds_count{handler="/v1/chat/completions",method="POST"} 322.0
|
||||
http_request_duration_seconds_sum{handler="/v1/chat/completions",method="POST"} 1420.287741311069
|
||||
# HELP http_request_duration_seconds_created Latency with only few buckets by handler. Made to be only used if aggregation by handler is important.
|
||||
# TYPE http_request_duration_seconds_created gauge
|
||||
http_request_duration_seconds_created{handler="/v1/models",method="GET"} 1.7798103326640909e+09
|
||||
http_request_duration_seconds_created{handler="/v1/chat/completions",method="POST"} 1.779810338509794e+09
|
||||
@@ -0,0 +1,322 @@
|
||||
{"req_id": "2ac2d78147b841f6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379468649694988, "t_first_token_ns": 379469452242112, "t_last_token_ns": 379473496585594, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "1c52b4ddae5440f8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379469420544863, "t_first_token_ns": 379469691460103, "t_last_token_ns": 379473724650153, "prompt_tokens": 4008, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "5cffa53da5014e2c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379470690440294, "t_first_token_ns": 379470950915747, "t_last_token_ns": 379476169570371, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "53d2e407d16a4933", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379470805683382, "t_first_token_ns": 379471195222775, "t_last_token_ns": 379476193877933, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "4cd485403dc24215", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379470944656726, "t_first_token_ns": 379471431104493, "t_last_token_ns": 379476205756409, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "7eb2388e11f841e0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379471651826433, "t_first_token_ns": 379471923091882, "t_last_token_ns": 379476524174872, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "aa97f9b587174a0b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379472319899708, "t_first_token_ns": 379472587464909, "t_last_token_ns": 379477220206965, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "05e72164f63b4f0f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379472389227765, "t_first_token_ns": 379472837638998, "t_last_token_ns": 379477240258598, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "fa8a1cded59a4e35", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379472940095463, "t_first_token_ns": 379473253549247, "t_last_token_ns": 379477359733621, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "83f14c7ede01493c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379473082219390, "t_first_token_ns": 379473453372031, "t_last_token_ns": 379477368204458, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "77790be138db4f95", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379473359148375, "t_first_token_ns": 379473711544016, "t_last_token_ns": 379477382282393, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "479c38eff81849b3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379474121097235, "t_first_token_ns": 379474386059291, "t_last_token_ns": 379477664864025, "prompt_tokens": 3950, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "3d7c583bdd404a2b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379476703205347, "t_first_token_ns": 379476964742959, "t_last_token_ns": 379478962313908, "prompt_tokens": 3956, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "e5684f4208f44689", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379477603322202, "t_first_token_ns": 379477865282655, "t_last_token_ns": 379480329644775, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "739a1abbbb7141ee", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379479249075513, "t_first_token_ns": 379479504761058, "t_last_token_ns": 379485412116554, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "f7f381e4e50e45a8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379479441549591, "t_first_token_ns": 379479744732983, "t_last_token_ns": 379485440448826, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "333b8994d3de4fe2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379479746056734, "t_first_token_ns": 379480007160352, "t_last_token_ns": 379485514505376, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "a07031ebddca46ed", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379479984468603, "t_first_token_ns": 379480251192796, "t_last_token_ns": 379485542505630, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "575fe33007874178", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379480326173934, "t_first_token_ns": 379480590931153, "t_last_token_ns": 379485691304536, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "a55f565390004afa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379480569297040, "t_first_token_ns": 379480838885570, "t_last_token_ns": 379485715817168, "prompt_tokens": 4019, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "75a3ca9c2cad4b77", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379481366567551, "t_first_token_ns": 379481636932973, "t_last_token_ns": 379486458209530, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "99d7ba7d963c4e27", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379481388216019, "t_first_token_ns": 379481872116126, "t_last_token_ns": 379486468887120, "prompt_tokens": 3956, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "f5d32eb77e4b47ee", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379481958962050, "t_first_token_ns": 379482233996784, "t_last_token_ns": 379486592104241, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "8a583593ee774d5d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379482216563307, "t_first_token_ns": 379482701640594, "t_last_token_ns": 379486610427703, "prompt_tokens": 4025, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "5d7dee767bfc40fd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379482038154041, "t_first_token_ns": 379482701357602, "t_last_token_ns": 379486610663197, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "3d986caae1b443ec", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379483190055168, "t_first_token_ns": 379483473152195, "t_last_token_ns": 379487106309161, "prompt_tokens": 4010, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "c2460439a0d947c1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379484840344275, "t_first_token_ns": 379485115864020, "t_last_token_ns": 379488879788234, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "dcf419674e294622", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379486785655707, "t_first_token_ns": 379487046239267, "t_last_token_ns": 379492523700523, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "35c81bdf37f847a7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379487101085185, "t_first_token_ns": 379487361566243, "t_last_token_ns": 379492689166814, "prompt_tokens": 4024, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "05ec2ebac3c54280", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379487415619444, "t_first_token_ns": 379487673385911, "t_last_token_ns": 379493073179715, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "ac5585c61a6244d4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379487805402364, "t_first_token_ns": 379488067847710, "t_last_token_ns": 379493377387437, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "8050ba6387aa4abe", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379488016503023, "t_first_token_ns": 379488311641264, "t_last_token_ns": 379493406367580, "prompt_tokens": 3960, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "491e1f81a38641b6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379489006998018, "t_first_token_ns": 379489267391514, "t_last_token_ns": 379494916079996, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "fc40660c73284987", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379489485093732, "t_first_token_ns": 379489754338125, "t_last_token_ns": 379495527143320, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "8ddef637b6bd4b43", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379489653378620, "t_first_token_ns": 379490218200813, "t_last_token_ns": 379495582240301, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "76a5ade1be5b48c6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379489714639489, "t_first_token_ns": 379490218730237, "t_last_token_ns": 379495582721462, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "c85748802f644491", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379490570970316, "t_first_token_ns": 379490848560078, "t_last_token_ns": 379496219946022, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "b48bfbb63afb4ffe", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379490730044659, "t_first_token_ns": 379491098647560, "t_last_token_ns": 379496243168282, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "cef2d47040b04603", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379491962536074, "t_first_token_ns": 379492244915516, "t_last_token_ns": 379497425401309, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "b18d500c76d94c5a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379492181265063, "t_first_token_ns": 379492497744071, "t_last_token_ns": 379497448212916, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "c6f9b566bbee4a8c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379492796833021, "t_first_token_ns": 379493072724215, "t_last_token_ns": 379498190144614, "prompt_tokens": 4019, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "00a8baa5840749b7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379493449325653, "t_first_token_ns": 379493714889241, "t_last_token_ns": 379498552735051, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "13c05274e515470e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379493959845333, "t_first_token_ns": 379494241625637, "t_last_token_ns": 379500410454827, "prompt_tokens": 4026, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "35a8ee4302614ed6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379494968013934, "t_first_token_ns": 379495245553410, "t_last_token_ns": 379503358345319, "prompt_tokens": 4030, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "317273ece7be48f7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379495519263954, "t_first_token_ns": 379495801438753, "t_last_token_ns": 379503835124817, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "8a8c8938163d4cc5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379496292481164, "t_first_token_ns": 379496553175477, "t_last_token_ns": 379504957898162, "prompt_tokens": 3945, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "e34d10b1cc524d5f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379497051700767, "t_first_token_ns": 379497320925344, "t_last_token_ns": 379506204668325, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "d46170dd86e54f17", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379497444991800, "t_first_token_ns": 379497710816148, "t_last_token_ns": 379506728088933, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "f69ac606222f4bc8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379497528661790, "t_first_token_ns": 379497959132127, "t_last_token_ns": 379506770526256, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "78ca54ac2cb44188", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379498535558962, "t_first_token_ns": 379498803697037, "t_last_token_ns": 379508756777639, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "9784587c77484464", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379498632032316, "t_first_token_ns": 379499271324369, "t_last_token_ns": 379508797302993, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "e6c41ba405bf4b92", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379498745414142, "t_first_token_ns": 379499270755696, "t_last_token_ns": 379508797538446, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "fed8788839e9419c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379499180713039, "t_first_token_ns": 379499789308944, "t_last_token_ns": 379508839907614, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "103b268194744402", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379499200437301, "t_first_token_ns": 379499788602453, "t_last_token_ns": 379508840118512, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "ec0c63ee45b44485", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379499247430665, "t_first_token_ns": 379499979235288, "t_last_token_ns": 379508856832628, "prompt_tokens": 3948, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "2baa8af736b14d3a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379499999612253, "t_first_token_ns": 379500276859791, "t_last_token_ns": 379508940194872, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "41984c3183dc4d64", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379500512337701, "t_first_token_ns": 379500789863022, "t_last_token_ns": 379509550100930, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "861c48c708924d27", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379500604104002, "t_first_token_ns": 379501045553262, "t_last_token_ns": 379509795269616, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "d1fe251ddaa84d8b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379501134887790, "t_first_token_ns": 379501419891265, "t_last_token_ns": 379509943319270, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "fbda8001903f4ca7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379501291677434, "t_first_token_ns": 379501716046830, "t_last_token_ns": 379509974797664, "prompt_tokens": 3952, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "230c8d5303ee42e3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379501494981905, "t_first_token_ns": 379502136845632, "t_last_token_ns": 379509991133851, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 18, "error": null}
|
||||
{"req_id": "68885fff3ca643e9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379501637409363, "t_first_token_ns": 379502137318400, "t_last_token_ns": 379509991529958, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 19, "error": null}
|
||||
{"req_id": "88e87f99a2ef460d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379502227089910, "t_first_token_ns": 379502517856625, "t_last_token_ns": 379510120928133, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 20, "error": null}
|
||||
{"req_id": "47aba0e5b42f4f49", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379502257055862, "t_first_token_ns": 379502761678617, "t_last_token_ns": 379510135431331, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "7f657028fb65458a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379503023922829, "t_first_token_ns": 379503318400226, "t_last_token_ns": 379510361570765, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 22, "error": null}
|
||||
{"req_id": "4b8702c8f76f4802", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379504496232151, "t_first_token_ns": 379504792123290, "t_last_token_ns": 379512123428490, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "31b1e0ffb92249ed", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379505554946406, "t_first_token_ns": 379505841960187, "t_last_token_ns": 379513393798279, "prompt_tokens": 3952, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "befbea4c267d40e3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379506354144337, "t_first_token_ns": 379506642622036, "t_last_token_ns": 379513818576544, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "2fc837dee4a04291", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379506761568579, "t_first_token_ns": 379507049011438, "t_last_token_ns": 379513954376771, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "54450530536b4d2f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379507291971641, "t_first_token_ns": 379507588104886, "t_last_token_ns": 379514181556610, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 21, "error": null}
|
||||
{"req_id": "d5fa07630f944b0c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379508145259539, "t_first_token_ns": 379508429769610, "t_last_token_ns": 379515280226837, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 22, "error": null}
|
||||
{"req_id": "38022ae2f4e148fa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379508318408858, "t_first_token_ns": 379508695461215, "t_last_token_ns": 379515306193495, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 23, "error": null}
|
||||
{"req_id": "c98b4affc7d9404a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379508962411756, "t_first_token_ns": 379509244738672, "t_last_token_ns": 379515573123165, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "60f880fd938d4826", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379509494794931, "t_first_token_ns": 379509777041983, "t_last_token_ns": 379515812926855, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 18, "error": null}
|
||||
{"req_id": "2ec4ca535d6c4745", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379510399516564, "t_first_token_ns": 379510673799575, "t_last_token_ns": 379516493645021, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "f431ed76f7c94271", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379510512920082, "t_first_token_ns": 379510925795533, "t_last_token_ns": 379516519912647, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "3ece7a3d08304750", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379510676165532, "t_first_token_ns": 379511168279088, "t_last_token_ns": 379516560177869, "prompt_tokens": 4026, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "fd0c3c85714e4870", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379511294965278, "t_first_token_ns": 379511566088575, "t_last_token_ns": 379516906233664, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "1d7e0644b00e471b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379512112947335, "t_first_token_ns": 379512395458094, "t_last_token_ns": 379517629287491, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "46c403f15ef54441", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379512219101360, "t_first_token_ns": 379512650533550, "t_last_token_ns": 379517652154746, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "507ad44ac2c24364", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379512565233409, "t_first_token_ns": 379512908356504, "t_last_token_ns": 379517670813677, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "046a0c5215f44a02", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379514226738241, "t_first_token_ns": 379514511542885, "t_last_token_ns": 379518584729390, "prompt_tokens": 3954, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "0acf18fcae604346", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379514377380742, "t_first_token_ns": 379514766543230, "t_last_token_ns": 379518601827779, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "89c7e16b56834885", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379514911214739, "t_first_token_ns": 379515189550719, "t_last_token_ns": 379518705855974, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "81f4c8a3f51e4b6c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379516501579164, "t_first_token_ns": 379516777751200, "t_last_token_ns": 379519287451560, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "2b0e75100f2142cb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379517254826086, "t_first_token_ns": 379517527394835, "t_last_token_ns": 379519506321628, "prompt_tokens": 4023, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "6b0a11a5cdf6400d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379519772176096, "t_first_token_ns": 379520024182943, "t_last_token_ns": 379522336101323, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "5765f84a285d4b33", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379519833771853, "t_first_token_ns": 379520262921897, "t_last_token_ns": 379522348519088, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "c866d79850f34b77", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379520569006605, "t_first_token_ns": 379520824982077, "t_last_token_ns": 379522628754332, "prompt_tokens": 3961, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "b63c38bc5a0e45b9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379523387211293, "t_first_token_ns": 379523640508482, "t_last_token_ns": 379526488873232, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "dddf1969ff2140d8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379523915074971, "t_first_token_ns": 379524173132297, "t_last_token_ns": 379526958599100, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "4ce25a3d666248ce", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379523954118805, "t_first_token_ns": 379524409151969, "t_last_token_ns": 379526966578556, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "109ca82baf12447f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379524168456945, "t_first_token_ns": 379524653243845, "t_last_token_ns": 379526978630933, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "b5a973ffabb64afc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379526080889585, "t_first_token_ns": 379526345756916, "t_last_token_ns": 379528235156479, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "11ba12a6321a48ae", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379527331818492, "t_first_token_ns": 379527588244073, "t_last_token_ns": 379529947714325, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "99081b2c5e874720", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379528226293765, "t_first_token_ns": 379528481664690, "t_last_token_ns": 379531324180086, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "6499fc508d164160", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379529108200012, "t_first_token_ns": 379529366952642, "t_last_token_ns": 379532692385361, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "913d1d2fb6824b5e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379529127087488, "t_first_token_ns": 379529602236030, "t_last_token_ns": 379532702840392, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "737fe3e1a3a74e42", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379530379038225, "t_first_token_ns": 379530639417470, "t_last_token_ns": 379534178995836, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "7728355dfec340f8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379530783064302, "t_first_token_ns": 379531045066858, "t_last_token_ns": 379534612973295, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "862f907fa2c44bbc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379531949880561, "t_first_token_ns": 379532217479007, "t_last_token_ns": 379535907391902, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "9920e5db28944ce8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379532001132801, "t_first_token_ns": 379532462600056, "t_last_token_ns": 379535926865041, "prompt_tokens": 3960, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "08c0866f5b1843b1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379532698344587, "t_first_token_ns": 379532964528073, "t_last_token_ns": 379536152051011, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "123c861bf5894706", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379533081638353, "t_first_token_ns": 379533343772183, "t_last_token_ns": 379536261652580, "prompt_tokens": 3953, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "e4e761872cf24d9d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379534247768142, "t_first_token_ns": 379534519633070, "t_last_token_ns": 379536836377052, "prompt_tokens": 4026, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "68c884e90a754e4e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379535399541610, "t_first_token_ns": 379535666645962, "t_last_token_ns": 379537304772711, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "e848fdfbf9264c59", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379537584724069, "t_first_token_ns": 379537837933651, "t_last_token_ns": 379541892659001, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "490eb6986b624820", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379538115798901, "t_first_token_ns": 379538374311012, "t_last_token_ns": 379542626198332, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "8504f80cfbbb4b2a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379538347290172, "t_first_token_ns": 379538831072720, "t_last_token_ns": 379542644885303, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "1279f01aa28e4857", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379538185193000, "t_first_token_ns": 379538830828977, "t_last_token_ns": 379542645261029, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "60c1c78e288c4843", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379538813948123, "t_first_token_ns": 379539285474350, "t_last_token_ns": 379542661837404, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "82f2ba5735984db8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379538760979055, "t_first_token_ns": 379539285813241, "t_last_token_ns": 379542662014461, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "0145107b62a2482b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379539469211528, "t_first_token_ns": 379539733778508, "t_last_token_ns": 379542817319328, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "0008169180354185", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379539886981023, "t_first_token_ns": 379540157598274, "t_last_token_ns": 379542903199280, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "774b66bc50de4523", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379543136965879, "t_first_token_ns": 379543389036747, "t_last_token_ns": 379544691705012, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "644154591ce54edf", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379544839423874, "t_first_token_ns": 379545093270573, "t_last_token_ns": 379548442763295, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "df1eb8313b904395", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379544926672241, "t_first_token_ns": 379545333278465, "t_last_token_ns": 379548462120286, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "54c3be0399d2413e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379545169024208, "t_first_token_ns": 379545787249835, "t_last_token_ns": 379548478343050, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "375a5f1bfa1149ba", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379545196596843, "t_first_token_ns": 379545787488044, "t_last_token_ns": 379548478461405, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "7a25c15edcd24001", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379547211770629, "t_first_token_ns": 379547479729028, "t_last_token_ns": 379550577466594, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "3978ff80196c41ce", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379547590167080, "t_first_token_ns": 379547858350368, "t_last_token_ns": 379550698218640, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "b4b53f096b91492e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379548811722786, "t_first_token_ns": 379549070886804, "t_last_token_ns": 379552086068654, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "c364b4fce5ec451a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379549446724311, "t_first_token_ns": 379549711759300, "t_last_token_ns": 379553061738584, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "5586b70901b342ae", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379549553459910, "t_first_token_ns": 379549955341189, "t_last_token_ns": 379553081642320, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "03418ade168f4d60", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379551578143702, "t_first_token_ns": 379551839852392, "t_last_token_ns": 379558255885487, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "6a194ef3c49f4062", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379551738983586, "t_first_token_ns": 379552086360043, "t_last_token_ns": 379558287931120, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "c71786624efd4e25", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379552145601896, "t_first_token_ns": 379552407064424, "t_last_token_ns": 379559143763860, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "4e99ca8cffa241e9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379552765039072, "t_first_token_ns": 379553034188287, "t_last_token_ns": 379559737459172, "prompt_tokens": 4014, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "ec7c9ae3309e4105", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379553246580865, "t_first_token_ns": 379553507662618, "t_last_token_ns": 379560639370733, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "607a467deb184b84", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379553350037553, "t_first_token_ns": 379554019082330, "t_last_token_ns": 379560669743128, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "c57054aff34c4f22", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379553301335032, "t_first_token_ns": 379554019336007, "t_last_token_ns": 379560670546044, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "e039d1d7c27a4c9b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379553371479151, "t_first_token_ns": 379554207923344, "t_last_token_ns": 379560684319747, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "63eb4b4636434910", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379553695098898, "t_first_token_ns": 379554661485988, "t_last_token_ns": 379560699803169, "prompt_tokens": 3998, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "757504291c874b88", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379553624626525, "t_first_token_ns": 379554661677292, "t_last_token_ns": 379560700220889, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "47cee03734c14916", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379554143719013, "t_first_token_ns": 379554900042344, "t_last_token_ns": 379560714442132, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "3432d212a0b541d6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379554649939403, "t_first_token_ns": 379555150371642, "t_last_token_ns": 379560738557392, "prompt_tokens": 3960, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "55461620b34e4360", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379555567941497, "t_first_token_ns": 379555848274850, "t_last_token_ns": 379561147846782, "prompt_tokens": 4021, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "d6bd076f9d584578", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379556126470110, "t_first_token_ns": 379556401657964, "t_last_token_ns": 379561428139317, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "4cb08867389a4680", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379556762784456, "t_first_token_ns": 379557037520773, "t_last_token_ns": 379561703151132, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "4d2b80185e5c4dda", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379558280577529, "t_first_token_ns": 379558560794537, "t_last_token_ns": 379563286548510, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "95dd6ac27c46461f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379558516376115, "t_first_token_ns": 379558816653587, "t_last_token_ns": 379563310932530, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "3a73c51726b04620", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379558834069858, "t_first_token_ns": 379559118165282, "t_last_token_ns": 379563358585348, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "2db5a8ebe63f40c9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379559995515050, "t_first_token_ns": 379560274462203, "t_last_token_ns": 379564229910543, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "9902820bb93c4eb6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379560342321965, "t_first_token_ns": 379560626648672, "t_last_token_ns": 379564313397338, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "cb85cd9a843748ff", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379562158572955, "t_first_token_ns": 379562430433606, "t_last_token_ns": 379565493907410, "prompt_tokens": 4041, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "7975f4411da744ae", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379562414091294, "t_first_token_ns": 379562676386421, "t_last_token_ns": 379565509194196, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "0dd126d9b0764a82", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379562528865413, "t_first_token_ns": 379562925337880, "t_last_token_ns": 379565521547068, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "ca4404cfda584f8e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379563949542305, "t_first_token_ns": 379564221211605, "t_last_token_ns": 379566025632257, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "8b6d27ec35bd411d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379566178328718, "t_first_token_ns": 379566428677660, "t_last_token_ns": 379568390882612, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "0ce8a0fa5cc54d1b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379566840377408, "t_first_token_ns": 379567095460148, "t_last_token_ns": 379569459673974, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "1d2dcb3c2fb941d7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379568090421527, "t_first_token_ns": 379568344819487, "t_last_token_ns": 379570964669873, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "7d34b6c3bdf4493a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379568633310259, "t_first_token_ns": 379568890366367, "t_last_token_ns": 379571348120664, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "b5475c197b9740a6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379568886349287, "t_first_token_ns": 379569147842957, "t_last_token_ns": 379571373439572, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "91810cef436846c0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379569873222818, "t_first_token_ns": 379570129049617, "t_last_token_ns": 379571872570991, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "650b1703848f4ab4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379572812286474, "t_first_token_ns": 379573066307770, "t_last_token_ns": 379575095392873, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "5cf138038a8d4397", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379573179547671, "t_first_token_ns": 379573433297013, "t_last_token_ns": 379575262292741, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "5cd500ccf1a042ae", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379574752177197, "t_first_token_ns": 379575011512239, "t_last_token_ns": 379576812685554, "prompt_tokens": 3943, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "9b671334d0074c47", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379575367732060, "t_first_token_ns": 379575625690152, "t_last_token_ns": 379577382377328, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "9f58633474514418", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379577060834647, "t_first_token_ns": 379577319062249, "t_last_token_ns": 379578633455405, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "45aa4f52aed14834", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379583967750347, "t_first_token_ns": 379584221742006, "t_last_token_ns": 379586567985546, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "2247614784fb4aa8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379584000878998, "t_first_token_ns": 379584455689500, "t_last_token_ns": 379586575118859, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "68fed35ff0de4f79", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379585636212112, "t_first_token_ns": 379585895204399, "t_last_token_ns": 379588308374626, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "0b9c6b508b914491", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379585681085707, "t_first_token_ns": 379586138457421, "t_last_token_ns": 379588558644126, "prompt_tokens": 4000, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "bad3bf1f1621423c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379587391528571, "t_first_token_ns": 379587650844761, "t_last_token_ns": 379590849086984, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "862c857e407b4bc8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379587410301505, "t_first_token_ns": 379587884463774, "t_last_token_ns": 379591062361237, "prompt_tokens": 3958, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "ca6f358addf742dc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379588248136172, "t_first_token_ns": 379588516620062, "t_last_token_ns": 379592246625741, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "dc076023898e4c6c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379588430211322, "t_first_token_ns": 379588763156754, "t_last_token_ns": 379592269252859, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "069856a3f6a14133", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379590046322293, "t_first_token_ns": 379590309236175, "t_last_token_ns": 379595807682979, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "24832cfd5c7f4f52", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379590224172576, "t_first_token_ns": 379590551191728, "t_last_token_ns": 379595834056859, "prompt_tokens": 3960, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "b6e79e7379924edc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379590791543249, "t_first_token_ns": 379591062661074, "t_last_token_ns": 379596245525789, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "597d27b329744143", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379591034214804, "t_first_token_ns": 379591307320918, "t_last_token_ns": 379596272846755, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "1f58cd1d1d844e90", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379591601477727, "t_first_token_ns": 379591869868136, "t_last_token_ns": 379596689524912, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "27cb1bc01915465f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379591674775033, "t_first_token_ns": 379592116438211, "t_last_token_ns": 379596710853653, "prompt_tokens": 3957, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "3c5d5bdf20474415", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379592792514740, "t_first_token_ns": 379593058189769, "t_last_token_ns": 379597778642399, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "a73b0dc20bcb4b43", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379593212190234, "t_first_token_ns": 379593481225065, "t_last_token_ns": 379597974792546, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "1b0c8a51e74a4325", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379593377628689, "t_first_token_ns": 379593732304651, "t_last_token_ns": 379597993834007, "prompt_tokens": 4031, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "cb7a320958c048db", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379593953363678, "t_first_token_ns": 379594221157400, "t_last_token_ns": 379598439109786, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "087e34f7f4f740bc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379594367833500, "t_first_token_ns": 379594643998267, "t_last_token_ns": 379598586626354, "prompt_tokens": 3959, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "b2732faad6794f10", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379594865371934, "t_first_token_ns": 379595141705112, "t_last_token_ns": 379598985835575, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "96ea4203e0ab4965", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379595357082508, "t_first_token_ns": 379595628413948, "t_last_token_ns": 379599132117634, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "a32491f8db604497", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379597068957620, "t_first_token_ns": 379597342644269, "t_last_token_ns": 379600268733384, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "38d66eab74d64171", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379597990413671, "t_first_token_ns": 379598248753003, "t_last_token_ns": 379600953978331, "prompt_tokens": 3952, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "e80133dfd1d547bd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379598657302073, "t_first_token_ns": 379598918642930, "t_last_token_ns": 379601558401714, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "595b5286a0d64206", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379599353374862, "t_first_token_ns": 379599610519100, "t_last_token_ns": 379602515802063, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "cbb28c4e0f7f4d36", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379600405300970, "t_first_token_ns": 379600664923643, "t_last_token_ns": 379603958833007, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "10d9f6cda9ca4d9a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379600964743123, "t_first_token_ns": 379601223054830, "t_last_token_ns": 379604386878220, "prompt_tokens": 3955, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "75d78b2fb2d341e2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379601756906759, "t_first_token_ns": 379602019064550, "t_last_token_ns": 379604972593923, "prompt_tokens": 3984, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "46bdf8cbb6f741cc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379601918266880, "t_first_token_ns": 379602258731645, "t_last_token_ns": 379604986832619, "prompt_tokens": 3963, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "5e5fc4d3ba9f4dbf", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379602674717484, "t_first_token_ns": 379602934148995, "t_last_token_ns": 379605759389266, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "44653d526a124ebc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379603611696419, "t_first_token_ns": 379603874856051, "t_last_token_ns": 379606847759988, "prompt_tokens": 3959, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "eb98824605ed492c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379605208617756, "t_first_token_ns": 379605466567250, "t_last_token_ns": 379609076683281, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "26a72eb966644118", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379605457996144, "t_first_token_ns": 379605717352176, "t_last_token_ns": 379609105506315, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "53f4b9aefda2426c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379605785728320, "t_first_token_ns": 379606044919645, "t_last_token_ns": 379609246454668, "prompt_tokens": 3975, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "bd7ac18fee534681", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379606240914819, "t_first_token_ns": 379606497127719, "t_last_token_ns": 379609487440519, "prompt_tokens": 3958, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "3c41324610784d41", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379607272656405, "t_first_token_ns": 379607534340716, "t_last_token_ns": 379610167484704, "prompt_tokens": 4023, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "aabd3ec6c88743d1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379607374125403, "t_first_token_ns": 379607778417676, "t_last_token_ns": 379610178435555, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "267cb39c75b7423b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379608665959444, "t_first_token_ns": 379608937240308, "t_last_token_ns": 379610653375922, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "38a1c46313194e5e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379612574962496, "t_first_token_ns": 379612828144236, "t_last_token_ns": 379615662650617, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "d5df566a1b8e43c5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379613324954675, "t_first_token_ns": 379613582561084, "t_last_token_ns": 379617408034078, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "7f59ba26bb1f44c3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379613668579683, "t_first_token_ns": 379613923668099, "t_last_token_ns": 379617850721629, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "6f8dada36a7841c6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379614286376321, "t_first_token_ns": 379614547585147, "t_last_token_ns": 379618510766925, "prompt_tokens": 4015, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "7affbd31b2814153", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379614447524192, "t_first_token_ns": 379614788734121, "t_last_token_ns": 379618528654350, "prompt_tokens": 3964, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "a81a75e3c8734be2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379614727800367, "t_first_token_ns": 379615032841167, "t_last_token_ns": 379618548047118, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "c7bcf67608714eb6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379616597932409, "t_first_token_ns": 379616862626363, "t_last_token_ns": 379620515672253, "prompt_tokens": 3949, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "df63fa31f8b14ff8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379616635198083, "t_first_token_ns": 379617099828396, "t_last_token_ns": 379620525208339, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "391fbc2408bd45f3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379616944955932, "t_first_token_ns": 379617345679351, "t_last_token_ns": 379620540768686, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "fe3814d660fd4764", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379617404867305, "t_first_token_ns": 379617679803160, "t_last_token_ns": 379620611563090, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "510bb4761ebc4043", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379618890806142, "t_first_token_ns": 379619150152574, "t_last_token_ns": 379621338301704, "prompt_tokens": 3953, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "eabf591da7c44cd7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379620225015022, "t_first_token_ns": 379620489824121, "t_last_token_ns": 379621892716534, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "b180702507324dec", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379622814373109, "t_first_token_ns": 379623067951651, "t_last_token_ns": 379625825334911, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "992d0b02b18945b7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379623102478951, "t_first_token_ns": 379623359238821, "t_last_token_ns": 379625951179132, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "7f988dbb21e8473a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379624344081256, "t_first_token_ns": 379624599302843, "t_last_token_ns": 379627604364548, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "c0468b4d7a404036", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379624867046225, "t_first_token_ns": 379625129818960, "t_last_token_ns": 379627939118893, "prompt_tokens": 4002, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "db17f2679d924cde", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379625188136997, "t_first_token_ns": 379625452528033, "t_last_token_ns": 379628027407026, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "9808046f376e492b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379625436711121, "t_first_token_ns": 379625709664374, "t_last_token_ns": 379628045041910, "prompt_tokens": 4005, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "ff100c87cbbc4a7a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379627142839558, "t_first_token_ns": 379627403239916, "t_last_token_ns": 379629206666196, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "6b19dccf60444521", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379628515992838, "t_first_token_ns": 379628773895363, "t_last_token_ns": 379630429148366, "prompt_tokens": 4013, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "9940f067b09f4ad6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379629501418169, "t_first_token_ns": 379629754729371, "t_last_token_ns": 379632866855183, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "ac20aa8038584a2c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379630626769624, "t_first_token_ns": 379630881023406, "t_last_token_ns": 379634719020827, "prompt_tokens": 4023, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "3c16e999f2164cdb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379630965881532, "t_first_token_ns": 379631221649690, "t_last_token_ns": 379634898131774, "prompt_tokens": 3950, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "6eda35287ce44c26", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379631015035593, "t_first_token_ns": 379631464717576, "t_last_token_ns": 379634917617099, "prompt_tokens": 4018, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "5030d39236c64d2f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379631537534094, "t_first_token_ns": 379631801091065, "t_last_token_ns": 379635028252625, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "4126db0db58541fc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379631895635030, "t_first_token_ns": 379632162330298, "t_last_token_ns": 379635130810145, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "d6d485fd79594628", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379632295024167, "t_first_token_ns": 379632560506195, "t_last_token_ns": 379635236864434, "prompt_tokens": 3963, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "ced0095294e54b47", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379633751674013, "t_first_token_ns": 379634014022603, "t_last_token_ns": 379636765032568, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "7460344293c14435", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379635429648478, "t_first_token_ns": 379635690026120, "t_last_token_ns": 379639160354160, "prompt_tokens": 4027, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "8b3f1124a6ac47d5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379635629272768, "t_first_token_ns": 379635929489341, "t_last_token_ns": 379639179028564, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "a254f74b25a249bc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379635755277396, "t_first_token_ns": 379636171619559, "t_last_token_ns": 379639199262504, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "e6f48c78d4c740e6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379636910459062, "t_first_token_ns": 379637171092055, "t_last_token_ns": 379640018180489, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "be97c02b04954105", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379637812053955, "t_first_token_ns": 379638071138647, "t_last_token_ns": 379640661831554, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "29c579f7a36b49b2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379637979067358, "t_first_token_ns": 379638316538821, "t_last_token_ns": 379640673186431, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "21403c131f484395", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379638657202061, "t_first_token_ns": 379638922897628, "t_last_token_ns": 379640863982183, "prompt_tokens": 4023, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "35f29e999be446ea", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379641161825991, "t_first_token_ns": 379641415785398, "t_last_token_ns": 379644702471571, "prompt_tokens": 4022, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "e557afd73ede4a53", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379641458643184, "t_first_token_ns": 379641717206953, "t_last_token_ns": 379644845759420, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "d67cdc232f7b4648", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379642389077100, "t_first_token_ns": 379642641239760, "t_last_token_ns": 379646191673166, "prompt_tokens": 3936, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "0c47f39f614a4f4f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379642405843000, "t_first_token_ns": 379642875383548, "t_last_token_ns": 379646201280269, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "2746f157973c448a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379642475836831, "t_first_token_ns": 379643111201088, "t_last_token_ns": 379646209047039, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "58ff2929a8f94380", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379643209815074, "t_first_token_ns": 379643469774181, "t_last_token_ns": 379646309993042, "prompt_tokens": 3962, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "e0ce25437c08484b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379644345872216, "t_first_token_ns": 379644610688963, "t_last_token_ns": 379646879900542, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "61c1bb62caea41b0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379645235992298, "t_first_token_ns": 379645500106428, "t_last_token_ns": 379647919441753, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "ffdef3b7719345fc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379647106089688, "t_first_token_ns": 379647364886747, "t_last_token_ns": 379651116245668, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "5aa75870fd914059", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379647133988428, "t_first_token_ns": 379647598990029, "t_last_token_ns": 379651125104572, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "2a66cfc1a2e543fb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379647314137958, "t_first_token_ns": 379647834733677, "t_last_token_ns": 379651135201276, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "d134a23a101a41ed", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379648031013239, "t_first_token_ns": 379648292761945, "t_last_token_ns": 379651389810090, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "83c7433a2dff434c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379648781164202, "t_first_token_ns": 379649041950926, "t_last_token_ns": 379652127434871, "prompt_tokens": 4012, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "bda5004819b84c5f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379649508073351, "t_first_token_ns": 379649777948249, "t_last_token_ns": 379652768031250, "prompt_tokens": 3997, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "b7c791ab1b00460a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379650108817605, "t_first_token_ns": 379650379018878, "t_last_token_ns": 379653041368685, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "c9533de9962c453d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379651672417002, "t_first_token_ns": 379651929219747, "t_last_token_ns": 379654363554947, "prompt_tokens": 3974, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "5bc6e930e1cb4dae", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379652344953368, "t_first_token_ns": 379652608756331, "t_last_token_ns": 379654793710582, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "e13a46efb31a4d1e", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379653593497823, "t_first_token_ns": 379653848847539, "t_last_token_ns": 379655740648098, "prompt_tokens": 3953, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "17a1681bdb1d46d1", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379653911743371, "t_first_token_ns": 379654166246079, "t_last_token_ns": 379655801886381, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "64416b6d21904a00", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379656651319510, "t_first_token_ns": 379656902545649, "t_last_token_ns": 379661426503340, "prompt_tokens": 3982, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "0093d2829fdb48e5", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379656871520926, "t_first_token_ns": 379657139631389, "t_last_token_ns": 379661446548801, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "212b1d0cce0140f4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379657293250181, "t_first_token_ns": 379657549834823, "t_last_token_ns": 379662033194688, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "d58f17523900470c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379657301169968, "t_first_token_ns": 379657787326474, "t_last_token_ns": 379662044517130, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "f0a5c306e6674eb8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379657746551243, "t_first_token_ns": 379658026151863, "t_last_token_ns": 379662067693696, "prompt_tokens": 3951, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "1c017ab857944ccc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379658723520966, "t_first_token_ns": 379658988595357, "t_last_token_ns": 379663024840000, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "bab0abee21bf4021", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379659661706502, "t_first_token_ns": 379659930600427, "t_last_token_ns": 379664225404434, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "b04f2cb2da2d4c99", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379659729094895, "t_first_token_ns": 379660178063089, "t_last_token_ns": 379664248723301, "prompt_tokens": 4004, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "2d45d86eaaaf4933", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379660499993696, "t_first_token_ns": 379660767517517, "t_last_token_ns": 379665250407880, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "322ae0a4c4744b40", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379661150562814, "t_first_token_ns": 379661426838125, "t_last_token_ns": 379665658636556, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "7817ed4da9c14c65", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379661665701984, "t_first_token_ns": 379661932277565, "t_last_token_ns": 379665949163766, "prompt_tokens": 3990, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "4c86675b76d24733", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379662689381385, "t_first_token_ns": 379662958380201, "t_last_token_ns": 379666732751188, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "0ba3b34d01514ca3", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379663405866280, "t_first_token_ns": 379663669588834, "t_last_token_ns": 379667223964978, "prompt_tokens": 4025, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "f8a5956f6afd4ae8", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379663734542490, "t_first_token_ns": 379664001710257, "t_last_token_ns": 379667300762333, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "d4d468270f694d20", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379664390778163, "t_first_token_ns": 379664662928889, "t_last_token_ns": 379667601961580, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "171c6d39abd742ea", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379664617485434, "t_first_token_ns": 379665125815764, "t_last_token_ns": 379667613288679, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "8ee3fdbbcd824327", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379664507637964, "t_first_token_ns": 379665126073807, "t_last_token_ns": 379667613420476, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "558bcc65469f430d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379668118576874, "t_first_token_ns": 379668370859389, "t_last_token_ns": 379671422445305, "prompt_tokens": 3999, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "3b4811eb1ba54a08", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379668739357240, "t_first_token_ns": 379668995158632, "t_last_token_ns": 379672453805104, "prompt_tokens": 3969, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "b69ea9e7a0d44502", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379668746599872, "t_first_token_ns": 379669228892942, "t_last_token_ns": 379672463641403, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "c38486a77ea94a16", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379669595331493, "t_first_token_ns": 379669862302634, "t_last_token_ns": 379672913336914, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "3b808dc6a48249a6", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379669642952245, "t_first_token_ns": 379670098723018, "t_last_token_ns": 379672920066586, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "112ae2ad627b4c06", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379669927574224, "t_first_token_ns": 379670339410927, "t_last_token_ns": 379672931744099, "prompt_tokens": 3943, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "df01cd979bfd42b7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379671809990345, "t_first_token_ns": 379672079240554, "t_last_token_ns": 379673676914587, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "a26817959de94c4f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379674536377484, "t_first_token_ns": 379674786925496, "t_last_token_ns": 379679161112979, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "0822877681c141bd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379674556568022, "t_first_token_ns": 379675020250200, "t_last_token_ns": 379679172577726, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "a54743f06c5e4fac", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379674718324143, "t_first_token_ns": 379675255088968, "t_last_token_ns": 379679183924146, "prompt_tokens": 4007, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "f5b9e3492d9a4bb2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379675555472408, "t_first_token_ns": 379675814290961, "t_last_token_ns": 379679625694115, "prompt_tokens": 3972, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "0d4358172e014565", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379675879343418, "t_first_token_ns": 379676158425580, "t_last_token_ns": 379679775359005, "prompt_tokens": 3983, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "00fd919c368342a9", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379676025020782, "t_first_token_ns": 379676615710071, "t_last_token_ns": 379679792451185, "prompt_tokens": 3980, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "daa44dba56d3435b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379676051507321, "t_first_token_ns": 379676615949476, "t_last_token_ns": 379679792694621, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "1cbb046f52274829", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379676317988072, "t_first_token_ns": 379676859649496, "t_last_token_ns": 379679804006206, "prompt_tokens": 3960, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "ecfaa55a07dd40c0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379677998785767, "t_first_token_ns": 379678272245187, "t_last_token_ns": 379680638243396, "prompt_tokens": 3995, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "c49520558b6b4bfa", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379679918436145, "t_first_token_ns": 379680172972467, "t_last_token_ns": 379682227711381, "prompt_tokens": 3978, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "f4b74dde72c7473a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379681338349666, "t_first_token_ns": 379681598721621, "t_last_token_ns": 379684464406246, "prompt_tokens": 4021, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "6ddbe3dee61c4efb", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379681576057905, "t_first_token_ns": 379681836864845, "t_last_token_ns": 379684479613553, "prompt_tokens": 3967, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "3bf26449e1ff4c14", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379681835854301, "t_first_token_ns": 379682092550272, "t_last_token_ns": 379684507755258, "prompt_tokens": 3981, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "5ff5a36e1e50456b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379683266145167, "t_first_token_ns": 379683523064586, "t_last_token_ns": 379689182251706, "prompt_tokens": 3958, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "eb3b1526f7114d0d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379683951544588, "t_first_token_ns": 379684214827445, "t_last_token_ns": 379690217551634, "prompt_tokens": 4017, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "eaa7b0baeff44765", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379684934038789, "t_first_token_ns": 379685194475350, "t_last_token_ns": 379691798703518, "prompt_tokens": 4009, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "8ca3b4b916914b15", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379685023895747, "t_first_token_ns": 379685648461954, "t_last_token_ns": 379691828776149, "prompt_tokens": 4011, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "5ac16bb9d5384d3c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379685004280987, "t_first_token_ns": 379685648638533, "t_last_token_ns": 379691829336236, "prompt_tokens": 3965, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "bb574629d5de4f2b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379685403964652, "t_first_token_ns": 379685891882657, "t_last_token_ns": 379691857448795, "prompt_tokens": 3989, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "c353d44f7f094adc", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379686431160202, "t_first_token_ns": 379686698381314, "t_last_token_ns": 379693023419621, "prompt_tokens": 4024, "completion_tokens": 256, "inflight_at_send": 7, "error": null}
|
||||
{"req_id": "113e167162704059", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379686472163740, "t_first_token_ns": 379686930207141, "t_last_token_ns": 379693037141795, "prompt_tokens": 3943, "completion_tokens": 256, "inflight_at_send": 8, "error": null}
|
||||
{"req_id": "e06db79ab9294196", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379686631108902, "t_first_token_ns": 379687168719178, "t_last_token_ns": 379693052824089, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 9, "error": null}
|
||||
{"req_id": "5e513f7e2fdf4d7d", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379687087260958, "t_first_token_ns": 379687418141101, "t_last_token_ns": 379693081993000, "prompt_tokens": 3985, "completion_tokens": 256, "inflight_at_send": 10, "error": null}
|
||||
{"req_id": "27410b63d43e45bd", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379687287673909, "t_first_token_ns": 379687881615353, "t_last_token_ns": 379693106341686, "prompt_tokens": 3976, "completion_tokens": 256, "inflight_at_send": 12, "error": null}
|
||||
{"req_id": "64fcefe5c96c4a47", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379687249315149, "t_first_token_ns": 379687881778163, "t_last_token_ns": 379693106553644, "prompt_tokens": 3994, "completion_tokens": 256, "inflight_at_send": 11, "error": null}
|
||||
{"req_id": "b85c9e990eb3449b", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379687758662679, "t_first_token_ns": 379688347761813, "t_last_token_ns": 379693127795472, "prompt_tokens": 3996, "completion_tokens": 256, "inflight_at_send": 14, "error": null}
|
||||
{"req_id": "e7c369e6f54a4568", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379687641712136, "t_first_token_ns": 379688348024852, "t_last_token_ns": 379693128026421, "prompt_tokens": 3992, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "8d43d7d91766402c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379688335779422, "t_first_token_ns": 379688612570332, "t_last_token_ns": 379693158043232, "prompt_tokens": 3973, "completion_tokens": 256, "inflight_at_send": 15, "error": null}
|
||||
{"req_id": "b1f3f68e54584ba4", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379688554582205, "t_first_token_ns": 379688867097628, "t_last_token_ns": 379693174224501, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 16, "error": null}
|
||||
{"req_id": "5228d8e364954cc0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379688657015553, "t_first_token_ns": 379689123204175, "t_last_token_ns": 379693188870431, "prompt_tokens": 4017, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "be2a16f5f0a64891", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379689948485195, "t_first_token_ns": 379690217823683, "t_last_token_ns": 379693564342143, "prompt_tokens": 3936, "completion_tokens": 256, "inflight_at_send": 17, "error": null}
|
||||
{"req_id": "4f216681a7804320", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379691910364919, "t_first_token_ns": 379692192211237, "t_last_token_ns": 379694409379205, "prompt_tokens": 3979, "completion_tokens": 256, "inflight_at_send": 13, "error": null}
|
||||
{"req_id": "c2a70627e0884742", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379693946201828, "t_first_token_ns": 379694201344098, "t_last_token_ns": 379695697673474, "prompt_tokens": 3986, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "1b6802e3463a4771", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379695361806786, "t_first_token_ns": 379695611757024, "t_last_token_ns": 379697461561232, "prompt_tokens": 3941, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "b7169bff604c4e5c", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379696486667160, "t_first_token_ns": 379696740389351, "t_last_token_ns": 379698550684329, "prompt_tokens": 3977, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "4d1de1dd8b9e445a", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379696534944671, "t_first_token_ns": 379696979743101, "t_last_token_ns": 379698559580532, "prompt_tokens": 3988, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "f473edb25a874574", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379698930568955, "t_first_token_ns": 379699181590969, "t_last_token_ns": 379700951650930, "prompt_tokens": 3993, "completion_tokens": 256, "inflight_at_send": 1, "error": null}
|
||||
{"req_id": "a235653bb03f4713", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379700184510706, "t_first_token_ns": 379700439813643, "t_last_token_ns": 379703609479358, "prompt_tokens": 3987, "completion_tokens": 256, "inflight_at_send": 2, "error": null}
|
||||
{"req_id": "f5242d0eb7894fc2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379700428566138, "t_first_token_ns": 379700687621759, "t_last_token_ns": 379703638469515, "prompt_tokens": 4016, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "764fb3dd3a9e4be0", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379701339990703, "t_first_token_ns": 379701598362366, "t_last_token_ns": 379704830128093, "prompt_tokens": 3970, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "478187bcfe88446f", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379701507227110, "t_first_token_ns": 379701839373333, "t_last_token_ns": 379704845717148, "prompt_tokens": 3968, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "9226fb9aa0294377", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379702023778768, "t_first_token_ns": 379702289425177, "t_last_token_ns": 379705052305807, "prompt_tokens": 3991, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "3b35543b4f4043d2", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379702589776796, "t_first_token_ns": 379702855939254, "t_last_token_ns": 379705739841513, "prompt_tokens": 3949, "completion_tokens": 256, "inflight_at_send": 6, "error": null}
|
||||
{"req_id": "023d925744904325", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379703799815224, "t_first_token_ns": 379704061815767, "t_last_token_ns": 379706759948025, "prompt_tokens": 4001, "completion_tokens": 256, "inflight_at_send": 5, "error": null}
|
||||
{"req_id": "5232e2d3624441c7", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379705177179151, "t_first_token_ns": 379705435363422, "t_last_token_ns": 379707742041795, "prompt_tokens": 4003, "completion_tokens": 256, "inflight_at_send": 3, "error": null}
|
||||
{"req_id": "acb02b4ac4304bea", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379705216456244, "t_first_token_ns": 379705671402352, "t_last_token_ns": 379707747785699, "prompt_tokens": 4006, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
{"req_id": "496d446698494b69", "rate_target": 1.5, "input_tokens_target": 4096, "output_tokens_target": 256, "t_send_ns": 379706425342294, "t_first_token_ns": 379706684134891, "t_last_token_ns": 379708263527492, "prompt_tokens": 3966, "completion_tokens": 256, "inflight_at_send": 4, "error": null}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"rate": 1.5,
|
||||
"input_tokens": 4096,
|
||||
"output_tokens": 256,
|
||||
"duration_target_s": 240.0,
|
||||
"duration_actual_s": 241.0230018220027,
|
||||
"n_requests": 322
|
||||
}
|
||||
133
microbench/connector_tax/cache_sweep/run_drfix.sh
Executable file
133
microbench/connector_tax/cache_sweep/run_drfix.sh
Executable file
@@ -0,0 +1,133 @@
|
||||
#!/bin/bash
|
||||
# A/B re-measurement after the direct-RDMA-read hash-sync env-gate.
|
||||
#
|
||||
# Applies all instrumentation (v1+v2) AND the CT_DR_FIX patch, then runs
|
||||
# the same workload as run_all.sh on:
|
||||
# plain (control)
|
||||
# mooncake_both (baseline: env not set → original behaviour)
|
||||
# mooncake_both_drfix (with VLLM_MOONCAKE_DISABLE_DIRECT_READ_SYNC=1)
|
||||
# Same vLLM lifetime per config; full apply→run→revert cycle at the end.
|
||||
#
|
||||
# Usage: bash run_drfix.sh
|
||||
# Env overrides: DURATION (default 240), RATE (default 1.5), CONFIGS (override list)
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
CT_DIR="$(cd "$HERE/.." && pwd)"
|
||||
PROJ_DIR="$(cd "$HERE/../../.." && pwd)"
|
||||
PYTHON="${PYTHON:-$PROJ_DIR/.venv/bin/python}"
|
||||
VLLM_ROOT="${VLLM_ROOT:-$PROJ_DIR/.venv/lib/python3.12/site-packages/vllm}"
|
||||
|
||||
DURATION="${DURATION:-240}"
|
||||
RATE="${RATE:-1.5}"
|
||||
PORT="${PORT:-8000}"
|
||||
GPU_ID="${GPU_ID:-0}"
|
||||
MODEL_PATH="${MODEL_PATH:-$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
|
||||
CONFIGS="${CONFIGS:-plain mooncake_both mooncake_both_drfix}"
|
||||
|
||||
DATE="$(date +%Y%m%d_%H%M)"
|
||||
RUN_ROOT="$HERE/results/${DATE}_drfix"
|
||||
mkdir -p "$RUN_ROOT"
|
||||
|
||||
echo "=== Cache-size sweep + DR-fix A/B ==="
|
||||
echo "Run dir : $RUN_ROOT"
|
||||
echo "Configs : $CONFIGS"
|
||||
echo "Rate : $RATE Duration: ${DURATION}s"
|
||||
echo ""
|
||||
|
||||
kill_all_vllm() {
|
||||
pkill -9 -f "VLLM::EngineCore" 2>/dev/null || true
|
||||
pkill -9 -f "vllm.entrypoints" 2>/dev/null || true
|
||||
pkill -9 -f "vllm serve" 2>/dev/null || true
|
||||
sleep 4
|
||||
for _ in $(seq 1 20); do
|
||||
used=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits -i "$GPU_ID" 2>/dev/null | tr -d ' ')
|
||||
[[ -n "$used" && "$used" -lt 1000 ]] && return 0
|
||||
sleep 3
|
||||
done
|
||||
echo "WARN: GPU $GPU_ID not free after kill" >&2
|
||||
}
|
||||
|
||||
cleanup_patches() {
|
||||
"$PYTHON" "$HERE/apply_direct_read_fix.py" --revert --vllm-root "$VLLM_ROOT" 2>/dev/null || true
|
||||
"$PYTHON" "$HERE/apply_step_timing_v2.py" --revert --vllm-root "$VLLM_ROOT" 2>/dev/null || true
|
||||
[[ -f "$CT_DIR/patches/apply_step_timing.py" ]] && \
|
||||
"$PYTHON" "$CT_DIR/patches/apply_step_timing.py" --revert --vllm-root "$VLLM_ROOT" 2>/dev/null || true
|
||||
}
|
||||
|
||||
trap 'kill_all_vllm; cleanup_patches' EXIT
|
||||
|
||||
echo "[stage 0] applying v1 + v2 + DR_FIX patches"
|
||||
"$PYTHON" "$CT_DIR/patches/apply_step_timing.py" --apply --vllm-root "$VLLM_ROOT"
|
||||
"$PYTHON" "$HERE/apply_step_timing_v2.py" --apply --vllm-root "$VLLM_ROOT"
|
||||
"$PYTHON" "$HERE/apply_direct_read_fix.py" --apply --vllm-root "$VLLM_ROOT"
|
||||
|
||||
kill_all_vllm
|
||||
for cfg in $CONFIGS; do
|
||||
cfg_dir="$RUN_ROOT/$cfg"
|
||||
mkdir -p "$cfg_dir"
|
||||
|
||||
# Pick the launch script. mooncake_both_drfix reuses mooncake_both's launcher.
|
||||
case "$cfg" in
|
||||
mooncake_both_drfix) launch_cfg="mooncake_both" ;;
|
||||
*) launch_cfg="$cfg" ;;
|
||||
esac
|
||||
|
||||
launch_script="$CT_DIR/launch/launch_${launch_cfg}.sh"
|
||||
if [[ ! -f "$launch_script" ]]; then
|
||||
echo "SKIP $cfg (no launch script at $launch_script)"
|
||||
continue
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "====== Config: $cfg (launcher=$launch_cfg) ======"
|
||||
|
||||
export RUN_DIR="$cfg_dir"
|
||||
export PORT GPU_ID MODEL_PATH
|
||||
export AGENTIC_STEP_LOG_PATH="$cfg_dir/engine_step.jsonl"
|
||||
export CT_WORKER_STEP_LOG_PATH="$cfg_dir/worker_step.jsonl"
|
||||
export PYTHONPATH="$PROJ_DIR:${PYTHONPATH:-}"
|
||||
|
||||
# The env-gated skip
|
||||
if [[ "$cfg" == "mooncake_both_drfix" ]]; then
|
||||
export VLLM_MOONCAKE_DISABLE_DIRECT_READ_SYNC=1
|
||||
echo " VLLM_MOONCAKE_DISABLE_DIRECT_READ_SYNC=1 (hash sync skipped)"
|
||||
else
|
||||
unset VLLM_MOONCAKE_DISABLE_DIRECT_READ_SYNC
|
||||
fi
|
||||
|
||||
: > "$cfg_dir/engine_step.jsonl"
|
||||
rm -f "$cfg_dir/worker_step.jsonl".*
|
||||
: > "$cfg_dir/requests.jsonl"
|
||||
|
||||
bash "$launch_script" 2>&1 | tail -5
|
||||
rc=$?
|
||||
if [[ $rc -ne 0 ]]; then
|
||||
echo "FAIL $cfg (launch rc=$rc) — skipping bench"
|
||||
kill_all_vllm
|
||||
continue
|
||||
fi
|
||||
|
||||
echo "[bench] running ${DURATION}s open-loop at rate=$RATE"
|
||||
"$PYTHON" "$HERE/run_cache_sweep.py" \
|
||||
--url "http://127.0.0.1:$PORT/v1/chat/completions" \
|
||||
--model "$MODEL_PATH" \
|
||||
--rate "$RATE" --duration "$DURATION" \
|
||||
--output-dir "$cfg_dir" 2>&1 | tail -8
|
||||
|
||||
curl -s "http://127.0.0.1:$PORT/metrics" > "$cfg_dir/metrics_final.txt" 2>&1 || true
|
||||
|
||||
echo "[teardown] $cfg"
|
||||
kill_all_vllm
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "[stage Z] reverting patches"
|
||||
cleanup_patches
|
||||
|
||||
echo ""
|
||||
echo "[analyze]"
|
||||
"$PYTHON" "$HERE/analyze.py" --run-root "$RUN_ROOT"
|
||||
echo ""
|
||||
echo "Done. Artifacts: $RUN_ROOT"
|
||||
Reference in New Issue
Block a user