gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
9ddabee6aecd50ce88636799edbdf52908de7516
agentic-kvc/analysis/characterization/elastic_migration_v2/data/breakdown_unified_v2.json
Gahow Wang d76eb02637 Elastic migration v2 section: PD-sep on agentic workload is net negative 
New analysis/characterization/elastic_migration_v2/ packages the
unified_v2 + unified_kv_both experiments into a self-contained
results section that the paper can cite as the "we tried selective
PD-sep migration" case study. The section finds three independent
reasons PD-sep doesn't help on agentic w600:

1. Mooncake kv_both substrate alone (no PD-sep ever firing) imposes
   TTFT p90 +45%, TPOT p90 +25%, hotspot index +19% vs plain
   unified. Per-step KVConnectorMetadata maintenance and block
   reservation semantics dominate even when no transfer is pending.
2. PD-sep gate fires only 0.16-0.41% of requests across two
   gate-tightness configurations. 88-76% are killed by
   new_local < threshold because 93% intra-session reuse on agentic
   traces leaves a small uncached tail; 19% are killed by
   chosen_no_active_decode (snapshot-time gate). Even relaxed
   thresholds can't grow trigger rate past 0.5%.
3. When PD-sep fires, the calibrated cost model
   (0.3s + bytes / 2.7 GB/s) is wrong by 10-20x. 5 triggered
   requests in v2.1 saw realized TTFT 12-45s vs model-predicted
   migrate cost 0.7-2.2s, consistent with the E2 audit's finding
   that D-side block pre-reservation and missing layerwise
   pipelining dominate the decode_sent -> first_token clock.

Three-way comparison (unified vs unified_kv_both vs unified_v2):
v2 vs the kv_both control is roughly net-zero (-10% hotspot,
-14% TPOT p90, +3% TTFT p90, +9% TTFT p99). v2 vs plain unified is
strictly worse by 27-49% across latency percentiles because the
kv_both substrate tax is unavoidable when the policy is enabled.

Contents:
- README.md: the four results sections, the three-way comparison
  table, an explicit "what this claims for the paper" list, and a
  cross-reference index to the earlier characterization documents.
- data/: b3_policy_comparison.json + per-policy breakdown.json
  + per-policy hotspot_index.json for the four policies in scope.
- figures/: 4 PNGs rendered by render_figures.py:
  * fig_kv_both_overhead.png   — 4-metric bar chart with delta
    annotations showing kv_both alone costs +45% TTFT p90.
  * fig_v2_trigger_funnel.png  — per-reason request count for the
    two gate configurations on log scale.
  * fig_v2_predicted_vs_actual.png  — scatter of model-predicted
    migrate cost vs realized TTFT for the 5 triggered requests,
    with y=x, 10x, and 20x reference lines.
  * fig_three_way_hotspot.png  — per-worker TTFT p90 grouped bars
    across the three policies.

The section is intentionally self-contained: it lists what the
experiment validates (cost model picks correct candidates;
shadow-drift fix is necessary; same-worker interference is real)
alongside what it disproves (per-request PD-sep on agentic via
Mooncake is not a net win in current implementation).

Refs: E1/E2 subagent audits, B2 microbench, unified_v2 commits
19f69a9 / 4b833d3 / 95c8ef8.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 13:28:37 +08:00

1 line
3.3 MiB
Raw Blame History

[{"request_id":"1237198:1:1237198:0","session_id":"1237198","input_length":8228,"t_proxy_recv":338667.10043106,"t_decision_unix":1779769532.114372,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8228,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338667.971445523,"t_first_token_unix":1779769532.985389,"t_done":338668.06685661,"t_done_unix":1779769533.0808012},{"request_id":"1237337:1:1237337:1","session_id":"1237337","input_length":3976,"t_proxy_recv":338667.519741965,"t_decision_unix":1779769532.533683,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3976 < 8000)","cache_hit":0,"estimated_new_tokens":3976,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":8228,"ongoing_decode_tokens":0,"pending_prefill_tokens":8228,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":8228.0,"score_lmetric":12204},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338668.046299937,"t_first_token_unix":1779769533.0602436,"t_done":338668.438404829,"t_done_unix":1779769533.4523494},{"request_id":"1237198:2:1237773:2","session_id":"1237198","input_length":10237,"t_proxy_recv":338669.069647319,"t_decision_unix":1779769534.0835886,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":8192,"affinity_cache_ratio":0.8002344436846732,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2045 < 8000)","cache_hit":8192,"estimated_new_tokens":2045,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":16,"cache_hit":8192,"new_prefill":2045,"score_linear":-8192.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":7,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":0,"t_first_token":338669.36577065,"t_first_token_unix":1779769534.3797143,"t_done":338669.500730136,"t_done_unix":1779769534.5146735},{"request_id":"1237877:1:1237877:3","session_id":"1237877","input_length":749,"t_proxy_recv":338669.52615391,"t_decision_unix":1779769534.5400956,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (749 < 8000)","cache_hit":0,"estimated_new_tokens":749,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":7,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338669.645184097,"t_first_token_unix":1779769534.6591275,"t_done":338669.773664965,"t_done_unix":1779769534.7876081},{"request_id":"1239034:1:1239034:4","session_id":"1239034","input_length":13489,"t_proxy_recv":338673.304630262,"t_decision_unix":1779769538.3185716,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":13489,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":8,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338674.788824342,"t_first_token_unix":1779769539.8027682,"t_done":338675.243570312,"t_done_unix":1779769540.2575135},{"request_id":"1237877:2:1239577:6","session_id":"1237877","input_length":1620,"t_proxy_recv":338675.188169765,"t_decision_unix":1779769540.2021112,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":512,"affinity_cache_ratio":0.3160493827160494,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1108 < 8000)","cache_hit":512,"estimated_new_tokens":1108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":8,"cache_hit":512,"new_prefill":1108,"score_linear":-512.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13489,"ongoing_decode_tokens":13489,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":13489.0,"score_lmetric":1620},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":37165.0,"score_lmetric":38785},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":338675.267612011,"t_first_token_unix":1779769540.2815557,"t_done":338675.719843724,"t_done_unix":1779769540.7337868},{"request_id":"1239704:1:1239704:7","session_id":"1239704","input_length":5194,"t_proxy_recv":338675.631392948,"t_decision_unix":1779769540.6453345,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5194 < 8000)","cache_hit":0,"estimated_new_tokens":5194,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1620,"ongoing_decode_tokens":1620,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":8,"cache_hit":0,"new_prefill":5194,"score_linear":1620.0,"score_lmetric":5194},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":37165.0,"score_lmetric":42359},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338676.242909951,"t_first_token_unix":1779769541.2568538,"t_done":338676.516126698,"t_done_unix":1779769541.5300698},{"request_id":"1240924:1:1240924:11","session_id":"1240924","input_length":7429,"t_proxy_recv":338679.651918984,"t_decision_unix":1779769544.6658607,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7429 < 8000)","cache_hit":0,"estimated_new_tokens":7429,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":66214.0,"score_lmetric":73643},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":30040,"ongoing_decode_tokens":0,"pending_prefill_tokens":30040,"num_requests":1,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":7429,"score_linear":30040.0,"score_lmetric":37469},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":7429,"score_linear":18420.0,"score_lmetric":7429},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":37165,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":37165.0,"score_lmetric":7429},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338680.194099124,"t_first_token_unix":1779769545.208043,"t_done":338680.367320351,"t_done_unix":1779769545.3812637},{"request_id":"1239100:1:1239100:5","session_id":"1239100","input_length":37165,"t_proxy_recv":338673.545980285,"t_decision_unix":1779769538.5599217,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":37165,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":8,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13489,"ongoing_decode_tokens":0,"pending_prefill_tokens":13489,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":13489.0,"score_lmetric":50654},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338679.505930021,"t_first_token_unix":1779769544.5198736,"t_done":338680.766974681,"t_done_unix":1779769545.7809181},{"request_id":"1240198:1:1240198:9","session_id":"1240198","input_length":30040,"t_proxy_recv":338677.166266208,"t_decision_unix":1779769542.1802077,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":30040,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":66214.0,"score_lmetric":96254},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":37165.0,"score_lmetric":67205},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338681.091025271,"t_first_token_unix":1779769546.104969,"t_done":338681.741733429,"t_done_unix":1779769546.7556767},{"request_id":"1241953:1:1241953:13","session_id":"1241953","input_length":1093,"t_proxy_recv":338683.06896983,"t_decision_unix":1779769548.0829113,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1093 < 8000)","cache_hit":0,"estimated_new_tokens":1093,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":34,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1093,"score_linear":66214.0,"score_lmetric":67307},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":1093,"score_linear":18420.0,"score_lmetric":1093},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7587,"ongoing_decode_tokens":0,"pending_prefill_tokens":7587,"num_requests":1,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":1093,"score_linear":7587.0,"score_lmetric":8680},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338683.181112882,"t_first_token_unix":1779769548.1950562,"t_done":338683.70550711,"t_done_unix":1779769548.7194538},{"request_id":"1241942:1:1241942:12","session_id":"1241942","input_length":7587,"t_proxy_recv":338683.023710922,"t_decision_unix":1779769548.0376523,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7587 < 8000)","cache_hit":0,"estimated_new_tokens":7587,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":34,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7587,"score_linear":66214.0,"score_lmetric":73801},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":7587,"score_linear":18420.0,"score_lmetric":7587},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338683.553314452,"t_first_token_unix":1779769548.5672581,"t_done":338683.745105717,"t_done_unix":1779769548.7590487},{"request_id":"1237877:3:1242953:15","session_id":"1237877","input_length":2659,"t_proxy_recv":338686.455737518,"t_decision_unix":1779769551.4696786,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":1536,"affinity_cache_ratio":0.5776607747273411,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1123 < 8000)","cache_hit":1536,"estimated_new_tokens":1123,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17157,"ongoing_decode_tokens":0,"pending_prefill_tokens":17157,"num_requests":1,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":2659,"score_linear":17157.0,"score_lmetric":19816},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":2659,"score_linear":66214.0,"score_lmetric":68873},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":1536,"new_prefill":1123,"score_linear":-1536.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":2659,"score_linear":18420.0,"score_lmetric":2659},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":338686.548476539,"t_first_token_unix":1779769551.5624197,"t_done":338686.61324635,"t_done_unix":1779769551.6271892},{"request_id":"1242838:1:1242838:14","session_id":"1242838","input_length":17157,"t_proxy_recv":338686.076013999,"t_decision_unix":1779769551.0899553,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":17157,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17157,"score_linear":66214.0,"score_lmetric":83371},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":17157,"score_linear":18420.0,"score_lmetric":17157},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338687.771314337,"t_first_token_unix":1779769552.7852578,"t_done":338688.679099142,"t_done_unix":1779769553.6930428},{"request_id":"1239704:2:1243943:17","session_id":"1239704","input_length":14812,"t_proxy_recv":338689.733988092,"t_decision_unix":1779769554.7479293,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5120,"affinity_cache_ratio":0.3456656764785309,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":5120,"estimated_new_tokens":9692,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":43,"cache_hit":5120,"new_prefill":9692,"score_linear":-5120.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":11074,"ongoing_decode_tokens":0,"pending_prefill_tokens":11074,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14812,"score_linear":11074.0,"score_lmetric":25886},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":14812,"score_linear":66214.0,"score_lmetric":81026},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":14812,"score_linear":18420.0,"score_lmetric":14812},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-5120.0,"chosen_score_lmetric":0,"t_first_token":338690.81467319,"t_first_token_unix":1779769555.8286178,"t_done":338690.932570273,"t_done_unix":1779769555.946514},{"request_id":"1243831:1:1243831:16","session_id":"1243831","input_length":11074,"t_proxy_recv":338689.332088267,"t_decision_unix":1779769554.3460298,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11074,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":43,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11074,"score_linear":66214.0,"score_lmetric":77288},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":11074,"score_linear":18420.0,"score_lmetric":11074},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338690.272657809,"t_first_token_unix":1779769555.2866013,"t_done":338691.750177029,"t_done_unix":1779769556.76412},{"request_id":"1239755:1:1239755:8","session_id":"1239755","input_length":66214,"t_proxy_recv":338675.840820589,"t_decision_unix":1779769540.854762,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":66214,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5194,"ongoing_decode_tokens":0,"pending_prefill_tokens":5194,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":5194.0,"score_lmetric":71408},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":37165.0,"score_lmetric":103379},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338691.693994211,"t_first_token_unix":1779769556.7079377,"t_done":338692.387445818,"t_done_unix":1779769557.4013896},{"request_id":"1239034:2:1240558:10","session_id":"1239034","input_length":18420,"t_proxy_recv":338678.401792285,"t_decision_unix":1779769543.4157336,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":13312,"affinity_cache_ratio":0.7226927252985885,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5108 < 8000)","cache_hit":13312,"estimated_new_tokens":5108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":66214.0,"score_lmetric":84634},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":30040,"ongoing_decode_tokens":0,"pending_prefill_tokens":30040,"num_requests":1,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":18420,"score_linear":30040.0,"score_lmetric":48460},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":13312,"new_prefill":5108,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":37165.0,"score_lmetric":55585},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-13312.0,"chosen_score_lmetric":0,"t_first_token":338679.102100818,"t_first_token_unix":1779769544.1160445,"t_done":338692.565673989,"t_done_unix":1779769557.5796173},{"request_id":"1244640:1:1244640:18","session_id":"1244640","input_length":8274,"t_proxy_recv":338692.227767542,"t_decision_unix":1779769557.2417088,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8274,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":66214,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8274,"score_linear":66214.0,"score_lmetric":8274},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":8274,"score_linear":18420.0,"score_lmetric":8274},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338692.820924204,"t_first_token_unix":1779769557.834868,"t_done":338693.025569636,"t_done_unix":1779769558.0395129},{"request_id":"1245572:1:1245572:19","session_id":"1245572","input_length":6109,"t_proxy_recv":338695.361701514,"t_decision_unix":1779769560.3756428,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6109 < 8000)","cache_hit":0,"estimated_new_tokens":6109,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338696.091696208,"t_first_token_unix":1779769561.105641,"t_done":338696.33587685,"t_done_unix":1779769561.34982},{"request_id":"1241953:2:1245624:20","session_id":"1241953","input_length":10382,"t_proxy_recv":338695.452086998,"t_decision_unix":1779769560.4660285,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":1024,"affinity_cache_ratio":0.09863224812174919,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":1024,"estimated_new_tokens":9358,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":1024,"new_prefill":9358,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":6109,"ongoing_decode_tokens":0,"pending_prefill_tokens":6109,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10382,"score_linear":6109.0,"score_lmetric":16491}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":338696.222670548,"t_first_token_unix":1779769561.2366154,"t_done":338696.408438355,"t_done_unix":1779769561.4223814},{"request_id":"1245572:2:1246348:22","session_id":"1245572","input_length":6182,"t_proxy_recv":338697.95797653,"t_decision_unix":1779769562.971918,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.9938531219670009,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (38 < 8000)","cache_hit":6144,"estimated_new_tokens":38,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":17447,"ongoing_decode_tokens":0,"pending_prefill_tokens":10279,"num_requests":1,"active_p_offloads":0,"cached_blocks":76,"cache_hit":0,"new_prefill":6182,"score_linear":17447.0,"score_lmetric":16461},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":6144,"new_prefill":38,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":338697.990065499,"t_first_token_unix":1779769563.004009,"t_done":338698.171226403,"t_done_unix":1779769563.1851697},{"request_id":"1240924:2:1246244:21","session_id":"1240924","input_length":17447,"t_proxy_recv":338697.584647897,"t_decision_unix":1779769562.5985892,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":7168,"affinity_cache_ratio":0.4108442712214134,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":7168,"estimated_new_tokens":10279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":76,"cache_hit":7168,"new_prefill":10279,"score_linear":-7168.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":338698.800287547,"t_first_token_unix":1779769563.8142314,"t_done":338699.17048763,"t_done_unix":1779769564.184431},{"request_id":"1241942:2:1247111:23","session_id":"1241942","input_length":17975,"t_proxy_recv":338700.631233046,"t_decision_unix":1779769565.6451743,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":7168,"affinity_cache_ratio":0.3987760778859527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":7168,"estimated_new_tokens":10807,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":7168,"new_prefill":10807,"score_linear":-7168.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":338701.922743698,"t_first_token_unix":1779769566.9366877,"t_done":338702.108838354,"t_done_unix":1779769567.1227815},{"request_id":"1244640:2:1247941:24","session_id":"1244640","input_length":10696,"t_proxy_recv":338703.447285353,"t_decision_unix":1779769568.4612265,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":8192,"affinity_cache_ratio":0.7658937920718025,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2504 < 8000)","cache_hit":8192,"estimated_new_tokens":2504,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":8192,"new_prefill":2504,"score_linear":-8192.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":0,"t_first_token":338703.776123363,"t_first_token_unix":1779769568.7900672,"t_done":338703.820479537,"t_done_unix":1779769568.8344228},{"request_id":"1248768:1:1248768:25","session_id":"1248768","input_length":11515,"t_proxy_recv":338706.228946773,"t_decision_unix":1779769571.2428882,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11515,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338707.21948301,"t_first_token_unix":1779769572.233427,"t_done":338707.484424874,"t_done_unix":1779769572.4983683},{"request_id":"1249058:1:1249058:26","session_id":"1249058","input_length":4126,"t_proxy_recv":338707.248088028,"t_decision_unix":1779769572.2620296,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4126 < 8000)","cache_hit":0,"estimated_new_tokens":4126,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11515,"ongoing_decode_tokens":11515,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":4126,"score_linear":11515.0,"score_lmetric":4126},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338707.511693473,"t_first_token_unix":1779769572.5256367,"t_done":338707.678695981,"t_done_unix":1779769572.6926394},{"request_id":"1248768:2:1249765:27","session_id":"1248768","input_length":12188,"t_proxy_recv":338709.511546188,"t_decision_unix":1779769574.5254877,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.924187725631769,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (924 < 8000)","cache_hit":11264,"estimated_new_tokens":924,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":83,"cache_hit":11264,"new_prefill":924,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":45,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":338709.651832203,"t_first_token_unix":1779769574.6657758,"t_done":338710.123259263,"t_done_unix":1779769575.1372027},{"request_id":"1249058:2:1249872:28","session_id":"1249058","input_length":7482,"t_proxy_recv":338709.903361986,"t_decision_unix":1779769574.9173033,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":4096,"affinity_cache_ratio":0.5474472066292435,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3386 < 8000)","cache_hit":4096,"estimated_new_tokens":3386,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":12188,"ongoing_decode_tokens":12188,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":83,"cache_hit":0,"new_prefill":7482,"score_linear":12188.0,"score_lmetric":7482},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":45,"cache_hit":4096,"new_prefill":3386,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":338710.221580917,"t_first_token_unix":1779769575.2355244,"t_done":338710.349834827,"t_done_unix":1779769575.363778},{"request_id":"1242838:2:1250503:29","session_id":"1242838","input_length":21578,"t_proxy_recv":338712.178068772,"t_decision_unix":1779769577.19201,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":16896,"affinity_cache_ratio":0.783019742330151,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4682 < 8000)","cache_hit":16896,"estimated_new_tokens":4682,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":16896,"new_prefill":4682,"score_linear":-16896.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":338712.887904428,"t_first_token_unix":1779769577.901848,"t_done":338713.382597462,"t_done_unix":1779769578.3965409},{"request_id":"1250772:1:1250772:30","session_id":"1250772","input_length":6829,"t_proxy_recv":338713.037759209,"t_decision_unix":1779769578.0517008,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6829 < 8000)","cache_hit":0,"estimated_new_tokens":6829,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21578,"ongoing_decode_tokens":21578,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":6829,"score_linear":21578.0,"score_lmetric":6829},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338714.016319145,"t_first_token_unix":1779769579.0302627,"t_done":338715.615382514,"t_done_unix":1779769580.629326},{"request_id":"1251059:1:1251059:32","session_id":"1251059","input_length":16512,"t_proxy_recv":338714.221661007,"t_decision_unix":1779769579.2356024,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":16512,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":93,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":6829,"ongoing_decode_tokens":6829,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":16512,"score_linear":6829.0,"score_lmetric":16512},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":0,"pending_prefill_tokens":20086,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":16512,"score_linear":20086.0,"score_lmetric":36598},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338715.779896815,"t_first_token_unix":1779769580.7938404,"t_done":338715.781579627,"t_done_unix":1779769580.7955225},{"request_id":"1248768:3:1251261:33","session_id":"1248768","input_length":16989,"t_proxy_recv":338714.795033134,"t_decision_unix":1779769579.8089743,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11776,"affinity_cache_ratio":0.6931543940196597,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5213 < 8000)","cache_hit":11776,"estimated_new_tokens":5213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":93,"cache_hit":11776,"new_prefill":5213,"score_linear":-11776.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16512,"ongoing_decode_tokens":0,"pending_prefill_tokens":16512,"num_requests":1,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":16989,"score_linear":16512.0,"score_lmetric":33501},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":6829,"ongoing_decode_tokens":6829,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":16989,"score_linear":6829.0,"score_lmetric":16989},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":0,"pending_prefill_tokens":20086,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":16989,"score_linear":20086.0,"score_lmetric":37075},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":338715.475562026,"t_first_token_unix":1779769580.4895058,"t_done":338716.454817366,"t_done_unix":1779769581.4687617},{"request_id":"1251828:1:1251828:35","session_id":"1251828","input_length":1090,"t_proxy_recv":338716.724525275,"t_decision_unix":1779769581.7384665,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1090 < 8000)","cache_hit":0,"estimated_new_tokens":1090,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":20086,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":1090,"score_linear":20086.0,"score_lmetric":1090},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":13780,"ongoing_decode_tokens":0,"pending_prefill_tokens":7636,"num_requests":1,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":1090,"score_linear":13780.0,"score_lmetric":8726}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338716.810683204,"t_first_token_unix":1779769581.824627,"t_done":338716.919296219,"t_done_unix":1779769581.9332392},{"request_id":"1250935:1:1250935:31","session_id":"1250935","input_length":20086,"t_proxy_recv":338713.662788935,"t_decision_unix":1779769578.6767306,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":20086,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":93,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":6829,"ongoing_decode_tokens":0,"pending_prefill_tokens":6829,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":20086,"score_linear":6829.0,"score_lmetric":26915},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338715.745230717,"t_first_token_unix":1779769580.759174,"t_done":338717.001870661,"t_done_unix":1779769582.015814},{"request_id":"1245572:3:1251662:34","session_id":"1245572","input_length":13780,"t_proxy_recv":338716.205529436,"t_decision_unix":1779769581.2194707,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.4458635703918723,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7636 < 8000)","cache_hit":6144,"estimated_new_tokens":7636,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":16989,"ongoing_decode_tokens":16989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":93,"cache_hit":0,"new_prefill":13780,"score_linear":16989.0,"score_lmetric":13780},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":20086,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":13780,"score_linear":20086.0,"score_lmetric":13780},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":6144,"new_prefill":7636,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":338717.020970386,"t_first_token_unix":1779769582.0349138,"t_done":338717.244452692,"t_done_unix":1779769582.2583961},{"request_id":"1252010:1:1252010:36","session_id":"1252010","input_length":9855,"t_proxy_recv":338717.332851932,"t_decision_unix":1779769582.3467934,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9855,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":144,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338718.151104773,"t_first_token_unix":1779769583.1650486,"t_done":338718.323406627,"t_done_unix":1779769583.33735},{"request_id":"1248768:4:1253385:37","session_id":"1248768","input_length":17309,"t_proxy_recv":338722.107478691,"t_decision_unix":1779769587.1214201,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":16896,"affinity_cache_ratio":0.9761395805650239,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (413 < 8000)","cache_hit":16896,"estimated_new_tokens":413,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":103,"cache_hit":16896,"new_prefill":413,"score_linear":-16896.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":338722.188116349,"t_first_token_unix":1779769587.2020595,"t_done":338725.164668707,"t_done_unix":1779769590.1786122},{"request_id":"1251828:2:1254272:41","session_id":"1251828","input_length":3934,"t_proxy_recv":338725.076708068,"t_decision_unix":1779769590.0906496,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":4,"affinity_cache_hit":1024,"affinity_cache_ratio":0.2602948652770717,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3934 < 8000)","cache_hit":0,"estimated_new_tokens":3934,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":3934,"score_linear":17309.0,"score_lmetric":3934},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":3934,"score_linear":35221.0,"score_lmetric":39155},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":1024,"new_prefill":2910,"score_linear":48600.0,"score_lmetric":52534},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10764,"ongoing_decode_tokens":0,"pending_prefill_tokens":4108,"num_requests":1,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":3934,"score_linear":10764.0,"score_lmetric":8042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338725.321230708,"t_first_token_unix":1779769590.3351738,"t_done":338725.467712534,"t_done_unix":1779769590.4816556},{"request_id":"1254584:1:1254584:42","session_id":"1254584","input_length":748,"t_proxy_recv":338726.209199739,"t_decision_unix":1779769591.2231412,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (748 < 8000)","cache_hit":0,"estimated_new_tokens":748,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":748,"score_linear":35221.0,"score_lmetric":35969},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":748,"score_linear":49624.0,"score_lmetric":50372},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10764,"ongoing_decode_tokens":10764,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":748,"score_linear":10764.0,"score_lmetric":748},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338726.276545774,"t_first_token_unix":1779769591.2904892,"t_done":338726.439675092,"t_done_unix":1779769591.4536183},{"request_id":"1252010:2:1254691:43","session_id":"1252010","input_length":13092,"t_proxy_recv":338726.661660483,"t_decision_unix":1779769591.675602,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":9728,"affinity_cache_ratio":0.743049190345249,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3364 < 8000)","cache_hit":9728,"estimated_new_tokens":3364,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":13092,"score_linear":35221.0,"score_lmetric":48313},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":13092,"score_linear":49624.0,"score_lmetric":62716},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10764,"ongoing_decode_tokens":10764,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":13092,"score_linear":10764.0,"score_lmetric":13092},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":9728,"new_prefill":3364,"score_linear":-9728.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":338727.087524303,"t_first_token_unix":1779769592.1014678,"t_done":338727.257404602,"t_done_unix":1779769592.271348},{"request_id":"1250772:2:1254198:40","session_id":"1250772","input_length":10764,"t_proxy_recv":338724.80445014,"t_decision_unix":1779769589.8183916,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":6656,"affinity_cache_ratio":0.6183574879227053,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4108 < 8000)","cache_hit":6656,"estimated_new_tokens":4108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":10764,"score_linear":17309.0,"score_lmetric":10764},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":10764,"score_linear":35221.0,"score_lmetric":45985},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":10764,"score_linear":49624.0,"score_lmetric":60388},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":6656,"new_prefill":4108,"score_linear":-6656.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-6656.0,"chosen_score_lmetric":0,"t_first_token":338725.18630031,"t_first_token_unix":1779769590.200244,"t_done":338727.464923705,"t_done_unix":1779769592.478867},{"request_id":"1253804:1:1253804:39","session_id":"1253804","input_length":35221,"t_proxy_recv":338723.695353738,"t_decision_unix":1779769588.709295,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":35221,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":35221,"score_linear":17309.0,"score_lmetric":35221},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":35221,"score_linear":49624.0,"score_lmetric":84845},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338728.86190479,"t_first_token_unix":1779769593.875849,"t_done":338731.083451555,"t_done_unix":1779769596.0973947},{"request_id":"1256273:1:1256273:46","session_id":"1256273","input_length":1694,"t_proxy_recv":338731.891840153,"t_decision_unix":1779769596.9057817,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1694 < 8000)","cache_hit":0,"estimated_new_tokens":1694,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":1694,"score_linear":49624.0,"score_lmetric":51318},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17786,"ongoing_decode_tokens":0,"pending_prefill_tokens":10618,"num_requests":1,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":1694,"score_linear":17786.0,"score_lmetric":12312},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":21,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338732.036342561,"t_first_token_unix":1779769597.0502858,"t_done":338732.322193332,"t_done_unix":1779769597.3361368},{"request_id":"1243831:2:1256324:47","session_id":"1243831","input_length":11448,"t_proxy_recv":338732.056720843,"t_decision_unix":1779769597.0706625,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":11264,"affinity_cache_ratio":0.983927323549965,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (184 < 8000)","cache_hit":11264,"estimated_new_tokens":184,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":11264,"new_prefill":184,"score_linear":-11264.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":11448,"score_linear":49624.0,"score_lmetric":61072},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17786,"ongoing_decode_tokens":0,"pending_prefill_tokens":10618,"num_requests":1,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":11448,"score_linear":17786.0,"score_lmetric":22066},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":1694,"ongoing_decode_tokens":1694,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":21,"cache_hit":0,"new_prefill":11448,"score_linear":1694.0,"score_lmetric":11448},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":338732.112411807,"t_first_token_unix":1779769597.1263552,"t_done":338732.554233626,"t_done_unix":1779769597.568177},{"request_id":"1253743:1:1253743:38","session_id":"1253743","input_length":49624,"t_proxy_recv":338723.451254913,"t_decision_unix":1779769588.4651964,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":49624,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":49624,"score_linear":17309.0,"score_lmetric":49624},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338732.700214653,"t_first_token_unix":1779769597.714158,"t_done":338733.096711166,"t_done_unix":1779769598.1106544},{"request_id":"1249058:3:1256222:45","session_id":"1249058","input_length":17786,"t_proxy_recv":338731.745074838,"t_decision_unix":1779769596.7590163,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":7168,"affinity_cache_ratio":0.4030136062071292,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":7168,"estimated_new_tokens":10618,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":17786,"score_linear":49624.0,"score_lmetric":67410},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":7168,"new_prefill":10618,"score_linear":-7168.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":21,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":338733.00986687,"t_first_token_unix":1779769598.023811,"t_done":338733.126377057,"t_done_unix":1779769598.1403203},{"request_id":"1253743:2:1256070:44","session_id":"1253743","input_length":49687,"t_proxy_recv":338733.106307134,"t_decision_unix":1779769598.1202488,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9995371022601486,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (23 < 8000)","cache_hit":49664,"estimated_new_tokens":23,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":3584,"new_prefill":46103,"score_linear":-3584.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":49664,"new_prefill":23,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17786,"ongoing_decode_tokens":17786,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":49687,"score_linear":17786.0,"score_lmetric":49687},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":338733.18257463,"t_first_token_unix":1779769598.196518,"t_done":338734.010923102,"t_done_unix":1779769599.0248663},{"request_id":"1256273:2:1257769:51","session_id":"1256273","input_length":6170,"t_proxy_recv":338736.873768315,"t_decision_unix":1779769601.8877096,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":1536,"affinity_cache_ratio":0.24894651539708265,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4634 < 8000)","cache_hit":1536,"estimated_new_tokens":4634,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":4434,"ongoing_decode_tokens":4434,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":6170,"score_linear":4434.0,"score_lmetric":6170},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":6170,"score_linear":27387.0,"score_lmetric":33557},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":6170,"score_linear":90488.0,"score_lmetric":96658},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":1536,"new_prefill":4634,"score_linear":-1536.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":338737.198856224,"t_first_token_unix":1779769602.2128,"t_done":338737.501803979,"t_done_unix":1779769602.5157473},{"request_id":"1257572:1:1257572:50","session_id":"1257572","input_length":4434,"t_proxy_recv":338736.240844909,"t_decision_unix":1779769601.2547865,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4434 < 8000)","cache_hit":0,"estimated_new_tokens":4434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":4434,"score_linear":27387.0,"score_lmetric":31821},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":4434,"score_linear":90488.0,"score_lmetric":94922},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338736.525669662,"t_first_token_unix":1779769601.539613,"t_done":338737.769864629,"t_done_unix":1779769602.7838085},{"request_id":"1254584:2:1258039:52","session_id":"1254584","input_length":1257,"t_proxy_recv":338737.895066612,"t_decision_unix":1779769602.9090078,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":512,"affinity_cache_ratio":0.40731901352426414,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (745 < 8000)","cache_hit":512,"estimated_new_tokens":745,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":113,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":1257,"score_linear":27387.0,"score_lmetric":28644},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":512,"new_prefill":745,"score_linear":-512.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":1257,"score_linear":90488.0,"score_lmetric":91745},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":338737.951309709,"t_first_token_unix":1779769602.9652534,"t_done":338738.286322928,"t_done_unix":1779769603.3002663},{"request_id":"1242838:3:1258161:53","session_id":"1242838","input_length":22262,"t_proxy_recv":338738.35486715,"t_decision_unix":1779769603.368809,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9659509478034318,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (758 < 8000)","cache_hit":21504,"estimated_new_tokens":758,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":113,"cache_hit":21504,"new_prefill":758,"score_linear":-21504.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":22262,"score_linear":27387.0,"score_lmetric":49649},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":22262,"score_linear":90488.0,"score_lmetric":112750},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":338738.514626544,"t_first_token_unix":1779769603.5285697,"t_done":338739.070953976,"t_done_unix":1779769604.0848973},{"request_id":"1256273:3:1258499:55","session_id":"1256273","input_length":7934,"t_proxy_recv":338739.518286656,"t_decision_unix":1779769604.532228,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":6144,"affinity_cache_ratio":0.7743887068313587,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1790 < 8000)","cache_hit":6144,"estimated_new_tokens":1790,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":0,"pending_prefill_tokens":45037,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":7934,"score_linear":80365.0,"score_lmetric":105942},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":7934,"score_linear":90488.0,"score_lmetric":98422},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":6144,"new_prefill":1790,"score_linear":-6144.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":338739.702895577,"t_first_token_unix":1779769604.7168388,"t_done":338740.321632512,"t_done_unix":1779769605.3355758},{"request_id":"1258908:1:1258908:58","session_id":"1258908","input_length":9250,"t_proxy_recv":338740.973199337,"t_decision_unix":1779769605.987141,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9250,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":9250,"score_linear":80365.0,"score_lmetric":53800},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":9250,"score_linear":68131.0,"score_lmetric":77381},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":9250,"score_linear":90488.0,"score_lmetric":99738},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":112793,"ongoing_decode_tokens":0,"pending_prefill_tokens":112793,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":9250,"score_linear":112793.0,"score_lmetric":122043},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338741.672361132,"t_first_token_unix":1779769606.6863048,"t_done":338743.674157648,"t_done_unix":1779769608.6881008},{"request_id":"1259178:1:1259178:59","session_id":"1259178","input_length":14291,"t_proxy_recv":338741.933745043,"t_decision_unix":1779769606.9476862,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":14291,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":14291,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":14291,"score_linear":80365.0,"score_lmetric":63882},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":14291,"score_linear":68131.0,"score_lmetric":82422},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9250,"ongoing_decode_tokens":9250,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":14291,"score_linear":9250.0,"score_lmetric":14291},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":14291,"score_linear":90488.0,"score_lmetric":104779},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":112793,"ongoing_decode_tokens":0,"pending_prefill_tokens":112793,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14291,"score_linear":112793.0,"score_lmetric":127084},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":14291,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":14291,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338743.182917701,"t_first_token_unix":1779769608.196862,"t_done":338745.137995018,"t_done_unix":1779769610.1519387},{"request_id":"1257525:1:1257525:49","session_id":"1257525","input_length":27387,"t_proxy_recv":338736.154064375,"t_decision_unix":1779769601.1680057,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":27387,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":27387,"score_linear":90488.0,"score_lmetric":117875},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338739.614406414,"t_first_token_unix":1779769604.6283498,"t_done":338745.243138762,"t_done_unix":1779769610.2570822},{"request_id":"1253804:2:1258346:54","session_id":"1253804","input_length":52978,"t_proxy_recv":338739.06533234,"t_decision_unix":1779769604.0792737,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":35328,"affinity_cache_ratio":0.6668428404243271,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":35328,"estimated_new_tokens":17650,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":22262,"ongoing_decode_tokens":22262,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":113,"cache_hit":0,"new_prefill":52978,"score_linear":22262.0,"score_lmetric":52978},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":35328,"new_prefill":17650,"score_linear":-7941.0,"score_lmetric":45037},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":3584,"new_prefill":49394,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":52978,"score_linear":90488.0,"score_lmetric":143466},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7941.0,"chosen_score_lmetric":45037,"t_first_token":338744.872805056,"t_first_token_unix":1779769609.8867488,"t_done":338745.798092911,"t_done_unix":1779769610.812036},{"request_id":"1260746:1:1260746:62","session_id":"1260746","input_length":9571,"t_proxy_recv":338747.477596232,"t_decision_unix":1779769612.4915376,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9571,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":9571,"score_linear":68131.0,"score_lmetric":77702},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":9571,"score_linear":90488.0,"score_lmetric":100059},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":9571,"score_linear":125063.0,"score_lmetric":253908},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":9571,"score_linear":30655.0,"score_lmetric":40226}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338748.224269637,"t_first_token_unix":1779769613.238213,"t_done":338748.617785838,"t_done_unix":1779769613.6317291},{"request_id":"1260792:1:1260792:63","session_id":"1260792","input_length":12522,"t_proxy_recv":338747.593480007,"t_decision_unix":1779769612.6074214,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":12522,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9571,"ongoing_decode_tokens":0,"pending_prefill_tokens":9571,"num_requests":1,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":12522,"score_linear":9571.0,"score_lmetric":22093},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":12522,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":12522,"score_linear":68131.0,"score_lmetric":80653},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":12522,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":12522,"score_linear":90488.0,"score_lmetric":103010},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":12522,"score_linear":125063.0,"score_lmetric":259810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":12522,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":12522,"score_linear":30655.0,"score_lmetric":43177}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338748.622168582,"t_first_token_unix":1779769613.6361115,"t_done":338748.959648413,"t_done_unix":1779769613.9735916},{"request_id":"1261126:1:1261126:65","session_id":"1261126","input_length":5640,"t_proxy_recv":338748.633658531,"t_decision_unix":1779769613.6476002,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5640 < 8000)","cache_hit":0,"estimated_new_tokens":5640,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":132,"cache_hit":0,"new_prefill":5640,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55138,"ongoing_decode_tokens":0,"pending_prefill_tokens":2402,"num_requests":1,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":5640,"score_linear":55138.0,"score_lmetric":8042},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":5640,"score_linear":68131.0,"score_lmetric":73771},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12522,"ongoing_decode_tokens":12522,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":5640,"score_linear":12522.0,"score_lmetric":5640},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":5640,"score_linear":90488.0,"score_lmetric":96128},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5640,"score_linear":125063.0,"score_lmetric":246046},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":5640,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":5640,"score_linear":30655.0,"score_lmetric":36295}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338749.001877299,"t_first_token_unix":1779769614.0158203,"t_done":338749.172125591,"t_done_unix":1779769614.1860685},{"request_id":"1253804:3:1260997:64","session_id":"1253804","input_length":55138,"t_proxy_recv":338748.331657695,"t_decision_unix":1779769613.3455992,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9564365773151003,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2402 < 8000)","cache_hit":52736,"estimated_new_tokens":2402,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9571,"ongoing_decode_tokens":9571,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":55138,"score_linear":9571.0,"score_lmetric":55138},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":52736,"new_prefill":2402,"score_linear":-52736.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":55138,"score_linear":68131.0,"score_lmetric":123269},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12522,"ongoing_decode_tokens":0,"pending_prefill_tokens":12522,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":3584,"new_prefill":51554,"score_linear":8938.0,"score_lmetric":64076},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":55138,"score_linear":90488.0,"score_lmetric":145626},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":55138,"score_linear":125063.0,"score_lmetric":345042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":55138,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":55138,"score_linear":30655.0,"score_lmetric":85793}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":338749.203627088,"t_first_token_unix":1779769614.21757,"t_done":338749.795546588,"t_done_unix":1779769614.8094902},{"request_id":"1260327:1:1260327:61","session_id":"1260327","input_length":30655,"t_proxy_recv":338745.757268033,"t_decision_unix":1779769610.7712095,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":30655,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":52978,"ongoing_decode_tokens":52978,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":218,"cache_hit":0,"new_prefill":30655,"score_linear":52978.0,"score_lmetric":30655},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":30655,"score_linear":68131.0,"score_lmetric":98786},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":30655,"score_linear":90488.0,"score_lmetric":121143},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":30655,"score_linear":125063.0,"score_lmetric":296076},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338749.849483769,"t_first_token_unix":1779769614.8634272,"t_done":338750.495278994,"t_done_unix":1779769615.5092218},{"request_id":"1257525:2:1262205:67","session_id":"1257525","input_length":29988,"t_proxy_recv":338752.445817076,"t_decision_unix":1779769617.4597583,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9048952914499133,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2852 < 8000)","cache_hit":27136,"estimated_new_tokens":2852,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":256,"cache_hit":27136,"new_prefill":2852,"score_linear":-27136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":29988,"score_linear":68131.0,"score_lmetric":98119},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":29988,"score_linear":90488.0,"score_lmetric":120476},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":29988,"score_linear":125063.0,"score_lmetric":294742},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":338753.07148901,"t_first_token_unix":1779769618.085432,"t_done":338753.10298866,"t_done_unix":1779769618.1169317},{"request_id":"1262757:1:1262757:73","session_id":"1262757","input_length":5465,"t_proxy_recv":338754.327855289,"t_decision_unix":1779769619.3417969,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5465 < 8000)","cache_hit":0,"estimated_new_tokens":5465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":0,"pending_prefill_tokens":1205,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":5465,"score_linear":55989.0,"score_lmetric":6670},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":5465,"score_linear":68131.0,"score_lmetric":73596},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":5465,"score_linear":15283.0,"score_lmetric":20748},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":5465,"score_linear":113053.0,"score_lmetric":196076},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5465,"score_linear":125063.0,"score_lmetric":245696},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":5465,"score_linear":21510.0,"score_lmetric":26975}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338754.685509855,"t_first_token_unix":1779769619.6994534,"t_done":338754.81625065,"t_done_unix":1779769619.8301947},{"request_id":"1261126:2:1262820:74","session_id":"1261126","input_length":6953,"t_proxy_recv":338754.497284271,"t_decision_unix":1779769619.511226,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5632,"affinity_cache_ratio":0.8100100675967209,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1321 < 8000)","cache_hit":5632,"estimated_new_tokens":1321,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":5632,"new_prefill":1321,"score_linear":-5632.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":0,"pending_prefill_tokens":1205,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":6953,"score_linear":55989.0,"score_lmetric":8158},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":6953,"score_linear":68131.0,"score_lmetric":75084},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":6953,"score_linear":15283.0,"score_lmetric":22236},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":6953,"score_linear":113053.0,"score_lmetric":199052},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":6953,"score_linear":125063.0,"score_lmetric":248672},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":5465,"ongoing_decode_tokens":0,"pending_prefill_tokens":5465,"num_requests":1,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":6953,"score_linear":5465.0,"score_lmetric":12418},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":6953,"score_linear":21510.0,"score_lmetric":28463}],"chosen_score_linear":-5632.0,"chosen_score_lmetric":0,"t_first_token":338754.65555696,"t_first_token_unix":1779769619.6695,"t_done":338754.893397741,"t_done_unix":1779769619.9073412},{"request_id":"1262354:1:1262354:69","session_id":"1262354","input_length":21510,"t_proxy_recv":338753.042335382,"t_decision_unix":1779769618.056277,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":21510,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":29988,"ongoing_decode_tokens":0,"pending_prefill_tokens":2852,"num_requests":1,"active_p_offloads":0,"cached_blocks":256,"cache_hit":0,"new_prefill":21510,"score_linear":29988.0,"score_lmetric":24362},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":21510,"score_linear":68131.0,"score_lmetric":89641},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":21510,"score_linear":113053.0,"score_lmetric":228166},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":21510,"score_linear":125063.0,"score_lmetric":277786},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338755.357167206,"t_first_token_unix":1779769620.3711112,"t_done":338755.959964698,"t_done_unix":1779769620.9739082},{"request_id":"1258611:1:1258611:56","session_id":"1258611","input_length":68131,"t_proxy_recv":338739.91782273,"t_decision_unix":1779769604.931764,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":68131,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":68131,"score_linear":80365.0,"score_lmetric":171562},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":68131,"score_linear":90488.0,"score_lmetric":158619},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":7934,"ongoing_decode_tokens":7934,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":68131,"score_linear":7934.0,"score_lmetric":68131},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338756.284707584,"t_first_token_unix":1779769621.2986515,"t_done":338757.463462089,"t_done_unix":1779769622.4774058},{"request_id":"1258611:2:1261594:66","session_id":"1258611","input_length":68320,"t_proxy_recv":338757.475103008,"t_decision_unix":1779769622.4890447,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9967213114754099,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (224 < 8000)","cache_hit":68096,"estimated_new_tokens":224,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":68320,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":68320,"score_linear":55989.0,"score_lmetric":68320},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":68096,"new_prefill":224,"score_linear":-68096.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":68320,"score_linear":15283.0,"score_lmetric":68320},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":68320,"score_linear":113053.0,"score_lmetric":321786},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":68320,"score_linear":125063.0,"score_lmetric":371406},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":68320,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":0,"pending_prefill_tokens":605,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":68320,"score_linear":22109.0,"score_lmetric":68925}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":338757.615849431,"t_first_token_unix":1779769622.6297925,"t_done":338757.656455728,"t_done_unix":1779769622.670399},{"request_id":"1258611:3:1262709:71","session_id":"1258611","input_length":68377,"t_proxy_recv":338757.667768251,"t_decision_unix":1779769622.6817098,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9958904309928778,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (281 < 8000)","cache_hit":68096,"estimated_new_tokens":281,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5990,"ongoing_decode_tokens":0,"pending_prefill_tokens":5990,"num_requests":1,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":68377,"score_linear":5990.0,"score_lmetric":74367},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":68377,"score_linear":55989.0,"score_lmetric":68377},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":68096,"new_prefill":281,"score_linear":-68096.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":68377,"score_linear":15283.0,"score_lmetric":68377},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":68377,"score_linear":113053.0,"score_lmetric":321900},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":68377,"score_linear":125063.0,"score_lmetric":371520},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":68377,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":22109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":68377,"score_linear":22109.0,"score_lmetric":68377}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":338757.807169256,"t_first_token_unix":1779769622.821113,"t_done":338758.247373022,"t_done_unix":1779769623.2613168},{"request_id":"1263611:1:1263611:76","session_id":"1263611","input_length":5990,"t_proxy_recv":338757.520431513,"t_decision_unix":1779769622.5343728,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5990 < 8000)","cache_hit":0,"estimated_new_tokens":5990,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":5990,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":5990,"score_linear":55989.0,"score_lmetric":5990},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68320,"ongoing_decode_tokens":0,"pending_prefill_tokens":224,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":5990,"score_linear":68320.0,"score_lmetric":6214},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":5990,"score_linear":15283.0,"score_lmetric":5990},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":5990,"score_linear":113053.0,"score_lmetric":197126},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5990,"score_linear":125063.0,"score_lmetric":246746},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":5990,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":0,"pending_prefill_tokens":605,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":5990,"score_linear":22109.0,"score_lmetric":6595}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338757.919967595,"t_first_token_unix":1779769622.9339116,"t_done":338758.311088458,"t_done_unix":1779769623.3250318},{"request_id":"1262354:2:1263593:75","session_id":"1262354","input_length":22109,"t_proxy_recv":338757.408820828,"t_decision_unix":1779769622.4227622,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9726355782712922,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (605 < 8000)","cache_hit":21504,"estimated_new_tokens":605,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":22109,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":22109,"score_linear":55989.0,"score_lmetric":22109},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":68131,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":22109,"score_linear":68131.0,"score_lmetric":22109},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":22109,"score_linear":15283.0,"score_lmetric":22109},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":22109,"score_linear":113053.0,"score_lmetric":229364},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":22109,"score_linear":125063.0,"score_lmetric":278984},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":22109,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":128,"cache_hit":21504,"new_prefill":605,"score_linear":-21504.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":338757.566084949,"t_first_token_unix":1779769622.580028,"t_done":338758.798762168,"t_done_unix":1779769623.8127062},{"request_id":"1263971:1:1263971:80","session_id":"1263971","input_length":6336,"t_proxy_recv":338758.828752403,"t_decision_unix":1779769623.8426938,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6336 < 8000)","cache_hit":0,"estimated_new_tokens":6336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":6336,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":6336,"score_linear":55989.0,"score_lmetric":6336},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":6336,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":6336,"score_linear":71575.0,"score_lmetric":125256},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":6336,"score_linear":113053.0,"score_lmetric":197818},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":6336,"score_linear":125063.0,"score_lmetric":247438},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":6336,"score_linear":50783.0,"score_lmetric":57119},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6336,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338759.246363041,"t_first_token_unix":1779769624.2603073,"t_done":338759.612093445,"t_done_unix":1779769624.626037},{"request_id":"1257323:1:1257323:48","session_id":"1257323","input_length":90488,"t_proxy_recv":338735.55363181,"t_decision_unix":1779769600.5675733,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":90488,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338763.261369359,"t_first_token_unix":1779769628.2753134,"t_done":338763.653882598,"t_done_unix":1779769628.6678264},{"request_id":"1239034:3:1262346:68","session_id":"1239034","input_length":22565,"t_proxy_recv":338753.011585069,"t_decision_unix":1779769618.0255265,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":20480,"affinity_cache_ratio":0.9076002658985154,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2085 < 8000)","cache_hit":20480,"estimated_new_tokens":2085,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":29988,"ongoing_decode_tokens":0,"pending_prefill_tokens":2852,"num_requests":1,"active_p_offloads":0,"cached_blocks":256,"cache_hit":0,"new_prefill":22565,"score_linear":29988.0,"score_lmetric":25417},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":22565,"score_linear":68131.0,"score_lmetric":90696},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":20480,"new_prefill":2085,"score_linear":70008.0,"score_lmetric":92573},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":22565,"score_linear":125063.0,"score_lmetric":279896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":70008.0,"chosen_score_lmetric":92573,"t_first_token":338763.263107516,"t_first_token_unix":1779769628.2770507,"t_done":338763.779568128,"t_done_unix":1779769628.7935114},{"request_id":"1265000:1:1265000:83","session_id":"1265000","input_length":301,"t_proxy_recv":338762.454764353,"t_decision_unix":1779769627.4687057,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (301 < 8000)","cache_hit":0,"estimated_new_tokens":301,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":301,"score_linear":110048.0,"score_lmetric":110349},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":301,"score_linear":55989.0,"score_lmetric":301},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":301,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":301,"score_linear":71575.0,"score_lmetric":113186},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":301,"score_linear":113053.0,"score_lmetric":185748},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":301,"score_linear":125063.0,"score_lmetric":235368},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":301,"score_linear":50783.0,"score_lmetric":51084},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":301,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338762.494888847,"t_first_token_unix":1779769627.508832,"t_done":338764.018099322,"t_done_unix":1779769629.0320425},{"request_id":"1265128:1:1265128:85","session_id":"1265128","input_length":14028,"t_proxy_recv":338762.782840987,"t_decision_unix":1779769627.796782,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":14028,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":14028,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":14028,"score_linear":110048.0,"score_lmetric":124076},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":14028,"score_linear":55989.0,"score_lmetric":14028},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":301,"ongoing_decode_tokens":301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":14028,"score_linear":301.0,"score_lmetric":14028},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":14028,"score_linear":71575.0,"score_lmetric":140640},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":14028,"score_linear":113053.0,"score_lmetric":213202},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14028,"score_linear":125063.0,"score_lmetric":262822},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":14028,"score_linear":50783.0,"score_lmetric":64811},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":23540,"ongoing_decode_tokens":0,"pending_prefill_tokens":23540,"num_requests":1,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":14028,"score_linear":23540.0,"score_lmetric":37568}],"chosen_score_linear":301.0,"chosen_score_lmetric":14028,"t_first_token":338764.000556096,"t_first_token_unix":1779769629.0145001,"t_done":338764.43744084,"t_done_unix":1779769629.4513845},{"request_id":"1266067:1:1266067:91","session_id":"1266067","input_length":579,"t_proxy_recv":338766.005248822,"t_decision_unix":1779769631.0191903,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":579,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (579 < 8000)","cache_hit":0,"estimated_new_tokens":579,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":579,"score_linear":135235.0,"score_lmetric":227596},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":60059,"ongoing_decode_tokens":55989,"pending_prefill_tokens":4070,"num_requests":2,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":579,"score_linear":60059.0,"score_lmetric":9298},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13178,"ongoing_decode_tokens":0,"pending_prefill_tokens":13178,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":579,"score_linear":13178.0,"score_lmetric":13757},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":579,"score_linear":92560.0,"score_lmetric":196704},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":579,"score_linear":27639.0,"score_lmetric":579},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":579,"score_linear":125063.0,"score_lmetric":235924},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":579,"score_linear":50783.0,"score_lmetric":51362},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":45884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":579,"score_linear":45884.0,"score_lmetric":1158}],"chosen_score_linear":27639.0,"chosen_score_lmetric":579,"t_first_token":338766.12983484,"t_first_token_unix":1779769631.143778,"t_done":338766.297386675,"t_done_unix":1779769631.3113303},{"request_id":"1263611:2:1265770:88","session_id":"1263611","input_length":13178,"t_proxy_recv":338765.103663449,"t_decision_unix":1779769630.1176043,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":2,"affinity_cache_hit":5632,"affinity_cache_ratio":0.4273789649415693,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":13178,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":5632,"new_prefill":7546,"score_linear":129603.0,"score_lmetric":241530},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":13178,"score_linear":55989.0,"score_lmetric":13178},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":13178,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":13178,"score_linear":71575.0,"score_lmetric":138940},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":13178,"score_linear":27639.0,"score_lmetric":13178},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":13178,"score_linear":125063.0,"score_lmetric":261122},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":13178,"score_linear":50783.0,"score_lmetric":63961},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":0,"pending_prefill_tokens":23868,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":13178,"score_linear":45884.0,"score_lmetric":74092}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338766.214815095,"t_first_token_unix":1779769631.2287586,"t_done":338766.468463234,"t_done_unix":1779769631.4824066},{"request_id":"1250772:3:1265019:84","session_id":"1250772","input_length":23540,"t_proxy_recv":338762.527221701,"t_decision_unix":1779769627.5411632,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":7,"affinity_cache_hit":10752,"affinity_cache_ratio":0.45675446049277824,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":23540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":23540,"score_linear":110048.0,"score_lmetric":133588},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":23540,"score_linear":55989.0,"score_lmetric":23540},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":301,"ongoing_decode_tokens":301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":23540,"score_linear":301.0,"score_lmetric":23540},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":23540,"score_linear":71575.0,"score_lmetric":159664},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":23540,"score_linear":113053.0,"score_lmetric":232226},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":10752,"new_prefill":12788,"score_linear":114311.0,"score_lmetric":260342},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":23540,"score_linear":50783.0,"score_lmetric":74323},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":23540,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338765.236723151,"t_first_token_unix":1779769630.2506669,"t_done":338766.88846295,"t_done_unix":1779769631.9024067},{"request_id":"1265942:1:1265942:89","session_id":"1265942","input_length":4070,"t_proxy_recv":338765.731098605,"t_decision_unix":1779769630.7450402,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":4070,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4070 < 8000)","cache_hit":0,"estimated_new_tokens":4070,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":4070,"score_linear":135235.0,"score_lmetric":234578},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":4070,"score_linear":55989.0,"score_lmetric":4070},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13178,"ongoing_decode_tokens":0,"pending_prefill_tokens":13178,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":4070,"score_linear":13178.0,"score_lmetric":17248},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":4070,"score_linear":71575.0,"score_lmetric":120724},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":4070,"score_linear":27639.0,"score_lmetric":4070},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":4070,"score_linear":125063.0,"score_lmetric":242906},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":4070,"score_linear":50783.0,"score_lmetric":54853},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":45884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":4070,"score_linear":45884.0,"score_lmetric":8140}],"chosen_score_linear":55989.0,"chosen_score_lmetric":4070,"t_first_token":338766.044187504,"t_first_token_unix":1779769631.0581307,"t_done":338766.89507774,"t_done_unix":1779769631.9090207},{"request_id":"1262354:3:1265178:86","session_id":"1262354","input_length":22344,"t_proxy_recv":338762.891179445,"t_decision_unix":1779769627.9051208,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9853204439670605,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (328 < 8000)","cache_hit":22016,"estimated_new_tokens":328,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":22344,"score_linear":110048.0,"score_lmetric":132392},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":22344,"score_linear":55989.0,"score_lmetric":22344},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14329,"ongoing_decode_tokens":301,"pending_prefill_tokens":14028,"num_requests":2,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":22344,"score_linear":14329.0,"score_lmetric":72744},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":22344,"score_linear":71575.0,"score_lmetric":157272},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":22344,"score_linear":113053.0,"score_lmetric":229834},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":22344,"score_linear":125063.0,"score_lmetric":279454},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":22344,"score_linear":50783.0,"score_lmetric":73127},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":23540,"ongoing_decode_tokens":0,"pending_prefill_tokens":23540,"num_requests":1,"active_p_offloads":0,"cached_blocks":141,"cache_hit":22016,"new_prefill":328,"score_linear":1524.0,"score_lmetric":23868}],"chosen_score_linear":1524.0,"chosen_score_lmetric":23868,"t_first_token":338765.238896187,"t_first_token_unix":1779769630.2528393,"t_done":338767.68498343,"t_done_unix":1779769632.698927},{"request_id":"1266440:1:1266440:92","session_id":"1266440","input_length":5381,"t_proxy_recv":338767.078618294,"t_decision_unix":1779769632.0925596,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5381 < 8000)","cache_hit":0,"estimated_new_tokens":5381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":5381,"score_linear":135235.0,"score_lmetric":237200},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":269,"cache_hit":0,"new_prefill":5381,"score_linear":55989.0,"score_lmetric":5381},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":348,"cache_hit":0,"new_prefill":5381,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":5381,"score_linear":92560.0,"score_lmetric":211110},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":5381,"score_linear":27639.0,"score_lmetric":5381},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5381,"score_linear":125063.0,"score_lmetric":245528},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":5381,"score_linear":50783.0,"score_lmetric":56164},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22344,"ongoing_decode_tokens":22344,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":187,"cache_hit":0,"new_prefill":5381,"score_linear":22344.0,"score_lmetric":5381}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338767.459142638,"t_first_token_unix":1779769632.4730856,"t_done":338767.762775179,"t_done_unix":1779769632.7767181},{"request_id":"1263727:1:1263727:77","session_id":"1263727","input_length":50783,"t_proxy_recv":338757.989256986,"t_decision_unix":1779769623.0031984,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":50783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5990,"ongoing_decode_tokens":5990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":50783,"score_linear":5990.0,"score_lmetric":50783},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":50783,"score_linear":55989.0,"score_lmetric":50783},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68377,"ongoing_decode_tokens":68377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":50783,"score_linear":68377.0,"score_lmetric":50783},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":50783,"score_linear":15283.0,"score_lmetric":50783},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":50783,"score_linear":113053.0,"score_lmetric":286712},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":50783,"score_linear":125063.0,"score_lmetric":336332},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":50783,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":22109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":50783,"score_linear":22109.0,"score_lmetric":50783}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338767.711995596,"t_first_token_unix":1779769632.725939,"t_done":338767.895555887,"t_done_unix":1779769632.909499},{"request_id":"1253804:4:1262690:70","session_id":"1253804","input_length":55989,"t_proxy_recv":338754.147426076,"t_decision_unix":1779769619.1613672,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":54784,"affinity_cache_ratio":0.978477915304792,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1205 < 8000)","cache_hit":54784,"estimated_new_tokens":1205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":55989,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":261,"cache_hit":54784,"new_prefill":1205,"score_linear":-54784.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":55989,"score_linear":68131.0,"score_lmetric":124120},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":3584,"new_prefill":52405,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":55989,"score_linear":113053.0,"score_lmetric":297124},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":55989,"score_linear":125063.0,"score_lmetric":346744},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":55989,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":55989,"score_linear":21510.0,"score_lmetric":77499}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":338754.577546759,"t_first_token_unix":1779769619.59149,"t_done":338768.433514895,"t_done_unix":1779769633.4474585},{"request_id":"1263971:2:1266797:94","session_id":"1263971","input_length":9504,"t_proxy_recv":338768.426280572,"t_decision_unix":1779769633.440222,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.6464646464646465,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3360 < 8000)","cache_hit":6144,"estimated_new_tokens":3360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":9504,"score_linear":135235.0,"score_lmetric":245446},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":269,"cache_hit":0,"new_prefill":9504,"score_linear":55989.0,"score_lmetric":9504},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9504,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":9504,"score_linear":92560.0,"score_lmetric":223479},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":9504,"score_linear":27639.0,"score_lmetric":9504},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":9504,"score_linear":125063.0,"score_lmetric":253774},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17466,"ongoing_decode_tokens":0,"pending_prefill_tokens":17466,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":9504,"score_linear":17466.0,"score_lmetric":26970},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":188,"cache_hit":6144,"new_prefill":3360,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":338768.796956882,"t_first_token_unix":1779769633.8109002,"t_done":338768.949367292,"t_done_unix":1779769633.9633112},{"request_id":"1260792:2:1265951:90","session_id":"1260792","input_length":20985,"t_proxy_recv":338765.757726675,"t_decision_unix":1779769630.7716682,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":12288,"affinity_cache_ratio":0.5855611150822015,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":12288,"estimated_new_tokens":8697,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":20985,"score_linear":135235.0,"score_lmetric":268408},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":60059,"ongoing_decode_tokens":55989,"pending_prefill_tokens":4070,"num_requests":2,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":20985,"score_linear":60059.0,"score_lmetric":50110},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13178,"ongoing_decode_tokens":0,"pending_prefill_tokens":13178,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":20985,"score_linear":13178.0,"score_lmetric":34163},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":12288,"new_prefill":8697,"score_linear":59287.0,"score_lmetric":129978},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":20985,"score_linear":27639.0,"score_lmetric":20985},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":20985,"score_linear":125063.0,"score_lmetric":276736},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":20985,"score_linear":50783.0,"score_lmetric":71768},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":45884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":20985,"score_linear":45884.0,"score_lmetric":41970}],"chosen_score_linear":59287.0,"chosen_score_lmetric":129978,"t_first_token":338770.687564042,"t_first_token_unix":1779769635.7015076,"t_done":338771.204633088,"t_done_unix":1779769636.2185767},{"request_id":"1263749:1:1263749:79","session_id":"1263749","input_length":56292,"t_proxy_recv":338758.064074511,"t_decision_unix":1779769623.0780158,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":56292,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":56292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5990,"ongoing_decode_tokens":5990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":56292,"score_linear":5990.0,"score_lmetric":56292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":56292,"score_linear":55989.0,"score_lmetric":56292},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68377,"ongoing_decode_tokens":68377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":56292,"score_linear":68377.0,"score_lmetric":56292},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":56292,"score_linear":15283.0,"score_lmetric":56292},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":56292,"score_linear":113053.0,"score_lmetric":297730},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":56292,"score_linear":125063.0,"score_lmetric":347350},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":56292,"score_linear":50783.0,"score_lmetric":107075},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":22109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":56292,"score_linear":22109.0,"score_lmetric":56292}],"chosen_score_linear":15283.0,"chosen_score_lmetric":56292,"t_first_token":338769.517631344,"t_first_token_unix":1779769634.5315745,"t_done":338772.149346372,"t_done_unix":1779769637.1632895},{"request_id":"1266668:1:1266668:93","session_id":"1266668","input_length":17466,"t_proxy_recv":338767.979078062,"t_decision_unix":1779769632.9930193,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":17466,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":13370,"score_linear":131139.0,"score_lmetric":253178},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":269,"cache_hit":0,"new_prefill":17466,"score_linear":55989.0,"score_lmetric":17466},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17466,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":17466,"score_linear":92560.0,"score_lmetric":247365},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":17466,"score_linear":27639.0,"score_lmetric":17466},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":17466,"score_linear":125063.0,"score_lmetric":269698},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":17466,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":188,"cache_hit":0,"new_prefill":17466,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338769.690359807,"t_first_token_unix":1779769634.7043033,"t_done":338773.605526703,"t_done_unix":1779769638.6194701},{"request_id":"1265942:2:1268069:97","session_id":"1265942","input_length":10457,"t_proxy_recv":338772.735975551,"t_decision_unix":1779769637.749917,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":4096,"affinity_cache_ratio":0.39169934015492014,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6361 < 8000)","cache_hit":4096,"estimated_new_tokens":6361,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":10457,"score_linear":135235.0,"score_lmetric":247352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":273,"cache_hit":4096,"new_prefill":6361,"score_linear":-4096.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10457,"score_linear":126924.0,"score_lmetric":118437},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10457,"score_linear":15283.0,"score_lmetric":10457},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":10457,"score_linear":27639.0,"score_lmetric":10457},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":10457,"score_linear":125063.0,"score_lmetric":255680},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":10457,"score_linear":17466.0,"score_lmetric":10457},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28671,"ongoing_decode_tokens":0,"pending_prefill_tokens":6143,"num_requests":1,"active_p_offloads":0,"cached_blocks":194,"cache_hit":0,"new_prefill":10457,"score_linear":28671.0,"score_lmetric":16600}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":338773.314376933,"t_first_token_unix":1779769638.3283205,"t_done":338773.674031978,"t_done_unix":1779769638.6879752},{"request_id":"1262354:4:1267821:95","session_id":"1262354","input_length":28671,"t_proxy_recv":338772.036125073,"t_decision_unix":1779769637.0500665,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":22528,"affinity_cache_ratio":0.7857416902096195,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6143 < 8000)","cache_hit":22528,"estimated_new_tokens":6143,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":28671,"score_linear":135235.0,"score_lmetric":283780},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":273,"cache_hit":0,"new_prefill":28671,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28671,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":71575,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":248,"cache_hit":0,"new_prefill":28671,"score_linear":71575.0,"score_lmetric":57342},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":28671,"score_linear":27639.0,"score_lmetric":28671},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":28671,"score_linear":125063.0,"score_lmetric":292108},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":28671,"score_linear":17466.0,"score_lmetric":28671},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":194,"cache_hit":22528,"new_prefill":6143,"score_linear":-22528.0,"score_lmetric":0}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":0,"t_first_token":338773.22266424,"t_first_token_unix":1779769638.236608,"t_done":338774.537783999,"t_done_unix":1779769639.5517273},{"request_id":"1268634:1:1268634:99","session_id":"1268634","input_length":2298,"t_proxy_recv":338774.988034664,"t_decision_unix":1779769640.0019758,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2298 < 8000)","cache_hit":0,"estimated_new_tokens":2298,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":2298,"score_linear":135235.0,"score_lmetric":231034},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":2298,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2298,"score_linear":126924.0,"score_lmetric":110278},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2298,"score_linear":15283.0,"score_lmetric":2298},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":2298,"score_linear":27639.0,"score_lmetric":2298},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":2298,"score_linear":125063.0,"score_lmetric":239362},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":2298,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":2298,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338775.205407527,"t_first_token_unix":1779769640.2193508,"t_done":338775.319998146,"t_done_unix":1779769640.3339489},{"request_id":"1268649:1:1268649:100","session_id":"1268649","input_length":18638,"t_proxy_recv":338774.990993757,"t_decision_unix":1779769640.0049355,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":18638,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":18638,"score_linear":135235.0,"score_lmetric":263714},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":18638,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18638,"score_linear":126924.0,"score_lmetric":126618},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18638,"score_linear":15283.0,"score_lmetric":18638},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":18638,"score_linear":27639.0,"score_lmetric":18638},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":18638,"score_linear":125063.0,"score_lmetric":272042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":18638,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":2298,"ongoing_decode_tokens":0,"pending_prefill_tokens":2298,"num_requests":1,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":18638,"score_linear":2298.0,"score_lmetric":20936}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338776.916012914,"t_first_token_unix":1779769641.9299562,"t_done":338777.057240094,"t_done_unix":1779769642.0711834},{"request_id":"1263971:3:1269088:104","session_id":"1263971","input_length":13955,"t_proxy_recv":338776.36795021,"t_decision_unix":1779769641.3818915,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":9216,"affinity_cache_ratio":0.660408455750627,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4739 < 8000)","cache_hit":9216,"estimated_new_tokens":4739,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":13955,"score_linear":135235.0,"score_lmetric":254348},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":13955,"score_linear":102074.0,"score_lmetric":112445},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":13955,"score_linear":126924.0,"score_lmetric":121935},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27603,"ongoing_decode_tokens":15283,"pending_prefill_tokens":12320,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":13955,"score_linear":27603.0,"score_lmetric":52550},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":13955,"score_linear":27639.0,"score_lmetric":13955},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":13955,"score_linear":125063.0,"score_lmetric":262676},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":13955,"score_linear":18638.0,"score_lmetric":32593},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":36224,"ongoing_decode_tokens":0,"pending_prefill_tokens":12672,"num_requests":2,"active_p_offloads":0,"cached_blocks":210,"cache_hit":9216,"new_prefill":4739,"score_linear":27008.0,"score_lmetric":34822}],"chosen_score_linear":27008.0,"chosen_score_lmetric":34822,"t_first_token":338777.076221665,"t_first_token_unix":1779769642.0901647,"t_done":338777.358732862,"t_done_unix":1779769642.3726766},{"request_id":"1250772:4:1268869:103","session_id":"1250772","input_length":24029,"t_proxy_recv":338775.627785658,"t_decision_unix":1779769640.6417272,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9801489866411419,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (477 < 8000)","cache_hit":23552,"estimated_new_tokens":477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":24029,"score_linear":135235.0,"score_lmetric":274496},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":24029,"score_linear":102074.0,"score_lmetric":122519},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24029,"score_linear":126924.0,"score_lmetric":132009},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27603,"ongoing_decode_tokens":15283,"pending_prefill_tokens":12320,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24029,"score_linear":27603.0,"score_lmetric":72698},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":24029,"score_linear":27639.0,"score_lmetric":24029},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":10752,"new_prefill":13277,"score_linear":114311.0,"score_lmetric":261320},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":24029,"score_linear":18638.0,"score_lmetric":42667},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":12195,"ongoing_decode_tokens":0,"pending_prefill_tokens":12195,"num_requests":1,"active_p_offloads":0,"cached_blocks":210,"cache_hit":23552,"new_prefill":477,"score_linear":-11357.0,"score_lmetric":12672}],"chosen_score_linear":-11357.0,"chosen_score_lmetric":12672,"t_first_token":338776.623665584,"t_first_token_unix":1779769641.637609,"t_done":338778.329499751,"t_done_unix":1779769643.3434436},{"request_id":"1262354:5:1269336:105","session_id":"1262354","input_length":28906,"t_proxy_recv":338777.324139575,"t_decision_unix":1779769642.3380811,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9919047948522798,"affinity_num_requests":3,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (234 < 8000)","cache_hit":28672,"estimated_new_tokens":234,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":28906,"score_linear":135235.0,"score_lmetric":284250},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":28906,"score_linear":102074.0,"score_lmetric":127396},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28906,"score_linear":126924.0,"score_lmetric":136886},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27603,"ongoing_decode_tokens":27603,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28906,"score_linear":27603.0,"score_lmetric":57812},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":28906,"score_linear":27639.0,"score_lmetric":28906},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":28906,"score_linear":125063.0,"score_lmetric":292578},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":377,"cache_hit":0,"new_prefill":28906,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":50179,"ongoing_decode_tokens":50179,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":210,"cache_hit":28672,"new_prefill":234,"score_linear":21507.0,"score_lmetric":702}],"chosen_score_linear":21507.0,"chosen_score_lmetric":702,"t_first_token":338777.399135091,"t_first_token_unix":1779769642.4130788,"t_done":338778.376168434,"t_done_unix":1779769643.3901138},{"request_id":"1268831:1:1268831:101","session_id":"1268831","input_length":12195,"t_proxy_recv":338775.470931429,"t_decision_unix":1779769640.484873,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":12195,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":12195,"score_linear":135235.0,"score_lmetric":250828},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":12195,"score_linear":102074.0,"score_lmetric":110685},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12195,"score_linear":126924.0,"score_lmetric":120175},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12195,"score_linear":15283.0,"score_lmetric":12195},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":12195,"score_linear":27639.0,"score_lmetric":12195},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":12195,"score_linear":125063.0,"score_lmetric":259156},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":12195,"score_linear":18638.0,"score_lmetric":30833},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":210,"cache_hit":0,"new_prefill":12195,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338776.483038347,"t_first_token_unix":1779769641.4969819,"t_done":338779.118372264,"t_done_unix":1779769644.1323156},{"request_id":"1268861:1:1268861:102","session_id":"1268861","input_length":12320,"t_proxy_recv":338775.590054145,"t_decision_unix":1779769640.6039956,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":12320,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":12320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":12320,"score_linear":135235.0,"score_lmetric":251078},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":12320,"score_linear":102074.0,"score_lmetric":110810},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12320,"score_linear":126924.0,"score_lmetric":120300},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12320,"score_linear":15283.0,"score_lmetric":12320},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":12320,"score_linear":27639.0,"score_lmetric":12320},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":12320,"score_linear":125063.0,"score_lmetric":259406},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":12320,"score_linear":18638.0,"score_lmetric":30958},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":12195,"ongoing_decode_tokens":0,"pending_prefill_tokens":12195,"num_requests":1,"active_p_offloads":0,"cached_blocks":210,"cache_hit":0,"new_prefill":12320,"score_linear":12195.0,"score_lmetric":24515}],"chosen_score_linear":15283.0,"chosen_score_lmetric":12320,"t_first_token":338776.613757083,"t_first_token_unix":1779769641.6277006,"t_done":338779.361750365,"t_done_unix":1779769644.375694},{"request_id":"1269373:1:1269373:106","session_id":"1269373","input_length":14104,"t_proxy_recv":338777.448513616,"t_decision_unix":1779769642.4624548,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6424 < 8000)","cache_hit":7680,"estimated_new_tokens":6424,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":10008,"score_linear":131139.0,"score_lmetric":246454},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":14104,"score_linear":102074.0,"score_lmetric":112594},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14104,"score_linear":126924.0,"score_lmetric":122084},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27603,"ongoing_decode_tokens":27603,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14104,"score_linear":27603.0,"score_lmetric":28208},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":14104,"score_linear":27639.0,"score_lmetric":14104},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14104,"score_linear":125063.0,"score_lmetric":262974},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":377,"cache_hit":7680,"new_prefill":6424,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":65130,"ongoing_decode_tokens":65130,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":219,"cache_hit":0,"new_prefill":14104,"score_linear":65130.0,"score_lmetric":42312}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":338778.176368226,"t_first_token_unix":1779769643.1903172,"t_done":338779.511073487,"t_done_unix":1779769644.5250168},{"request_id":"1269901:1:1269901:107","session_id":"1269901","input_length":1323,"t_proxy_recv":338779.007185289,"t_decision_unix":1779769644.021126,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":1323,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1323 < 8000)","cache_hit":0,"estimated_new_tokens":1323,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":1323,"score_linear":135235.0,"score_lmetric":229084},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":1323,"score_linear":102074.0,"score_lmetric":99813},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":1323,"score_linear":126924.0,"score_lmetric":109303},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27603,"ongoing_decode_tokens":27603,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":1323,"score_linear":27603.0,"score_lmetric":2646},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":1323,"score_linear":27639.0,"score_lmetric":1323},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":1323,"score_linear":125063.0,"score_lmetric":237412},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14104,"ongoing_decode_tokens":14104,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":377,"cache_hit":0,"new_prefill":1323,"score_linear":14104.0,"score_lmetric":1323},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":12195,"ongoing_decode_tokens":12195,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":220,"cache_hit":0,"new_prefill":1323,"score_linear":12195.0,"score_lmetric":1323}],"chosen_score_linear":12195.0,"chosen_score_lmetric":1323,"t_first_token":338779.161502027,"t_first_token_unix":1779769644.1754453,"t_done":338779.783292825,"t_done_unix":1779769644.7972367},{"request_id":"1260327:2:1270005:108","session_id":"1260327","input_length":30828,"t_proxy_recv":338779.431981484,"t_decision_unix":1779769644.445923,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9964966913195796,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (108 < 8000)","cache_hit":30720,"estimated_new_tokens":108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":30828,"score_linear":135235.0,"score_lmetric":288094},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":30828,"score_linear":102074.0,"score_lmetric":129318},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":30828,"score_linear":126924.0,"score_lmetric":138808},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":30828,"score_linear":15283.0,"score_lmetric":30828},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":30828,"score_linear":27639.0,"score_lmetric":30828},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":30828,"score_linear":125063.0,"score_lmetric":296422},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14104,"ongoing_decode_tokens":14104,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":377,"cache_hit":0,"new_prefill":30828,"score_linear":14104.0,"score_lmetric":30828},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":1323,"ongoing_decode_tokens":1323,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":244,"cache_hit":30720,"new_prefill":108,"score_linear":-29397.0,"score_lmetric":108}],"chosen_score_linear":-29397.0,"chosen_score_lmetric":108,"t_first_token":338779.53205885,"t_first_token_unix":1779769644.5460024,"t_done":338779.917530779,"t_done_unix":1779769644.9314742},{"request_id":"1266440:2:1270501:112","session_id":"1266440","input_length":11128,"t_proxy_recv":338781.201287233,"t_decision_unix":1779769646.2152286,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":3,"affinity_cache_hit":5120,"affinity_cache_ratio":0.46010064701653486,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":11128,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"local_cost 1.39s <= migrate_cost 1.22s + margin 0.20s","v2_chosen_cache_hit":0,"v2_src_idx":2,"v2_src_cache_hit":5120,"v2_new_local":11128,"v2_new_src":6008,"v2_cost_local_s":1.391,"v2_cost_src_interf_s":0.751,"v2_cost_xfer_s":0.4736111111111111,"v2_cost_migrate_s":1.224611111111111,"cache_hit":0,"estimated_new_tokens":11128,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":11128,"score_linear":135235.0,"score_lmetric":248694},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":11128,"score_linear":102074.0,"score_lmetric":109618},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":5120,"new_prefill":6008,"score_linear":121804.0,"score_lmetric":113988},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":11128,"score_linear":15283.0,"score_lmetric":11128},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":11128,"score_linear":27639.0,"score_lmetric":11128},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":11128,"score_linear":125063.0,"score_lmetric":257022},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":11128,"score_linear":57803.0,"score_lmetric":18243},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":27927,"ongoing_decode_tokens":0,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":11128,"score_linear":27927.0,"score_lmetric":39055}],"chosen_score_linear":15283.0,"chosen_score_lmetric":11128,"t_first_token":338782.103041081,"t_first_token_unix":1779769647.1169848,"t_done":338782.320850664,"t_done_unix":1779769647.334794},{"request_id":"1256273:4:1259632:60","session_id":"1256273","input_length":12270,"t_proxy_recv":338743.475132859,"t_decision_unix":1779769608.4890742,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7680,"affinity_cache_ratio":0.6259168704156479,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4590 < 8000)","cache_hit":7680,"estimated_new_tokens":4590,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":12270,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":12270,"score_linear":80365.0,"score_lmetric":59840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":12270,"score_linear":68131.0,"score_lmetric":80401},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9250,"ongoing_decode_tokens":9250,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":12270,"score_linear":9250.0,"score_lmetric":12270},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":12270,"score_linear":90488.0,"score_lmetric":102758},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":112793,"ongoing_decode_tokens":0,"pending_prefill_tokens":112793,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":7680,"new_prefill":4590,"score_linear":105113.0,"score_lmetric":117383},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14291,"ongoing_decode_tokens":14291,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":12270,"score_linear":14291.0,"score_lmetric":12270},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":12270,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":105113.0,"chosen_score_lmetric":117383,"t_first_token":338782.238069557,"t_first_token_unix":1779769647.2520132,"t_done":338783.288377856,"t_done_unix":1779769648.3023214},{"request_id":"1258859:1:1258859:57","session_id":"1258859","input_length":112793,"t_proxy_recv":338740.886416397,"t_decision_unix":1779769605.9003575,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":112793,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":112793,"score_linear":80365.0,"score_lmetric":260886},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":112793,"score_linear":68131.0,"score_lmetric":180924},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":112793,"score_linear":90488.0,"score_lmetric":203281},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338781.999357363,"t_first_token_unix":1779769647.0133016,"t_done":338783.403580236,"t_done_unix":1779769648.4175243},{"request_id":"1263727:2:1270052:109","session_id":"1263727","input_length":57803,"t_proxy_recv":338779.627530479,"t_decision_unix":1779769644.641472,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":50688,"affinity_cache_ratio":0.876909502966974,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7115 < 8000)","cache_hit":50688,"estimated_new_tokens":7115,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":57803,"score_linear":135235.0,"score_lmetric":342044},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":57803,"score_linear":102074.0,"score_lmetric":156293},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":57803,"score_linear":126924.0,"score_lmetric":165783},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":57803,"score_linear":15283.0,"score_lmetric":57803},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":57803,"score_linear":27639.0,"score_lmetric":57803},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":57803,"score_linear":125063.0,"score_lmetric":350372},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":390,"cache_hit":50688,"new_prefill":7115,"score_linear":-50688.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":32151,"ongoing_decode_tokens":32151,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":244,"cache_hit":0,"new_prefill":57803,"score_linear":32151.0,"score_lmetric":115606}],"chosen_score_linear":-50688.0,"chosen_score_lmetric":0,"t_first_token":338782.189638855,"t_first_token_unix":1779769647.2035825,"t_done":338784.261116628,"t_done_unix":1779769649.2750602},{"request_id":"1271459:1:1271459:118","session_id":"1271459","input_length":751,"t_proxy_recv":338784.593776942,"t_decision_unix":1779769649.6077185,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":751,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (751 < 8000)","cache_hit":0,"estimated_new_tokens":751,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":751,"score_linear":135235.0,"score_lmetric":227940},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":751,"score_linear":102074.0,"score_lmetric":99241},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":751,"score_linear":126924.0,"score_lmetric":108731},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":751,"score_linear":15283.0,"score_lmetric":751},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":751,"score_linear":121069.0,"score_lmetric":188362},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":751,"score_linear":130878.0,"score_lmetric":36914},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28385,"ongoing_decode_tokens":28385,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":751,"score_linear":28385.0,"score_lmetric":751},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80149,"ongoing_decode_tokens":0,"pending_prefill_tokens":39189,"num_requests":3,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":751,"score_linear":80149.0,"score_lmetric":119820}],"chosen_score_linear":15283.0,"chosen_score_lmetric":751,"t_first_token":338784.695915577,"t_first_token_unix":1779769649.7098587,"t_done":338785.660228456,"t_done_unix":1779769650.6741722},{"request_id":"1271847:1:1271847:123","session_id":"1271847","input_length":2524,"t_proxy_recv":338785.889284295,"t_decision_unix":1779769650.9032254,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":2524,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2524 < 8000)","cache_hit":0,"estimated_new_tokens":2524,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":2524,"score_linear":135235.0,"score_lmetric":231486},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":2524,"score_linear":102074.0,"score_lmetric":101014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2524,"score_linear":150574.0,"score_lmetric":240660},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":404,"cache_hit":0,"new_prefill":2524,"score_linear":15283.0,"score_lmetric":2524},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":2524,"score_linear":121069.0,"score_lmetric":191908},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":2524,"score_linear":130878.0,"score_lmetric":40460},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":2524,"score_linear":64508.0,"score_lmetric":72933},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80149,"ongoing_decode_tokens":60701,"pending_prefill_tokens":7160,"num_requests":3,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":2524,"score_linear":80149.0,"score_lmetric":29052}],"chosen_score_linear":15283.0,"chosen_score_lmetric":2524,"t_first_token":338786.089665066,"t_first_token_unix":1779769651.1036084,"t_done":338786.265026336,"t_done_unix":1779769651.2789693},{"request_id":"1270340:1:1270340:111","session_id":"1270340","input_length":27927,"t_proxy_recv":338780.633531644,"t_decision_unix":1779769645.6474729,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":27927,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":27927,"score_linear":135235.0,"score_lmetric":282292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":27927,"score_linear":102074.0,"score_lmetric":126417},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":27927,"score_linear":126924.0,"score_lmetric":135907},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":27927,"score_linear":15283.0,"score_lmetric":27927},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":27927,"score_linear":27639.0,"score_lmetric":27927},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":27927,"score_linear":125063.0,"score_lmetric":290620},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":27927,"score_linear":57803.0,"score_lmetric":35042},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":27927,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338785.009009054,"t_first_token_unix":1779769650.022952,"t_done":338786.519937575,"t_done_unix":1779769651.533881},{"request_id":"1271991:1:1271991:124","session_id":"1271991","input_length":2851,"t_proxy_recv":338786.421097345,"t_decision_unix":1779769651.435039,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":2851,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2851 < 8000)","cache_hit":0,"estimated_new_tokens":2851,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":2851,"score_linear":135235.0,"score_lmetric":232140},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":2851,"score_linear":102074.0,"score_lmetric":101341},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2851,"score_linear":150574.0,"score_lmetric":241314},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":2851,"score_linear":15283.0,"score_lmetric":2851},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":2851,"score_linear":121069.0,"score_lmetric":192562},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":130878,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":2851,"score_linear":130878.0,"score_lmetric":5702},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":2851,"score_linear":64508.0,"score_lmetric":73914},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80149,"ongoing_decode_tokens":80149,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":2851,"score_linear":80149.0,"score_lmetric":8553}],"chosen_score_linear":15283.0,"chosen_score_lmetric":2851,"t_first_token":338786.609066588,"t_first_token_unix":1779769651.6230097,"t_done":338786.808402006,"t_done_unix":1779769651.8223455},{"request_id":"1271328:1:1271328:117","session_id":"1271328","input_length":17706,"t_proxy_recv":338784.18315839,"t_decision_unix":1779769649.1970997,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":17706,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":17706,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":17706,"score_linear":135235.0,"score_lmetric":261850},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":17706,"score_linear":102074.0,"score_lmetric":116196},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17706,"score_linear":126924.0,"score_lmetric":125686},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":17706,"score_linear":15283.0,"score_lmetric":17706},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":17706,"score_linear":121069.0,"score_lmetric":222272},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":17706,"score_linear":113172.0,"score_lmetric":17706},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86188,"ongoing_decode_tokens":86188,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":17706,"score_linear":86188.0,"score_lmetric":35412},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80149,"ongoing_decode_tokens":0,"pending_prefill_tokens":39189,"num_requests":3,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":17706,"score_linear":80149.0,"score_lmetric":170685}],"chosen_score_linear":113172.0,"chosen_score_lmetric":17706,"t_first_token":338786.018234244,"t_first_token_unix":1779769651.0321777,"t_done":338787.503847325,"t_done_unix":1779769652.5177906},{"request_id":"1272085:1:1272085:125","session_id":"1272085","input_length":5302,"t_proxy_recv":338786.758558261,"t_decision_unix":1779769651.7725,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":10604,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5302 < 8000)","cache_hit":0,"estimated_new_tokens":5302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":5302,"score_linear":135235.0,"score_lmetric":237042},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":5302,"score_linear":102074.0,"score_lmetric":103792},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":5302,"score_linear":150574.0,"score_lmetric":246216},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18134,"ongoing_decode_tokens":18134,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":5302,"score_linear":18134.0,"score_lmetric":10604},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":5302,"score_linear":121069.0,"score_lmetric":197464},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":130878,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":5302,"score_linear":130878.0,"score_lmetric":10604},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":5302,"score_linear":64508.0,"score_lmetric":81267},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80545,"ongoing_decode_tokens":80545,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":300,"cache_hit":0,"new_prefill":5302,"score_linear":80545.0,"score_lmetric":15906}],"chosen_score_linear":130878.0,"chosen_score_lmetric":10604,"t_first_token":338787.218659715,"t_first_token_unix":1779769652.2326033,"t_done":338788.149891355,"t_done_unix":1779769653.1638346},{"request_id":"1268831:2:1271087:116","session_id":"1268831","input_length":19448,"t_proxy_recv":338783.447835933,"t_decision_unix":1779769648.4617774,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":12288,"affinity_cache_ratio":0.6318387494858083,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7160 < 8000)","cache_hit":12288,"estimated_new_tokens":7160,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":19448,"score_linear":135235.0,"score_lmetric":265334},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":19448,"score_linear":102074.0,"score_lmetric":117938},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":19448,"score_linear":126924.0,"score_lmetric":127428},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":19448,"score_linear":15283.0,"score_lmetric":19448},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":19448,"score_linear":121069.0,"score_lmetric":225756},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":0,"pending_prefill_tokens":532,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":19448,"score_linear":113172.0,"score_lmetric":19980},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86188,"ongoing_decode_tokens":57803,"pending_prefill_tokens":10465,"num_requests":2,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":19448,"score_linear":86188.0,"score_lmetric":59826},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":0,"pending_prefill_tokens":32029,"num_requests":2,"active_p_offloads":0,"cached_blocks":246,"cache_hit":12288,"new_prefill":7160,"score_linear":48413.0,"score_lmetric":78378}],"chosen_score_linear":48413.0,"chosen_score_lmetric":78378,"t_first_token":338785.906825112,"t_first_token_unix":1779769650.9207683,"t_done":338788.776403503,"t_done_unix":1779769653.7903469},{"request_id":"1270340:2:1271641:121","session_id":"1270340","input_length":28323,"t_proxy_recv":338786.525247454,"t_decision_unix":1779769651.539189,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":27648,"affinity_cache_ratio":0.9761677788369876,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (675 < 8000)","cache_hit":27648,"estimated_new_tokens":675,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":28323,"score_linear":135235.0,"score_lmetric":283084},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":28323,"score_linear":102074.0,"score_lmetric":126813},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28323,"score_linear":150574.0,"score_lmetric":292258},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18134,"ongoing_decode_tokens":15283,"pending_prefill_tokens":2851,"num_requests":2,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":28323,"score_linear":18134.0,"score_lmetric":62348},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":28323,"score_linear":121069.0,"score_lmetric":243506},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":130878,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":28323,"score_linear":130878.0,"score_lmetric":56646},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":28323,"score_linear":64508.0,"score_lmetric":150330},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":52222,"ongoing_decode_tokens":52222,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":300,"cache_hit":27648,"new_prefill":675,"score_linear":24574.0,"score_lmetric":1350}],"chosen_score_linear":24574.0,"chosen_score_lmetric":1350,"t_first_token":338786.705774623,"t_first_token_unix":1779769651.719718,"t_done":338789.597374815,"t_done_unix":1779769654.6113183},{"request_id":"1271486:1:1271486:119","session_id":"1271486","input_length":21719,"t_proxy_recv":338784.693327601,"t_decision_unix":1779769649.707269,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":21719,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":21719,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":21719,"score_linear":135235.0,"score_lmetric":269876},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":21719,"score_linear":102074.0,"score_lmetric":120209},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21719,"score_linear":126924.0,"score_lmetric":129699},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":16034,"ongoing_decode_tokens":15283,"pending_prefill_tokens":751,"num_requests":2,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":21719,"score_linear":16034.0,"score_lmetric":44940},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":21719,"score_linear":121069.0,"score_lmetric":230298},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":21719,"score_linear":130878.0,"score_lmetric":78850},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28385,"ongoing_decode_tokens":28385,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":21719,"score_linear":28385.0,"score_lmetric":21719},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80149,"ongoing_decode_tokens":0,"pending_prefill_tokens":39189,"num_requests":3,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":21719,"score_linear":80149.0,"score_lmetric":182724}],"chosen_score_linear":28385.0,"chosen_score_lmetric":21719,"t_first_token":338787.07537177,"t_first_token_unix":1779769652.0893152,"t_done":338790.172712378,"t_done_unix":1779769655.186656},{"request_id":"1266668:2:1270661:115","session_id":"1266668","input_length":28385,"t_proxy_recv":338781.816406271,"t_decision_unix":1779769646.8303478,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17920,"affinity_cache_ratio":0.6313193588162762,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":17920,"estimated_new_tokens":10465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":24289,"score_linear":131139.0,"score_lmetric":275016},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":28385,"score_linear":102074.0,"score_lmetric":126875},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28385,"score_linear":126924.0,"score_lmetric":136365},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26411,"ongoing_decode_tokens":15283,"pending_prefill_tokens":11128,"num_requests":2,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":28385,"score_linear":26411.0,"score_lmetric":79026},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":28385,"score_linear":121069.0,"score_lmetric":243630},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":28385,"score_linear":125063.0,"score_lmetric":291536},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":390,"cache_hit":17920,"new_prefill":10465,"score_linear":39883.0,"score_lmetric":17580},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":0,"pending_prefill_tokens":32029,"num_requests":2,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":28385,"score_linear":60701.0,"score_lmetric":120828}],"chosen_score_linear":39883.0,"chosen_score_lmetric":17580,"t_first_token":338784.018164516,"t_first_token_unix":1779769649.032108,"t_done":338790.525588441,"t_done_unix":1779769655.5395317},{"request_id":"1262354:6:1270643:114","session_id":"1262354","input_length":32774,"t_proxy_recv":338781.736730776,"t_decision_unix":1779769646.7506723,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28672,"affinity_cache_ratio":0.8748398120461341,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4102 < 8000)","cache_hit":28672,"estimated_new_tokens":4102,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":32774,"score_linear":135235.0,"score_lmetric":291986},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":32774,"score_linear":102074.0,"score_lmetric":131264},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":32774,"score_linear":126924.0,"score_lmetric":140754},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26411,"ongoing_decode_tokens":15283,"pending_prefill_tokens":11128,"num_requests":2,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":32774,"score_linear":26411.0,"score_lmetric":87804},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":32774,"score_linear":121069.0,"score_lmetric":252408},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":32774,"score_linear":125063.0,"score_lmetric":300314},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":32774,"score_linear":57803.0,"score_lmetric":39889},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":27927,"ongoing_decode_tokens":0,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":246,"cache_hit":28672,"new_prefill":4102,"score_linear":-745.0,"score_lmetric":32029}],"chosen_score_linear":-745.0,"chosen_score_lmetric":32029,"t_first_token":338785.009590477,"t_first_token_unix":1779769650.0235336,"t_done":338792.513249315,"t_done_unix":1779769657.5271928},{"request_id":"1262727:1:1262727:72","session_id":"1262727","input_length":15283,"t_proxy_recv":338754.247879612,"t_decision_unix":1779769619.261821,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":15283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":0,"pending_prefill_tokens":1205,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":15283,"score_linear":55989.0,"score_lmetric":16488},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":15283,"score_linear":68131.0,"score_lmetric":83414},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":15283,"score_linear":113053.0,"score_lmetric":215712},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":15283,"score_linear":125063.0,"score_lmetric":265332},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":15283,"score_linear":21510.0,"score_lmetric":36793}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338755.656378492,"t_first_token_unix":1779769620.6703224,"t_done":338792.811905008,"t_done_unix":1779769657.8258483},{"request_id":"1260327:3:1273741:132","session_id":"1260327","input_length":31283,"t_proxy_recv":338792.791653233,"t_decision_unix":1779769657.8055947,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9820030048269028,"affinity_num_requests":0,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (563 < 8000)","cache_hit":30720,"estimated_new_tokens":563,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":31283,"score_linear":135235.0,"score_lmetric":289004},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":31283,"score_linear":120167.0,"score_lmetric":275252},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31283,"score_linear":150574.0,"score_lmetric":298178},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":413,"cache_hit":0,"new_prefill":31283,"score_linear":15283.0,"score_lmetric":31283},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":31283,"score_linear":121069.0,"score_lmetric":249426},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168260,"ongoing_decode_tokens":121189,"pending_prefill_tokens":7700,"num_requests":4,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":31283,"score_linear":168260.0,"score_lmetric":155932},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":79796,"ongoing_decode_tokens":57833,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":466,"cache_hit":0,"new_prefill":31283,"score_linear":79796.0,"score_lmetric":125132},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":324,"cache_hit":30720,"new_prefill":563,"score_linear":-30720.0,"score_lmetric":0}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":338792.996610911,"t_first_token_unix":1779769658.0105548,"t_done":338793.650331921,"t_done_unix":1779769658.6642752},{"request_id":"1272085:2:1273645:130","session_id":"1272085","input_length":8017,"t_proxy_recv":338792.390234399,"t_decision_unix":1779769657.4041755,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":5120,"affinity_cache_ratio":0.6386428838717725,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2897 < 8000)","cache_hit":5120,"estimated_new_tokens":2897,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":8017,"score_linear":135235.0,"score_lmetric":242472},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":8017,"score_linear":120167.0,"score_lmetric":228720},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8017,"score_linear":150574.0,"score_lmetric":251646},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":413,"cache_hit":0,"new_prefill":8017,"score_linear":15283.0,"score_lmetric":8017},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":8017,"score_linear":121069.0,"score_lmetric":202894},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135135,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":309,"cache_hit":5120,"new_prefill":2897,"score_linear":130015.0,"score_lmetric":5794},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":79796,"ongoing_decode_tokens":57833,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":466,"cache_hit":0,"new_prefill":8017,"score_linear":79796.0,"score_lmetric":32068},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":315,"cache_hit":0,"new_prefill":8017,"score_linear":32774.0,"score_lmetric":8017}],"chosen_score_linear":130015.0,"chosen_score_lmetric":5794,"t_first_token":338792.768892094,"t_first_token_unix":1779769657.7828355,"t_done":338795.015167037,"t_done_unix":1779769660.0291102},{"request_id":"1272313:1:1272313:126","session_id":"1272313","input_length":9718,"t_proxy_recv":338787.670394986,"t_decision_unix":1779769652.6843362,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":6114,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2038 < 8000)","cache_hit":7680,"estimated_new_tokens":2038,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":5622,"score_linear":131139.0,"score_lmetric":237682},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":9718,"score_linear":102074.0,"score_lmetric":108208},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9718,"score_linear":150574.0,"score_lmetric":255048},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":413,"cache_hit":0,"new_prefill":9718,"score_linear":15283.0,"score_lmetric":9718},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":9718,"score_linear":121069.0,"score_lmetric":206296},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118474,"ongoing_decode_tokens":118474,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":299,"cache_hit":0,"new_prefill":9718,"score_linear":118474.0,"score_lmetric":19436},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":64508,"ongoing_decode_tokens":64508,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":403,"cache_hit":7680,"new_prefill":2038,"score_linear":56828.0,"score_lmetric":6114},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80545,"ongoing_decode_tokens":80545,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":300,"cache_hit":0,"new_prefill":9718,"score_linear":80545.0,"score_lmetric":29154}],"chosen_score_linear":56828.0,"chosen_score_lmetric":6114,"t_first_token":338787.920667235,"t_first_token_unix":1779769652.9346104,"t_done":338795.089560646,"t_done_unix":1779769660.103504},{"request_id":"1269373:2:1271580:120","session_id":"1269373","input_length":14404,"t_proxy_recv":338785.006830108,"t_decision_unix":1779769650.0207713,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.995279089141905,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (68 < 8000)","cache_hit":14336,"estimated_new_tokens":68,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":10308,"score_linear":131139.0,"score_lmetric":247054},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":14404,"score_linear":102074.0,"score_lmetric":112894},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14404,"score_linear":126924.0,"score_lmetric":122384},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":16034,"ongoing_decode_tokens":16034,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":14404,"score_linear":16034.0,"score_lmetric":28808},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":14404,"score_linear":121069.0,"score_lmetric":215668},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":14404,"score_linear":130878.0,"score_lmetric":64220},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50104,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21719,"num_requests":2,"active_p_offloads":0,"cached_blocks":403,"cache_hit":14336,"new_prefill":68,"score_linear":35768.0,"score_lmetric":43574},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80149,"ongoing_decode_tokens":0,"pending_prefill_tokens":39189,"num_requests":3,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":14404,"score_linear":80149.0,"score_lmetric":160779}],"chosen_score_linear":35768.0,"chosen_score_lmetric":43574,"t_first_token":338787.077226255,"t_first_token_unix":1779769652.0911694,"t_done":338795.238305346,"t_done_unix":1779769660.2522485},{"request_id":"1258859:2:1264639:81","session_id":"1258859","input_length":113172,"t_proxy_recv":338783.421357581,"t_decision_unix":1779769648.4352992,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":112640,"affinity_cache_ratio":0.995299190612519,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (532 < 8000)","cache_hit":112640,"estimated_new_tokens":532,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":113172,"score_linear":135235.0,"score_lmetric":452782},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":113172,"score_linear":102074.0,"score_lmetric":211662},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":113172,"score_linear":126924.0,"score_lmetric":221152},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":113172,"score_linear":15283.0,"score_lmetric":113172},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":113172,"score_linear":121069.0,"score_lmetric":413204},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":112640,"new_prefill":532,"score_linear":-112640.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86188,"ongoing_decode_tokens":57803,"pending_prefill_tokens":10465,"num_requests":2,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":113172,"score_linear":86188.0,"score_lmetric":247274},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60701,"ongoing_decode_tokens":0,"pending_prefill_tokens":32029,"num_requests":2,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":113172,"score_linear":60701.0,"score_lmetric":290402}],"chosen_score_linear":-112640.0,"chosen_score_lmetric":0,"t_first_token":338783.771656619,"t_first_token_unix":1779769648.7856011,"t_done":338795.753154667,"t_done_unix":1779769660.7670984},{"request_id":"1274405:1:1274405:134","session_id":"1274405","input_length":6208,"t_proxy_recv":338795.188033232,"t_decision_unix":1779769660.2019749,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6208 < 8000)","cache_hit":0,"estimated_new_tokens":6208,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":6208,"score_linear":135235.0,"score_lmetric":238854},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":6208,"score_linear":120167.0,"score_lmetric":225102},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":6208,"score_linear":150574.0,"score_lmetric":248028},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":448,"cache_hit":0,"new_prefill":6208,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":6208,"score_linear":121069.0,"score_lmetric":199276},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160243,"ongoing_decode_tokens":138280,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":314,"cache_hit":0,"new_prefill":6208,"score_linear":160243.0,"score_lmetric":18624},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":48115,"ongoing_decode_tokens":48115,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":487,"cache_hit":0,"new_prefill":6208,"score_linear":48115.0,"score_lmetric":12416},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":325,"cache_hit":0,"new_prefill":6208,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338795.601408505,"t_first_token_unix":1779769660.615352,"t_done":338796.079644746,"t_done_unix":1779769661.0935879},{"request_id":"1266668:3:1273119:128","session_id":"1266668","input_length":33711,"t_proxy_recv":338790.535087904,"t_decision_unix":1779769655.5490294,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":28672,"affinity_cache_ratio":0.850523567974845,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5039 < 8000)","cache_hit":28672,"estimated_new_tokens":5039,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":29615,"score_linear":131139.0,"score_lmetric":285668},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":33711,"score_linear":120167.0,"score_lmetric":280108},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33711,"score_linear":150574.0,"score_lmetric":303034},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":413,"cache_hit":0,"new_prefill":33711,"score_linear":15283.0,"score_lmetric":33711},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":33711,"score_linear":121069.0,"score_lmetric":254282},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":33711,"score_linear":113172.0,"score_lmetric":33711},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24122,"ongoing_decode_tokens":24122,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":466,"cache_hit":28672,"new_prefill":5039,"score_linear":-4550.0,"score_lmetric":10078},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":315,"cache_hit":0,"new_prefill":33711,"score_linear":32774.0,"score_lmetric":33711}],"chosen_score_linear":-4550.0,"chosen_score_lmetric":10078,"t_first_token":338791.688912856,"t_first_token_unix":1779769656.7028568,"t_done":338796.434468807,"t_done_unix":1779769661.448412},{"request_id":"1274890:1:1274890:135","session_id":"1274890","input_length":10506,"t_proxy_recv":338796.712702405,"t_decision_unix":1779769661.7266438,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":10506,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":10506,"score_linear":135235.0,"score_lmetric":247450},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":10506,"score_linear":120167.0,"score_lmetric":233698},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10506,"score_linear":150574.0,"score_lmetric":256624},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":448,"cache_hit":0,"new_prefill":10506,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":10506,"score_linear":121069.0,"score_lmetric":207872},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":47071,"ongoing_decode_tokens":25108,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":316,"cache_hit":0,"new_prefill":10506,"score_linear":47071.0,"score_lmetric":21012},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":10506,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":337,"cache_hit":0,"new_prefill":10506,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338797.532872578,"t_first_token_unix":1779769662.5468156,"t_done":338797.78666339,"t_done_unix":1779769662.8006072},{"request_id":"1271328:2:1273711:131","session_id":"1271328","input_length":25108,"t_proxy_recv":338792.65214769,"t_decision_unix":1779769657.666089,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":17408,"affinity_cache_ratio":0.6933248367054325,"affinity_num_requests":3,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7700 < 8000)","cache_hit":17408,"estimated_new_tokens":7700,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":25108,"score_linear":135235.0,"score_lmetric":276654},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":25108,"score_linear":120167.0,"score_lmetric":262902},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":25108,"score_linear":150574.0,"score_lmetric":285828},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":413,"cache_hit":0,"new_prefill":25108,"score_linear":15283.0,"score_lmetric":25108},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":25108,"score_linear":121069.0,"score_lmetric":237076},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":143152,"ongoing_decode_tokens":113172,"pending_prefill_tokens":2897,"num_requests":3,"active_p_offloads":0,"cached_blocks":309,"cache_hit":17408,"new_prefill":7700,"score_linear":125744.0,"score_lmetric":31791},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":79796,"ongoing_decode_tokens":57833,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":466,"cache_hit":0,"new_prefill":25108,"score_linear":79796.0,"score_lmetric":100432},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":324,"cache_hit":0,"new_prefill":25108,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":125744.0,"chosen_score_lmetric":31791,"t_first_token":338794.000447058,"t_first_token_unix":1779769659.0143907,"t_done":338798.347206196,"t_done_unix":1779769663.3611498},{"request_id":"1275433:1:1275433:137","session_id":"1275433","input_length":7794,"t_proxy_recv":338798.462734433,"t_decision_unix":1779769663.476676,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7794 < 8000)","cache_hit":0,"estimated_new_tokens":7794,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":7794,"score_linear":135235.0,"score_lmetric":242026},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":7794,"score_linear":120167.0,"score_lmetric":228274},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":7794,"score_linear":150574.0,"score_lmetric":251200},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":7794,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":7794,"score_linear":121069.0,"score_lmetric":202448},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21963,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":331,"cache_hit":0,"new_prefill":7794,"score_linear":21963.0,"score_lmetric":7794},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":7794,"score_linear":114612.0,"score_lmetric":122406},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":337,"cache_hit":0,"new_prefill":7794,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338799.003894711,"t_first_token_unix":1779769664.0178382,"t_done":338799.05597362,"t_done_unix":1779769664.0699167},{"request_id":"1273203:1:1273203:129","session_id":"1273203","input_length":21963,"t_proxy_recv":338790.747699017,"t_decision_unix":1779769655.7616405,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":21963,"tie_break_used":true,"v2_pd_sep":true,"v2_decision":"pd_sep","v2_reason":"local_cost 4.39s > migrate_cost 4.17s + margin 0.20s","v2_chosen_cache_hit":0,"v2_src_idx":6,"v2_src_cache_hit":12800,"v2_new_local":21963,"v2_new_src":9163,"v2_cost_local_s":4.392600000000001,"v2_cost_src_interf_s":3.436125,"v2_cost_xfer_s":0.7340277777777777,"v2_cost_migrate_s":4.170152777777778,"cache_hit":0,"estimated_new_tokens":21963,"route_class":"PD_SEP_V2","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":21963,"score_linear":135235.0,"score_lmetric":270364},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":21963,"score_linear":120167.0,"score_lmetric":256612},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21963,"score_linear":150574.0,"score_lmetric":279538},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":413,"cache_hit":0,"new_prefill":21963,"score_linear":15283.0,"score_lmetric":21963},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":21963,"score_linear":121069.0,"score_lmetric":230786},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":21963,"score_linear":113172.0,"score_lmetric":21963},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57833,"ongoing_decode_tokens":24122,"pending_prefill_tokens":5039,"num_requests":3,"active_p_offloads":0,"cached_blocks":466,"cache_hit":12800,"new_prefill":9163,"score_linear":45033.0,"score_lmetric":42606},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":315,"cache_hit":0,"new_prefill":21963,"score_linear":32774.0,"score_lmetric":21963}],"chosen_score_linear":113172.0,"chosen_score_lmetric":21963,"v2_src_url":"http://127.0.0.1:8006","t_prefill_sent":338790.748202167,"t_prefill_sent_unix":1779769655.7621453,"t_prefill_done":338793.096791479,"t_prefill_done_unix":1779769658.110735,"t_decode_sent":338793.097086976,"t_decode_sent_unix":1779769658.11103,"t_first_token":338794.433215252,"t_first_token_unix":1779769659.4471593,"t_done":338799.23040825,"t_done_unix":1779769664.2443516},{"request_id":"1275433:2:1276027:139","session_id":"1275433","input_length":7463,"t_proxy_recv":338800.735455909,"t_decision_unix":1779769665.7493975,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":7168,"affinity_cache_ratio":0.960471660190272,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (295 < 8000)","cache_hit":7168,"estimated_new_tokens":295,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":7463,"score_linear":135235.0,"score_lmetric":241364},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":7463,"score_linear":120167.0,"score_lmetric":227612},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":7463,"score_linear":150574.0,"score_lmetric":250538},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":483,"cache_hit":7168,"new_prefill":295,"score_linear":-7168.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":7463,"score_linear":121069.0,"score_lmetric":201786},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":374,"cache_hit":0,"new_prefill":7463,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":7463,"score_linear":125251.0,"score_lmetric":245972},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":337,"cache_hit":0,"new_prefill":7463,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":338800.763517992,"t_first_token_unix":1779769665.7774613,"t_done":338800.77298309,"t_done_unix":1779769665.786926},{"request_id":"1264679:1:1264679:82","session_id":"1264679","input_length":110048,"t_proxy_recv":338761.352442044,"t_decision_unix":1779769626.3663828,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":110048,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":110048,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":110048,"score_linear":55989.0,"score_lmetric":110048},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":110048,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":110048,"score_linear":71575.0,"score_lmetric":332680},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":110048,"score_linear":113053.0,"score_lmetric":405242},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":110048,"score_linear":125063.0,"score_lmetric":454862},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":110048,"score_linear":50783.0,"score_lmetric":160831},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":110048,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338801.078130001,"t_first_token_unix":1779769666.0920732,"t_done":338802.165039486,"t_done_unix":1779769667.1789832},{"request_id":"1276111:1:1276111:140","session_id":"1276111","input_length":15783,"t_proxy_recv":338801.0936575,"t_decision_unix":1779769666.107599,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":15783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":135235,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":15783,"score_linear":135235.0,"score_lmetric":31566},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":15783,"score_linear":120167.0,"score_lmetric":244252},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":15783,"score_linear":150574.0,"score_lmetric":267178},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":483,"cache_hit":0,"new_prefill":15783,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":15783,"score_linear":121069.0,"score_lmetric":218426},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":374,"cache_hit":0,"new_prefill":15783,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":15783,"score_linear":125251.0,"score_lmetric":262612},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":337,"cache_hit":0,"new_prefill":15783,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338802.527236187,"t_first_token_unix":1779769667.5411797,"t_done":338802.713933335,"t_done_unix":1779769667.727877},{"request_id":"1276136:1:1276136:141","session_id":"1276136","input_length":17280,"t_proxy_recv":338801.168413043,"t_decision_unix":1779769666.1823545,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":17280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":135235,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":17280,"score_linear":135235.0,"score_lmetric":34560},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":17280,"score_linear":120167.0,"score_lmetric":247246},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17280,"score_linear":150574.0,"score_lmetric":270172},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15783,"ongoing_decode_tokens":0,"pending_prefill_tokens":15783,"num_requests":1,"active_p_offloads":0,"cached_blocks":483,"cache_hit":0,"new_prefill":17280,"score_linear":15783.0,"score_lmetric":33063},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":17280,"score_linear":121069.0,"score_lmetric":221420},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":374,"cache_hit":0,"new_prefill":17280,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":17280,"score_linear":125251.0,"score_lmetric":265606},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":337,"cache_hit":0,"new_prefill":17280,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338802.827684479,"t_first_token_unix":1779769667.8416278,"t_done":338803.209168731,"t_done_unix":1779769668.2231119},{"request_id":"1276498:1:1276498:142","session_id":"1276498","input_length":8704,"t_proxy_recv":338802.415108363,"t_decision_unix":1779769667.42905,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8704,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":8704,"score_linear":25187.0,"score_lmetric":8704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":8704,"score_linear":120167.0,"score_lmetric":230094},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8704,"score_linear":150574.0,"score_lmetric":253020},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15783,"ongoing_decode_tokens":0,"pending_prefill_tokens":15783,"num_requests":1,"active_p_offloads":0,"cached_blocks":483,"cache_hit":0,"new_prefill":8704,"score_linear":15783.0,"score_lmetric":24487},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":8704,"score_linear":121069.0,"score_lmetric":204268},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":17280,"ongoing_decode_tokens":0,"pending_prefill_tokens":17280,"num_requests":1,"active_p_offloads":0,"cached_blocks":374,"cache_hit":0,"new_prefill":8704,"score_linear":17280.0,"score_lmetric":25984},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":8704,"score_linear":125251.0,"score_lmetric":248454},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":337,"cache_hit":0,"new_prefill":8704,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338803.057115464,"t_first_token_unix":1779769668.0710588,"t_done":338803.34830719,"t_done_unix":1779769668.3622506},{"request_id":"1274405:2:1276717:144","session_id":"1274405","input_length":6360,"t_proxy_recv":338803.197913944,"t_decision_unix":1779769668.2118554,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.9660377358490566,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (216 < 8000)","cache_hit":6144,"estimated_new_tokens":216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":6360,"score_linear":25187.0,"score_lmetric":6360},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":6360,"score_linear":120167.0,"score_lmetric":225406},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":6360,"score_linear":150574.0,"score_lmetric":248332},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":513,"cache_hit":0,"new_prefill":6360,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":6360,"score_linear":121069.0,"score_lmetric":199580},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":17280,"ongoing_decode_tokens":17280,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":374,"cache_hit":0,"new_prefill":6360,"score_linear":17280.0,"score_lmetric":6360},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":6360,"score_linear":174631.0,"score_lmetric":412413},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":8704,"ongoing_decode_tokens":8704,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":337,"cache_hit":6144,"new_prefill":216,"score_linear":2560.0,"score_lmetric":216}],"chosen_score_linear":2560.0,"chosen_score_lmetric":216,"t_first_token":338803.238684137,"t_first_token_unix":1779769668.252627,"t_done":338803.427365262,"t_done_unix":1779769668.4413087},{"request_id":"1271991:2:1276999:145","session_id":"1271991","input_length":11292,"t_proxy_recv":338804.185056785,"t_decision_unix":1779769669.198998,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":2560,"affinity_cache_ratio":0.2267091746369111,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":2560,"estimated_new_tokens":8732,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":11292,"score_linear":25187.0,"score_lmetric":11292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":11292,"score_linear":120167.0,"score_lmetric":235270},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":11292,"score_linear":150574.0,"score_lmetric":258196},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":513,"cache_hit":2560,"new_prefill":8732,"score_linear":-2560.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":11292,"score_linear":121069.0,"score_lmetric":209444},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":407,"cache_hit":0,"new_prefill":11292,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":11292,"score_linear":174631.0,"score_lmetric":427209},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":354,"cache_hit":0,"new_prefill":11292,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-2560.0,"chosen_score_lmetric":0,"t_first_token":338804.936808186,"t_first_token_unix":1779769669.9507525,"t_done":338805.098623856,"t_done_unix":1779769670.112568},{"request_id":"1277533:1:1277533:150","session_id":"1277533","input_length":4729,"t_proxy_recv":338806.219067618,"t_decision_unix":1779769671.233009,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4729 < 8000)","cache_hit":0,"estimated_new_tokens":4729,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":4729,"score_linear":25187.0,"score_lmetric":4729},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":4729,"score_linear":120167.0,"score_lmetric":222144},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":4729,"score_linear":150574.0,"score_lmetric":245070},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":9813,"num_requests":1,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":4729,"score_linear":27221.0,"score_lmetric":14542},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":4729,"score_linear":121069.0,"score_lmetric":196318},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":407,"cache_hit":0,"new_prefill":4729,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":4729,"score_linear":191531.0,"score_lmetric":551568},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":65369,"ongoing_decode_tokens":33524,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":354,"cache_hit":0,"new_prefill":4729,"score_linear":65369.0,"score_lmetric":34236}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338806.521334194,"t_first_token_unix":1779769671.5352776,"t_done":338806.664684942,"t_done_unix":1779769671.6786282},{"request_id":"1277428:1:1277428:149","session_id":"1277428","input_length":27221,"t_proxy_recv":338805.788108067,"t_decision_unix":1779769670.8020494,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":17408,"estimated_new_tokens":9813,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":27221,"score_linear":25187.0,"score_lmetric":27221},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":3584,"new_prefill":23637,"score_linear":116583.0,"score_lmetric":259960},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":27221,"score_linear":150574.0,"score_lmetric":290054},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":530,"cache_hit":17408,"new_prefill":9813,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":27221,"score_linear":121069.0,"score_lmetric":241302},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":407,"cache_hit":0,"new_prefill":27221,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":27221,"score_linear":191531.0,"score_lmetric":641536},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":65369,"ongoing_decode_tokens":33524,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":354,"cache_hit":0,"new_prefill":27221,"score_linear":65369.0,"score_lmetric":79220}],"chosen_score_linear":-17408.0,"chosen_score_lmetric":0,"t_first_token":338807.573897187,"t_first_token_unix":1779769672.5878406,"t_done":338808.188803358,"t_done_unix":1779769673.2027469},{"request_id":"1260327:4:1277974:154","session_id":"1260327","input_length":31430,"t_proxy_recv":338807.841166685,"t_decision_unix":1779769672.855108,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9937002863506205,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (198 < 8000)","cache_hit":31232,"estimated_new_tokens":198,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":31430,"score_linear":34071.0,"score_lmetric":62860},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":31430,"score_linear":120167.0,"score_lmetric":275546},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31430,"score_linear":150574.0,"score_lmetric":298472},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27221,"ongoing_decode_tokens":27221,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":31430,"score_linear":27221.0,"score_lmetric":31430},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":31430,"score_linear":121069.0,"score_lmetric":249720},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46795,"ongoing_decode_tokens":22573,"pending_prefill_tokens":24222,"num_requests":2,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":31430,"score_linear":46795.0,"score_lmetric":111304},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":31430,"score_linear":191531.0,"score_lmetric":658372},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":65369,"ongoing_decode_tokens":65369,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":354,"cache_hit":31232,"new_prefill":198,"score_linear":34137.0,"score_lmetric":396}],"chosen_score_linear":34137.0,"chosen_score_lmetric":396,"t_first_token":338807.93155346,"t_first_token_unix":1779769672.9454966,"t_done":338808.667738283,"t_done_unix":1779769673.6816819},{"request_id":"1268831:3:1277112:147","session_id":"1268831","input_length":31845,"t_proxy_recv":338804.632816324,"t_decision_unix":1779769669.6467578,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19456,"affinity_cache_ratio":0.6109593342753965,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":19456,"estimated_new_tokens":12389,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":31845,"score_linear":25187.0,"score_lmetric":31845},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":31845,"score_linear":120167.0,"score_lmetric":276376},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31845,"score_linear":150574.0,"score_lmetric":299302},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":11292,"ongoing_decode_tokens":0,"pending_prefill_tokens":8732,"num_requests":1,"active_p_offloads":0,"cached_blocks":513,"cache_hit":0,"new_prefill":31845,"score_linear":11292.0,"score_lmetric":40577},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":31845,"score_linear":121069.0,"score_lmetric":250550},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":407,"cache_hit":0,"new_prefill":31845,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":31845,"score_linear":174631.0,"score_lmetric":488868},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33524,"ongoing_decode_tokens":0,"pending_prefill_tokens":244,"num_requests":1,"active_p_offloads":0,"cached_blocks":354,"cache_hit":19456,"new_prefill":12389,"score_linear":14068.0,"score_lmetric":12633}],"chosen_score_linear":14068.0,"chosen_score_lmetric":12633,"t_first_token":338807.000926329,"t_first_token_unix":1779769672.0148697,"t_done":338809.306619718,"t_done_unix":1779769674.3205638},{"request_id":"1262354:7:1277104:146","session_id":"1262354","input_length":33524,"t_proxy_recv":338804.592284975,"t_decision_unix":1779769669.6062264,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33280,"affinity_cache_ratio":0.992721632263453,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (244 < 8000)","cache_hit":33280,"estimated_new_tokens":244,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":33524,"score_linear":25187.0,"score_lmetric":33524},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":33524,"score_linear":120167.0,"score_lmetric":279734},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33524,"score_linear":150574.0,"score_lmetric":302660},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":11292,"ongoing_decode_tokens":0,"pending_prefill_tokens":8732,"num_requests":1,"active_p_offloads":0,"cached_blocks":513,"cache_hit":0,"new_prefill":33524,"score_linear":11292.0,"score_lmetric":42256},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":33524,"score_linear":121069.0,"score_lmetric":253908},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":407,"cache_hit":0,"new_prefill":33524,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":33524,"score_linear":174631.0,"score_lmetric":493905},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":354,"cache_hit":33280,"new_prefill":244,"score_linear":-33280.0,"score_lmetric":0}],"chosen_score_linear":-33280.0,"chosen_score_lmetric":0,"t_first_token":338804.673615171,"t_first_token_unix":1779769669.6875584,"t_done":338810.199176648,"t_done_unix":1779769675.213123},{"request_id":"1271459:2:1277544:151","session_id":"1271459","input_length":8884,"t_proxy_recv":338806.258709045,"t_decision_unix":1779769671.2726507,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":0,"affinity_cache_hit":512,"affinity_cache_ratio":0.05763169743358847,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":8884,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (512 < 4000)","cache_hit":0,"estimated_new_tokens":8884,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":8884,"score_linear":25187.0,"score_lmetric":8884},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":8884,"score_linear":120167.0,"score_lmetric":230454},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8884,"score_linear":150574.0,"score_lmetric":253380},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":9813,"num_requests":1,"active_p_offloads":0,"cached_blocks":530,"cache_hit":512,"new_prefill":8372,"score_linear":26709.0,"score_lmetric":18185},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":8884,"score_linear":121069.0,"score_lmetric":204628},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":4729,"ongoing_decode_tokens":0,"pending_prefill_tokens":4729,"num_requests":1,"active_p_offloads":0,"cached_blocks":407,"cache_hit":0,"new_prefill":8884,"score_linear":4729.0,"score_lmetric":13613},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":8884,"score_linear":191531.0,"score_lmetric":568188},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":65369,"ongoing_decode_tokens":33524,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":354,"cache_hit":0,"new_prefill":8884,"score_linear":65369.0,"score_lmetric":42546}],"chosen_score_linear":25187.0,"chosen_score_lmetric":8884,"t_first_token":338806.940227628,"t_first_token_unix":1779769671.9541717,"t_done":338810.35115404,"t_done_unix":1779769675.365097},{"request_id":"1265942:3:1272453:127","session_id":"1265942","input_length":18093,"t_proxy_recv":338788.221203513,"t_decision_unix":1779769653.2351453,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":10240,"affinity_cache_ratio":0.5659647377438789,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7853 < 8000)","cache_hit":10240,"estimated_new_tokens":7853,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":18093,"score_linear":135235.0,"score_lmetric":262624},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":10240,"new_prefill":7853,"score_linear":91834.0,"score_lmetric":106343},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18093,"score_linear":150574.0,"score_lmetric":271798},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":413,"cache_hit":0,"new_prefill":18093,"score_linear":15283.0,"score_lmetric":18093},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":18093,"score_linear":121069.0,"score_lmetric":223046},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":18093,"score_linear":113172.0,"score_lmetric":18093},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":74226,"ongoing_decode_tokens":74226,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":18093,"score_linear":74226.0,"score_lmetric":72372},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80545,"ongoing_decode_tokens":80545,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":300,"cache_hit":0,"new_prefill":18093,"score_linear":80545.0,"score_lmetric":54279}],"chosen_score_linear":91834.0,"chosen_score_lmetric":106343,"t_first_token":338809.857082159,"t_first_token_unix":1779769674.8710253,"t_done":338810.942779943,"t_done_unix":1779769675.9567254},{"request_id":"1276498:2:1278996:157","session_id":"1276498","input_length":16650,"t_proxy_recv":338811.369316207,"t_decision_unix":1779769676.3832576,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":8704,"affinity_cache_ratio":0.5227627627627628,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7946 < 8000)","cache_hit":8704,"estimated_new_tokens":7946,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":16650,"score_linear":25187.0,"score_lmetric":16650},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":182055,"ongoing_decode_tokens":160826,"pending_prefill_tokens":3309,"num_requests":3,"active_p_offloads":0,"cached_blocks":300,"cache_hit":0,"new_prefill":16650,"score_linear":182055.0,"score_lmetric":59877},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":16650,"score_linear":150574.0,"score_lmetric":268912},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":16650,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":121069,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":16650,"score_linear":121069.0,"score_lmetric":33300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":164892,"ongoing_decode_tokens":46795,"pending_prefill_tokens":4433,"num_requests":3,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":16650,"score_linear":164892.0,"score_lmetric":63249},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":16650,"score_linear":191531.0,"score_lmetric":599252},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":379,"cache_hit":8704,"new_prefill":7946,"score_linear":-8704.0,"score_lmetric":0}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":338812.322286565,"t_first_token_unix":1779769677.3362308,"t_done":338812.656348638,"t_done_unix":1779769677.6702921},{"request_id":"1270606:1:1270606:113","session_id":"1270606","input_length":93430,"t_proxy_recv":338781.643211138,"t_decision_unix":1779769646.6571524,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":93430,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":93430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":93430,"score_linear":135235.0,"score_lmetric":413298},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":93430,"score_linear":102074.0,"score_lmetric":191920},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":93430,"score_linear":126924.0,"score_lmetric":201410},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26411,"ongoing_decode_tokens":15283,"pending_prefill_tokens":11128,"num_requests":2,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":93430,"score_linear":26411.0,"score_lmetric":209116},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":93430,"score_linear":27639.0,"score_lmetric":93430},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":93430,"score_linear":125063.0,"score_lmetric":421626},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57803,"ongoing_decode_tokens":0,"pending_prefill_tokens":7115,"num_requests":1,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":93430,"score_linear":57803.0,"score_lmetric":100545},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":27927,"ongoing_decode_tokens":0,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":93430,"score_linear":27927.0,"score_lmetric":121357}],"chosen_score_linear":27639.0,"chosen_score_lmetric":93430,"t_first_token":338810.687522972,"t_first_token_unix":1779769675.701466,"t_done":338812.742746679,"t_done_unix":1779769677.7566903},{"request_id":"1265942:4:1274182:133","session_id":"1265942","input_length":21229,"t_proxy_recv":338810.956265414,"t_decision_unix":1779769675.9702072,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":17920,"affinity_cache_ratio":0.8441283150407461,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3309 < 8000)","cache_hit":17920,"estimated_new_tokens":3309,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":21229,"score_linear":25187.0,"score_lmetric":21229},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":160826,"ongoing_decode_tokens":160826,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":300,"cache_hit":17920,"new_prefill":3309,"score_linear":142906.0,"score_lmetric":6618},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21229,"score_linear":150574.0,"score_lmetric":278070},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":21229,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":121069,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":21229,"score_linear":121069.0,"score_lmetric":42458},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":164892,"ongoing_decode_tokens":46795,"pending_prefill_tokens":4433,"num_requests":3,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":21229,"score_linear":164892.0,"score_lmetric":76986},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":21229,"score_linear":191531.0,"score_lmetric":617568},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":379,"cache_hit":0,"new_prefill":21229,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":142906.0,"chosen_score_lmetric":6618,"t_first_token":338811.521131825,"t_first_token_unix":1779769676.5350757,"t_done":338812.779449443,"t_done_unix":1779769677.7933924},{"request_id":"1268630:1:1268630:98","session_id":"1268630","input_length":102074,"t_proxy_recv":338775.000853309,"t_decision_unix":1779769640.014795,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":3584,"estimated_new_tokens":98490,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":102074,"score_linear":135235.0,"score_lmetric":430586},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":285,"cache_hit":3584,"new_prefill":98490,"score_linear":-3584.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":102074,"score_linear":126924.0,"score_lmetric":210054},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":4608,"new_prefill":97466,"score_linear":10675.0,"score_lmetric":97466},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":102074,"score_linear":27639.0,"score_lmetric":102074},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":102074,"score_linear":125063.0,"score_lmetric":438914},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":341,"cache_hit":0,"new_prefill":102074,"score_linear":18638.0,"score_lmetric":120712},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":2298,"ongoing_decode_tokens":0,"pending_prefill_tokens":2298,"num_requests":1,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":102074,"score_linear":2298.0,"score_lmetric":104372}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":338809.855783013,"t_first_token_unix":1779769674.8697264,"t_done":338813.114359191,"t_done_unix":1779769678.1283026},{"request_id":"1268831:4:1279168:158","session_id":"1268831","input_length":32346,"t_proxy_recv":338811.855723921,"t_decision_unix":1779769676.8696654,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":31744,"affinity_cache_ratio":0.9813887343102702,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (602 < 8000)","cache_hit":31744,"estimated_new_tokens":602,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":32346,"score_linear":25187.0,"score_lmetric":32346},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":182055,"ongoing_decode_tokens":182055,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":300,"cache_hit":0,"new_prefill":32346,"score_linear":182055.0,"score_lmetric":97038},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":32346,"score_linear":150574.0,"score_lmetric":300304},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":32346,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":121069,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":32346,"score_linear":121069.0,"score_lmetric":64692},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":164892,"ongoing_decode_tokens":46795,"pending_prefill_tokens":4433,"num_requests":3,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":32346,"score_linear":164892.0,"score_lmetric":110337},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":32346,"score_linear":191531.0,"score_lmetric":662036},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":16650,"ongoing_decode_tokens":0,"pending_prefill_tokens":7946,"num_requests":1,"active_p_offloads":0,"cached_blocks":379,"cache_hit":31744,"new_prefill":602,"score_linear":-15094.0,"score_lmetric":8548}],"chosen_score_linear":-15094.0,"chosen_score_lmetric":8548,"t_first_token":338812.417789887,"t_first_token_unix":1779769677.431734,"t_done":338813.649435596,"t_done_unix":1779769678.6633787},{"request_id":"1253804:5:1278488:156","session_id":"1253804","input_length":58752,"t_proxy_recv":338809.519690128,"t_decision_unix":1779769674.5336313,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9760348583877996,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1408 < 8000)","cache_hit":57344,"estimated_new_tokens":1408,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":58752,"score_linear":34071.0,"score_lmetric":117504},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":57344,"new_prefill":1408,"score_linear":62823.0,"score_lmetric":215502},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":58752,"score_linear":150574.0,"score_lmetric":353116},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":549,"cache_hit":3584,"new_prefill":55168,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":58752,"score_linear":121069.0,"score_lmetric":304364},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":164892,"ongoing_decode_tokens":22573,"pending_prefill_tokens":28655,"num_requests":3,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":58752,"score_linear":164892.0,"score_lmetric":262221},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":58752,"score_linear":191531.0,"score_lmetric":767660},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33524,"ongoing_decode_tokens":33524,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":378,"cache_hit":0,"new_prefill":58752,"score_linear":33524.0,"score_lmetric":58752}],"chosen_score_linear":62823.0,"chosen_score_lmetric":215502,"t_first_token":338810.323421257,"t_first_token_unix":1779769675.3373644,"t_done":338816.094208163,"t_done_unix":1779769681.1081522},{"request_id":"1279412:1:1279412:160","session_id":"1279412","input_length":20136,"t_proxy_recv":338812.836789387,"t_decision_unix":1779769677.850731,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":20136,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":20136,"score_linear":25187.0,"score_lmetric":20136},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":160826,"ongoing_decode_tokens":160826,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":20136,"score_linear":160826.0,"score_lmetric":40272},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":20136,"score_linear":150574.0,"score_lmetric":275884},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":20136,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":20136,"score_linear":27639.0,"score_lmetric":20136},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":164892,"ongoing_decode_tokens":46795,"pending_prefill_tokens":4433,"num_requests":3,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":20136,"score_linear":164892.0,"score_lmetric":73707},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":20136,"score_linear":191531.0,"score_lmetric":613196},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":32346,"ongoing_decode_tokens":32346,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":394,"cache_hit":12800,"new_prefill":7336,"score_linear":19546.0,"score_lmetric":7336}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338814.945483513,"t_first_token_unix":1779769679.959427,"t_done":338816.371086344,"t_done_unix":1779769681.3850293},{"request_id":"1276136:2:1280145:162","session_id":"1276136","input_length":24519,"t_proxy_recv":338815.145273461,"t_decision_unix":1779769680.1592147,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":16896,"affinity_cache_ratio":0.6890982503364738,"affinity_num_requests":3,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7623 < 8000)","cache_hit":16896,"estimated_new_tokens":7623,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":24519,"score_linear":25187.0,"score_lmetric":24519},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58752,"ongoing_decode_tokens":58752,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":24519,"score_linear":58752.0,"score_lmetric":24519},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24519,"score_linear":150574.0,"score_lmetric":284650},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":20136,"ongoing_decode_tokens":20136,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":24519,"score_linear":20136.0,"score_lmetric":24519},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":24519,"score_linear":27639.0,"score_lmetric":24519},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":164892,"ongoing_decode_tokens":164892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":416,"cache_hit":16896,"new_prefill":7623,"score_linear":147996.0,"score_lmetric":22869},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":24519,"score_linear":191531.0,"score_lmetric":630728},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":88317,"ongoing_decode_tokens":0,"pending_prefill_tokens":88317,"num_requests":1,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":24519,"score_linear":88317.0,"score_lmetric":112836}],"chosen_score_linear":147996.0,"chosen_score_lmetric":22869,"t_first_token":338816.37676626,"t_first_token_unix":1779769681.390709,"t_done":338817.154455235,"t_done_unix":1779769682.1683984},{"request_id":"1273203:2:1277744:152","session_id":"1273203","input_length":22573,"t_proxy_recv":338806.942649988,"t_decision_unix":1779769671.9565911,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9753245027244939,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (557 < 8000)","cache_hit":22016,"estimated_new_tokens":557,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":22573,"score_linear":34071.0,"score_lmetric":45146},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":22573,"score_linear":120167.0,"score_lmetric":257832},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":22573,"score_linear":150574.0,"score_lmetric":280758},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":9813,"num_requests":1,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":22573,"score_linear":27221.0,"score_lmetric":32386},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":22573,"score_linear":121069.0,"score_lmetric":232006},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":416,"cache_hit":22016,"new_prefill":557,"score_linear":-22016.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":21504,"new_prefill":1069,"score_linear":170027.0,"score_lmetric":536928},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":65369,"ongoing_decode_tokens":33524,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":354,"cache_hit":0,"new_prefill":22573,"score_linear":65369.0,"score_lmetric":69924}],"chosen_score_linear":-22016.0,"chosen_score_lmetric":0,"t_first_token":338807.013568038,"t_first_token_unix":1779769672.0275114,"t_done":338817.314821506,"t_done_unix":1779769682.328765},{"request_id":"1276136:3:1280987:166","session_id":"1276136","input_length":25923,"t_proxy_recv":338818.092442363,"t_decision_unix":1779769683.1063836,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9282876210315164,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1859 < 8000)","cache_hit":24064,"estimated_new_tokens":1859,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":25923,"score_linear":25187.0,"score_lmetric":25923},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":25923,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":25923,"score_linear":150574.0,"score_lmetric":287458},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12765,"ongoing_decode_tokens":12765,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":588,"cache_hit":0,"new_prefill":25923,"score_linear":12765.0,"score_lmetric":25923},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":25923,"score_linear":27639.0,"score_lmetric":25923},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":142319,"ongoing_decode_tokens":142319,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":431,"cache_hit":24064,"new_prefill":1859,"score_linear":118255.0,"score_lmetric":3718},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":25923,"score_linear":191531.0,"score_lmetric":636344},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":122245,"ongoing_decode_tokens":0,"pending_prefill_tokens":88453,"num_requests":2,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":25923,"score_linear":122245.0,"score_lmetric":228752}],"chosen_score_linear":118255.0,"chosen_score_lmetric":3718,"t_first_token":338818.44308448,"t_first_token_unix":1779769683.4570284,"t_done":338819.708451172,"t_done_unix":1779769684.7223942},{"request_id":"1268861:2:1280530:164","session_id":"1268861","input_length":12765,"t_proxy_recv":338816.365423969,"t_decision_unix":1779769681.3793652,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":12288,"affinity_cache_ratio":0.9626321974148061,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (477 < 8000)","cache_hit":12288,"estimated_new_tokens":477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":12765,"score_linear":25187.0,"score_lmetric":12765},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":12765,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12765,"score_linear":150574.0,"score_lmetric":261142},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":20136,"ongoing_decode_tokens":20136,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":549,"cache_hit":12288,"new_prefill":477,"score_linear":7848.0,"score_lmetric":477},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":12765,"score_linear":27639.0,"score_lmetric":12765},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":189411,"ongoing_decode_tokens":164892,"pending_prefill_tokens":7623,"num_requests":4,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":12765,"score_linear":189411.0,"score_lmetric":81552},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":12765,"score_linear":191531.0,"score_lmetric":583712},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":122245,"ongoing_decode_tokens":0,"pending_prefill_tokens":88453,"num_requests":2,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":12765,"score_linear":122245.0,"score_lmetric":202436}],"chosen_score_linear":7848.0,"chosen_score_lmetric":477,"t_first_token":338816.420829867,"t_first_token_unix":1779769681.434773,"t_done":338819.852727247,"t_done_unix":1779769684.8666704},{"request_id":"1281887:1:1281887:171","session_id":"1281887","input_length":9370,"t_proxy_recv":338821.013408796,"t_decision_unix":1779769686.02735,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9370,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":9370,"score_linear":25187.0,"score_lmetric":9370},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":9370,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9370,"score_linear":150574.0,"score_lmetric":254352},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26426,"ongoing_decode_tokens":26426,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":9370,"score_linear":26426.0,"score_lmetric":9370},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":9370,"score_linear":27639.0,"score_lmetric":9370},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":142319,"ongoing_decode_tokens":142319,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":434,"cache_hit":0,"new_prefill":9370,"score_linear":142319.0,"score_lmetric":18740},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":9370,"score_linear":191531.0,"score_lmetric":570132},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":122245,"ongoing_decode_tokens":0,"pending_prefill_tokens":88453,"num_requests":2,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":9370,"score_linear":122245.0,"score_lmetric":195646}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338821.724412419,"t_first_token_unix":1779769686.7383559,"t_done":338821.893316459,"t_done_unix":1779769686.90726},{"request_id":"1282082:1:1282082:173","session_id":"1282082","input_length":4972,"t_proxy_recv":338821.683165788,"t_decision_unix":1779769686.6971073,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":4972,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4972 < 8000)","cache_hit":0,"estimated_new_tokens":4972,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":4972,"score_linear":25187.0,"score_lmetric":4972},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":9370,"ongoing_decode_tokens":0,"pending_prefill_tokens":9370,"num_requests":1,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":4972,"score_linear":9370.0,"score_lmetric":14342},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":4972,"score_linear":150574.0,"score_lmetric":245556},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26426,"ongoing_decode_tokens":26426,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":4972,"score_linear":26426.0,"score_lmetric":4972},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":4972,"score_linear":27639.0,"score_lmetric":4972},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":142319,"ongoing_decode_tokens":142319,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":434,"cache_hit":0,"new_prefill":4972,"score_linear":142319.0,"score_lmetric":9944},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":4972,"score_linear":191531.0,"score_lmetric":552540},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":4972,"score_linear":164268.0,"score_lmetric":312648}],"chosen_score_linear":25187.0,"chosen_score_lmetric":4972,"t_first_token":338822.0089254,"t_first_token_unix":1779769687.0228686,"t_done":338822.253768791,"t_done_unix":1779769687.2677124},{"request_id":"1279412:2:1281113:167","session_id":"1279412","input_length":26426,"t_proxy_recv":338818.628237378,"t_decision_unix":1779769683.642178,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":19968,"affinity_cache_ratio":0.7556194656777416,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6458 < 8000)","cache_hit":19968,"estimated_new_tokens":6458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":26426,"score_linear":25187.0,"score_lmetric":26426},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":26426,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":26426,"score_linear":150574.0,"score_lmetric":288464},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12765,"ongoing_decode_tokens":12765,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":588,"cache_hit":19968,"new_prefill":6458,"score_linear":-7203.0,"score_lmetric":6458},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":26426,"score_linear":27639.0,"score_lmetric":26426},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":168242,"ongoing_decode_tokens":168242,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":431,"cache_hit":0,"new_prefill":26426,"score_linear":168242.0,"score_lmetric":79278},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":26426,"score_linear":191531.0,"score_lmetric":638356},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":122245,"ongoing_decode_tokens":0,"pending_prefill_tokens":88453,"num_requests":2,"active_p_offloads":0,"cached_blocks":395,"cache_hit":12800,"new_prefill":13626,"score_linear":109445.0,"score_lmetric":204158}],"chosen_score_linear":-7203.0,"chosen_score_lmetric":6458,"t_first_token":338819.746020363,"t_first_token_unix":1779769684.7599638,"t_done":338823.753982416,"t_done_unix":1779769688.767926},{"request_id":"1258859:3:1278118:155","session_id":"1258859","input_length":118097,"t_proxy_recv":338808.361171524,"t_decision_unix":1779769673.375113,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":113664,"affinity_cache_ratio":0.962463060026927,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4433 < 8000)","cache_hit":113664,"estimated_new_tokens":4433,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":118097,"score_linear":34071.0,"score_lmetric":236194},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":118097,"score_linear":120167.0,"score_lmetric":448880},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":118097,"score_linear":150574.0,"score_lmetric":471806},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":118097,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":118097,"score_linear":121069.0,"score_lmetric":423054},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46795,"ongoing_decode_tokens":22573,"pending_prefill_tokens":24222,"num_requests":2,"active_p_offloads":0,"cached_blocks":416,"cache_hit":113664,"new_prefill":4433,"score_linear":-66869.0,"score_lmetric":57310},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":118097,"score_linear":191531.0,"score_lmetric":1005040},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96799,"ongoing_decode_tokens":96799,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":354,"cache_hit":0,"new_prefill":118097,"score_linear":96799.0,"score_lmetric":354291}],"chosen_score_linear":-66869.0,"chosen_score_lmetric":57310,"t_first_token":338813.449387023,"t_first_token_unix":1779769678.4633312,"t_done":338823.967290595,"t_done_unix":1779769688.981234},{"request_id":"1267915:1:1267915:96","session_id":"1267915","input_length":126924,"t_proxy_recv":338772.362876671,"t_decision_unix":1779769637.376818,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":18944,"estimated_new_tokens":107980,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":126924,"score_linear":135235.0,"score_lmetric":480286},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":273,"cache_hit":0,"new_prefill":126924,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":18944,"new_prefill":107980,"score_linear":-18944.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":126924,"score_linear":15283.0,"score_lmetric":126924},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":126924,"score_linear":27639.0,"score_lmetric":126924},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":126924,"score_linear":125063.0,"score_lmetric":488614},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":126924,"score_linear":17466.0,"score_lmetric":126924},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28671,"ongoing_decode_tokens":0,"pending_prefill_tokens":6143,"num_requests":1,"active_p_offloads":0,"cached_blocks":194,"cache_hit":0,"new_prefill":126924,"score_linear":28671.0,"score_lmetric":133067}],"chosen_score_linear":-18944.0,"chosen_score_lmetric":0,"t_first_token":338822.783435489,"t_first_token_unix":1779769687.7973788,"t_done":338824.099036017,"t_done_unix":1779769689.1129797},{"request_id":"1277909:1:1277909:153","session_id":"1277909","input_length":24222,"t_proxy_recv":338807.643114733,"t_decision_unix":1779769672.6570559,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":24222,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"local_cost 4.84s <= migrate_cost 13.79s + margin 0.20s","v2_chosen_cache_hit":0,"v2_src_idx":6,"v2_src_cache_hit":7680,"v2_new_local":24222,"v2_new_src":16542,"v2_cost_local_s":4.8444,"v2_cost_src_interf_s":13.233600000000003,"v2_cost_xfer_s":0.5604166666666666,"v2_cost_migrate_s":13.79401666666667,"cache_hit":0,"estimated_new_tokens":24222,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":4096,"new_prefill":20126,"score_linear":29975.0,"score_lmetric":40252},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":24222,"score_linear":120167.0,"score_lmetric":261130},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24222,"score_linear":150574.0,"score_lmetric":284056},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":27221,"ongoing_decode_tokens":27221,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":24222,"score_linear":27221.0,"score_lmetric":24222},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":24222,"score_linear":121069.0,"score_lmetric":235304},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":22573,"ongoing_decode_tokens":22573,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":24222,"score_linear":22573.0,"score_lmetric":24222},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":7680,"new_prefill":16542,"score_linear":183851.0,"score_lmetric":598820},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":65369,"ongoing_decode_tokens":65369,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":354,"cache_hit":0,"new_prefill":24222,"score_linear":65369.0,"score_lmetric":48444}],"chosen_score_linear":22573.0,"chosen_score_lmetric":24222,"t_first_token":338810.517083819,"t_first_token_unix":1779769675.5310273,"t_done":338824.285475797,"t_done_unix":1779769689.2994192},{"request_id":"1265128:2:1271655:122","session_id":"1265128","input_length":23650,"t_proxy_recv":338785.201878882,"t_decision_unix":1779769650.21582,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":13824,"affinity_cache_ratio":0.5845243128964059,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":13824,"estimated_new_tokens":9826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":23650,"score_linear":135235.0,"score_lmetric":273738},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":23650,"score_linear":102074.0,"score_lmetric":122140},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":13824,"new_prefill":9826,"score_linear":113100.0,"score_lmetric":117806},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":16034,"ongoing_decode_tokens":16034,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":23650,"score_linear":16034.0,"score_lmetric":47300},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":23650,"score_linear":121069.0,"score_lmetric":234160},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":130878,"ongoing_decode_tokens":113172,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":23650,"score_linear":130878.0,"score_lmetric":82712},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":403,"cache_hit":0,"new_prefill":23650,"score_linear":64508.0,"score_lmetric":136311},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80149,"ongoing_decode_tokens":60701,"pending_prefill_tokens":7160,"num_requests":3,"active_p_offloads":0,"cached_blocks":246,"cache_hit":0,"new_prefill":23650,"score_linear":80149.0,"score_lmetric":92430}],"chosen_score_linear":113100.0,"chosen_score_lmetric":117806,"t_first_token":338823.281260551,"t_first_token_unix":1779769688.2952042,"t_done":338824.460201006,"t_done_unix":1779769689.4741442},{"request_id":"1277533:2:1282872:175","session_id":"1277533","input_length":16299,"t_proxy_recv":338824.427476737,"t_decision_unix":1779769689.4414184,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":4608,"affinity_cache_ratio":0.2827167310877968,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":4608,"estimated_new_tokens":11691,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":398,"cache_hit":0,"new_prefill":16299,"score_linear":25187.0,"score_lmetric":16299},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":16299,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23650,"ongoing_decode_tokens":23650,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":547,"cache_hit":0,"new_prefill":16299,"score_linear":23650.0,"score_lmetric":16299},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":16299,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":16299,"score_linear":27639.0,"score_lmetric":16299},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":491,"cache_hit":4608,"new_prefill":11691,"score_linear":-4608.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":16299,"score_linear":191531.0,"score_lmetric":597848},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":16299,"score_linear":164268.0,"score_lmetric":346629}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":338825.714360768,"t_first_token_unix":1779769690.7283044,"t_done":338825.819666355,"t_done_unix":1779769690.8336103},{"request_id":"1242838:4:1265709:87","session_id":"1242838","input_length":25187,"t_proxy_recv":338764.83201252,"t_decision_unix":1779769629.8459537,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":22016,"affinity_cache_ratio":0.8741017191408266,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3171 < 8000)","cache_hit":22016,"estimated_new_tokens":3171,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":22016,"new_prefill":3171,"score_linear":88032.0,"score_lmetric":113219},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":25187,"score_linear":55989.0,"score_lmetric":25187},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":25187,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":25187,"score_linear":71575.0,"score_lmetric":162958},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":25187,"score_linear":27639.0,"score_lmetric":25187},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":25187,"score_linear":125063.0,"score_lmetric":285140},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":25187,"score_linear":50783.0,"score_lmetric":75970},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":0,"pending_prefill_tokens":23868,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":25187,"score_linear":45884.0,"score_lmetric":98110}],"chosen_score_linear":88032.0,"chosen_score_lmetric":113219,"t_first_token":338801.080032894,"t_first_token_unix":1779769666.0939763,"t_done":338827.680454185,"t_done_unix":1779769692.6943982},{"request_id":"1283642:1:1283642:178","session_id":"1283642","input_length":8510,"t_proxy_recv":338826.862908466,"t_decision_unix":1779769691.8768497,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8510,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":398,"cache_hit":0,"new_prefill":8510,"score_linear":25187.0,"score_lmetric":8510},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":8510,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":8510,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":8510,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":8510,"score_linear":27639.0,"score_lmetric":8510},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118736,"ongoing_decode_tokens":118736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":513,"cache_hit":0,"new_prefill":8510,"score_linear":118736.0,"score_lmetric":8510},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":8510,"score_linear":191531.0,"score_lmetric":566692},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":8510,"score_linear":164268.0,"score_lmetric":323262}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338827.497591635,"t_first_token_unix":1779769692.5115361,"t_done":338827.811820604,"t_done_unix":1779769692.8257647},{"request_id":"1258859:4:1282900:176","session_id":"1258859","input_length":118736,"t_proxy_recv":338824.501814322,"t_decision_unix":1779769689.515756,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":118272,"affinity_cache_ratio":0.9960921708664601,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (464 < 8000)","cache_hit":118272,"estimated_new_tokens":464,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":398,"cache_hit":0,"new_prefill":118736,"score_linear":25187.0,"score_lmetric":118736},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":118736,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":118736,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":118736,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":118736,"score_linear":27639.0,"score_lmetric":118736},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":16299,"ongoing_decode_tokens":0,"pending_prefill_tokens":11691,"num_requests":1,"active_p_offloads":0,"cached_blocks":491,"cache_hit":118272,"new_prefill":464,"score_linear":-101973.0,"score_lmetric":12155},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":118736,"score_linear":191531.0,"score_lmetric":1007596},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":118736,"score_linear":164268.0,"score_lmetric":653940}],"chosen_score_linear":-101973.0,"chosen_score_lmetric":12155,"t_first_token":338825.816160411,"t_first_token_unix":1779769690.8301039,"t_done":338831.677486949,"t_done_unix":1779769696.6914313},{"request_id":"1279412:3:1285475:187","session_id":"1279412","input_length":27077,"t_proxy_recv":338833.328912986,"t_decision_unix":1779769698.3428538,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9832699338922333,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (453 < 8000)","cache_hit":26624,"estimated_new_tokens":453,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":27077,"score_linear":49619.0,"score_lmetric":76696},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":42264,"ongoing_decode_tokens":0,"pending_prefill_tokens":42264,"num_requests":1,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":27077,"score_linear":42264.0,"score_lmetric":69341},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":27077,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":26624,"new_prefill":453,"score_linear":-26624.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":27077,"score_linear":27639.0,"score_lmetric":27077},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":30965,"ongoing_decode_tokens":30965,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":514,"cache_hit":0,"new_prefill":27077,"score_linear":30965.0,"score_lmetric":27077},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":27077,"score_linear":191531.0,"score_lmetric":640960},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":12800,"new_prefill":14277,"score_linear":151468.0,"score_lmetric":340563}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":338833.399132035,"t_first_token_unix":1779769698.4130762,"t_done":338833.465422525,"t_done_unix":1779769698.4793658},{"request_id":"1239034:4:1263738:78","session_id":"1239034","input_length":27639,"t_proxy_recv":338763.787706702,"t_decision_unix":1779769628.8016481,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":22528,"affinity_cache_ratio":0.8150801403813452,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5111 < 8000)","cache_hit":22528,"estimated_new_tokens":5111,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":27639,"score_linear":110048.0,"score_lmetric":137687},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":27639,"score_linear":55989.0,"score_lmetric":27639},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14329,"ongoing_decode_tokens":301,"pending_prefill_tokens":14028,"num_requests":2,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":27639,"score_linear":14329.0,"score_lmetric":83334},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":27639,"score_linear":71575.0,"score_lmetric":167862},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":258,"cache_hit":22528,"new_prefill":5111,"score_linear":-22528.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":27639,"score_linear":125063.0,"score_lmetric":290044},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":27639,"score_linear":50783.0,"score_lmetric":78422},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":0,"pending_prefill_tokens":23868,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":27639,"score_linear":45884.0,"score_lmetric":103014}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":0,"t_first_token":338764.796351967,"t_first_token_unix":1779769629.8102958,"t_done":338833.543141808,"t_done_unix":1779769698.5570855},{"request_id":"1277909:2:1283679:179","session_id":"1277909","input_length":30965,"t_proxy_recv":338826.970408669,"t_decision_unix":1779769691.98435,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":24576,"affinity_cache_ratio":0.7936702728887454,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6389 < 8000)","cache_hit":24576,"estimated_new_tokens":6389,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":398,"cache_hit":4096,"new_prefill":26869,"score_linear":21091.0,"score_lmetric":26869},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":30965,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":8510,"ongoing_decode_tokens":0,"pending_prefill_tokens":8510,"num_requests":1,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":30965,"score_linear":8510.0,"score_lmetric":39475},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":30965,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":30965,"score_linear":27639.0,"score_lmetric":30965},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118736,"ongoing_decode_tokens":118736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":513,"cache_hit":24576,"new_prefill":6389,"score_linear":94160.0,"score_lmetric":6389},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":7680,"new_prefill":23285,"score_linear":183851.0,"score_lmetric":625792},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":30965,"score_linear":164268.0,"score_lmetric":390627}],"chosen_score_linear":94160.0,"chosen_score_lmetric":6389,"t_first_token":338828.249160671,"t_first_token_unix":1779769693.2631047,"t_done":338834.649391277,"t_done_unix":1779769699.6633348},{"request_id":"1285800:1:1285800:191","session_id":"1285800","input_length":4510,"t_proxy_recv":338834.438814692,"t_decision_unix":1779769699.452756,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4510 < 8000)","cache_hit":0,"estimated_new_tokens":4510,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":4510,"score_linear":49619.0,"score_lmetric":54129},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":42264,"ongoing_decode_tokens":0,"pending_prefill_tokens":42264,"num_requests":1,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":4510,"score_linear":42264.0,"score_lmetric":46774},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14203,"ongoing_decode_tokens":11,"pending_prefill_tokens":6000,"num_requests":2,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":4510,"score_linear":14203.0,"score_lmetric":21020},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":4510,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":451,"cache_hit":0,"new_prefill":4510,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":30965,"ongoing_decode_tokens":30965,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":514,"cache_hit":0,"new_prefill":4510,"score_linear":30965.0,"score_lmetric":4510},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":4510,"score_linear":191531.0,"score_lmetric":550692},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":4510,"score_linear":164268.0,"score_lmetric":311262}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338834.731847064,"t_first_token_unix":1779769699.7457905,"t_done":338835.014051289,"t_done_unix":1779769700.0279953},{"request_id":"1283642:2:1285731:190","session_id":"1283642","input_length":14192,"t_proxy_recv":338834.185072533,"t_decision_unix":1779769699.1990137,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":8192,"affinity_cache_ratio":0.5772266065388951,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6000 < 8000)","cache_hit":8192,"estimated_new_tokens":6000,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":14192,"score_linear":49619.0,"score_lmetric":63811},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":42264,"ongoing_decode_tokens":0,"pending_prefill_tokens":42264,"num_requests":1,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":14192,"score_linear":42264.0,"score_lmetric":56456},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":11,"ongoing_decode_tokens":11,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":582,"cache_hit":8192,"new_prefill":6000,"score_linear":-8181.0,"score_lmetric":6000},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":14192,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":451,"cache_hit":0,"new_prefill":14192,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":30965,"ongoing_decode_tokens":30965,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":514,"cache_hit":0,"new_prefill":14192,"score_linear":30965.0,"score_lmetric":14192},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":14192,"score_linear":191531.0,"score_lmetric":589420},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":14192,"score_linear":164268.0,"score_lmetric":340308}],"chosen_score_linear":-8181.0,"chosen_score_lmetric":6000,"t_first_token":338834.841358402,"t_first_token_unix":1779769699.8553023,"t_done":338835.309353715,"t_done_unix":1779769700.3232977},{"request_id":"1285558:1:1285558:188","session_id":"1285558","input_length":11,"t_proxy_recv":338833.531161958,"t_decision_unix":1779769698.5451035,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (11 < 8000)","cache_hit":0,"estimated_new_tokens":11,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":11,"score_linear":49619.0,"score_lmetric":49630},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":42264,"ongoing_decode_tokens":0,"pending_prefill_tokens":42264,"num_requests":1,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":11,"score_linear":42264.0,"score_lmetric":42275},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":11,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":11,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":11,"score_linear":27639.0,"score_lmetric":11},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":30965,"ongoing_decode_tokens":30965,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":514,"cache_hit":0,"new_prefill":11,"score_linear":30965.0,"score_lmetric":11},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":11,"score_linear":191531.0,"score_lmetric":532696},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":11,"score_linear":164268.0,"score_lmetric":297765}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338833.552071477,"t_first_token_unix":1779769698.566015,"t_done":338836.031212907,"t_done_unix":1779769701.0451567},{"request_id":"1286551:1:1286551:194","session_id":"1286551","input_length":1700,"t_proxy_recv":338837.404087374,"t_decision_unix":1779769702.4180286,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1700 < 8000)","cache_hit":0,"estimated_new_tokens":1700,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":1700,"score_linear":49619.0,"score_lmetric":51319},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":42264,"ongoing_decode_tokens":0,"pending_prefill_tokens":42264,"num_requests":1,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":1700,"score_linear":42264.0,"score_lmetric":43964},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":31898,"ongoing_decode_tokens":0,"pending_prefill_tokens":5274,"num_requests":1,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":1700,"score_linear":31898.0,"score_lmetric":6974},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":459,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":1700,"score_linear":191531.0,"score_lmetric":539452},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":1700,"score_linear":164268.0,"score_lmetric":302832}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338837.550456322,"t_first_token_unix":1779769702.5644,"t_done":338837.745795427,"t_done_unix":1779769702.759739},{"request_id":"1268831:5:1284640:184","session_id":"1268831","input_length":42264,"t_proxy_recv":338830.379280347,"t_decision_unix":1779769695.3932216,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":1,"affinity_cache_hit":32256,"affinity_cache_ratio":0.7632027257240205,"affinity_num_requests":3,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":42264,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":42264,"score_linear":49619.0,"score_lmetric":91883},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":42264,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":42264,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":42264,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":42264,"score_linear":27639.0,"score_lmetric":42264},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":149701,"ongoing_decode_tokens":149701,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":513,"cache_hit":0,"new_prefill":42264,"score_linear":149701.0,"score_lmetric":84528},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":42264,"score_linear":191531.0,"score_lmetric":701708},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":32256,"new_prefill":10008,"score_linear":132012.0,"score_lmetric":327756}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338837.469228962,"t_first_token_unix":1779769702.4831727,"t_done":338838.807540358,"t_done_unix":1779769703.8214843},{"request_id":"1279412:4:1286447:193","session_id":"1279412","input_length":31898,"t_proxy_recv":338837.003256844,"t_decision_unix":1779769702.0171978,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":26624,"affinity_cache_ratio":0.8346604802808953,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5274 < 8000)","cache_hit":26624,"estimated_new_tokens":5274,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":31898,"score_linear":49619.0,"score_lmetric":81517},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":42264,"ongoing_decode_tokens":0,"pending_prefill_tokens":42264,"num_requests":1,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":31898,"score_linear":42264.0,"score_lmetric":74162},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":31898,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":26624,"new_prefill":5274,"score_linear":-26624.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":459,"cache_hit":0,"new_prefill":31898,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":31898,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":31898,"score_linear":191531.0,"score_lmetric":660244},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":12800,"new_prefill":19098,"score_linear":151468.0,"score_lmetric":355026}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":338838.079812216,"t_first_token_unix":1779769703.0937564,"t_done":338839.710681659,"t_done_unix":1779769704.7246256},{"request_id":"1286804:1:1286804:195","session_id":"1286804","input_length":13962,"t_proxy_recv":338838.135384435,"t_decision_unix":1779769703.1493254,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":13962,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":13962,"score_linear":49619.0,"score_lmetric":13962},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":42264,"ongoing_decode_tokens":42264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":13962,"score_linear":42264.0,"score_lmetric":13962},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":31898,"ongoing_decode_tokens":31898,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":13962,"score_linear":31898.0,"score_lmetric":13962},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":459,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":13962,"score_linear":191531.0,"score_lmetric":588500},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":13962,"score_linear":164268.0,"score_lmetric":339618}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338839.346832087,"t_first_token_unix":1779769704.3607764,"t_done":338840.051673727,"t_done_unix":1779769705.0656173},{"request_id":"1287588:1:1287588:198","session_id":"1287588","input_length":9925,"t_proxy_recv":338840.684114949,"t_decision_unix":1779769705.6980565,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9925,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":9925,"score_linear":49619.0,"score_lmetric":9925},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":486,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":9925,"score_linear":77805.0,"score_lmetric":87730},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":114612,"pending_prefill_tokens":18551,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":9925,"score_linear":191531.0,"score_lmetric":113904},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":9925,"score_linear":164268.0,"score_lmetric":327507}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338841.494504279,"t_first_token_unix":1779769706.5084484,"t_done":338842.827319462,"t_done_unix":1779769707.8412626},{"request_id":"1288143:1:1288143:200","session_id":"1288143","input_length":448,"t_proxy_recv":338842.649745845,"t_decision_unix":1779769707.6636875,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (448 < 8000)","cache_hit":0,"estimated_new_tokens":448,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":448,"score_linear":49619.0,"score_lmetric":448},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":448,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":448,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":448,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":9925,"ongoing_decode_tokens":9925,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":486,"cache_hit":0,"new_prefill":448,"score_linear":9925.0,"score_lmetric":448},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":448,"score_linear":77805.0,"score_lmetric":78253},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":125251,"pending_prefill_tokens":17640,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":448,"score_linear":191531.0,"score_lmetric":72352},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":122245,"pending_prefill_tokens":10791,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":448,"score_linear":164268.0,"score_lmetric":33717}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338842.698512978,"t_first_token_unix":1779769707.712456,"t_done":338843.220506281,"t_done_unix":1779769708.2344494},{"request_id":"1262354:8:1280463:163","session_id":"1262354","input_length":33928,"t_proxy_recv":338816.151149325,"t_decision_unix":1779769681.1650908,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9959915114359821,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (136 < 8000)","cache_hit":33792,"estimated_new_tokens":136,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":33928,"score_linear":25187.0,"score_lmetric":33928},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":33928,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33928,"score_linear":150574.0,"score_lmetric":303468},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":20136,"ongoing_decode_tokens":20136,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":33928,"score_linear":20136.0,"score_lmetric":33928},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":33928,"score_linear":27639.0,"score_lmetric":33928},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":189411,"ongoing_decode_tokens":164892,"pending_prefill_tokens":7623,"num_requests":4,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":33928,"score_linear":189411.0,"score_lmetric":166204},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":33928,"score_linear":191531.0,"score_lmetric":668364},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":88317,"ongoing_decode_tokens":0,"pending_prefill_tokens":88317,"num_requests":1,"active_p_offloads":0,"cached_blocks":395,"cache_hit":33792,"new_prefill":136,"score_linear":54525.0,"score_lmetric":88453}],"chosen_score_linear":54525.0,"chosen_score_lmetric":88453,"t_first_token":338840.988481924,"t_first_token_unix":1779769706.0024254,"t_done":338844.109635422,"t_done_unix":1779769709.123583},{"request_id":"1284132:1:1284132:182","session_id":"1284132","input_length":49619,"t_proxy_recv":338828.655325005,"t_decision_unix":1779769693.6692662,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":49619,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":519,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":49619,"score_linear":27639.0,"score_lmetric":49619},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":149701,"ongoing_decode_tokens":149701,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":513,"cache_hit":0,"new_prefill":49619,"score_linear":149701.0,"score_lmetric":99238},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":49619,"score_linear":191531.0,"score_lmetric":731128},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":49619,"score_linear":164268.0,"score_lmetric":446589}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338837.910524291,"t_first_token_unix":1779769702.924468,"t_done":338844.654736305,"t_done_unix":1779769709.6686802},{"request_id":"1279994:1:1279994:161","session_id":"1279994","input_length":88317,"t_proxy_recv":338814.712543418,"t_decision_unix":1779769679.7264843,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":88317,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":88317,"score_linear":25187.0,"score_lmetric":88317},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58752,"ongoing_decode_tokens":58752,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":88317,"score_linear":58752.0,"score_lmetric":88317},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":88317,"score_linear":150574.0,"score_lmetric":412246},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":20136,"ongoing_decode_tokens":0,"pending_prefill_tokens":20136,"num_requests":1,"active_p_offloads":0,"cached_blocks":549,"cache_hit":0,"new_prefill":88317,"score_linear":20136.0,"score_lmetric":108453},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":88317,"score_linear":27639.0,"score_lmetric":88317},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":164892,"ongoing_decode_tokens":164892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":88317,"score_linear":164892.0,"score_lmetric":264951},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":88317,"score_linear":191531.0,"score_lmetric":885920},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":88317,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338840.986471674,"t_first_token_unix":1779769706.000415,"t_done":338844.664874212,"t_done_unix":1779769709.6788175},{"request_id":"1260327:5:1281915:172","session_id":"1260327","input_length":42023,"t_proxy_recv":338821.133083351,"t_decision_unix":1779769686.1470246,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":31232,"affinity_cache_ratio":0.7432120505437498,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=2 decode_tok=0)","cache_hit":31232,"estimated_new_tokens":10791,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":42023,"score_linear":25187.0,"score_lmetric":42023},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":9370,"ongoing_decode_tokens":0,"pending_prefill_tokens":9370,"num_requests":1,"active_p_offloads":0,"cached_blocks":501,"cache_hit":0,"new_prefill":42023,"score_linear":9370.0,"score_lmetric":51393},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":42023,"score_linear":150574.0,"score_lmetric":319658},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26426,"ongoing_decode_tokens":26426,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":42023,"score_linear":26426.0,"score_lmetric":42023},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":42023,"score_linear":27639.0,"score_lmetric":42023},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":142319,"ongoing_decode_tokens":142319,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":434,"cache_hit":0,"new_prefill":42023,"score_linear":142319.0,"score_lmetric":84046},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":42023,"score_linear":191531.0,"score_lmetric":700744},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":122245,"ongoing_decode_tokens":0,"pending_prefill_tokens":88453,"num_requests":2,"active_p_offloads":0,"cached_blocks":395,"cache_hit":31232,"new_prefill":10791,"score_linear":91013.0,"score_lmetric":198488}],"chosen_score_linear":91013.0,"chosen_score_lmetric":198488,"t_first_token":338843.307035844,"t_first_token_unix":1779769708.320979,"t_done":338844.69811366,"t_done_unix":1779769709.7120569},{"request_id":"1279994:2:1283011:177","session_id":"1279994","input_length":88940,"t_proxy_recv":338844.683364223,"t_decision_unix":1779769709.6973054,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":88064,"affinity_cache_ratio":0.990150663368563,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (876 < 8000)","cache_hit":88064,"estimated_new_tokens":876,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":88940,"score_linear":77805.0,"score_lmetric":166745},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":125251,"pending_prefill_tokens":17640,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":88940,"score_linear":191531.0,"score_lmetric":426320},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":76051,"ongoing_decode_tokens":76051,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":567,"cache_hit":88064,"new_prefill":876,"score_linear":-12013.0,"score_lmetric":1752}],"chosen_score_linear":-12013.0,"chosen_score_lmetric":1752,"t_first_token":338845.185174937,"t_first_token_unix":1779769710.1991184,"t_done":338846.197787741,"t_done_unix":1779769711.211731},{"request_id":"1262354:9:1281632:169","session_id":"1262354","input_length":34028,"t_proxy_recv":338844.115976923,"t_decision_unix":1779769709.1299186,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9930645350887505,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (236 < 8000)","cache_hit":33792,"estimated_new_tokens":236,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":34028,"score_linear":49619.0,"score_lmetric":34028},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":34028,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":34028,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":12800,"new_prefill":21228,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":34028,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":34028,"score_linear":77805.0,"score_lmetric":111833},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":125251,"pending_prefill_tokens":17640,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":34028,"score_linear":191531.0,"score_lmetric":206672},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130340,"ongoing_decode_tokens":130340,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":395,"cache_hit":33792,"new_prefill":236,"score_linear":96548.0,"score_lmetric":472}],"chosen_score_linear":96548.0,"chosen_score_lmetric":472,"t_first_token":338844.223114644,"t_first_token_unix":1779769709.2370582,"t_done":338847.503600594,"t_done_unix":1779769712.517544},{"request_id":"1286551:2:1289450:202","session_id":"1286551","input_length":9610,"t_proxy_recv":338846.994351411,"t_decision_unix":1779769712.008293,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":1536,"affinity_cache_ratio":0.1598335067637877,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":1536,"estimated_new_tokens":8074,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":9610,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":9610,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":1536,"new_prefill":8074,"score_linear":-1536.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":9610,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":9610,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":9610,"score_linear":122725.0,"score_lmetric":202206},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":9610,"score_linear":191531.0,"score_lmetric":38440},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123254,"ongoing_decode_tokens":123254,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":9610,"score_linear":123254.0,"score_lmetric":19220}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":338847.622116734,"t_first_token_unix":1779769712.63606,"t_done":338847.9922919,"t_done_unix":1779769713.0062351},{"request_id":"1279412:4:1289482:203","session_id":"1279412","input_length":32191,"t_proxy_recv":338847.074162992,"t_decision_unix":1779769712.0881045,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":31744,"affinity_cache_ratio":0.9861141312789289,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (447 < 8000)","cache_hit":31744,"estimated_new_tokens":447,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":32191,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":32191,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9610,"ongoing_decode_tokens":0,"pending_prefill_tokens":8074,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":32191,"score_linear":9610.0,"score_lmetric":40265},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":31744,"new_prefill":447,"score_linear":-31744.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":32191,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":32191,"score_linear":122725.0,"score_lmetric":247368},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":32191,"score_linear":191531.0,"score_lmetric":128764},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123254,"ongoing_decode_tokens":123254,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":589,"cache_hit":12800,"new_prefill":19391,"score_linear":110454.0,"score_lmetric":38782}],"chosen_score_linear":-31744.0,"chosen_score_lmetric":0,"t_first_token":338847.15615297,"t_first_token_unix":1779769712.170096,"t_done":338848.190795288,"t_done_unix":1779769713.2047386},{"request_id":"1262354:10:1283773:180","session_id":"1262354","input_length":34282,"t_proxy_recv":338847.510580009,"t_decision_unix":1779769712.5245218,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9857067849016977,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (490 < 8000)","cache_hit":33792,"estimated_new_tokens":490,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":34282,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":34282,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9610,"ongoing_decode_tokens":0,"pending_prefill_tokens":8074,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":34282,"score_linear":9610.0,"score_lmetric":42356},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32191,"ongoing_decode_tokens":32191,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":612,"cache_hit":12800,"new_prefill":21482,"score_linear":19391.0,"score_lmetric":21482},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":34282,"score_linear":82014.0,"score_lmetric":116296},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":34282,"score_linear":122725.0,"score_lmetric":251550},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":34282,"score_linear":191531.0,"score_lmetric":137128},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":89226,"ongoing_decode_tokens":89226,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":33792,"new_prefill":490,"score_linear":55434.0,"score_lmetric":490}],"chosen_score_linear":55434.0,"chosen_score_lmetric":490,"t_first_token":338847.606258404,"t_first_token_unix":1779769712.6202018,"t_done":338848.395693979,"t_done_unix":1779769713.4096375},{"request_id":"1272313:2:1275926:138","session_id":"1272313","input_length":10639,"t_proxy_recv":338800.402778228,"t_decision_unix":1779769665.4167197,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":9728,"affinity_cache_ratio":0.914371651471003,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (911 < 8000)","cache_hit":9728,"estimated_new_tokens":911,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":6543,"score_linear":131139.0,"score_lmetric":239524},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":10639,"score_linear":120167.0,"score_lmetric":233964},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10639,"score_linear":150574.0,"score_lmetric":256890},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":483,"cache_hit":0,"new_prefill":10639,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":10639,"score_linear":121069.0,"score_lmetric":208138},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":374,"cache_hit":0,"new_prefill":10639,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":498,"cache_hit":9728,"new_prefill":911,"score_linear":104884.0,"score_lmetric":115523},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":337,"cache_hit":0,"new_prefill":10639,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":104884.0,"chosen_score_lmetric":115523,"t_first_token":338842.51337784,"t_first_token_unix":1779769707.5273244,"t_done":338849.146367832,"t_done_unix":1779769714.1603112},{"request_id":"1290442:1:1290442:210","session_id":"1290442","input_length":7261,"t_proxy_recv":338850.267417963,"t_decision_unix":1779769715.2813594,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7261 < 8000)","cache_hit":0,"estimated_new_tokens":7261,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11549,"ongoing_decode_tokens":11549,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":7261,"score_linear":11549.0,"score_lmetric":7261},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":7261,"score_linear":23228.0,"score_lmetric":30489},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13552,"ongoing_decode_tokens":0,"pending_prefill_tokens":12016,"num_requests":1,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":7261,"score_linear":13552.0,"score_lmetric":19277},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":613,"cache_hit":0,"new_prefill":7261,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":7261,"score_linear":92209.0,"score_lmetric":179484},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":7261,"score_linear":122725.0,"score_lmetric":197508},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":500,"cache_hit":0,"new_prefill":7261,"score_linear":180892.0,"score_lmetric":21783},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123579,"ongoing_decode_tokens":123579,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":7261,"score_linear":123579.0,"score_lmetric":14522}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338850.763255869,"t_first_token_unix":1779769715.7771993,"t_done":338850.907644886,"t_done_unix":1779769715.9215882},{"request_id":"1272313:3:1279208:159","session_id":"1272313","input_length":11549,"t_proxy_recv":338849.150573217,"t_decision_unix":1779769714.1645148,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":0,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9309896960775825,"affinity_num_requests":3,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7453 < 8000)","cache_hit":4096,"estimated_new_tokens":7453,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":4096,"new_prefill":7453,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":11549,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":11549,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":613,"cache_hit":0,"new_prefill":11549,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":11549,"score_linear":92209.0,"score_lmetric":188060},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":7680,"new_prefill":3869,"score_linear":115045.0,"score_lmetric":190724},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":500,"cache_hit":10752,"new_prefill":797,"score_linear":170140.0,"score_lmetric":2391},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123579,"ongoing_decode_tokens":123579,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":11549,"score_linear":123579.0,"score_lmetric":23098}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":338849.835090249,"t_first_token_unix":1779769714.8490338,"t_done":338851.018443051,"t_done_unix":1779769716.0323863},{"request_id":"1290426:1:1290426:209","session_id":"1290426","input_length":13552,"t_proxy_recv":338850.240291906,"t_decision_unix":1779769715.2542331,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":1536,"estimated_new_tokens":12016,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11549,"ongoing_decode_tokens":11549,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":13552,"score_linear":11549.0,"score_lmetric":13552},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":13552,"score_linear":23228.0,"score_lmetric":36780},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":1536,"new_prefill":12016,"score_linear":-1536.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":613,"cache_hit":0,"new_prefill":13552,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":13552,"score_linear":92209.0,"score_lmetric":192066},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":13552,"score_linear":122725.0,"score_lmetric":210090},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":500,"cache_hit":0,"new_prefill":13552,"score_linear":180892.0,"score_lmetric":40656},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123579,"ongoing_decode_tokens":123579,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":13552,"score_linear":123579.0,"score_lmetric":27104}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":338851.330520168,"t_first_token_unix":1779769716.3444638,"t_done":338851.708811018,"t_done_unix":1779769716.722754},{"request_id":"1279994:3:1286181:192","session_id":"1279994","input_length":89226,"t_proxy_recv":338846.220280605,"t_decision_unix":1779769711.2342222,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":88576,"affinity_cache_ratio":0.9927151278775245,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (650 < 8000)","cache_hit":88576,"estimated_new_tokens":650,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":89226,"score_linear":122725.0,"score_lmetric":361438},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":89226,"score_linear":191531.0,"score_lmetric":356904},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":34028,"ongoing_decode_tokens":34028,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":589,"cache_hit":88576,"new_prefill":650,"score_linear":-54548.0,"score_lmetric":650}],"chosen_score_linear":-54548.0,"chosen_score_lmetric":650,"t_first_token":338846.507624588,"t_first_token_unix":1779769711.5215678,"t_done":338852.479653243,"t_done_unix":1779769717.493597},{"request_id":"1272313:4:1281720:170","session_id":"1272313","input_length":13901,"t_proxy_recv":338851.023262915,"t_decision_unix":1779769716.0372045,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.8103014171642328,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2637 < 8000)","cache_hit":11264,"estimated_new_tokens":2637,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":520,"cache_hit":11264,"new_prefill":2637,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":13901,"score_linear":23228.0,"score_lmetric":37129},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13552,"ongoing_decode_tokens":0,"pending_prefill_tokens":12016,"num_requests":1,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":13901,"score_linear":13552.0,"score_lmetric":25917},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":627,"cache_hit":0,"new_prefill":13901,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":13901,"score_linear":92209.0,"score_lmetric":192764},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":7680,"new_prefill":6221,"score_linear":115045.0,"score_lmetric":195428},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":500,"cache_hit":10752,"new_prefill":3149,"score_linear":170140.0,"score_lmetric":9447},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123579,"ongoing_decode_tokens":123579,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":13901,"score_linear":123579.0,"score_lmetric":27802}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":338851.364231434,"t_first_token_unix":1779769716.3781748,"t_done":338852.940757409,"t_done_unix":1779769717.9547005},{"request_id":"1291100:1:1291100:212","session_id":"1291100","input_length":1744,"t_proxy_recv":338852.494136518,"t_decision_unix":1779769717.5080783,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1744 < 8000)","cache_hit":0,"estimated_new_tokens":1744,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13901,"ongoing_decode_tokens":13901,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":520,"cache_hit":0,"new_prefill":1744,"score_linear":13901.0,"score_lmetric":1744},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":1744,"score_linear":23228.0,"score_lmetric":24972},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":1744,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":627,"cache_hit":0,"new_prefill":1744,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":1744,"score_linear":92209.0,"score_lmetric":168450},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":1744,"score_linear":122725.0,"score_lmetric":186474},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":500,"cache_hit":0,"new_prefill":1744,"score_linear":180892.0,"score_lmetric":5232},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":592,"cache_hit":0,"new_prefill":1744,"score_linear":34353.0,"score_lmetric":1744}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338852.651282552,"t_first_token_unix":1779769717.6652257,"t_done":338853.048559568,"t_done_unix":1779769718.0625026},{"request_id":"1290365:1:1290365:208","session_id":"1290365","input_length":23228,"t_proxy_recv":338850.059549308,"t_decision_unix":1779769715.0734904,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":23228,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11549,"ongoing_decode_tokens":11549,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":23228,"score_linear":11549.0,"score_lmetric":23228},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":23228,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":23228,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":613,"cache_hit":0,"new_prefill":23228,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":23228,"score_linear":92209.0,"score_lmetric":211418},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":23228,"score_linear":122725.0,"score_lmetric":229442},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":500,"cache_hit":0,"new_prefill":23228,"score_linear":180892.0,"score_lmetric":69684},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123579,"ongoing_decode_tokens":123579,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":590,"cache_hit":0,"new_prefill":23228,"score_linear":123579.0,"score_lmetric":46456}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338852.643935856,"t_first_token_unix":1779769717.6578794,"t_done":338853.376516521,"t_done_unix":1779769718.39046},{"request_id":"1266668:4:1276519:143","session_id":"1266668","input_length":49380,"t_proxy_recv":338802.477073007,"t_decision_unix":1779769667.4910145,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":33792,"affinity_cache_ratio":0.684325637910085,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=2 decode_tok=0)","cache_hit":33792,"estimated_new_tokens":15588,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":4096,"new_prefill":45284,"score_linear":21091.0,"score_lmetric":45284},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":49380,"score_linear":120167.0,"score_lmetric":311446},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":49380,"score_linear":150574.0,"score_lmetric":334372},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15783,"ongoing_decode_tokens":0,"pending_prefill_tokens":15783,"num_requests":1,"active_p_offloads":0,"cached_blocks":483,"cache_hit":0,"new_prefill":49380,"score_linear":15783.0,"score_lmetric":65163},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":49380,"score_linear":121069.0,"score_lmetric":285620},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":17280,"ongoing_decode_tokens":0,"pending_prefill_tokens":17280,"num_requests":1,"active_p_offloads":0,"cached_blocks":374,"cache_hit":0,"new_prefill":49380,"score_linear":17280.0,"score_lmetric":66660},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":498,"cache_hit":33792,"new_prefill":15588,"score_linear":91459.0,"score_lmetric":262222},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":8704,"ongoing_decode_tokens":0,"pending_prefill_tokens":8704,"num_requests":1,"active_p_offloads":0,"cached_blocks":337,"cache_hit":0,"new_prefill":49380,"score_linear":8704.0,"score_lmetric":58084}],"chosen_score_linear":91459.0,"chosen_score_lmetric":262222,"t_first_token":338844.912885509,"t_first_token_unix":1779769709.9268296,"t_done":338854.075922545,"t_done_unix":1779769719.0898764},{"request_id":"1279412:5:1291651:214","session_id":"1279412","input_length":32359,"t_proxy_recv":338854.406128506,"t_decision_unix":1779769719.42007,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":32256,"affinity_cache_ratio":0.9968169597329954,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (103 < 8000)","cache_hit":32256,"estimated_new_tokens":103,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":525,"cache_hit":0,"new_prefill":32359,"score_linear":14885.0,"score_lmetric":32359},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":32359,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":32359,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":630,"cache_hit":32256,"new_prefill":103,"score_linear":-32256.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":32359,"score_linear":92209.0,"score_lmetric":229680},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":32359,"score_linear":122725.0,"score_lmetric":247704},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":32359,"score_linear":189544.0,"score_lmetric":122181},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124102,"ongoing_decode_tokens":124102,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":592,"cache_hit":12800,"new_prefill":19559,"score_linear":111302.0,"score_lmetric":39118}],"chosen_score_linear":-32256.0,"chosen_score_lmetric":0,"t_first_token":338854.471632041,"t_first_token_unix":1779769719.4855752,"t_done":338855.053634252,"t_done_unix":1779769720.0675778},{"request_id":"1279994:4:1290711:211","session_id":"1279994","input_length":89749,"t_proxy_recv":338852.50036553,"t_decision_unix":1779769717.5143068,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":89600,"affinity_cache_ratio":0.9983398143711908,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (149 < 8000)","cache_hit":89600,"estimated_new_tokens":149,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13901,"ongoing_decode_tokens":13901,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":520,"cache_hit":0,"new_prefill":89749,"score_linear":13901.0,"score_lmetric":89749},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":89749,"score_linear":23228.0,"score_lmetric":112977},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":89749,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1744,"ongoing_decode_tokens":0,"pending_prefill_tokens":1744,"num_requests":1,"active_p_offloads":0,"cached_blocks":627,"cache_hit":0,"new_prefill":89749,"score_linear":1744.0,"score_lmetric":91493},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":89749,"score_linear":92209.0,"score_lmetric":344460},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":89749,"score_linear":122725.0,"score_lmetric":362484},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":500,"cache_hit":0,"new_prefill":89749,"score_linear":180892.0,"score_lmetric":269247},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":592,"cache_hit":89600,"new_prefill":149,"score_linear":-55247.0,"score_lmetric":149}],"chosen_score_linear":-55247.0,"chosen_score_lmetric":149,"t_first_token":338852.688776665,"t_first_token_unix":1779769717.7027204,"t_done":338855.275631815,"t_done_unix":1779769720.2895753},{"request_id":"1260327:6:1291827:215","session_id":"1260327","input_length":42153,"t_proxy_recv":338855.090654945,"t_decision_unix":1779769720.1045876,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9959907954356748,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (169 < 8000)","cache_hit":41984,"estimated_new_tokens":169,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":525,"cache_hit":0,"new_prefill":42153,"score_linear":14885.0,"score_lmetric":42153},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":42153,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":42153,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":630,"cache_hit":0,"new_prefill":42153,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":42153,"score_linear":92209.0,"score_lmetric":249268},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":42153,"score_linear":122725.0,"score_lmetric":267292},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":42153,"score_linear":189544.0,"score_lmetric":151563},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124102,"ongoing_decode_tokens":124102,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":592,"cache_hit":41984,"new_prefill":169,"score_linear":82118.0,"score_lmetric":338}],"chosen_score_linear":82118.0,"chosen_score_lmetric":338,"t_first_token":338855.191696235,"t_first_token_unix":1779769720.2056394,"t_done":338855.876188832,"t_done_unix":1779769720.8901327},{"request_id":"1272313:5:1284744:185","session_id":"1272313","input_length":14885,"t_proxy_recv":338852.945790998,"t_decision_unix":1779769717.9597328,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":13824,"affinity_cache_ratio":0.9287201881088344,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1061 < 8000)","cache_hit":13824,"estimated_new_tokens":1061,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":525,"cache_hit":13824,"new_prefill":1061,"score_linear":-13824.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23228,"ongoing_decode_tokens":23228,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":14885,"score_linear":23228.0,"score_lmetric":14885},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":14885,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1744,"ongoing_decode_tokens":1744,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":627,"cache_hit":0,"new_prefill":14885,"score_linear":1744.0,"score_lmetric":14885},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":14885,"score_linear":92209.0,"score_lmetric":194732},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":7680,"new_prefill":7205,"score_linear":115045.0,"score_lmetric":197396},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":500,"cache_hit":10752,"new_prefill":4133,"score_linear":170140.0,"score_lmetric":12399},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124102,"ongoing_decode_tokens":124102,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":592,"cache_hit":0,"new_prefill":14885,"score_linear":124102.0,"score_lmetric":29770}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":338853.104666732,"t_first_token_unix":1779769718.11861,"t_done":338855.932598543,"t_done_unix":1779769720.946542},{"request_id":"1290365:2:1292000:217","session_id":"1290365","input_length":23414,"t_proxy_recv":338855.658555572,"t_decision_unix":1779769720.6724975,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9840266507217904,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (374 < 8000)","cache_hit":23040,"estimated_new_tokens":374,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":525,"cache_hit":0,"new_prefill":23414,"score_linear":14885.0,"score_lmetric":23414},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":642,"cache_hit":23040,"new_prefill":374,"score_linear":-23040.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":23414,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8144,"ongoing_decode_tokens":0,"pending_prefill_tokens":8144,"num_requests":1,"active_p_offloads":0,"cached_blocks":630,"cache_hit":0,"new_prefill":23414,"score_linear":8144.0,"score_lmetric":31558},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":23414,"score_linear":92209.0,"score_lmetric":211790},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":23414,"score_linear":122725.0,"score_lmetric":229814},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":23414,"score_linear":189544.0,"score_lmetric":95346},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":76506,"ongoing_decode_tokens":76506,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":592,"cache_hit":0,"new_prefill":23414,"score_linear":76506.0,"score_lmetric":46828}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":338855.735893944,"t_first_token_unix":1779769720.7498374,"t_done":338856.051472149,"t_done_unix":1779769721.0654154},{"request_id":"1291944:1:1291944:216","session_id":"1291944","input_length":8144,"t_proxy_recv":338855.460737027,"t_decision_unix":1779769720.4746783,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":525,"cache_hit":0,"new_prefill":8144,"score_linear":14885.0,"score_lmetric":8144},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":8144,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":8144,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":630,"cache_hit":0,"new_prefill":8144,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":8144,"score_linear":92209.0,"score_lmetric":181250},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":8144,"score_linear":122725.0,"score_lmetric":199274},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":8144,"score_linear":189544.0,"score_lmetric":49536},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":76506,"ongoing_decode_tokens":76506,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":592,"cache_hit":0,"new_prefill":8144,"score_linear":76506.0,"score_lmetric":16288}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338856.03248548,"t_first_token_unix":1779769721.046429,"t_done":338856.177266417,"t_done_unix":1779769721.1912098},{"request_id":"1262354:11:1284987:186","session_id":"1262354","input_length":34353,"t_proxy_recv":338848.402530631,"t_decision_unix":1779769713.4164724,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9985736325793962,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (49 < 8000)","cache_hit":34304,"estimated_new_tokens":49,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":34353,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":34353,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":34353,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":613,"cache_hit":12800,"new_prefill":21553,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":34353,"score_linear":92209.0,"score_lmetric":233668},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":34353,"score_linear":122725.0,"score_lmetric":251692},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":34353,"score_linear":191531.0,"score_lmetric":137412},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":89226,"ongoing_decode_tokens":89226,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":590,"cache_hit":34304,"new_prefill":49,"score_linear":54922.0,"score_lmetric":49}],"chosen_score_linear":54922.0,"chosen_score_lmetric":49,"t_first_token":338848.487576829,"t_first_token_unix":1779769713.5015202,"t_done":338856.471860006,"t_done_unix":1779769721.4858038},{"request_id":"1262354:12:1291548:213","session_id":"1262354","input_length":35099,"t_proxy_recv":338856.479196252,"t_decision_unix":1779769721.4931378,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9919370922248497,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (283 < 8000)","cache_hit":34816,"estimated_new_tokens":283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15547,"ongoing_decode_tokens":15547,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":528,"cache_hit":0,"new_prefill":35099,"score_linear":15547.0,"score_lmetric":35099},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":35099,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":35099,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":645,"cache_hit":12800,"new_prefill":22299,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":35099,"score_linear":92209.0,"score_lmetric":235160},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":35099,"score_linear":122725.0,"score_lmetric":253184},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":35099,"score_linear":189544.0,"score_lmetric":130401},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":34816,"new_prefill":283,"score_linear":-34816.0,"score_lmetric":0}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":338856.58592279,"t_first_token_unix":1779769721.5998662,"t_done":338857.106893117,"t_done_unix":1779769722.1208365},{"request_id":"1272313:6:1287745:199","session_id":"1272313","input_length":15547,"t_proxy_recv":338855.936893988,"t_decision_unix":1779769720.9508352,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":15360,"affinity_cache_ratio":0.9879719560043738,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (187 < 8000)","cache_hit":15360,"estimated_new_tokens":187,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":528,"cache_hit":15360,"new_prefill":187,"score_linear":-15360.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":23414,"ongoing_decode_tokens":23414,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":15547,"score_linear":23414.0,"score_lmetric":15547},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":15547,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8144,"ongoing_decode_tokens":0,"pending_prefill_tokens":8144,"num_requests":1,"active_p_offloads":0,"cached_blocks":630,"cache_hit":0,"new_prefill":15547,"score_linear":8144.0,"score_lmetric":23691},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":15547,"score_linear":92209.0,"score_lmetric":196056},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":7680,"new_prefill":7867,"score_linear":115045.0,"score_lmetric":198720},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":531,"cache_hit":10752,"new_prefill":4795,"score_linear":178792.0,"score_lmetric":39489},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":592,"cache_hit":0,"new_prefill":15547,"score_linear":34353.0,"score_lmetric":15547}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":338856.014559036,"t_first_token_unix":1779769721.0285025,"t_done":338857.591396616,"t_done_unix":1779769722.6053402},{"request_id":"1269373:3:1277365:148","session_id":"1269373","input_length":16900,"t_proxy_recv":338805.508299253,"t_decision_unix":1779769670.5222402,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14848,"affinity_cache_ratio":0.8785798816568048,"affinity_num_requests":3,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2052 < 8000)","cache_hit":14848,"estimated_new_tokens":2052,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":4096,"new_prefill":12804,"score_linear":21091.0,"score_lmetric":12804},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":16900,"score_linear":120167.0,"score_lmetric":246486},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":16900,"score_linear":150574.0,"score_lmetric":269412},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":530,"cache_hit":0,"new_prefill":16900,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":16900,"score_linear":121069.0,"score_lmetric":220660},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":407,"cache_hit":0,"new_prefill":16900,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":498,"cache_hit":14848,"new_prefill":2052,"score_linear":159783.0,"score_lmetric":399489},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":65369,"ongoing_decode_tokens":33524,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":354,"cache_hit":0,"new_prefill":16900,"score_linear":65369.0,"score_lmetric":58578}],"chosen_score_linear":159783.0,"chosen_score_lmetric":399489,"t_first_token":338845.098511036,"t_first_token_unix":1779769710.1124547,"t_done":338858.214865638,"t_done_unix":1779769723.2288094},{"request_id":"1286551:3:1292589:219","session_id":"1286551","input_length":16914,"t_proxy_recv":338857.32749103,"t_decision_unix":1779769722.3414323,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":9216,"affinity_cache_ratio":0.54487406881873,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7698 < 8000)","cache_hit":9216,"estimated_new_tokens":7698,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15547,"ongoing_decode_tokens":15547,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":528,"cache_hit":0,"new_prefill":16914,"score_linear":15547.0,"score_lmetric":16914},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":16914,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":9216,"new_prefill":7698,"score_linear":-9216.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":645,"cache_hit":0,"new_prefill":16914,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":16914,"score_linear":92209.0,"score_lmetric":198790},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":16914,"score_linear":122725.0,"score_lmetric":216814},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":189544,"ongoing_decode_tokens":189544,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":16914,"score_linear":189544.0,"score_lmetric":50742},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":16914,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":338858.223126464,"t_first_token_unix":1779769723.23707,"t_done":338858.407717315,"t_done_unix":1779769723.4216614},{"request_id":"1292750:1:1292750:220","session_id":"1292750","input_length":4684,"t_proxy_recv":338857.962068758,"t_decision_unix":1779769722.97601,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4684 < 8000)","cache_hit":0,"estimated_new_tokens":4684,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":528,"cache_hit":0,"new_prefill":4684,"score_linear":15969.0,"score_lmetric":4684},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":4684,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16914,"ongoing_decode_tokens":0,"pending_prefill_tokens":7698,"num_requests":1,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":4684,"score_linear":16914.0,"score_lmetric":12382},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":645,"cache_hit":0,"new_prefill":4684,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":4684,"score_linear":92209.0,"score_lmetric":174330},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":4684,"score_linear":122725.0,"score_lmetric":192354},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":189544,"ongoing_decode_tokens":189544,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":531,"cache_hit":0,"new_prefill":4684,"score_linear":189544.0,"score_lmetric":14052},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":4684,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338858.262199845,"t_first_token_unix":1779769723.276143,"t_done":338858.709341968,"t_done_unix":1779769723.7232854},{"request_id":"1262354:13:1293000:221","session_id":"1262354","input_length":35210,"t_proxy_recv":338858.807178307,"t_decision_unix":1779769723.8211195,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9888099971598977,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (394 < 8000)","cache_hit":34816,"estimated_new_tokens":394,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":528,"cache_hit":0,"new_prefill":35210,"score_linear":15969.0,"score_lmetric":35210},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":651,"cache_hit":0,"new_prefill":35210,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":629,"cache_hit":0,"new_prefill":35210,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":645,"cache_hit":12800,"new_prefill":22410,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":35210,"score_linear":92209.0,"score_lmetric":235382},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":35210,"score_linear":122725.0,"score_lmetric":253406},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":202906,"ongoing_decode_tokens":172644,"pending_prefill_tokens":12854,"num_requests":3,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":35210,"score_linear":202906.0,"score_lmetric":144192},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":34816,"new_prefill":394,"score_linear":-34816.0,"score_lmetric":0}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":338858.901172138,"t_first_token_unix":1779769723.915116,"t_done":338859.489140793,"t_done_unix":1779769724.5030847},{"request_id":"1272313:7:1290281:206","session_id":"1272313","input_length":15969,"t_proxy_recv":338857.59578186,"t_decision_unix":1779769722.6097233,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":15360,"affinity_cache_ratio":0.96186361074582,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (609 < 8000)","cache_hit":15360,"estimated_new_tokens":609,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":528,"cache_hit":15360,"new_prefill":609,"score_linear":-15360.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":15969,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16914,"ongoing_decode_tokens":0,"pending_prefill_tokens":7698,"num_requests":1,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":15969,"score_linear":16914.0,"score_lmetric":23667},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":645,"cache_hit":0,"new_prefill":15969,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":15969,"score_linear":92209.0,"score_lmetric":196900},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":7680,"new_prefill":8289,"score_linear":115045.0,"score_lmetric":199564},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":189544,"ongoing_decode_tokens":189544,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":531,"cache_hit":10752,"new_prefill":5217,"score_linear":178792.0,"score_lmetric":15651},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":15969,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":338857.672474801,"t_first_token_unix":1779769722.6864178,"t_done":338859.668596399,"t_done_unix":1779769724.6825397},{"request_id":"1291100:2:1293458:223","session_id":"1291100","input_length":2572,"t_proxy_recv":338860.451265034,"t_decision_unix":1779769725.4652064,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":1536,"affinity_cache_ratio":0.5972006220839814,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1036 < 8000)","cache_hit":1536,"estimated_new_tokens":1036,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":529,"cache_hit":0,"new_prefill":2572,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":651,"cache_hit":0,"new_prefill":2572,"score_linear":24482.0,"score_lmetric":2572},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":629,"cache_hit":0,"new_prefill":2572,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":645,"cache_hit":1536,"new_prefill":1036,"score_linear":-1536.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":2572,"score_linear":92209.0,"score_lmetric":170106},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":2572,"score_linear":122725.0,"score_lmetric":188130},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":202906,"ongoing_decode_tokens":172644,"pending_prefill_tokens":12854,"num_requests":3,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":2572,"score_linear":202906.0,"score_lmetric":46278},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":2572,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":338860.531193636,"t_first_token_unix":1779769725.5451367,"t_done":338860.68266532,"t_done_unix":1779769725.6966088},{"request_id":"1293507:1:1293507:224","session_id":"1293507","input_length":6460,"t_proxy_recv":338860.638360552,"t_decision_unix":1779769725.652302,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6460 < 8000)","cache_hit":0,"estimated_new_tokens":6460,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":529,"cache_hit":0,"new_prefill":6460,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":651,"cache_hit":0,"new_prefill":6460,"score_linear":24482.0,"score_lmetric":6460},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":629,"cache_hit":0,"new_prefill":6460,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":2572,"ongoing_decode_tokens":2572,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":645,"cache_hit":0,"new_prefill":6460,"score_linear":2572.0,"score_lmetric":6460},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":6460,"score_linear":92209.0,"score_lmetric":177882},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":77805,"pending_prefill_tokens":13688,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":6460,"score_linear":122725.0,"score_lmetric":40296},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":6460,"score_linear":202906.0,"score_lmetric":19380},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":6460,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338861.07376266,"t_first_token_unix":1779769726.087706,"t_done":338861.194771829,"t_done_unix":1779769726.208715},{"request_id":"1290365:3:1293216:222","session_id":"1290365","input_length":24482,"t_proxy_recv":338859.591186018,"t_decision_unix":1779769724.6051273,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":23040,"affinity_cache_ratio":0.941099583367372,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1442 < 8000)","cache_hit":23040,"estimated_new_tokens":1442,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":528,"cache_hit":0,"new_prefill":24482,"score_linear":15969.0,"score_lmetric":24482},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":651,"cache_hit":23040,"new_prefill":1442,"score_linear":-23040.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":629,"cache_hit":0,"new_prefill":24482,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":645,"cache_hit":0,"new_prefill":24482,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":24482,"score_linear":92209.0,"score_lmetric":213926},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":24482,"score_linear":122725.0,"score_lmetric":231950},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":202906,"ongoing_decode_tokens":172644,"pending_prefill_tokens":12854,"num_requests":3,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":24482,"score_linear":202906.0,"score_lmetric":112008},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":24482,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":338859.860818725,"t_first_token_unix":1779769724.8747633,"t_done":338861.926073987,"t_done_unix":1779769726.9400177},{"request_id":"1287100:1:1287100:196","session_id":"1287100","input_length":77805,"t_proxy_recv":338839.104941551,"t_decision_unix":1779769704.118883,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":77805,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":408,"cache_hit":0,"new_prefill":77805,"score_linear":49619.0,"score_lmetric":77805},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":77805,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":77805,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":31898,"ongoing_decode_tokens":31898,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":602,"cache_hit":0,"new_prefill":77805,"score_linear":31898.0,"score_lmetric":77805},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13962,"ongoing_decode_tokens":0,"pending_prefill_tokens":13962,"num_requests":1,"active_p_offloads":0,"cached_blocks":459,"cache_hit":0,"new_prefill":77805,"score_linear":13962.0,"score_lmetric":91767},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":77805,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":77805,"score_linear":191531.0,"score_lmetric":843872},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":164268,"ongoing_decode_tokens":0,"pending_prefill_tokens":99244,"num_requests":3,"active_p_offloads":0,"cached_blocks":395,"cache_hit":0,"new_prefill":77805,"score_linear":164268.0,"score_lmetric":531147}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338860.627765114,"t_first_token_unix":1779769725.6417086,"t_done":338863.909309649,"t_done_unix":1779769728.9232535},{"request_id":"1291944:2:1293798:227","session_id":"1291944","input_length":12388,"t_proxy_recv":338861.724083821,"t_decision_unix":1779769726.7380252,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":7680,"affinity_cache_ratio":0.6199547949628673,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4708 < 8000)","cache_hit":7680,"estimated_new_tokens":4708,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28796,"ongoing_decode_tokens":0,"pending_prefill_tokens":12924,"num_requests":1,"active_p_offloads":0,"cached_blocks":529,"cache_hit":0,"new_prefill":12388,"score_linear":28796.0,"score_lmetric":25312},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":651,"cache_hit":0,"new_prefill":12388,"score_linear":24482.0,"score_lmetric":12388},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":641,"cache_hit":0,"new_prefill":12388,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40278,"ongoing_decode_tokens":0,"pending_prefill_tokens":8022,"num_requests":1,"active_p_offloads":0,"cached_blocks":647,"cache_hit":7680,"new_prefill":4708,"score_linear":32598.0,"score_lmetric":12730},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":12388,"score_linear":92209.0,"score_lmetric":189738},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":77805,"pending_prefill_tokens":13688,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":12388,"score_linear":122725.0,"score_lmetric":52152},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":12388,"score_linear":202906.0,"score_lmetric":37164},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":12388,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":32598.0,"chosen_score_lmetric":12730,"t_first_token":338863.680329196,"t_first_token_unix":1779769728.6942725,"t_done":338864.189419309,"t_done_unix":1779769729.2033625},{"request_id":"1262354:14:1294243:228","session_id":"1262354","input_length":35435,"t_proxy_recv":338862.890066202,"t_decision_unix":1779769727.9040074,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9825313955129109,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (619 < 8000)","cache_hit":34816,"estimated_new_tokens":619,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28796,"ongoing_decode_tokens":0,"pending_prefill_tokens":12924,"num_requests":1,"active_p_offloads":0,"cached_blocks":529,"cache_hit":0,"new_prefill":35435,"score_linear":28796.0,"score_lmetric":48359},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":654,"cache_hit":0,"new_prefill":35435,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":641,"cache_hit":0,"new_prefill":35435,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":52666,"ongoing_decode_tokens":0,"pending_prefill_tokens":12730,"num_requests":2,"active_p_offloads":0,"cached_blocks":647,"cache_hit":12800,"new_prefill":22635,"score_linear":39866.0,"score_lmetric":70730},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":35435,"score_linear":92209.0,"score_lmetric":235832},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":77805,"pending_prefill_tokens":13688,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":35435,"score_linear":122725.0,"score_lmetric":98246},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":35435,"score_linear":202906.0,"score_lmetric":106305},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":34816,"new_prefill":619,"score_linear":-34816.0,"score_lmetric":0}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":338863.001443898,"t_first_token_unix":1779769728.0153873,"t_done":338864.390289063,"t_done_unix":1779769729.4042325},{"request_id":"1272313:8:1293738:226","session_id":"1272313","input_length":28796,"t_proxy_recv":338861.57709554,"t_decision_unix":1779769726.5910373,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":15872,"affinity_cache_ratio":0.5511876649534657,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":15872,"estimated_new_tokens":12924,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":529,"cache_hit":15872,"new_prefill":12924,"score_linear":-15872.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":651,"cache_hit":0,"new_prefill":28796,"score_linear":24482.0,"score_lmetric":28796},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":641,"cache_hit":0,"new_prefill":28796,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40278,"ongoing_decode_tokens":0,"pending_prefill_tokens":8022,"num_requests":1,"active_p_offloads":0,"cached_blocks":647,"cache_hit":0,"new_prefill":28796,"score_linear":40278.0,"score_lmetric":36818},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":28796,"score_linear":92209.0,"score_lmetric":222554},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":77805,"pending_prefill_tokens":13688,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":7680,"new_prefill":21116,"score_linear":115045.0,"score_lmetric":69608},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":536,"cache_hit":10752,"new_prefill":18044,"score_linear":192154.0,"score_lmetric":54132},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":28796,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-15872.0,"chosen_score_lmetric":0,"t_first_token":338863.766678238,"t_first_token_unix":1779769728.7806218,"t_done":338865.590807619,"t_done_unix":1779769730.6047513},{"request_id":"1269373:4:1280765:165","session_id":"1269373","input_length":30262,"t_proxy_recv":338858.226780473,"t_decision_unix":1779769723.240722,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17408,"affinity_cache_ratio":0.5752428788579736,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=7680 chosen=17408)","cache_hit":17408,"estimated_new_tokens":12854,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":528,"cache_hit":7680,"new_prefill":22582,"score_linear":8289.0,"score_lmetric":22582},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":4684,"ongoing_decode_tokens":0,"pending_prefill_tokens":4684,"num_requests":1,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":30262,"score_linear":4684.0,"score_lmetric":34946},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16914,"ongoing_decode_tokens":16914,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":30262,"score_linear":16914.0,"score_lmetric":30262},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":645,"cache_hit":0,"new_prefill":30262,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":30262,"score_linear":92209.0,"score_lmetric":225486},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":7680,"new_prefill":22582,"score_linear":115045.0,"score_lmetric":228150},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172644,"ongoing_decode_tokens":172644,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":536,"cache_hit":17408,"new_prefill":12854,"score_linear":155236.0,"score_lmetric":25708},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":30262,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":155236.0,"chosen_score_lmetric":25708,"t_first_token":338860.586245216,"t_first_token_unix":1779769725.600189,"t_done":338866.433014065,"t_done_unix":1779769731.446958},{"request_id":"1266668:5:1281133:168","session_id":"1266668","input_length":58032,"t_proxy_recv":338854.088834721,"t_decision_unix":1779769719.1027763,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":49664,"affinity_cache_ratio":0.8558036945133719,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=7680 chosen=49664)","cache_hit":49664,"estimated_new_tokens":8368,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":525,"cache_hit":7680,"new_prefill":50352,"score_linear":7205.0,"score_lmetric":50352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":642,"cache_hit":0,"new_prefill":58032,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":621,"cache_hit":0,"new_prefill":58032,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":630,"cache_hit":0,"new_prefill":58032,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":58032,"score_linear":92209.0,"score_lmetric":281026},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":7680,"new_prefill":50352,"score_linear":115045.0,"score_lmetric":283690},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":131512,"ongoing_decode_tokens":131512,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":531,"cache_hit":49664,"new_prefill":8368,"score_linear":81848.0,"score_lmetric":16736},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124102,"ongoing_decode_tokens":124102,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":592,"cache_hit":0,"new_prefill":58032,"score_linear":124102.0,"score_lmetric":116064}],"chosen_score_linear":81848.0,"chosen_score_lmetric":16736,"t_first_token":338857.024685239,"t_first_token_unix":1779769722.0386295,"t_done":338867.164385127,"t_done_unix":1779769732.1783292},{"request_id":"1294611:1:1294611:229","session_id":"1294611","input_length":21947,"t_proxy_recv":338864.184119596,"t_decision_unix":1779769729.1980608,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":21947,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28796,"ongoing_decode_tokens":28796,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":529,"cache_hit":0,"new_prefill":21947,"score_linear":28796.0,"score_lmetric":21947},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":654,"cache_hit":0,"new_prefill":21947,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":641,"cache_hit":0,"new_prefill":21947,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":52666,"ongoing_decode_tokens":52666,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":647,"cache_hit":0,"new_prefill":21947,"score_linear":52666.0,"score_lmetric":43894},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":21947,"score_linear":92209.0,"score_lmetric":208856},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":12800,"new_prefill":9147,"score_linear":32120.0,"score_lmetric":9147},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":536,"cache_hit":12800,"new_prefill":9147,"score_linear":190106.0,"score_lmetric":27441},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":35435,"ongoing_decode_tokens":35435,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":21947,"score_linear":35435.0,"score_lmetric":21947}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338866.548428737,"t_first_token_unix":1779769731.562373,"t_done":338867.217851475,"t_done_unix":1779769732.2317948},{"request_id":"1279412:6:1293655:225","session_id":"1279412","input_length":40278,"t_proxy_recv":338861.217545338,"t_decision_unix":1779769726.231487,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":32256,"affinity_cache_ratio":0.8008342022940563,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":32256,"estimated_new_tokens":8022,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":529,"cache_hit":0,"new_prefill":40278,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":651,"cache_hit":0,"new_prefill":40278,"score_linear":24482.0,"score_lmetric":40278},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":641,"cache_hit":0,"new_prefill":40278,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":647,"cache_hit":32256,"new_prefill":8022,"score_linear":-32256.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":40278,"score_linear":92209.0,"score_lmetric":245518},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":77805,"pending_prefill_tokens":13688,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":40278,"score_linear":122725.0,"score_lmetric":107932},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":40278,"score_linear":202906.0,"score_lmetric":120834},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":12800,"new_prefill":27478,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-32256.0,"chosen_score_lmetric":0,"t_first_token":338863.237048966,"t_first_token_unix":1779769728.250992,"t_done":338867.280305012,"t_done_unix":1779769732.2942488},{"request_id":"1295690:1:1295690:232","session_id":"1295690","input_length":10857,"t_proxy_recv":338868.086516572,"t_decision_unix":1779769733.100458,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":10857,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":654,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":684,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":10857,"score_linear":92209.0,"score_lmetric":186676},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":10857,"score_linear":44920.0,"score_lmetric":10857},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":10857,"score_linear":225261.0,"score_lmetric":97632},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338868.987095743,"t_first_token_unix":1779769734.0010393,"t_done":338869.435137956,"t_done_unix":1779769734.4490817},{"request_id":"1294611:2:1295852:233","session_id":"1294611","input_length":23610,"t_proxy_recv":338868.592207999,"t_decision_unix":1779769733.6061492,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9324862346463363,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1594 < 8000)","cache_hit":22016,"estimated_new_tokens":1594,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":23610,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":654,"cache_hit":0,"new_prefill":23610,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":684,"cache_hit":22016,"new_prefill":1594,"score_linear":-22016.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":23610,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":23610,"score_linear":92209.0,"score_lmetric":212182},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":12800,"new_prefill":10810,"score_linear":32120.0,"score_lmetric":10810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":12800,"new_prefill":10810,"score_linear":212461.0,"score_lmetric":97491},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":10857,"ongoing_decode_tokens":0,"pending_prefill_tokens":10857,"num_requests":1,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":23610,"score_linear":10857.0,"score_lmetric":34467}],"chosen_score_linear":-22016.0,"chosen_score_lmetric":0,"t_first_token":338868.95204705,"t_first_token_unix":1779769733.9659905,"t_done":338870.330443734,"t_done_unix":1779769735.344387},{"request_id":"1287588:2:1289701:205","session_id":"1287588","input_length":10195,"t_proxy_recv":338847.910707931,"t_decision_unix":1779769712.9246492,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":9728,"affinity_cache_ratio":0.954193231976459,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (467 < 8000)","cache_hit":9728,"estimated_new_tokens":467,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":10195,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":10195,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9610,"ongoing_decode_tokens":9610,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":10195,"score_linear":9610.0,"score_lmetric":10195},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32191,"ongoing_decode_tokens":32191,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":10195,"score_linear":32191.0,"score_lmetric":10195},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":505,"cache_hit":9728,"new_prefill":467,"score_linear":72286.0,"score_lmetric":82481},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":10195,"score_linear":122725.0,"score_lmetric":203376},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":10195,"score_linear":191531.0,"score_lmetric":40780},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123508,"ongoing_decode_tokens":123508,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":10195,"score_linear":123508.0,"score_lmetric":20390}],"chosen_score_linear":72286.0,"chosen_score_lmetric":82481,"t_first_token":338870.268274769,"t_first_token_unix":1779769735.2822182,"t_done":338870.343633051,"t_done_unix":1779769735.3575761},{"request_id":"1289570:1:1289570:204","session_id":"1289570","input_length":82014,"t_proxy_recv":338847.429965341,"t_decision_unix":1779769712.4439065,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":82014,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":82014,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":82014,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9610,"ongoing_decode_tokens":0,"pending_prefill_tokens":8074,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":82014,"score_linear":9610.0,"score_lmetric":90088},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32191,"ongoing_decode_tokens":32191,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":82014,"score_linear":32191.0,"score_lmetric":82014},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":82014,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122725,"ongoing_decode_tokens":0,"pending_prefill_tokens":91493,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":82014,"score_linear":122725.0,"score_lmetric":347014},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":82014,"score_linear":191531.0,"score_lmetric":328056},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123254,"ongoing_decode_tokens":123254,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":589,"cache_hit":0,"new_prefill":82014,"score_linear":123254.0,"score_lmetric":164028}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338870.266486652,"t_first_token_unix":1779769735.2804308,"t_done":338870.808874899,"t_done_unix":1779769735.8228192},{"request_id":"1262354:15:1296106:234","session_id":"1262354","input_length":35669,"t_proxy_recv":338869.539344923,"t_decision_unix":1779769734.5532863,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9904398777650061,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (341 < 8000)","cache_hit":35328,"estimated_new_tokens":341,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":35669,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":654,"cache_hit":0,"new_prefill":35669,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23610,"ongoing_decode_tokens":23610,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":684,"cache_hit":0,"new_prefill":35669,"score_linear":23610.0,"score_lmetric":35669},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":672,"cache_hit":12800,"new_prefill":22869,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":35669,"score_linear":92209.0,"score_lmetric":236300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":35669,"score_linear":44920.0,"score_lmetric":35669},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":35669,"score_linear":225261.0,"score_lmetric":172068},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":615,"cache_hit":35328,"new_prefill":341,"score_linear":-35328.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":338869.614735277,"t_first_token_unix":1779769734.6286786,"t_done":338872.065257422,"t_done_unix":1779769737.0792007},{"request_id":"1296872:1:1296872:235","session_id":"1296872","input_length":624,"t_proxy_recv":338872.150683701,"t_decision_unix":1779769737.1646252,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (624 < 8000)","cache_hit":0,"estimated_new_tokens":624,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":654,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":687,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":624,"score_linear":44920.0,"score_lmetric":624},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":624,"score_linear":225261.0,"score_lmetric":66933},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338872.220321753,"t_first_token_unix":1779769737.2342649,"t_done":338872.323993869,"t_done_unix":1779769737.3379369},{"request_id":"1296983:1:1296983:236","session_id":"1296983","input_length":1791,"t_proxy_recv":338872.583899317,"t_decision_unix":1779769737.5978408,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1791 < 8000)","cache_hit":0,"estimated_new_tokens":1791,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":687,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":1791,"score_linear":44920.0,"score_lmetric":1791},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":1791,"score_linear":225261.0,"score_lmetric":70434},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338872.73322331,"t_first_token_unix":1779769737.7471664,"t_done":338872.962259832,"t_done_unix":1779769737.9762034},{"request_id":"1260327:7:1297673:242","session_id":"1260327","input_length":42363,"t_proxy_recv":338874.772769838,"t_decision_unix":1779769739.786711,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":41984,"affinity_cache_ratio":0.99105351367939,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (379 < 8000)","cache_hit":41984,"estimated_new_tokens":379,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":42363,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":42363,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":42363,"score_linear":86277.0,"score_lmetric":172288},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":119700,"ongoing_decode_tokens":0,"pending_prefill_tokens":79252,"num_requests":2,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":42363,"score_linear":119700.0,"score_lmetric":243230},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":42363,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":42363,"score_linear":44920.0,"score_lmetric":42363},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":42363,"score_linear":225261.0,"score_lmetric":192150},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":41984,"new_prefill":379,"score_linear":-41984.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":338874.927584802,"t_first_token_unix":1779769739.9415286,"t_done":338875.158052155,"t_done_unix":1779769740.1719959},{"request_id":"1272313:9:1297810:243","session_id":"1272313","input_length":29149,"t_proxy_recv":338875.250236797,"t_decision_unix":1779769740.2641785,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9836358022573674,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (477 < 8000)","cache_hit":28672,"estimated_new_tokens":477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":28672,"new_prefill":477,"score_linear":-28672.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":29149,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":29149,"score_linear":86277.0,"score_lmetric":145860},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":119700,"ongoing_decode_tokens":0,"pending_prefill_tokens":79252,"num_requests":2,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":29149,"score_linear":119700.0,"score_lmetric":216802},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":29149,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":7680,"new_prefill":21469,"score_linear":37240.0,"score_lmetric":21469},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":10752,"new_prefill":18397,"score_linear":214509.0,"score_lmetric":120252},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":29149,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-28672.0,"chosen_score_lmetric":0,"t_first_token":338875.34217981,"t_first_token_unix":1779769740.356123,"t_done":338876.274085007,"t_done_unix":1779769741.2880282},{"request_id":"1290442:2:1298058:246","session_id":"1290442","input_length":11994,"t_proxy_recv":338876.121297667,"t_decision_unix":1779769741.1352394,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":1,"affinity_cache_hit":7168,"affinity_cache_ratio":0.5976321494080373,"affinity_num_requests":3,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11994,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":29149,"ongoing_decode_tokens":29149,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":11994,"score_linear":29149.0,"score_lmetric":11994},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":11994,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":11994,"score_linear":86277.0,"score_lmetric":111550},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":7168,"new_prefill":4826,"score_linear":134162.0,"score_lmetric":278724},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13992,"ongoing_decode_tokens":0,"pending_prefill_tokens":12456,"num_requests":1,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":11994,"score_linear":13992.0,"score_lmetric":24450},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":11994,"score_linear":44920.0,"score_lmetric":11994},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":11994,"score_linear":225261.0,"score_lmetric":101043},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":11994,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338877.11640652,"t_first_token_unix":1779769742.1303499,"t_done":338877.204051909,"t_done_unix":1779769742.217995},{"request_id":"1297811:1:1297811:244","session_id":"1297811","input_length":13992,"t_proxy_recv":338875.25565302,"t_decision_unix":1779769740.2695947,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":1536,"estimated_new_tokens":12456,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":29149,"ongoing_decode_tokens":0,"pending_prefill_tokens":477,"num_requests":1,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":13992,"score_linear":29149.0,"score_lmetric":14469},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":13992,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":13992,"score_linear":86277.0,"score_lmetric":115546},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":119700,"ongoing_decode_tokens":0,"pending_prefill_tokens":79252,"num_requests":2,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":13992,"score_linear":119700.0,"score_lmetric":186488},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":665,"cache_hit":1536,"new_prefill":12456,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":13992,"score_linear":44920.0,"score_lmetric":13992},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":13992,"score_linear":225261.0,"score_lmetric":107037},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":13992,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":338876.402960447,"t_first_token_unix":1779769741.416904,"t_done":338877.705857795,"t_done_unix":1779769742.7198012},{"request_id":"1290365:4:1298540:247","session_id":"1290365","input_length":25103,"t_proxy_recv":338877.827951898,"t_decision_unix":1779769742.8418934,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":24576,"affinity_cache_ratio":0.979006493247819,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (527 < 8000)","cache_hit":24576,"estimated_new_tokens":527,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":25103,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":678,"cache_hit":24576,"new_prefill":527,"score_linear":-24576.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":25103,"score_linear":86277.0,"score_lmetric":137768},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":25103,"score_linear":141330.0,"score_lmetric":339555},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":689,"cache_hit":0,"new_prefill":25103,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":25103,"score_linear":44920.0,"score_lmetric":25103},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":25103,"score_linear":225261.0,"score_lmetric":75309},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":25103,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":338877.939554377,"t_first_token_unix":1779769742.953498,"t_done":338879.011065259,"t_done_unix":1779769744.0250099},{"request_id":"1298970:1:1298970:248","session_id":"1298970","input_length":9530,"t_proxy_recv":338879.390594266,"t_decision_unix":1779769744.4045353,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9530,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":9530,"score_linear":86277.0,"score_lmetric":106622},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":9530,"score_linear":141330.0,"score_lmetric":292836},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":689,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":9530,"score_linear":44920.0,"score_lmetric":9530},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":9530,"score_linear":225261.0,"score_lmetric":28590},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338880.120276474,"t_first_token_unix":1779769745.1342201,"t_done":338880.307198572,"t_done_unix":1779769745.321142},{"request_id":"1299073:1:1299073:249","session_id":"1299073","input_length":881,"t_proxy_recv":338879.767832264,"t_decision_unix":1779769744.7817733,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (881 < 8000)","cache_hit":0,"estimated_new_tokens":881,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":9530,"ongoing_decode_tokens":0,"pending_prefill_tokens":9530,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":881,"score_linear":9530.0,"score_lmetric":10411},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":881,"score_linear":86277.0,"score_lmetric":89324},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":881,"score_linear":141330.0,"score_lmetric":266889},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":689,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":881,"score_linear":44920.0,"score_lmetric":881},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":881,"score_linear":225261.0,"score_lmetric":2643},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338879.848938873,"t_first_token_unix":1779769744.8628824,"t_done":338880.97628045,"t_done_unix":1779769745.9902246},{"request_id":"1296983:2:1299368:250","session_id":"1296983","input_length":7963,"t_proxy_recv":338880.575346213,"t_decision_unix":1779769745.5892878,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":4,"affinity_cache_hit":1536,"affinity_cache_ratio":0.1928921260831345,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7963 < 8000)","cache_hit":0,"estimated_new_tokens":7963,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":7963,"score_linear":881.0,"score_lmetric":7963},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":697,"cache_hit":0,"new_prefill":7963,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":1536,"new_prefill":6427,"score_linear":84741.0,"score_lmetric":100416},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":7963,"score_linear":141330.0,"score_lmetric":288135},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":689,"cache_hit":0,"new_prefill":7963,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":7963,"score_linear":44920.0,"score_lmetric":7963},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":7963,"score_linear":225261.0,"score_lmetric":23889},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":7963,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338881.138760741,"t_first_token_unix":1779769746.152704,"t_done":338881.333580718,"t_done_unix":1779769746.3475242},{"request_id":"1296872:2:1299394:251","session_id":"1296872","input_length":8352,"t_proxy_recv":338880.68841772,"t_decision_unix":1779769745.702359,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":512,"affinity_cache_ratio":0.06130268199233716,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7840 < 8000)","cache_hit":512,"estimated_new_tokens":7840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":8352,"score_linear":881.0,"score_lmetric":8352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":697,"cache_hit":512,"new_prefill":7840,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":8352,"score_linear":86277.0,"score_lmetric":104266},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":8352,"score_linear":141330.0,"score_lmetric":289302},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":7963,"ongoing_decode_tokens":0,"pending_prefill_tokens":7963,"num_requests":1,"active_p_offloads":0,"cached_blocks":689,"cache_hit":0,"new_prefill":8352,"score_linear":7963.0,"score_lmetric":16315},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":8352,"score_linear":44920.0,"score_lmetric":8352},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":8352,"score_linear":225261.0,"score_lmetric":25056},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":8352,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":338881.254373606,"t_first_token_unix":1779769746.268317,"t_done":338881.38785866,"t_done_unix":1779769746.401802},{"request_id":"1292750:2:1299406:252","session_id":"1292750","input_length":19157,"t_proxy_recv":338880.728910306,"t_decision_unix":1779769745.7428517,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":7,"affinity_cache_hit":4608,"affinity_cache_ratio":0.24053870647804979,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":19157,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":555,"cache_hit":0,"new_prefill":19157,"score_linear":881.0,"score_lmetric":19157},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":8352,"ongoing_decode_tokens":0,"pending_prefill_tokens":7840,"num_requests":1,"active_p_offloads":0,"cached_blocks":697,"cache_hit":4608,"new_prefill":14549,"score_linear":3744.0,"score_lmetric":22389},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":19157,"score_linear":86277.0,"score_lmetric":125876},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":19157,"score_linear":141330.0,"score_lmetric":321717},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":7963,"ongoing_decode_tokens":0,"pending_prefill_tokens":7963,"num_requests":1,"active_p_offloads":0,"cached_blocks":689,"cache_hit":0,"new_prefill":19157,"score_linear":7963.0,"score_lmetric":27120},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":19157,"score_linear":44920.0,"score_lmetric":19157},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":19157,"score_linear":225261.0,"score_lmetric":57471},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":19157,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338882.655454037,"t_first_token_unix":1779769747.6693978,"t_done":338882.745032994,"t_done_unix":1779769747.7589767},{"request_id":"1277909:3:1289205:201","session_id":"1277909","input_length":44920,"t_proxy_recv":338846.129351429,"t_decision_unix":1779769711.1432927,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":31232,"affinity_cache_ratio":0.6952804986642921,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":31232,"estimated_new_tokens":13688,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":4096,"new_prefill":40824,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":601,"cache_hit":0,"new_prefill":44920,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":44920,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":612,"cache_hit":0,"new_prefill":44920,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":44920,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":527,"cache_hit":31232,"new_prefill":13688,"score_linear":46573.0,"score_lmetric":91493},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":498,"cache_hit":7680,"new_prefill":37240,"score_linear":183851.0,"score_lmetric":148960},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":122968,"ongoing_decode_tokens":122968,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":588,"cache_hit":0,"new_prefill":44920,"score_linear":122968.0,"score_lmetric":89840}],"chosen_score_linear":46573.0,"chosen_score_lmetric":91493,"t_first_token":338863.217933612,"t_first_token_unix":1779769728.2318826,"t_done":338883.866207198,"t_done_unix":1779769748.8801508},{"request_id":"1286804:2:1300234:256","session_id":"1286804","input_length":14213,"t_proxy_recv":338883.555009119,"t_decision_unix":1779769748.5689406,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":13824,"affinity_cache_ratio":0.9726306902131852,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (389 < 8000)","cache_hit":13824,"estimated_new_tokens":389,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":48379,"ongoing_decode_tokens":48379,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":14213,"score_linear":48379.0,"score_lmetric":28426},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":14213,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":14213,"score_linear":86277.0,"score_lmetric":115988},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":14213,"score_linear":141330.0,"score_lmetric":306885},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":704,"cache_hit":13824,"new_prefill":389,"score_linear":-13824.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":14213,"score_linear":44920.0,"score_lmetric":14213},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":14213,"score_linear":225261.0,"score_lmetric":42639},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":14213,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":338883.613575105,"t_first_token_unix":1779769748.6275184,"t_done":338884.262562026,"t_done_unix":1779769749.2765057},{"request_id":"1272313:10:1299814:254","session_id":"1272313","input_length":29486,"t_proxy_recv":338882.10870189,"t_decision_unix":1779769747.1226432,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9897578511836126,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (302 < 8000)","cache_hit":29184,"estimated_new_tokens":302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18893,"ongoing_decode_tokens":0,"pending_prefill_tokens":18893,"num_requests":1,"active_p_offloads":0,"cached_blocks":557,"cache_hit":29184,"new_prefill":302,"score_linear":-10291.0,"score_lmetric":19195},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":29486,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":29486,"score_linear":86277.0,"score_lmetric":146534},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":29486,"score_linear":141330.0,"score_lmetric":352704},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":29486,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":7680,"new_prefill":21806,"score_linear":37240.0,"score_lmetric":21806},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":10752,"new_prefill":18734,"score_linear":214509.0,"score_lmetric":56202},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":19157,"ongoing_decode_tokens":0,"pending_prefill_tokens":19157,"num_requests":1,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":29486,"score_linear":19157.0,"score_lmetric":48643}],"chosen_score_linear":-10291.0,"chosen_score_lmetric":19195,"t_first_token":338882.982709149,"t_first_token_unix":1779769747.996653,"t_done":338884.808825072,"t_done_unix":1779769749.8227684},{"request_id":"1297473:1:1297473:239","session_id":"1297473","input_length":62414,"t_proxy_recv":338874.333586842,"t_decision_unix":1779769739.3475282,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":18944,"estimated_new_tokens":43470,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":690,"cache_hit":18944,"new_prefill":43470,"score_linear":-18944.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":119700,"ongoing_decode_tokens":0,"pending_prefill_tokens":79252,"num_requests":2,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":62414,"score_linear":119700.0,"score_lmetric":283332},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":62414,"score_linear":44920.0,"score_lmetric":62414},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":62414,"score_linear":225261.0,"score_lmetric":252303},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-18944.0,"chosen_score_lmetric":0,"t_first_token":338886.410520785,"t_first_token_unix":1779769751.4244645,"t_done":338886.822318733,"t_done_unix":1779769751.8362625},{"request_id":"1269373:5:1282812:174","session_id":"1269373","input_length":30594,"t_proxy_recv":338866.440859641,"t_decision_unix":1779769731.4548013,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":30208,"affinity_cache_ratio":0.9873831470222919,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (386 < 8000)","cache_hit":30208,"estimated_new_tokens":386,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":7680,"new_prefill":22914,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":654,"cache_hit":0,"new_prefill":30594,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21947,"ongoing_decode_tokens":0,"pending_prefill_tokens":21947,"num_requests":1,"active_p_offloads":0,"cached_blocks":641,"cache_hit":0,"new_prefill":30594,"score_linear":21947.0,"score_lmetric":52541},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40278,"ongoing_decode_tokens":40278,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":656,"cache_hit":0,"new_prefill":30594,"score_linear":40278.0,"score_lmetric":30594},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":30594,"score_linear":92209.0,"score_lmetric":226150},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":7680,"new_prefill":22914,"score_linear":37240.0,"score_lmetric":22914},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172644,"ongoing_decode_tokens":172644,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":561,"cache_hit":30208,"new_prefill":386,"score_linear":142436.0,"score_lmetric":772},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":30594,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":142436.0,"chosen_score_lmetric":772,"t_first_token":338866.570679755,"t_first_token_unix":1779769731.584623,"t_done":338887.675339476,"t_done_unix":1779769752.6892831},{"request_id":"1290365:5:1300807:259","session_id":"1290365","input_length":25948,"t_proxy_recv":338885.750889366,"t_decision_unix":1779769750.7648308,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9668567905040851,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (860 < 8000)","cache_hit":25088,"estimated_new_tokens":860,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":25948,"score_linear":18893.0,"score_lmetric":25948},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":25088,"new_prefill":860,"score_linear":-25088.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":25948,"score_linear":86277.0,"score_lmetric":139458},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":25948,"score_linear":141330.0,"score_lmetric":342090},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":25948,"score_linear":43607.0,"score_lmetric":55731},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":25948,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":25948,"score_linear":225261.0,"score_lmetric":77844},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":25948,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":0,"t_first_token":338885.976780645,"t_first_token_unix":1779769750.9907238,"t_done":338888.095052442,"t_done_unix":1779769753.1089966},{"request_id":"1299507:1:1299507:253","session_id":"1299507","input_length":18893,"t_proxy_recv":338880.989546333,"t_decision_unix":1779769746.0034878,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":18893,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":557,"cache_hit":0,"new_prefill":18893,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":8352,"ongoing_decode_tokens":0,"pending_prefill_tokens":7840,"num_requests":1,"active_p_offloads":0,"cached_blocks":697,"cache_hit":0,"new_prefill":18893,"score_linear":8352.0,"score_lmetric":26733},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":18893,"score_linear":86277.0,"score_lmetric":125348},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":18893,"score_linear":141330.0,"score_lmetric":320925},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":7963,"ongoing_decode_tokens":0,"pending_prefill_tokens":7963,"num_requests":1,"active_p_offloads":0,"cached_blocks":689,"cache_hit":0,"new_prefill":18893,"score_linear":7963.0,"score_lmetric":26856},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":18893,"score_linear":44920.0,"score_lmetric":18893},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":11776,"new_prefill":7117,"score_linear":213485.0,"score_lmetric":21351},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":19157,"ongoing_decode_tokens":0,"pending_prefill_tokens":19157,"num_requests":1,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":18893,"score_linear":19157.0,"score_lmetric":38050}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338882.900910637,"t_first_token_unix":1779769747.9148548,"t_done":338889.157820078,"t_done_unix":1779769754.1717637},{"request_id":"1294611:3:1297534:240","session_id":"1294611","input_length":23863,"t_proxy_recv":338874.428188655,"t_decision_unix":1779769739.4421303,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9869672715081926,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (311 < 8000)","cache_hit":23552,"estimated_new_tokens":311,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":690,"cache_hit":23552,"new_prefill":311,"score_linear":38862.0,"score_lmetric":43781},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":119700,"ongoing_decode_tokens":0,"pending_prefill_tokens":79252,"num_requests":2,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":23863,"score_linear":119700.0,"score_lmetric":206230},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":12800,"new_prefill":11063,"score_linear":32120.0,"score_lmetric":11063},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":12800,"new_prefill":11063,"score_linear":212461.0,"score_lmetric":98250},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":38862.0,"chosen_score_lmetric":43781,"t_first_token":338886.411847654,"t_first_token_unix":1779769751.4257908,"t_done":338889.640470378,"t_done_unix":1779769754.654414},{"request_id":"1266668:6:1284003:181","session_id":"1266668","input_length":80055,"t_proxy_recv":338867.186201571,"t_decision_unix":1779769732.200143,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58368,"affinity_cache_ratio":0.7290987446130786,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=7680 chosen=58368)","cache_hit":58368,"estimated_new_tokens":21687,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":7680,"new_prefill":72375,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":654,"cache_hit":0,"new_prefill":80055,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21947,"ongoing_decode_tokens":21947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":641,"cache_hit":0,"new_prefill":80055,"score_linear":21947.0,"score_lmetric":80055},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40278,"ongoing_decode_tokens":40278,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":656,"cache_hit":0,"new_prefill":80055,"score_linear":40278.0,"score_lmetric":80055},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":92209,"ongoing_decode_tokens":0,"pending_prefill_tokens":82481,"num_requests":2,"active_p_offloads":0,"cached_blocks":505,"cache_hit":0,"new_prefill":80055,"score_linear":92209.0,"score_lmetric":325072},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":7680,"new_prefill":72375,"score_linear":37240.0,"score_lmetric":72375},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145206,"ongoing_decode_tokens":145206,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":578,"cache_hit":58368,"new_prefill":21687,"score_linear":86838.0,"score_lmetric":43374},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":80055,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":86838.0,"chosen_score_lmetric":43374,"t_first_token":338876.738853511,"t_first_token_unix":1779769741.752797,"t_done":338889.724671224,"t_done_unix":1779769754.7386146},{"request_id":"1294611:4:1300505:257","session_id":"1294611","input_length":24384,"t_proxy_recv":338889.64563504,"t_decision_unix":1779769754.659577,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9868766404199475,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (320 < 8000)","cache_hit":24064,"estimated_new_tokens":320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":24384,"score_linear":34706.0,"score_lmetric":55506},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":24384,"score_linear":28972.0,"score_lmetric":53356},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":775,"cache_hit":24064,"new_prefill":320,"score_linear":-24064.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":24384,"score_linear":141330.0,"score_lmetric":158877},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":24384,"score_linear":43607.0,"score_lmetric":54167},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":12800,"new_prefill":11584,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":233455,"ongoing_decode_tokens":233455,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":579,"cache_hit":12800,"new_prefill":11584,"score_linear":220655.0,"score_lmetric":34752},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":24384,"score_linear":43181.0,"score_lmetric":24384}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":338889.703211969,"t_first_token_unix":1779769754.7171552,"t_done":338890.068764933,"t_done_unix":1779769755.0827081},{"request_id":"1294611:5:1301356:262","session_id":"1294611","input_length":25650,"t_proxy_recv":338890.075461885,"t_decision_unix":1779769755.0894036,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9381676413255361,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1586 < 8000)","cache_hit":24064,"estimated_new_tokens":1586,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":25650,"score_linear":34706.0,"score_lmetric":56772},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":25650,"score_linear":28972.0,"score_lmetric":54622},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":775,"cache_hit":24064,"new_prefill":1586,"score_linear":-24064.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":25650,"score_linear":141330.0,"score_lmetric":162675},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":25650,"score_linear":43607.0,"score_lmetric":55433},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":12800,"new_prefill":12850,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":12800,"new_prefill":12850,"score_linear":228341.0,"score_lmetric":60621},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":25650,"score_linear":43181.0,"score_lmetric":25650}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":338890.360371772,"t_first_token_unix":1779769755.3743157,"t_done":338890.666956647,"t_done_unix":1779769755.6809},{"request_id":"1302071:1:1302071:267","session_id":"1302071","input_length":6527,"t_proxy_recv":338890.108025659,"t_decision_unix":1779769755.1219673,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":6527,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6527 < 8000)","cache_hit":0,"estimated_new_tokens":6527,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":6527,"score_linear":34706.0,"score_lmetric":37649},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":6527,"score_linear":28972.0,"score_lmetric":35499},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":25650,"ongoing_decode_tokens":0,"pending_prefill_tokens":1586,"num_requests":1,"active_p_offloads":0,"cached_blocks":775,"cache_hit":0,"new_prefill":6527,"score_linear":25650.0,"score_lmetric":8113},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":6527,"score_linear":141330.0,"score_lmetric":105306},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":6527,"score_linear":43607.0,"score_lmetric":36310},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":6527,"score_linear":26441.0,"score_lmetric":32968},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":6527,"score_linear":241141.0,"score_lmetric":41652},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":6527,"score_linear":43181.0,"score_lmetric":6527}],"chosen_score_linear":43181.0,"chosen_score_lmetric":6527,"t_first_token":338890.54988081,"t_first_token_unix":1779769755.5638247,"t_done":338890.964661862,"t_done_unix":1779769755.9786053},{"request_id":"1302449:1:1302449:269","session_id":"1302449","input_length":1105,"t_proxy_recv":338891.651089788,"t_decision_unix":1779769756.6650314,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":1105,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1105 < 8000)","cache_hit":0,"estimated_new_tokens":1105,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":1105,"score_linear":34706.0,"score_lmetric":32227},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":1105,"score_linear":28972.0,"score_lmetric":30077},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":49855,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":1105,"score_linear":49855.0,"score_lmetric":50960},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":1105,"score_linear":141330.0,"score_lmetric":89040},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":1105,"score_linear":43607.0,"score_lmetric":1105},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":1105,"score_linear":26441.0,"score_lmetric":27546},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":1105,"score_linear":241141.0,"score_lmetric":25386},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":1105,"score_linear":43181.0,"score_lmetric":1105}],"chosen_score_linear":43181.0,"chosen_score_lmetric":1105,"t_first_token":338891.813979455,"t_first_token_unix":1779769756.8279226,"t_done":338892.559895691,"t_done_unix":1779769757.573839},{"request_id":"1295690:2:1301706:264","session_id":"1295690","input_length":28972,"t_proxy_recv":338888.683824336,"t_decision_unix":1779769753.6977656,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":1,"affinity_cache_hit":10752,"affinity_cache_ratio":0.37111694049427035,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":28972,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":28972,"score_linear":18893.0,"score_lmetric":28972},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":774,"cache_hit":0,"new_prefill":28972,"score_linear":23863.0,"score_lmetric":28972},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":28972,"score_linear":141330.0,"score_lmetric":172641},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":28972,"score_linear":43607.0,"score_lmetric":58755},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":233455,"ongoing_decode_tokens":194667,"pending_prefill_tokens":8068,"num_requests":3,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":28972,"score_linear":233455.0,"score_lmetric":111120},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":0,"pending_prefill_tokens":1197,"num_requests":1,"active_p_offloads":0,"cached_blocks":653,"cache_hit":10752,"new_prefill":18220,"score_linear":32429.0,"score_lmetric":19417}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338892.408442821,"t_first_token_unix":1779769757.4223866,"t_done":338892.680100492,"t_done_unix":1779769757.694044},{"request_id":"1302063:1:1302063:266","session_id":"1302063","input_length":26441,"t_proxy_recv":338890.10509143,"t_decision_unix":1779769755.119033,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":26441,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":26441,"score_linear":34706.0,"score_lmetric":57563},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":26441,"score_linear":28972.0,"score_lmetric":55413},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":25650,"ongoing_decode_tokens":0,"pending_prefill_tokens":1586,"num_requests":1,"active_p_offloads":0,"cached_blocks":775,"cache_hit":0,"new_prefill":26441,"score_linear":25650.0,"score_lmetric":28027},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":26441,"score_linear":141330.0,"score_lmetric":165048},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":26441,"score_linear":43607.0,"score_lmetric":56224},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":26441,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":26441,"score_linear":241141.0,"score_lmetric":101394},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":26441,"score_linear":43181.0,"score_lmetric":26441}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338893.324355724,"t_first_token_unix":1779769758.338299,"t_done":338893.580460871,"t_done_unix":1779769758.5944045},{"request_id":"1260327:8:1301686:263","session_id":"1260327","input_length":43181,"t_proxy_recv":338888.597680586,"t_decision_unix":1779769753.6116214,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9722794747689957,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1197 < 8000)","cache_hit":41984,"estimated_new_tokens":1197,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":43181,"score_linear":18893.0,"score_lmetric":43181},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":774,"cache_hit":0,"new_prefill":43181,"score_linear":23863.0,"score_lmetric":43181},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":43181,"score_linear":141330.0,"score_lmetric":215268},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":43181,"score_linear":43607.0,"score_lmetric":72964},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":233455,"ongoing_decode_tokens":194667,"pending_prefill_tokens":8068,"num_requests":3,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":43181,"score_linear":233455.0,"score_lmetric":153747},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":653,"cache_hit":41984,"new_prefill":1197,"score_linear":-41984.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":338888.93815194,"t_first_token_unix":1779769753.9520955,"t_done":338894.525802207,"t_done_unix":1779769759.5397456},{"request_id":"1302907:1:1302907:272","session_id":"1302907","input_length":15958,"t_proxy_recv":338893.017083825,"t_decision_unix":1779769758.0310252,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":15958,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":0,"pending_prefill_tokens":31374,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":15958,"score_linear":64654.0,"score_lmetric":94664},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":770,"cache_hit":0,"new_prefill":15958,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":15958,"score_linear":80500.0,"score_lmetric":141716},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":119700,"pending_prefill_tokens":8830,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":15958,"score_linear":141330.0,"score_lmetric":74364},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":15958,"score_linear":43607.0,"score_lmetric":15958},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":15958,"score_linear":26441.0,"score_lmetric":42399},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":15958,"score_linear":241141.0,"score_lmetric":69945},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":667,"cache_hit":0,"new_prefill":15958,"score_linear":43181.0,"score_lmetric":15958}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338894.476544827,"t_first_token_unix":1779769759.4904883,"t_done":338894.680071149,"t_done_unix":1779769759.6940148},{"request_id":"1302449:2:1303563:277","session_id":"1302449","input_length":3709,"t_proxy_recv":338895.34071219,"t_decision_unix":1779769760.3546534,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":1,"affinity_cache_hit":1024,"affinity_cache_ratio":0.27608519816662175,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3709 < 8000)","cache_hit":0,"estimated_new_tokens":3709,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":3709,"score_linear":64654.0,"score_lmetric":7418},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":3709,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":3709,"score_linear":80500.0,"score_lmetric":117218},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":141330,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":3709,"score_linear":141330.0,"score_lmetric":11127},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":3709,"score_linear":43607.0,"score_lmetric":3709},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":758,"cache_hit":0,"new_prefill":3709,"score_linear":14944.0,"score_lmetric":18653},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":3709,"score_linear":241141.0,"score_lmetric":11127},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":15081,"ongoing_decode_tokens":0,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":670,"cache_hit":1024,"new_prefill":2685,"score_linear":14057.0,"score_lmetric":17766}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338895.57504205,"t_first_token_unix":1779769760.5889854,"t_done":338896.000079365,"t_done_unix":1779769761.0140226},{"request_id":"1303651:1:1303651:278","session_id":"1303651","input_length":3065,"t_proxy_recv":338895.648468385,"t_decision_unix":1779769760.66241,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":3065,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3065 < 8000)","cache_hit":0,"estimated_new_tokens":3065,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":3065,"score_linear":64654.0,"score_lmetric":6130},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":3709,"ongoing_decode_tokens":3709,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":3065,"score_linear":3709.0,"score_lmetric":3065},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":3065,"score_linear":80500.0,"score_lmetric":115930},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":141330,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":3065,"score_linear":141330.0,"score_lmetric":9195},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":3065,"score_linear":43607.0,"score_lmetric":3065},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":758,"cache_hit":0,"new_prefill":3065,"score_linear":14944.0,"score_lmetric":18009},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":3065,"score_linear":241141.0,"score_lmetric":9195},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":15081,"ongoing_decode_tokens":0,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":670,"cache_hit":0,"new_prefill":3065,"score_linear":15081.0,"score_lmetric":18146}],"chosen_score_linear":3709.0,"chosen_score_lmetric":3065,"t_first_token":338895.840098972,"t_first_token_unix":1779769760.8540423,"t_done":338896.09735228,"t_done_unix":1779769761.1112955},{"request_id":"1297117:1:1297117:237","session_id":"1297117","input_length":59507,"t_proxy_recv":338873.063754747,"t_decision_unix":1779769738.0776963,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":59507,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":59507,"score_linear":44920.0,"score_lmetric":59507},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":59507,"score_linear":225261.0,"score_lmetric":243582},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338887.47528064,"t_first_token_unix":1779769752.4892242,"t_done":338896.386068228,"t_done_unix":1779769761.4000115},{"request_id":"1279412:7:1297236:238","session_id":"1279412","input_length":60193,"t_proxy_recv":338873.586896141,"t_decision_unix":1779769738.6008377,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":40448,"affinity_cache_ratio":0.6719718239662419,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":40448,"estimated_new_tokens":19745,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":1,"active_p_offloads":0,"cached_blocks":672,"cache_hit":40448,"new_prefill":19745,"score_linear":19059.0,"score_lmetric":79252},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":60193,"score_linear":44920.0,"score_lmetric":60193},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":60193,"score_linear":225261.0,"score_lmetric":245640},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":12800,"new_prefill":47393,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":19059.0,"chosen_score_lmetric":79252,"t_first_token":338892.469210142,"t_first_token_unix":1779769757.4831533,"t_done":338896.406012248,"t_done_unix":1779769761.4199557},{"request_id":"1303517:1:1303517:275","session_id":"1303517","input_length":15081,"t_proxy_recv":338895.213273695,"t_decision_unix":1779769760.227215,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":15081,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":15081,"score_linear":64654.0,"score_lmetric":30162},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":15081,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":15081,"score_linear":80500.0,"score_lmetric":139962},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":141330,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":15081,"score_linear":141330.0,"score_lmetric":45243},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":15081,"score_linear":43607.0,"score_lmetric":15081},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":758,"cache_hit":0,"new_prefill":15081,"score_linear":14944.0,"score_lmetric":30025},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":15081,"score_linear":241141.0,"score_lmetric":45243},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":670,"cache_hit":0,"new_prefill":15081,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338896.568197337,"t_first_token_unix":1779769761.5821412,"t_done":338896.899769631,"t_done_unix":1779769761.9137132},{"request_id":"1303462:1:1303462:274","session_id":"1303462","input_length":14944,"t_proxy_recv":338895.048231407,"t_decision_unix":1779769760.0621727,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":14944,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":14944,"score_linear":64654.0,"score_lmetric":29888},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":14944,"score_linear":80500.0,"score_lmetric":139688},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":141330,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":14944,"score_linear":141330.0,"score_lmetric":44832},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":14944,"score_linear":43607.0,"score_lmetric":14944},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":758,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":14944,"score_linear":241141.0,"score_lmetric":44832},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":670,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338896.373372473,"t_first_token_unix":1779769761.3873158,"t_done":338897.577626717,"t_done_unix":1779769762.59157},{"request_id":"1302063:2:1303816:279","session_id":"1302063","input_length":30248,"t_proxy_recv":338896.320927838,"t_decision_unix":1779769761.3348691,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":26112,"affinity_cache_ratio":0.8632636868553293,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4136 < 8000)","cache_hit":26112,"estimated_new_tokens":4136,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":30248,"score_linear":64654.0,"score_lmetric":60496},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":30248,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":30248,"score_linear":80500.0,"score_lmetric":170296},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":141330,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":30248,"score_linear":141330.0,"score_lmetric":90744},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":30248,"score_linear":43607.0,"score_lmetric":30248},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":758,"cache_hit":26112,"new_prefill":4136,"score_linear":-11168.0,"score_lmetric":19080},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":30248,"score_linear":241141.0,"score_lmetric":90744},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":15081,"ongoing_decode_tokens":0,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":670,"cache_hit":0,"new_prefill":30248,"score_linear":15081.0,"score_lmetric":45329}],"chosen_score_linear":-11168.0,"chosen_score_lmetric":19080,"t_first_token":338897.197967949,"t_first_token_unix":1779769762.2119112,"t_done":338897.578423287,"t_done_unix":1779769762.592367},{"request_id":"1268861:3:1298057:245","session_id":"1268861","input_length":21630,"t_proxy_recv":338876.10802351,"t_decision_unix":1779769741.1219647,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":12800,"affinity_cache_ratio":0.5917706888580675,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=2 decode_tok=0)","cache_hit":12800,"estimated_new_tokens":8830,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":29149,"ongoing_decode_tokens":29149,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":554,"cache_hit":0,"new_prefill":21630,"score_linear":29149.0,"score_lmetric":21630},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":21630,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":21630,"score_linear":86277.0,"score_lmetric":130822},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":119700,"ongoing_decode_tokens":0,"pending_prefill_tokens":79252,"num_requests":2,"active_p_offloads":0,"cached_blocks":672,"cache_hit":12800,"new_prefill":8830,"score_linear":106900.0,"score_lmetric":176164},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13992,"ongoing_decode_tokens":0,"pending_prefill_tokens":12456,"num_requests":1,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":21630,"score_linear":13992.0,"score_lmetric":34086},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":679,"cache_hit":0,"new_prefill":21630,"score_linear":44920.0,"score_lmetric":21630},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":21630,"score_linear":225261.0,"score_lmetric":129951},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":616,"cache_hit":0,"new_prefill":21630,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":106900.0,"chosen_score_lmetric":176164,"t_first_token":338893.263473344,"t_first_token_unix":1779769758.2774172,"t_done":338897.875412537,"t_done_unix":1779769762.8893557},{"request_id":"1304239:1:1304239:280","session_id":"1304239","input_length":1300,"t_proxy_recv":338897.84672954,"t_decision_unix":1779769762.8606708,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1300 < 8000)","cache_hit":0,"estimated_new_tokens":1300,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":1300,"score_linear":64654.0,"score_lmetric":2600},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":1300,"score_linear":80500.0,"score_lmetric":112400},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82308,"ongoing_decode_tokens":82308,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":827,"cache_hit":0,"new_prefill":1300,"score_linear":82308.0,"score_lmetric":2600},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":1300,"score_linear":43607.0,"score_lmetric":1300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":1300,"score_linear":241141.0,"score_lmetric":3900},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":699,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338897.945754847,"t_first_token_unix":1779769762.959699,"t_done":338898.089807523,"t_done_unix":1779769763.1037507},{"request_id":"1279412:8:1300001:255","session_id":"1279412","input_length":60678,"t_proxy_recv":338896.416063671,"t_decision_unix":1779769761.4300056,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":60416,"affinity_cache_ratio":0.9956821253172484,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (262 < 8000)","cache_hit":60416,"estimated_new_tokens":262,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":60678,"score_linear":64654.0,"score_lmetric":121356},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":60678,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":60678,"score_linear":80500.0,"score_lmetric":231156},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21630,"ongoing_decode_tokens":21630,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":827,"cache_hit":60416,"new_prefill":262,"score_linear":-38786.0,"score_lmetric":262},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":60678,"score_linear":43607.0,"score_lmetric":60678},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":45192,"ongoing_decode_tokens":14944,"pending_prefill_tokens":4136,"num_requests":2,"active_p_offloads":0,"cached_blocks":758,"cache_hit":0,"new_prefill":60678,"score_linear":45192.0,"score_lmetric":129628},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":60678,"score_linear":241141.0,"score_lmetric":182034},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":15081,"ongoing_decode_tokens":0,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":670,"cache_hit":12800,"new_prefill":47878,"score_linear":2281.0,"score_lmetric":62959}],"chosen_score_linear":-38786.0,"chosen_score_lmetric":262,"t_first_token":338896.629312655,"t_first_token_unix":1779769761.6432567,"t_done":338898.614503302,"t_done_unix":1779769763.6284473},{"request_id":"1304239:2:1304866:281","session_id":"1304239","input_length":7798,"t_proxy_recv":338899.9958599,"t_decision_unix":1779769765.0098011,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":1024,"affinity_cache_ratio":0.13131572197999486,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6774 < 8000)","cache_hit":1024,"estimated_new_tokens":6774,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":7798,"score_linear":64654.0,"score_lmetric":15596},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":7798,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":7798,"score_linear":80500.0,"score_lmetric":125396},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":844,"cache_hit":0,"new_prefill":7798,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":7798,"score_linear":43607.0,"score_lmetric":7798},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":797,"cache_hit":1024,"new_prefill":6774,"score_linear":-1024.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":7798,"score_linear":241141.0,"score_lmetric":23394},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":699,"cache_hit":0,"new_prefill":7798,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":338900.477525793,"t_first_token_unix":1779769765.4914691,"t_done":338900.62808826,"t_done_unix":1779769765.6420317},{"request_id":"1304943:1:1304943:283","session_id":"1304943","input_length":2772,"t_proxy_recv":338900.247761675,"t_decision_unix":1779769765.261703,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2772 < 8000)","cache_hit":0,"estimated_new_tokens":2772,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":2772,"score_linear":64654.0,"score_lmetric":5544},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":2772,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":2772,"score_linear":80500.0,"score_lmetric":115344},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":844,"cache_hit":0,"new_prefill":2772,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":2772,"score_linear":43607.0,"score_lmetric":2772},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":7798,"ongoing_decode_tokens":0,"pending_prefill_tokens":6774,"num_requests":1,"active_p_offloads":0,"cached_blocks":797,"cache_hit":0,"new_prefill":2772,"score_linear":7798.0,"score_lmetric":9546},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":2772,"score_linear":241141.0,"score_lmetric":8316},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":699,"cache_hit":0,"new_prefill":2772,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338900.427381201,"t_first_token_unix":1779769765.4413247,"t_done":338900.728310471,"t_done_unix":1779769765.7422538},{"request_id":"1266668:7:1287468:197","session_id":"1266668","input_length":87741,"t_proxy_recv":338889.744932281,"t_decision_unix":1779769754.758874,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":80384,"affinity_cache_ratio":0.916150944256391,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7357 < 8000)","cache_hit":80384,"estimated_new_tokens":7357,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":595,"cache_hit":7680,"new_prefill":80061,"score_linear":27026.0,"score_lmetric":111183},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":87741,"score_linear":28972.0,"score_lmetric":116713},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24384,"ongoing_decode_tokens":24384,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":775,"cache_hit":0,"new_prefill":87741,"score_linear":24384.0,"score_lmetric":87741},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":87741,"score_linear":141330.0,"score_lmetric":348948},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":87741,"score_linear":43607.0,"score_lmetric":117524},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":7680,"new_prefill":80061,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":153400,"ongoing_decode_tokens":153400,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":622,"cache_hit":80384,"new_prefill":7357,"score_linear":73016.0,"score_lmetric":14714},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":87741,"score_linear":43181.0,"score_lmetric":87741}],"chosen_score_linear":73016.0,"chosen_score_lmetric":14714,"t_first_token":338893.629285218,"t_first_token_unix":1779769758.6432288,"t_done":338901.387109465,"t_done_unix":1779769766.4010532},{"request_id":"1253743:3:1302294:268","session_id":"1253743","input_length":49855,"t_proxy_recv":338891.035180813,"t_decision_unix":1779769756.0491223,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":2,"affinity_cache_hit":49664,"affinity_cache_ratio":0.996168889780363,"affinity_num_requests":3,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":49855,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":49855,"score_linear":34706.0,"score_lmetric":80977},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":4608,"new_prefill":45247,"score_linear":24364.0,"score_lmetric":74219},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":49855,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":49664,"new_prefill":191,"score_linear":91666.0,"score_lmetric":86298},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":49855,"score_linear":43607.0,"score_lmetric":79638},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":49855,"score_linear":26441.0,"score_lmetric":76296},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":0,"new_prefill":49855,"score_linear":241141.0,"score_lmetric":171636},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":49855,"score_linear":43181.0,"score_lmetric":49855}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338901.335813922,"t_first_token_unix":1779769766.3497577,"t_done":338902.118460124,"t_done_unix":1779769767.1324036},{"request_id":"1294611:6:1302460:270","session_id":"1294611","input_length":30645,"t_proxy_recv":338891.670307189,"t_decision_unix":1779769756.684249,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":25600,"affinity_cache_ratio":0.8353728177516724,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5045 < 8000)","cache_hit":25600,"estimated_new_tokens":5045,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":30645,"score_linear":34706.0,"score_lmetric":61767},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":30645,"score_linear":28972.0,"score_lmetric":59617},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":49855,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":25600,"new_prefill":5045,"score_linear":24255.0,"score_lmetric":54900},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":30645,"score_linear":141330.0,"score_lmetric":177660},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":30645,"score_linear":43607.0,"score_lmetric":30645},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":707,"cache_hit":12800,"new_prefill":17845,"score_linear":13641.0,"score_lmetric":44286},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":12800,"new_prefill":17845,"score_linear":228341.0,"score_lmetric":75606},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44286,"ongoing_decode_tokens":43181,"pending_prefill_tokens":1105,"num_requests":2,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":30645,"score_linear":44286.0,"score_lmetric":63500}],"chosen_score_linear":24255.0,"chosen_score_lmetric":54900,"t_first_token":338901.340531632,"t_first_token_unix":1779769766.3544748,"t_done":338903.708814072,"t_done_unix":1779769768.7227576},{"request_id":"1279412:9:1305633:286","session_id":"1279412","input_length":61291,"t_proxy_recv":338902.925283612,"t_decision_unix":1779769767.9392252,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":60416,"affinity_cache_ratio":0.985723841999641,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (875 < 8000)","cache_hit":60416,"estimated_new_tokens":875,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":61291,"score_linear":64654.0,"score_lmetric":122582},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":61291,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":30645,"ongoing_decode_tokens":30645,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":875,"cache_hit":0,"new_prefill":61291,"score_linear":30645.0,"score_lmetric":61291},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":844,"cache_hit":60416,"new_prefill":875,"score_linear":-60416.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":61291,"score_linear":43607.0,"score_lmetric":61291},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":61291,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":61291,"score_linear":249505.0,"score_lmetric":207996},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":12800,"new_prefill":48491,"score_linear":63026.0,"score_lmetric":124317}],"chosen_score_linear":-60416.0,"chosen_score_lmetric":0,"t_first_token":338903.248624322,"t_first_token_unix":1779769768.2625678,"t_done":338904.121141478,"t_done_unix":1779769769.135085},{"request_id":"1303651:2:1305707:287","session_id":"1303651","input_length":12795,"t_proxy_recv":338903.190274646,"t_decision_unix":1779769768.2042146,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":3072,"affinity_cache_ratio":0.24009378663540445,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":3072,"estimated_new_tokens":9723,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":12795,"score_linear":64654.0,"score_lmetric":25590},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":814,"cache_hit":3072,"new_prefill":9723,"score_linear":-3072.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":30645,"ongoing_decode_tokens":30645,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":875,"cache_hit":0,"new_prefill":12795,"score_linear":30645.0,"score_lmetric":12795},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61291,"ongoing_decode_tokens":0,"pending_prefill_tokens":875,"num_requests":1,"active_p_offloads":0,"cached_blocks":844,"cache_hit":0,"new_prefill":12795,"score_linear":61291.0,"score_lmetric":13670},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":12795,"score_linear":43607.0,"score_lmetric":12795},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":12795,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":12795,"score_linear":249505.0,"score_lmetric":62508},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":12795,"score_linear":75826.0,"score_lmetric":88621}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":338904.131141147,"t_first_token_unix":1779769769.1450846,"t_done":338904.252093604,"t_done_unix":1779769769.266037},{"request_id":"1294611:6:1305080:284","session_id":"1294611","input_length":30995,"t_proxy_recv":338903.714513792,"t_decision_unix":1779769768.7284553,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9911276012260042,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (275 < 8000)","cache_hit":30720,"estimated_new_tokens":275,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83569,"ongoing_decode_tokens":83569,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":30995,"score_linear":83569.0,"score_lmetric":92985},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12795,"ongoing_decode_tokens":0,"pending_prefill_tokens":9723,"num_requests":1,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":30995,"score_linear":12795.0,"score_lmetric":40718},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":885,"cache_hit":30720,"new_prefill":275,"score_linear":-30720.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61291,"ongoing_decode_tokens":61291,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":844,"cache_hit":0,"new_prefill":30995,"score_linear":61291.0,"score_lmetric":30995},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":30995,"score_linear":43607.0,"score_lmetric":30995},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":810,"cache_hit":12800,"new_prefill":18195,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":637,"cache_hit":12800,"new_prefill":18195,"score_linear":236705.0,"score_lmetric":78708},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":30995,"score_linear":75826.0,"score_lmetric":106821}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":338903.780658523,"t_first_token_unix":1779769768.794602,"t_done":338905.10251758,"t_done_unix":1779769770.116461},{"request_id":"1303462:2:1306126:290","session_id":"1303462","input_length":18631,"t_proxy_recv":338904.418831152,"t_decision_unix":1779769769.4327726,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":14848,"affinity_cache_ratio":0.7969513176963126,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3783 < 8000)","cache_hit":14848,"estimated_new_tokens":3783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83569,"ongoing_decode_tokens":83569,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":18631,"score_linear":83569.0,"score_lmetric":55893},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":18631,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":30995,"ongoing_decode_tokens":30995,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":885,"cache_hit":0,"new_prefill":18631,"score_linear":30995.0,"score_lmetric":18631},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":18631,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":18631,"score_linear":43607.0,"score_lmetric":18631},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":14027,"ongoing_decode_tokens":0,"pending_prefill_tokens":14027,"num_requests":1,"active_p_offloads":0,"cached_blocks":810,"cache_hit":14848,"new_prefill":3783,"score_linear":-821.0,"score_lmetric":17810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":18631,"score_linear":249505.0,"score_lmetric":80016},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":18631,"score_linear":75826.0,"score_lmetric":94457}],"chosen_score_linear":-821.0,"chosen_score_lmetric":17810,"t_first_token":338905.538415563,"t_first_token_unix":1779769770.5523593,"t_done":338905.585653848,"t_done_unix":1779769770.5995972},{"request_id":"1306347:1:1306347:291","session_id":"1306347","input_length":897,"t_proxy_recv":338905.205682514,"t_decision_unix":1779769770.219624,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (897 < 8000)","cache_hit":0,"estimated_new_tokens":897,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83569,"ongoing_decode_tokens":83569,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":897,"score_linear":83569.0,"score_lmetric":2691},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":897,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":885,"cache_hit":0,"new_prefill":897,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":897,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":897,"score_linear":43607.0,"score_lmetric":897},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32658,"ongoing_decode_tokens":14027,"pending_prefill_tokens":3783,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":897,"score_linear":32658.0,"score_lmetric":9360},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":897,"score_linear":249505.0,"score_lmetric":26814},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":897,"score_linear":75826.0,"score_lmetric":76723}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338905.319725258,"t_first_token_unix":1779769770.333668,"t_done":338905.68776669,"t_done_unix":1779769770.70171},{"request_id":"1299507:2:1305770:288","session_id":"1299507","input_length":18915,"t_proxy_recv":338903.384629569,"t_decision_unix":1779769768.3985713,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9744647105471848,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (483 < 8000)","cache_hit":18432,"estimated_new_tokens":483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":18432,"new_prefill":483,"score_linear":46222.0,"score_lmetric":966},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12795,"ongoing_decode_tokens":0,"pending_prefill_tokens":9723,"num_requests":1,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":18915,"score_linear":12795.0,"score_lmetric":28638},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":30645,"ongoing_decode_tokens":30645,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":875,"cache_hit":0,"new_prefill":18915,"score_linear":30645.0,"score_lmetric":18915},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61291,"ongoing_decode_tokens":61291,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":844,"cache_hit":0,"new_prefill":18915,"score_linear":61291.0,"score_lmetric":18915},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":18915,"score_linear":43607.0,"score_lmetric":18915},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":18915,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":637,"cache_hit":11776,"new_prefill":7139,"score_linear":237729.0,"score_lmetric":45540},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":18915,"score_linear":75826.0,"score_lmetric":94741}],"chosen_score_linear":46222.0,"chosen_score_lmetric":966,"t_first_token":338903.441735398,"t_first_token_unix":1779769768.4556787,"t_done":338905.877078636,"t_done_unix":1779769770.8910215},{"request_id":"1269373:6:1285668:189","session_id":"1269373","input_length":38788,"t_proxy_recv":338887.685031408,"t_decision_unix":1779769752.6989722,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":30720,"affinity_cache_ratio":0.7919975250077343,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=7680 chosen=30720)","cache_hit":30720,"estimated_new_tokens":8068,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":558,"cache_hit":7680,"new_prefill":31108,"score_linear":11213.0,"score_lmetric":31108},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":25948,"ongoing_decode_tokens":25948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":38788,"score_linear":25948.0,"score_lmetric":38788},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":774,"cache_hit":0,"new_prefill":38788,"score_linear":23863.0,"score_lmetric":38788},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":38788,"score_linear":141330.0,"score_lmetric":202089},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":38788,"score_linear":43607.0,"score_lmetric":68571},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":7680,"new_prefill":31108,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":194667,"ongoing_decode_tokens":194667,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":579,"cache_hit":30720,"new_prefill":8068,"score_linear":163947.0,"score_lmetric":16136},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":38788,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":163947.0,"chosen_score_lmetric":16136,"t_first_token":338889.608275497,"t_first_token_unix":1779769754.6222188,"t_done":338906.32162657,"t_done_unix":1779769771.33557},{"request_id":"1305906:1:1305906:289","session_id":"1305906","input_length":14027,"t_proxy_recv":338903.802251269,"t_decision_unix":1779769768.8161929,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":14027,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83569,"ongoing_decode_tokens":83569,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":14027,"score_linear":83569.0,"score_lmetric":42081},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12795,"ongoing_decode_tokens":0,"pending_prefill_tokens":9723,"num_requests":1,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":14027,"score_linear":12795.0,"score_lmetric":23750},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":30995,"ongoing_decode_tokens":30995,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":885,"cache_hit":0,"new_prefill":14027,"score_linear":30995.0,"score_lmetric":14027},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61291,"ongoing_decode_tokens":61291,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":844,"cache_hit":0,"new_prefill":14027,"score_linear":61291.0,"score_lmetric":14027},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":14027,"score_linear":43607.0,"score_lmetric":14027},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":14027,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":14027,"score_linear":249505.0,"score_lmetric":66204},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":14027,"score_linear":75826.0,"score_lmetric":89853}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338905.01402833,"t_first_token_unix":1779769770.0279717,"t_done":338906.586223612,"t_done_unix":1779769771.6001668},{"request_id":"1275274:1:1275274:136","session_id":"1275274","input_length":114612,"t_proxy_recv":338797.963414144,"t_decision_unix":1779769662.9773555,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":114612,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":114612,"score_linear":135235.0,"score_lmetric":455662},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":120167,"ongoing_decode_tokens":0,"pending_prefill_tokens":106343,"num_requests":2,"active_p_offloads":0,"cached_blocks":285,"cache_hit":0,"new_prefill":114612,"score_linear":120167.0,"score_lmetric":441910},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":114612,"score_linear":150574.0,"score_lmetric":464836},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121069,"ongoing_decode_tokens":27639,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":259,"cache_hit":0,"new_prefill":114612,"score_linear":121069.0,"score_lmetric":416084},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":47071,"ongoing_decode_tokens":25108,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":316,"cache_hit":0,"new_prefill":114612,"score_linear":47071.0,"score_lmetric":229224},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":498,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":337,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338840.499720743,"t_first_token_unix":1779769705.513664,"t_done":338908.078714592,"t_done_unix":1779769773.0926585},{"request_id":"1304239:3:1307007:294","session_id":"1304239","input_length":13840,"t_proxy_recv":338907.56226075,"t_decision_unix":1779769772.576202,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7680,"affinity_cache_ratio":0.5549132947976878,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6160 < 8000)","cache_hit":7680,"estimated_new_tokens":6160,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":13840,"score_linear":64654.0,"score_lmetric":27680},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":13840,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43378,"ongoing_decode_tokens":0,"pending_prefill_tokens":12658,"num_requests":1,"active_p_offloads":0,"cached_blocks":886,"cache_hit":0,"new_prefill":13840,"score_linear":43378.0,"score_lmetric":26498},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":31058,"ongoing_decode_tokens":31058,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":13840,"score_linear":31058.0,"score_lmetric":13840},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":13840,"score_linear":43607.0,"score_lmetric":13840},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":844,"cache_hit":7680,"new_prefill":6160,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":253462,"ongoing_decode_tokens":253462,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":13840,"score_linear":253462.0,"score_lmetric":41520},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":13840,"score_linear":75826.0,"score_lmetric":89666}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":338908.229233611,"t_first_token_unix":1779769773.2431777,"t_done":338908.393361043,"t_done_unix":1779769773.407305},{"request_id":"1266668:8:1292408:218","session_id":"1266668","input_length":96105,"t_proxy_recv":338901.407546582,"t_decision_unix":1779769766.4214883,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":88064,"affinity_cache_ratio":0.916331096196868,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=7680 chosen=88064)","cache_hit":88064,"estimated_new_tokens":8041,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":7680,"new_prefill":88425,"score_linear":56974.0,"score_lmetric":176850},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":96105,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":80500,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":96105,"score_linear":80500.0,"score_lmetric":192210},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":844,"cache_hit":0,"new_prefill":96105,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":96105,"score_linear":43607.0,"score_lmetric":96105},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":810,"cache_hit":7680,"new_prefill":88425,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":153400,"ongoing_decode_tokens":153400,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":637,"cache_hit":88064,"new_prefill":8041,"score_linear":65336.0,"score_lmetric":16082},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":96105,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":65336.0,"chosen_score_lmetric":16082,"t_first_token":338905.99726113,"t_first_token_unix":1779769771.0112047,"t_done":338908.60962631,"t_done_unix":1779769773.6235702},{"request_id":"1266668:9:1295576:231","session_id":"1266668","input_length":96717,"t_proxy_recv":338908.626583572,"t_decision_unix":1779769773.640525,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9899397210418024,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (973 < 8000)","cache_hit":95744,"estimated_new_tokens":973,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":7680,"new_prefill":89037,"score_linear":56974.0,"score_lmetric":178074},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":96717,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":93353,"ongoing_decode_tokens":0,"pending_prefill_tokens":12969,"num_requests":2,"active_p_offloads":0,"cached_blocks":886,"cache_hit":0,"new_prefill":96717,"score_linear":93353.0,"score_lmetric":219372},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":31058,"ongoing_decode_tokens":31058,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":96717,"score_linear":31058.0,"score_lmetric":96717},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":96717,"score_linear":43607.0,"score_lmetric":96717},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":7680,"new_prefill":89037,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":42745,"ongoing_decode_tokens":42745,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":893,"cache_hit":95744,"new_prefill":973,"score_linear":-52999.0,"score_lmetric":973},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":96717,"score_linear":75826.0,"score_lmetric":172543}],"chosen_score_linear":-52999.0,"chosen_score_lmetric":973,"t_first_token":338909.144124974,"t_first_token_unix":1779769774.1580684,"t_done":338909.507784328,"t_done_unix":1779769774.5217283},{"request_id":"1277428:2:1306373:292","session_id":"1277428","input_length":31058,"t_proxy_recv":338905.315828841,"t_decision_unix":1779769770.32977,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":27136,"affinity_cache_ratio":0.8737201365187713,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3922 < 8000)","cache_hit":27136,"estimated_new_tokens":3922,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83569,"ongoing_decode_tokens":83569,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":31058,"score_linear":83569.0,"score_lmetric":93174},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":4608,"new_prefill":26450,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":897,"ongoing_decode_tokens":0,"pending_prefill_tokens":897,"num_requests":1,"active_p_offloads":0,"cached_blocks":885,"cache_hit":17408,"new_prefill":13650,"score_linear":-16511.0,"score_lmetric":14547},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":845,"cache_hit":27136,"new_prefill":3922,"score_linear":-27136.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":31058,"score_linear":43607.0,"score_lmetric":31058},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32658,"ongoing_decode_tokens":14027,"pending_prefill_tokens":3783,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":31058,"score_linear":32658.0,"score_lmetric":69682},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":31058,"score_linear":249505.0,"score_lmetric":117297},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":31058,"score_linear":75826.0,"score_lmetric":106884}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":338906.197148728,"t_first_token_unix":1779769771.211092,"t_done":338909.699349253,"t_done_unix":1779769774.7132928},{"request_id":"1253743:4:1307058:295","session_id":"1253743","input_length":49975,"t_proxy_recv":338907.759756121,"t_decision_unix":1779769772.7736974,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9937768884442221,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (311 < 8000)","cache_hit":49664,"estimated_new_tokens":311,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":49975,"score_linear":64654.0,"score_lmetric":99950},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":4608,"new_prefill":45367,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43378,"ongoing_decode_tokens":0,"pending_prefill_tokens":12658,"num_requests":1,"active_p_offloads":0,"cached_blocks":886,"cache_hit":49664,"new_prefill":311,"score_linear":-6286.0,"score_lmetric":12969},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":31058,"ongoing_decode_tokens":31058,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":49664,"new_prefill":311,"score_linear":-18606.0,"score_lmetric":311},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":49975,"score_linear":43607.0,"score_lmetric":49975},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13840,"ongoing_decode_tokens":0,"pending_prefill_tokens":6160,"num_requests":1,"active_p_offloads":0,"cached_blocks":844,"cache_hit":0,"new_prefill":49975,"score_linear":13840.0,"score_lmetric":56135},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":253462,"ongoing_decode_tokens":253462,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":49975,"score_linear":253462.0,"score_lmetric":149925},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":49975,"score_linear":75826.0,"score_lmetric":125801}],"chosen_score_linear":-6286.0,"chosen_score_lmetric":12969,"t_first_token":338910.258956811,"t_first_token_unix":1779769775.2729006,"t_done":338910.884241483,"t_done_unix":1779769775.898185},{"request_id":"1272313:11:1302571:271","session_id":"1272313","input_length":29948,"t_proxy_recv":338891.974902239,"t_decision_unix":1779769756.9888437,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29696,"affinity_cache_ratio":0.991585414718846,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (252 < 8000)","cache_hit":29696,"estimated_new_tokens":252,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":595,"cache_hit":29696,"new_prefill":252,"score_linear":5010.0,"score_lmetric":31374},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":29948,"score_linear":28972.0,"score_lmetric":58920},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":0,"pending_prefill_tokens":54900,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":29948,"score_linear":80500.0,"score_lmetric":169696},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":29948,"score_linear":141330.0,"score_lmetric":175569},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":29948,"score_linear":43607.0,"score_lmetric":29948},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":707,"cache_hit":7680,"new_prefill":22268,"score_linear":18761.0,"score_lmetric":48709},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":622,"cache_hit":10752,"new_prefill":19196,"score_linear":230389.0,"score_lmetric":79659},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44286,"ongoing_decode_tokens":44286,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":665,"cache_hit":0,"new_prefill":29948,"score_linear":44286.0,"score_lmetric":59896}],"chosen_score_linear":5010.0,"chosen_score_lmetric":31374,"t_first_token":338894.446013786,"t_first_token_unix":1779769759.4599574,"t_done":338911.421787281,"t_done_unix":1779769776.4357307},{"request_id":"1294611:7:1306822:293","session_id":"1294611","input_length":43378,"t_proxy_recv":338907.028102682,"t_decision_unix":1779769772.0420444,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":30720,"affinity_cache_ratio":0.7081930932730878,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":30720,"estimated_new_tokens":12658,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":43378,"score_linear":64654.0,"score_lmetric":86756},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":43378,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":886,"cache_hit":30720,"new_prefill":12658,"score_linear":-30720.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":31058,"ongoing_decode_tokens":31058,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":43378,"score_linear":31058.0,"score_lmetric":43378},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":43378,"score_linear":43607.0,"score_lmetric":43378},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":844,"cache_hit":12800,"new_prefill":30578,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":253462,"ongoing_decode_tokens":210717,"pending_prefill_tokens":3833,"num_requests":3,"active_p_offloads":0,"cached_blocks":653,"cache_hit":12800,"new_prefill":30578,"score_linear":240662.0,"score_lmetric":103233},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":43378,"score_linear":75826.0,"score_lmetric":119204}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":338910.19284447,"t_first_token_unix":1779769775.2067876,"t_done":338911.556912999,"t_done_unix":1779769776.5708563},{"request_id":"1279412:10:1308091:298","session_id":"1279412","input_length":61505,"t_proxy_recv":338911.397944663,"t_decision_unix":1779769776.4118857,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9906186488903341,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (577 < 8000)","cache_hit":60928,"estimated_new_tokens":577,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":61505,"score_linear":64654.0,"score_lmetric":123010},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":61505,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43378,"ongoing_decode_tokens":43378,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":886,"cache_hit":0,"new_prefill":61505,"score_linear":43378.0,"score_lmetric":61505},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":853,"cache_hit":60928,"new_prefill":577,"score_linear":-60928.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":61505,"score_linear":43607.0,"score_lmetric":61505},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":61505,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":139890,"ongoing_decode_tokens":139890,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":894,"cache_hit":0,"new_prefill":61505,"score_linear":139890.0,"score_lmetric":123010},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":12800,"new_prefill":48705,"score_linear":86157.0,"score_lmetric":265628}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":338911.568852972,"t_first_token_unix":1779769776.582796,"t_done":338912.343547164,"t_done_unix":1779769777.3574905},{"request_id":"1266668:10:1297641:241","session_id":"1266668","input_length":97145,"t_proxy_recv":338909.525998642,"t_decision_unix":1779769774.53994,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":96256,"affinity_cache_ratio":0.9908487312779866,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (889 < 8000)","cache_hit":96256,"estimated_new_tokens":889,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":7680,"new_prefill":89465,"score_linear":56974.0,"score_lmetric":178930},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":97145,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":93353,"ongoing_decode_tokens":0,"pending_prefill_tokens":12969,"num_requests":2,"active_p_offloads":0,"cached_blocks":886,"cache_hit":0,"new_prefill":97145,"score_linear":93353.0,"score_lmetric":220228},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":31058,"ongoing_decode_tokens":31058,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":97145,"score_linear":31058.0,"score_lmetric":97145},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":97145,"score_linear":43607.0,"score_lmetric":97145},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":7680,"new_prefill":89465,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":42745,"ongoing_decode_tokens":42745,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":894,"cache_hit":96256,"new_prefill":889,"score_linear":-53511.0,"score_lmetric":889},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":97145,"score_linear":75826.0,"score_lmetric":172971}],"chosen_score_linear":-53511.0,"chosen_score_lmetric":889,"t_first_token":338910.013648566,"t_first_token_unix":1779769775.027592,"t_done":338913.449047525,"t_done_unix":1779769778.4629908},{"request_id":"1302449:3:1308121:299","session_id":"1302449","input_length":8729,"t_proxy_recv":338911.494123675,"t_decision_unix":1779769776.508065,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":3584,"affinity_cache_ratio":0.41058540497193263,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5145 < 8000)","cache_hit":3584,"estimated_new_tokens":5145,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":8729,"score_linear":34706.0,"score_lmetric":8729},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":3584,"new_prefill":5145,"score_linear":-3584.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43378,"ongoing_decode_tokens":43378,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":886,"cache_hit":0,"new_prefill":8729,"score_linear":43378.0,"score_lmetric":8729},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61505,"ongoing_decode_tokens":0,"pending_prefill_tokens":577,"num_requests":1,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":8729,"score_linear":61505.0,"score_lmetric":9306},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":8729,"score_linear":43607.0,"score_lmetric":8729},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":8729,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":139890,"ongoing_decode_tokens":139890,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":894,"cache_hit":0,"new_prefill":8729,"score_linear":139890.0,"score_lmetric":17458},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":1024,"new_prefill":7705,"score_linear":97933.0,"score_lmetric":183628}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":338911.922982913,"t_first_token_unix":1779769776.936926,"t_done":338913.530805355,"t_done_unix":1779769778.5447483},{"request_id":"1303651:3:1308202:300","session_id":"1303651","input_length":22896,"t_proxy_recv":338911.748909541,"t_decision_unix":1779769776.762851,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":12800,"affinity_cache_ratio":0.5590496156533893,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":12800,"estimated_new_tokens":10096,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":22896,"score_linear":34706.0,"score_lmetric":22896},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":8729,"ongoing_decode_tokens":0,"pending_prefill_tokens":5145,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":12800,"new_prefill":10096,"score_linear":-4071.0,"score_lmetric":15241},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":22896,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":61505,"ongoing_decode_tokens":61505,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":22896,"score_linear":61505.0,"score_lmetric":22896},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":22896,"score_linear":43607.0,"score_lmetric":22896},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":22896,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":139890,"ongoing_decode_tokens":139890,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":894,"cache_hit":0,"new_prefill":22896,"score_linear":139890.0,"score_lmetric":45792},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":22896,"score_linear":98957.0,"score_lmetric":214010}],"chosen_score_linear":-4071.0,"chosen_score_lmetric":15241,"t_first_token":338913.444714055,"t_first_token_unix":1779769778.4586577,"t_done":338913.536616866,"t_done_unix":1779769778.55056},{"request_id":"1309040:1:1309040:302","session_id":"1309040","input_length":4679,"t_proxy_recv":338914.864292786,"t_decision_unix":1779769779.8782344,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4679 < 8000)","cache_hit":0,"estimated_new_tokens":4679,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":4679,"score_linear":34706.0,"score_lmetric":4679},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43710,"ongoing_decode_tokens":43710,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":4679,"score_linear":43710.0,"score_lmetric":4679},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":854,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":4679,"score_linear":43607.0,"score_lmetric":4679},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":140556,"ongoing_decode_tokens":140556,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":4679,"score_linear":140556.0,"score_lmetric":9358},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":4679,"score_linear":98957.0,"score_lmetric":177576}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338915.161252984,"t_first_token_unix":1779769780.1751966,"t_done":338915.660201086,"t_done_unix":1779769780.6741445},{"request_id":"1269373:7:1290314:207","session_id":"1269373","input_length":42745,"t_proxy_recv":338906.331648694,"t_decision_unix":1779769771.3455906,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":38912,"affinity_cache_ratio":0.9103286934144345,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3833 < 8000)","cache_hit":38912,"estimated_new_tokens":3833,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":7680,"new_prefill":35065,"score_linear":56974.0,"score_lmetric":70130},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":42745,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":886,"cache_hit":0,"new_prefill":42745,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":31058,"ongoing_decode_tokens":31058,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":42745,"score_linear":31058.0,"score_lmetric":42745},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":42745,"score_linear":43607.0,"score_lmetric":42745},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":14027,"ongoing_decode_tokens":14027,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":817,"cache_hit":7680,"new_prefill":35065,"score_linear":6347.0,"score_lmetric":35065},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":210717,"ongoing_decode_tokens":210717,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":653,"cache_hit":38912,"new_prefill":3833,"score_linear":171805.0,"score_lmetric":7666},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":42745,"score_linear":75826.0,"score_lmetric":118571}],"chosen_score_linear":171805.0,"chosen_score_lmetric":7666,"t_first_token":338907.461362323,"t_first_token_unix":1779769772.4753058,"t_done":338915.784519467,"t_done_unix":1779769780.7984626},{"request_id":"1294611:8:1308970:301","session_id":"1294611","input_length":43710,"t_proxy_recv":338914.535203784,"t_decision_unix":1779769779.549145,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9956531686113017,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (190 < 8000)","cache_hit":43520,"estimated_new_tokens":190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":43710,"score_linear":34706.0,"score_lmetric":43710},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":43710,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":43520,"new_prefill":190,"score_linear":-43520.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":854,"cache_hit":0,"new_prefill":43710,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":43710,"score_linear":43607.0,"score_lmetric":43710},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":12800,"new_prefill":30910,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":140556,"ongoing_decode_tokens":140556,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":896,"cache_hit":12800,"new_prefill":30910,"score_linear":127756.0,"score_lmetric":61820},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":43710,"score_linear":98957.0,"score_lmetric":255638}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":338914.696379718,"t_first_token_unix":1779769779.7103229,"t_done":338915.975045142,"t_done_unix":1779769780.9889889},{"request_id":"1301929:1:1301929:265","session_id":"1301929","input_length":34706,"t_proxy_recv":338889.551166505,"t_decision_unix":1779769754.565108,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":3584,"estimated_new_tokens":31122,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":595,"cache_hit":3584,"new_prefill":31122,"score_linear":-3584.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28972,"ongoing_decode_tokens":0,"pending_prefill_tokens":28972,"num_requests":1,"active_p_offloads":0,"cached_blocks":714,"cache_hit":0,"new_prefill":34706,"score_linear":28972.0,"score_lmetric":63678},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":774,"cache_hit":0,"new_prefill":34706,"score_linear":23863.0,"score_lmetric":34706},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":59507,"pending_prefill_tokens":28575,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":34706,"score_linear":141330.0,"score_lmetric":189843},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":34706,"score_linear":43607.0,"score_lmetric":64489},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":34706,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":233455,"ongoing_decode_tokens":194667,"pending_prefill_tokens":8068,"num_requests":3,"active_p_offloads":0,"cached_blocks":579,"cache_hit":0,"new_prefill":34706,"score_linear":233455.0,"score_lmetric":128322},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":34706,"score_linear":43181.0,"score_lmetric":34706}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":338894.368258397,"t_first_token_unix":1779769759.3822021,"t_done":338917.347589468,"t_done_unix":1779769782.361537},{"request_id":"1266668:11:1300918:260","session_id":"1266668","input_length":97811,"t_proxy_recv":338913.469516364,"t_decision_unix":1779769778.483458,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":97280,"affinity_cache_ratio":0.9945711627526557,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (531 < 8000)","cache_hit":97280,"estimated_new_tokens":531,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":7680,"new_prefill":90131,"score_linear":27026.0,"score_lmetric":90131},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":31625,"ongoing_decode_tokens":31625,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":97811,"score_linear":31625.0,"score_lmetric":195622},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":97811,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":854,"cache_hit":0,"new_prefill":97811,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":97811,"score_linear":43607.0,"score_lmetric":97811},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":7680,"new_prefill":90131,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":42745,"ongoing_decode_tokens":42745,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":896,"cache_hit":97280,"new_prefill":531,"score_linear":-54535.0,"score_lmetric":531},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":97811,"score_linear":98957.0,"score_lmetric":363840}],"chosen_score_linear":-54535.0,"chosen_score_lmetric":531,"t_first_token":338913.966095979,"t_first_token_unix":1779769778.9800394,"t_done":338920.327771878,"t_done_unix":1779769785.3417156},{"request_id":"1309252:1:1309252:304","session_id":"1309252","input_length":24545,"t_proxy_recv":338915.696357457,"t_decision_unix":1779769780.7102993,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":24545,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":0,"new_prefill":24545,"score_linear":34706.0,"score_lmetric":24545},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43710,"ongoing_decode_tokens":43710,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":24545,"score_linear":43710.0,"score_lmetric":24545},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":854,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":24545,"score_linear":43607.0,"score_lmetric":24545},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":140556,"ongoing_decode_tokens":140556,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":24545,"score_linear":140556.0,"score_lmetric":49090},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":24545,"score_linear":98957.0,"score_lmetric":217308}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338918.525932391,"t_first_token_unix":1779769783.5398757,"t_done":338920.408661544,"t_done_unix":1779769785.4226048},{"request_id":"1279412:11:1310014:305","session_id":"1279412","input_length":64558,"t_proxy_recv":338918.332829922,"t_decision_unix":1779769783.3467715,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9517023451779795,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3118 < 8000)","cache_hit":61440,"estimated_new_tokens":3118,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":64558,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24545,"ongoing_decode_tokens":0,"pending_prefill_tokens":24545,"num_requests":1,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":64558,"score_linear":24545.0,"score_lmetric":89103},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":64558,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":854,"cache_hit":61440,"new_prefill":3118,"score_linear":-61440.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":64558,"score_linear":43607.0,"score_lmetric":64558},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":64558,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":148491,"ongoing_decode_tokens":148491,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":64558,"score_linear":148491.0,"score_lmetric":129116},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":12800,"new_prefill":51758,"score_linear":86157.0,"score_lmetric":271734}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":338919.639171511,"t_first_token_unix":1779769784.6531148,"t_done":338921.028696034,"t_done_unix":1779769786.0426397},{"request_id":"1310590:1:1310590:306","session_id":"1310590","input_length":18089,"t_proxy_recv":338920.278963547,"t_decision_unix":1779769785.2929046,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":18089,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24545,"ongoing_decode_tokens":24545,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":18089,"score_linear":24545.0,"score_lmetric":18089},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":64558,"ongoing_decode_tokens":64558,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":854,"cache_hit":0,"new_prefill":18089,"score_linear":64558.0,"score_lmetric":18089},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":18089,"score_linear":43607.0,"score_lmetric":18089},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":148491,"ongoing_decode_tokens":148491,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":18089,"score_linear":148491.0,"score_lmetric":36178},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":18089,"score_linear":98957.0,"score_lmetric":204396}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338922.062881782,"t_first_token_unix":1779769787.0768251,"t_done":338922.560896543,"t_done_unix":1779769787.5748398},{"request_id":"1297811:2:1300578:258","session_id":"1297811","input_length":43607,"t_proxy_recv":338884.89742947,"t_decision_unix":1779769749.9113708,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":13824,"affinity_cache_ratio":0.3170133235489715,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":13824,"estimated_new_tokens":29783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":558,"cache_hit":0,"new_prefill":43607,"score_linear":18893.0,"score_lmetric":43607},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":43607,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86277,"ongoing_decode_tokens":0,"pending_prefill_tokens":43781,"num_requests":2,"active_p_offloads":0,"cached_blocks":690,"cache_hit":0,"new_prefill":43607,"score_linear":86277.0,"score_lmetric":174776},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":141330,"ongoing_decode_tokens":0,"pending_prefill_tokens":88082,"num_requests":3,"active_p_offloads":0,"cached_blocks":672,"cache_hit":0,"new_prefill":43607,"score_linear":141330.0,"score_lmetric":395067},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":704,"cache_hit":13824,"new_prefill":29783,"score_linear":-13824.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":707,"cache_hit":0,"new_prefill":43607,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":578,"cache_hit":0,"new_prefill":43607,"score_linear":225261.0,"score_lmetric":130821},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":653,"cache_hit":0,"new_prefill":43607,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":338891.1013923,"t_first_token_unix":1779769756.1153357,"t_done":338922.935805221,"t_done_unix":1779769787.9497538},{"request_id":"1303517:2:1307703:297","session_id":"1303517","input_length":23131,"t_proxy_recv":338910.08594269,"t_decision_unix":1779769775.099884,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":14848,"affinity_cache_ratio":0.6419091262807488,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":14848,"estimated_new_tokens":8283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":23131,"score_linear":64654.0,"score_lmetric":46262},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":23131,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":93353,"ongoing_decode_tokens":0,"pending_prefill_tokens":12969,"num_requests":2,"active_p_offloads":0,"cached_blocks":886,"cache_hit":0,"new_prefill":23131,"score_linear":93353.0,"score_lmetric":72200},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":23131,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":23131,"score_linear":43607.0,"score_lmetric":23131},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":856,"cache_hit":0,"new_prefill":23131,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":139890,"ongoing_decode_tokens":139890,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":894,"cache_hit":0,"new_prefill":23131,"score_linear":139890.0,"score_lmetric":46262},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":14848,"new_prefill":8283,"score_linear":60978.0,"score_lmetric":84109}],"chosen_score_linear":60978.0,"chosen_score_lmetric":84109,"t_first_token":338922.871396362,"t_first_token_unix":1779769787.8853397,"t_done":338923.558174478,"t_done_unix":1779769788.5721178},{"request_id":"1304943:2:1311189:307","session_id":"1304943","input_length":13164,"t_proxy_recv":338922.411964246,"t_decision_unix":1779769787.4259057,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":0,"affinity_cache_hit":2560,"affinity_cache_ratio":0.19446976602856275,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":13164,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18089,"ongoing_decode_tokens":18089,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":13164,"score_linear":18089.0,"score_lmetric":13164},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":860,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":13164,"score_linear":43607.0,"score_lmetric":13164},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":13164,"score_linear":172648.0,"score_lmetric":74680},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":2560,"new_prefill":10604,"score_linear":96397.0,"score_lmetric":189426}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338923.517345025,"t_first_token_unix":1779769788.5312886,"t_done":338923.569358071,"t_done_unix":1779769788.5833008},{"request_id":"1305420:1:1305420:285","session_id":"1305420","input_length":75826,"t_proxy_recv":338902.012714089,"t_decision_unix":1779769767.0266554,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":75826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":75826,"score_linear":64654.0,"score_lmetric":151652},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":814,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80500,"ongoing_decode_tokens":80500,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":75826,"score_linear":80500.0,"score_lmetric":151652},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":844,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":75826,"score_linear":43607.0,"score_lmetric":75826},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":637,"cache_hit":0,"new_prefill":75826,"score_linear":249505.0,"score_lmetric":251601},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338922.564191691,"t_first_token_unix":1779769787.5781348,"t_done":338923.687633864,"t_done_unix":1779769788.7015772},{"request_id":"1303517:3:1309213:303","session_id":"1303517","input_length":23259,"t_proxy_recv":338923.563320322,"t_decision_unix":1779769788.5772622,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9905842899522765,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (219 < 8000)","cache_hit":23040,"estimated_new_tokens":219,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13164,"ongoing_decode_tokens":13164,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":659,"cache_hit":0,"new_prefill":23259,"score_linear":13164.0,"score_lmetric":23259},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":946,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":860,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":23259,"score_linear":172648.0,"score_lmetric":94870},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75826,"ongoing_decode_tokens":75826,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":720,"cache_hit":23040,"new_prefill":219,"score_linear":52786.0,"score_lmetric":219}],"chosen_score_linear":52786.0,"chosen_score_lmetric":219,"t_first_token":338923.642307608,"t_first_token_unix":1779769788.6562507,"t_done":338923.721421713,"t_done_unix":1779769788.735365},{"request_id":"1303517:4:1311293:308","session_id":"1303517","input_length":24638,"t_proxy_recv":338923.726740408,"t_decision_unix":1779769788.7406821,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9351408393538436,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1598 < 8000)","cache_hit":23040,"estimated_new_tokens":1598,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":684,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":24638,"score_linear":44792.0,"score_lmetric":44854},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":946,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":860,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":24638,"score_linear":172648.0,"score_lmetric":97628},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":868,"cache_hit":23040,"new_prefill":1598,"score_linear":-23040.0,"score_lmetric":0}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":338924.091562299,"t_first_token_unix":1779769789.1055057,"t_done":338924.265036701,"t_done_unix":1779769789.27898},{"request_id":"1277428:3:1311644:311","session_id":"1277428","input_length":31611,"t_proxy_recv":338924.034062053,"t_decision_unix":1779769789.0480037,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9880105026731201,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (379 < 8000)","cache_hit":31232,"estimated_new_tokens":379,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":684,"cache_hit":0,"new_prefill":31611,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":910,"cache_hit":4608,"new_prefill":27003,"score_linear":40184.0,"score_lmetric":47219},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":946,"cache_hit":17408,"new_prefill":14203,"score_linear":-17408.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8996,"ongoing_decode_tokens":0,"pending_prefill_tokens":292,"num_requests":1,"active_p_offloads":0,"cached_blocks":860,"cache_hit":31232,"new_prefill":379,"score_linear":-22236.0,"score_lmetric":671},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":31611,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":31611,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":31611,"score_linear":172648.0,"score_lmetric":111574},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24638,"ongoing_decode_tokens":0,"pending_prefill_tokens":1598,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":31611,"score_linear":24638.0,"score_lmetric":33209}],"chosen_score_linear":-22236.0,"chosen_score_lmetric":671,"t_first_token":338924.576959401,"t_first_token_unix":1779769789.5909026,"t_done":338924.980475958,"t_done_unix":1779769789.994419},{"request_id":"1258908:2:1311606:310","session_id":"1258908","input_length":8996,"t_proxy_recv":338923.853952835,"t_decision_unix":1779769788.8678942,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":8704,"affinity_cache_ratio":0.9675411293908404,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (292 < 8000)","cache_hit":8704,"estimated_new_tokens":292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":684,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":8996,"score_linear":44792.0,"score_lmetric":29212},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":946,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":860,"cache_hit":8704,"new_prefill":292,"score_linear":-8704.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":8996,"score_linear":172648.0,"score_lmetric":66344},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24638,"ongoing_decode_tokens":0,"pending_prefill_tokens":1598,"num_requests":1,"active_p_offloads":0,"cached_blocks":868,"cache_hit":0,"new_prefill":8996,"score_linear":24638.0,"score_lmetric":10594}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":338924.531942876,"t_first_token_unix":1779769789.5458863,"t_done":338926.598393521,"t_done_unix":1779769791.6123369},{"request_id":"1311753:1:1311753:312","session_id":"1311753","input_length":20175,"t_proxy_recv":338924.431207438,"t_decision_unix":1779769789.445149,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":20175,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":684,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":20175,"score_linear":44792.0,"score_lmetric":40391},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":946,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40607,"ongoing_decode_tokens":0,"pending_prefill_tokens":671,"num_requests":2,"active_p_offloads":0,"cached_blocks":860,"cache_hit":0,"new_prefill":20175,"score_linear":40607.0,"score_lmetric":41692},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":20175,"score_linear":172648.0,"score_lmetric":88702},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338926.535219914,"t_first_token_unix":1779769791.5491636,"t_done":338926.718811496,"t_done_unix":1779769791.7327545},{"request_id":"1294611:9:1312398:313","session_id":"1294611","input_length":44335,"t_proxy_recv":338926.713030359,"t_decision_unix":1779769791.7269719,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9816172324348709,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (815 < 8000)","cache_hit":43520,"estimated_new_tokens":815,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":20175,"ongoing_decode_tokens":20175,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":684,"cache_hit":0,"new_prefill":44335,"score_linear":20175.0,"score_lmetric":44335},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":44335,"score_linear":44792.0,"score_lmetric":64551},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":946,"cache_hit":43520,"new_prefill":815,"score_linear":-43520.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":861,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":12800,"new_prefill":31535,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":12800,"new_prefill":31535,"score_linear":159848.0,"score_lmetric":111422},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":338926.963413747,"t_first_token_unix":1779769791.9773574,"t_done":338928.268745971,"t_done_unix":1779769793.2826896},{"request_id":"1303517:5:1313040:317","session_id":"1303517","input_length":25637,"t_proxy_recv":338928.774292739,"t_decision_unix":1779769793.7882342,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24576,"affinity_cache_ratio":0.9586145024768888,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1061 < 8000)","cache_hit":24576,"estimated_new_tokens":1061,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44792,"ongoing_decode_tokens":44792,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":25637,"score_linear":44792.0,"score_lmetric":25637},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40245,"ongoing_decode_tokens":13360,"pending_prefill_tokens":5381,"num_requests":2,"active_p_offloads":0,"cached_blocks":861,"cache_hit":0,"new_prefill":25637,"score_linear":40245.0,"score_lmetric":62036},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":25637,"score_linear":172648.0,"score_lmetric":99626},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":871,"cache_hit":24576,"new_prefill":1061,"score_linear":-24576.0,"score_lmetric":0}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":338929.051756093,"t_first_token_unix":1779769794.0657003,"t_done":338929.082544678,"t_done_unix":1779769794.0964882},{"request_id":"1309252:2:1311531:309","session_id":"1309252","input_length":44792,"t_proxy_recv":338923.623927145,"t_decision_unix":1779769788.637869,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":24576,"affinity_cache_ratio":0.5486694052509377,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":24576,"estimated_new_tokens":20216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":684,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":910,"cache_hit":24576,"new_prefill":20216,"score_linear":-24576.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":946,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":860,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":44792,"score_linear":172648.0,"score_lmetric":137936},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99085,"ongoing_decode_tokens":75826,"pending_prefill_tokens":219,"num_requests":2,"active_p_offloads":0,"cached_blocks":720,"cache_hit":0,"new_prefill":44792,"score_linear":99085.0,"score_lmetric":90022}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":338928.527209976,"t_first_token_unix":1779769793.5411532,"t_done":338929.206050754,"t_done_unix":1779769794.219994},{"request_id":"1306347:2:1312642:314","session_id":"1306347","input_length":13360,"t_proxy_recv":338927.607477702,"t_decision_unix":1779769792.6214192,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":3,"affinity_cache_hit":512,"affinity_cache_ratio":0.03832335329341317,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":13360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":13360,"score_linear":44792.0,"score_lmetric":33576},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44335,"ongoing_decode_tokens":44335,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":946,"cache_hit":512,"new_prefill":12848,"score_linear":43823.0,"score_lmetric":12848},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":861,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":13360,"score_linear":172648.0,"score_lmetric":75072},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338928.741522594,"t_first_token_unix":1779769793.755466,"t_done":338930.407212802,"t_done_unix":1779769795.4211566},{"request_id":"1268861:4:1312689:315","session_id":"1268861","input_length":26885,"t_proxy_recv":338927.768503805,"t_decision_unix":1779769792.782445,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":21504,"affinity_cache_ratio":0.7998512181513855,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5381 < 8000)","cache_hit":21504,"estimated_new_tokens":5381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":910,"cache_hit":0,"new_prefill":26885,"score_linear":44792.0,"score_lmetric":47101},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44335,"ongoing_decode_tokens":44335,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":946,"cache_hit":0,"new_prefill":26885,"score_linear":44335.0,"score_lmetric":26885},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13360,"ongoing_decode_tokens":0,"pending_prefill_tokens":13360,"num_requests":1,"active_p_offloads":0,"cached_blocks":861,"cache_hit":21504,"new_prefill":5381,"score_linear":-8144.0,"score_lmetric":18741},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":26885,"score_linear":172648.0,"score_lmetric":102122},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":871,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8144.0,"chosen_score_lmetric":18741,"t_first_token":338929.646815861,"t_first_token_unix":1779769794.660759,"t_done":338931.444166869,"t_done_unix":1779769796.4581106},{"request_id":"1313181:1:1313181:318","session_id":"1313181","input_length":22686,"t_proxy_recv":338929.390037082,"t_decision_unix":1779769794.4039788,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":12800,"estimated_new_tokens":9886,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40245,"ongoing_decode_tokens":13360,"pending_prefill_tokens":5381,"num_requests":2,"active_p_offloads":0,"cached_blocks":861,"cache_hit":12800,"new_prefill":9886,"score_linear":27445.0,"score_lmetric":30534},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":22686,"score_linear":172648.0,"score_lmetric":93724},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":873,"cache_hit":12800,"new_prefill":9886,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":338930.889747535,"t_first_token_unix":1779769795.9036913,"t_done":338931.554660834,"t_done_unix":1779769796.568605},{"request_id":"1279412:12:1313518:319","session_id":"1279412","input_length":64755,"t_proxy_recv":338930.511593622,"t_decision_unix":1779769795.5255346,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":64512,"affinity_cache_ratio":0.9962473940236275,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (243 < 8000)","cache_hit":64512,"estimated_new_tokens":243,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":26885,"ongoing_decode_tokens":26885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":887,"cache_hit":64512,"new_prefill":243,"score_linear":-37627.0,"score_lmetric":243},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":64755,"score_linear":172648.0,"score_lmetric":177862},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22686,"ongoing_decode_tokens":0,"pending_prefill_tokens":9886,"num_requests":1,"active_p_offloads":0,"cached_blocks":873,"cache_hit":12800,"new_prefill":51955,"score_linear":9886.0,"score_lmetric":61841}],"chosen_score_linear":-37627.0,"chosen_score_lmetric":243,"t_first_token":338930.640391946,"t_first_token_unix":1779769795.6543355,"t_done":338932.119994962,"t_done_unix":1779769797.1339388},{"request_id":"1294611:10:1314495:322","session_id":"1294611","input_length":45362,"t_proxy_recv":338934.168348239,"t_decision_unix":1779769799.1822896,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":44032,"affinity_cache_ratio":0.970680305101186,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1330 < 8000)","cache_hit":44032,"estimated_new_tokens":1330,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":45362,"score_linear":57051.0,"score_lmetric":98829},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":44032,"new_prefill":1330,"score_linear":-44032.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":12800,"new_prefill":32562,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":12800,"new_prefill":32562,"score_linear":159848.0,"score_lmetric":113476},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":338934.521474186,"t_first_token_unix":1779769799.5354178,"t_done":338935.995442623,"t_done_unix":1779769801.009386},{"request_id":"1313181:2:1314940:323","session_id":"1313181","input_length":24306,"t_proxy_recv":338935.579543248,"t_decision_unix":1779769800.5934849,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":22528,"affinity_cache_ratio":0.9268493376121122,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1778 < 8000)","cache_hit":22528,"estimated_new_tokens":1778,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":24306,"score_linear":57051.0,"score_lmetric":77773},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":45362,"ongoing_decode_tokens":45362,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":24306,"score_linear":45362.0,"score_lmetric":24306},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":12800,"new_prefill":11506,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":24306,"score_linear":172648.0,"score_lmetric":96964},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":22528,"new_prefill":1778,"score_linear":-22528.0,"score_lmetric":0}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":0,"t_first_token":338935.918831088,"t_first_token_unix":1779769800.9327743,"t_done":338936.513287993,"t_done_unix":1779769801.527231},{"request_id":"1315313:1:1315313:325","session_id":"1315313","input_length":7,"t_proxy_recv":338936.810458873,"t_decision_unix":1779769801.8244004,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7 < 8000)","cache_hit":0,"estimated_new_tokens":7,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":7,"score_linear":57051.0,"score_lmetric":53474},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":32440,"ongoing_decode_tokens":0,"pending_prefill_tokens":14520,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":7,"score_linear":32440.0,"score_lmetric":14527},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":7,"score_linear":172648.0,"score_lmetric":14},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338936.831007869,"t_first_token_unix":1779769801.8449507,"t_done":338936.835136032,"t_done_unix":1779769801.8490791},{"request_id":"1310590:2:1315212:324","session_id":"1310590","input_length":32440,"t_proxy_recv":338936.45792614,"t_decision_unix":1779769801.4718676,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":17920,"affinity_cache_ratio":0.5524044389642416,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":17920,"estimated_new_tokens":14520,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":32440,"score_linear":57051.0,"score_lmetric":85907},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":17920,"new_prefill":14520,"score_linear":-17920.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":32440,"score_linear":172648.0,"score_lmetric":64880},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24306,"ongoing_decode_tokens":24306,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":32440,"score_linear":24306.0,"score_lmetric":32440}],"chosen_score_linear":-17920.0,"chosen_score_lmetric":0,"t_first_token":338939.172598931,"t_first_token_unix":1779769804.1865427,"t_done":338939.322598401,"t_done_unix":1779769804.3365417},{"request_id":"1309040:2:1315488:326","session_id":"1309040","input_length":23097,"t_proxy_recv":338937.534875027,"t_decision_unix":1779769802.5488162,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":4608,"affinity_cache_ratio":0.1995064294064164,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":4608,"estimated_new_tokens":18489,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":23097,"score_linear":57051.0,"score_lmetric":76564},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":32440,"ongoing_decode_tokens":0,"pending_prefill_tokens":14520,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":23097,"score_linear":32440.0,"score_lmetric":37617},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":4608,"new_prefill":18489,"score_linear":-4608.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":23097,"score_linear":172648.0,"score_lmetric":46194},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":338939.852127267,"t_first_token_unix":1779769804.8660705,"t_done":338940.217625976,"t_done_unix":1779769805.231569},{"request_id":"1279412:13:1315780:328","session_id":"1279412","input_length":64949,"t_proxy_recv":338938.595463436,"t_decision_unix":1779769803.6094046,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":64512,"affinity_cache_ratio":0.9932716439052179,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (437 < 8000)","cache_hit":64512,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":16960,"ongoing_decode_tokens":0,"pending_prefill_tokens":9280,"num_requests":1,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":64949,"score_linear":16960.0,"score_lmetric":74229},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":64949,"score_linear":57051.0,"score_lmetric":118416},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":32440,"ongoing_decode_tokens":0,"pending_prefill_tokens":14520,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":64949,"score_linear":32440.0,"score_lmetric":79469},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":64512,"new_prefill":437,"score_linear":-64512.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":64949,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23097,"ongoing_decode_tokens":0,"pending_prefill_tokens":18489,"num_requests":1,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":64949,"score_linear":23097.0,"score_lmetric":83438},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":64949,"score_linear":172648.0,"score_lmetric":129898},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":12800,"new_prefill":52149,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-64512.0,"chosen_score_lmetric":0,"t_first_token":338938.733998041,"t_first_token_unix":1779769803.7479563,"t_done":338941.34996111,"t_done_unix":1779769806.3639047},{"request_id":"1269373:8:1295332:230","session_id":"1269373","input_length":50680,"t_proxy_recv":338915.796664144,"t_decision_unix":1779769780.8106058,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":43008,"affinity_cache_ratio":0.8486187845303867,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7672 < 8000)","cache_hit":43008,"estimated_new_tokens":7672,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":597,"cache_hit":7680,"new_prefill":43000,"score_linear":27026.0,"score_lmetric":43000},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24545,"ongoing_decode_tokens":0,"pending_prefill_tokens":24545,"num_requests":1,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":50680,"score_linear":24545.0,"score_lmetric":75225},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43710,"ongoing_decode_tokens":43710,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":50680,"score_linear":43710.0,"score_lmetric":50680},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":854,"cache_hit":0,"new_prefill":50680,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":50680,"score_linear":43607.0,"score_lmetric":50680},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":7680,"new_prefill":43000,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":97811,"ongoing_decode_tokens":97811,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":904,"cache_hit":43008,"new_prefill":7672,"score_linear":54803.0,"score_lmetric":7672},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":50680,"score_linear":98957.0,"score_lmetric":269578}],"chosen_score_linear":54803.0,"chosen_score_lmetric":7672,"t_first_token":338918.136455226,"t_first_token_unix":1779769783.1503987,"t_done":338941.447875867,"t_done_unix":1779769806.4618192},{"request_id":"1315567:1:1315567:327","session_id":"1315567","input_length":16960,"t_proxy_recv":338937.799903269,"t_decision_unix":1779769802.8138447,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":7680,"estimated_new_tokens":9280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":7680,"new_prefill":9280,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":16960,"score_linear":57051.0,"score_lmetric":70427},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":32440,"ongoing_decode_tokens":0,"pending_prefill_tokens":14520,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":16960,"score_linear":32440.0,"score_lmetric":31480},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23097,"ongoing_decode_tokens":0,"pending_prefill_tokens":18489,"num_requests":1,"active_p_offloads":0,"cached_blocks":865,"cache_hit":7680,"new_prefill":9280,"score_linear":15417.0,"score_lmetric":27769},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":7680,"new_prefill":9280,"score_linear":164968.0,"score_lmetric":18560},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":338938.914632529,"t_first_token_unix":1779769803.928576,"t_done":338942.058141302,"t_done_unix":1779769807.072085},{"request_id":"1316848:1:1316848:331","session_id":"1316848","input_length":5985,"t_proxy_recv":338942.151752099,"t_decision_unix":1779769807.1656938,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4961 < 8000)","cache_hit":1024,"estimated_new_tokens":4961,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":5985,"score_linear":57051.0,"score_lmetric":59452},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24487,"ongoing_decode_tokens":0,"pending_prefill_tokens":24487,"num_requests":1,"active_p_offloads":0,"cached_blocks":977,"cache_hit":0,"new_prefill":5985,"score_linear":24487.0,"score_lmetric":30472},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":121968,"pending_prefill_tokens":3763,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":5985,"score_linear":176419.0,"score_lmetric":19496},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":1024,"new_prefill":4961,"score_linear":-1024.0,"score_lmetric":0}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":338942.497771492,"t_first_token_unix":1779769807.5117147,"t_done":338942.681915779,"t_done_unix":1779769807.6958592},{"request_id":"1316201:1:1316201:329","session_id":"1316201","input_length":24487,"t_proxy_recv":338940.063682621,"t_decision_unix":1779769805.0776238,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":24487,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":16960,"ongoing_decode_tokens":16960,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":24487,"score_linear":16960.0,"score_lmetric":24487},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":24487,"score_linear":57051.0,"score_lmetric":77954},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":977,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":64949,"ongoing_decode_tokens":64949,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":24487,"score_linear":64949.0,"score_lmetric":24487},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23097,"ongoing_decode_tokens":23097,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":24487,"score_linear":23097.0,"score_lmetric":24487},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":24487,"score_linear":172648.0,"score_lmetric":48974},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338942.859250894,"t_first_token_unix":1779769807.873194,"t_done":338943.153808773,"t_done_unix":1779769808.1677523},{"request_id":"1317160:1:1317160:332","session_id":"1317160","input_length":3441,"t_proxy_recv":338943.306219634,"t_decision_unix":1779769808.3201613,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3441 < 8000)","cache_hit":0,"estimated_new_tokens":3441,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":3441,"score_linear":57051.0,"score_lmetric":56908},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1024,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":3441,"score_linear":176419.0,"score_lmetric":6882},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338943.521124922,"t_first_token_unix":1779769808.5350683,"t_done":338943.679527588,"t_done_unix":1779769808.6934714},{"request_id":"1316848:2:1317438:333","session_id":"1316848","input_length":6040,"t_proxy_recv":338944.387755934,"t_decision_unix":1779769809.401697,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":5632,"affinity_cache_ratio":0.9324503311258279,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (408 < 8000)","cache_hit":5632,"estimated_new_tokens":408,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":6040,"score_linear":57051.0,"score_lmetric":59507},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1030,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":6040,"score_linear":176419.0,"score_lmetric":12080},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":5632,"new_prefill":408,"score_linear":-5632.0,"score_lmetric":0}],"chosen_score_linear":-5632.0,"chosen_score_lmetric":0,"t_first_token":338944.420977051,"t_first_token_unix":1779769809.4349203,"t_done":338944.833851974,"t_done_unix":1779769809.8477957},{"request_id":"1317160:2:1317773:336","session_id":"1317160","input_length":7440,"t_proxy_recv":338945.705188643,"t_decision_unix":1779769810.71913,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":3072,"affinity_cache_ratio":0.4129032258064516,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4368 < 8000)","cache_hit":3072,"estimated_new_tokens":4368,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":57051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":7440,"score_linear":57051.0,"score_lmetric":7440},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1030,"cache_hit":3072,"new_prefill":4368,"score_linear":-3072.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":7440,"score_linear":79663.0,"score_lmetric":86591},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":7440,"score_linear":176419.0,"score_lmetric":14880},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":338946.042577205,"t_first_token_unix":1779769811.0565205,"t_done":338946.277735271,"t_done_unix":1779769811.2916784},{"request_id":"1316201:2:1317776:337","session_id":"1316201","input_length":24768,"t_proxy_recv":338945.723782112,"t_decision_unix":1779769810.7377236,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9715762273901809,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (704 < 8000)","cache_hit":24064,"estimated_new_tokens":704,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":57051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":24768,"score_linear":57051.0,"score_lmetric":24768},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":7440,"ongoing_decode_tokens":0,"pending_prefill_tokens":4368,"num_requests":1,"active_p_offloads":0,"cached_blocks":1030,"cache_hit":24064,"new_prefill":704,"score_linear":-16624.0,"score_lmetric":5072},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":24768,"score_linear":79663.0,"score_lmetric":103919},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":24768,"score_linear":176419.0,"score_lmetric":49536},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16624.0,"chosen_score_lmetric":5072,"t_first_token":338946.122785238,"t_first_token_unix":1779769811.1367283,"t_done":338946.689441624,"t_done_unix":1779769811.7033849},{"request_id":"1318031:1:1318031:338","session_id":"1318031","input_length":5219,"t_proxy_recv":338946.366874292,"t_decision_unix":1779769811.380816,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5219 < 8000)","cache_hit":0,"estimated_new_tokens":5219,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":57051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":5219,"score_linear":57051.0,"score_lmetric":5219},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24768,"ongoing_decode_tokens":24768,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":5219,"score_linear":24768.0,"score_lmetric":5219},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":5219,"score_linear":79663.0,"score_lmetric":84370},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":5219,"score_linear":176419.0,"score_lmetric":10438},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338946.701774591,"t_first_token_unix":1779769811.7157173,"t_done":338946.924232211,"t_done_unix":1779769811.9381754},{"request_id":"1318036:1:1318036:339","session_id":"1318036","input_length":2696,"t_proxy_recv":338946.370917784,"t_decision_unix":1779769811.3848593,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2696 < 8000)","cache_hit":0,"estimated_new_tokens":2696,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":57051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":2696,"score_linear":57051.0,"score_lmetric":2696},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24768,"ongoing_decode_tokens":24768,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1038,"cache_hit":0,"new_prefill":2696,"score_linear":24768.0,"score_lmetric":2696},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":2696,"score_linear":79663.0,"score_lmetric":81847},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":5219,"ongoing_decode_tokens":0,"pending_prefill_tokens":5219,"num_requests":1,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":2696,"score_linear":5219.0,"score_lmetric":7915},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":2696,"score_linear":176419.0,"score_lmetric":5392},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338946.586926507,"t_first_token_unix":1779769811.6008701,"t_done":338947.079741102,"t_done_unix":1779769812.093684},{"request_id":"1314357:1:1314357:321","session_id":"1314357","input_length":57051,"t_proxy_recv":338933.706728476,"t_decision_unix":1779769798.72067,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":3584,"estimated_new_tokens":53467,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":3584,"new_prefill":53467,"score_linear":-3584.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":57051,"score_linear":172648.0,"score_lmetric":162454},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":892,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":338945.28912801,"t_first_token_unix":1779769810.3030715,"t_done":338947.209511234,"t_done_unix":1779769812.2234547},{"request_id":"1318604:1:1318604:341","session_id":"1318604","input_length":14412,"t_proxy_recv":338948.415197435,"t_decision_unix":1779769813.429139,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":14412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":14412,"score_linear":62618.0,"score_lmetric":18150},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":14412,"score_linear":79663.0,"score_lmetric":93563},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":14412,"score_linear":176419.0,"score_lmetric":28824},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338949.665854062,"t_first_token_unix":1779769814.6797974,"t_done":338949.852483128,"t_done_unix":1779769814.8664267},{"request_id":"1269373:9:1300958:261","session_id":"1269373","input_length":54451,"t_proxy_recv":338941.4583397,"t_decision_unix":1779769806.4722815,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":50688,"affinity_cache_ratio":0.9308919946373804,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3763 < 8000)","cache_hit":50688,"estimated_new_tokens":3763,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":16960,"ongoing_decode_tokens":16960,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":723,"cache_hit":7680,"new_prefill":46771,"score_linear":9280.0,"score_lmetric":46771},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":54451,"score_linear":57051.0,"score_lmetric":107918},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24487,"ongoing_decode_tokens":0,"pending_prefill_tokens":24487,"num_requests":1,"active_p_offloads":0,"cached_blocks":977,"cache_hit":0,"new_prefill":54451,"score_linear":24487.0,"score_lmetric":78938},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":54451,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":54451,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":7680,"new_prefill":46771,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":920,"cache_hit":50688,"new_prefill":3763,"score_linear":71280.0,"score_lmetric":3763},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":895,"cache_hit":0,"new_prefill":54451,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":71280.0,"chosen_score_lmetric":3763,"t_first_token":338942.699713796,"t_first_token_unix":1779769807.7136574,"t_done":338949.956388985,"t_done_unix":1779769814.9703324},{"request_id":"1316848:3:1318975:343","session_id":"1316848","input_length":6298,"t_proxy_recv":338949.787521924,"t_decision_unix":1779769814.8014634,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":5632,"affinity_cache_ratio":0.8942521435376309,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (666 < 8000)","cache_hit":5632,"estimated_new_tokens":666,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":6298,"score_linear":62618.0,"score_lmetric":10036},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":14412,"ongoing_decode_tokens":14412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":6298,"score_linear":14412.0,"score_lmetric":6298},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":6298,"score_linear":79663.0,"score_lmetric":85449},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":6298,"score_linear":176419.0,"score_lmetric":12596},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":5632,"new_prefill":666,"score_linear":-5632.0,"score_lmetric":0}],"chosen_score_linear":-5632.0,"chosen_score_lmetric":0,"t_first_token":338949.843506443,"t_first_token_unix":1779769814.85745,"t_done":338950.117060493,"t_done_unix":1779769815.1310039},{"request_id":"1318604:2:1319895:347","session_id":"1318604","input_length":15745,"t_proxy_recv":338952.930708721,"t_decision_unix":1779769817.9446504,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":14336,"affinity_cache_ratio":0.9105112734201334,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1409 < 8000)","cache_hit":14336,"estimated_new_tokens":1409,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":37138,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":15745,"score_linear":54546.0,"score_lmetric":52883},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":15745,"score_linear":62618.0,"score_lmetric":19483},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":15745,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71854,"ongoing_decode_tokens":0,"pending_prefill_tokens":6830,"num_requests":1,"active_p_offloads":0,"cached_blocks":926,"cache_hit":14336,"new_prefill":1409,"score_linear":57518.0,"score_lmetric":8239},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":15745,"score_linear":79663.0,"score_lmetric":94896},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":15745,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181114,"ongoing_decode_tokens":181114,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":928,"cache_hit":0,"new_prefill":15745,"score_linear":181114.0,"score_lmetric":31490},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21140,"ongoing_decode_tokens":0,"pending_prefill_tokens":14996,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":15745,"score_linear":21140.0,"score_lmetric":30741}],"chosen_score_linear":57518.0,"chosen_score_lmetric":8239,"t_first_token":338953.288939227,"t_first_token_unix":1779769818.3028827,"t_done":338953.634213642,"t_done_unix":1779769818.648157},{"request_id":"1269373:10:1304899:282","session_id":"1269373","input_length":59146,"t_proxy_recv":338949.971117508,"t_decision_unix":1779769814.9850593,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9262502958779968,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4362 < 8000)","cache_hit":54784,"estimated_new_tokens":4362,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":747,"cache_hit":7680,"new_prefill":51466,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":59146,"score_linear":62618.0,"score_lmetric":62884},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":59146,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":59146,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":59146,"score_linear":79663.0,"score_lmetric":138297},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":7680,"new_prefill":51466,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":928,"cache_hit":54784,"new_prefill":4362,"score_linear":67184.0,"score_lmetric":4362},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":6298,"ongoing_decode_tokens":6298,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":59146,"score_linear":6298.0,"score_lmetric":59146}],"chosen_score_linear":67184.0,"chosen_score_lmetric":4362,"t_first_token":338951.622915726,"t_first_token_unix":1779769816.6368594,"t_done":338955.372698441,"t_done_unix":1779769820.3866456},{"request_id":"1320630:1:1320630:348","session_id":"1320630","input_length":3099,"t_proxy_recv":338955.706024542,"t_decision_unix":1779769820.7199657,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3099 < 8000)","cache_hit":0,"estimated_new_tokens":3099,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":37138,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":3099,"score_linear":54546.0,"score_lmetric":40237},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":3099,"score_linear":62618.0,"score_lmetric":6837},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":3099,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71854,"ongoing_decode_tokens":71854,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":928,"cache_hit":0,"new_prefill":3099,"score_linear":71854.0,"score_lmetric":3099},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":3099,"score_linear":79663.0,"score_lmetric":82250},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":3099,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":3099,"score_linear":181414.0,"score_lmetric":6198},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21140,"ongoing_decode_tokens":21140,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":3099,"score_linear":21140.0,"score_lmetric":3099}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338955.903231039,"t_first_token_unix":1779769820.9171746,"t_done":338956.100734295,"t_done_unix":1779769821.114678},{"request_id":"1279412:14:1319059:344","session_id":"1279412","input_length":71854,"t_proxy_recv":338950.146471611,"t_decision_unix":1779769815.1604128,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9049461407854816,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6830 < 8000)","cache_hit":65024,"estimated_new_tokens":6830,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":71854,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":71854,"score_linear":62618.0,"score_lmetric":75592},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":71854,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":65024,"new_prefill":6830,"score_linear":-65024.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":71854,"score_linear":79663.0,"score_lmetric":151005},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":71854,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181114,"ongoing_decode_tokens":121968,"pending_prefill_tokens":4362,"num_requests":2,"active_p_offloads":0,"cached_blocks":928,"cache_hit":0,"new_prefill":71854,"score_linear":181114.0,"score_lmetric":152432},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":12800,"new_prefill":59054,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":338953.108641191,"t_first_token_unix":1779769818.1225843,"t_done":338956.251295579,"t_done_unix":1779769821.2652395},{"request_id":"1316848:4:1319820:345","session_id":"1316848","input_length":21140,"t_proxy_recv":338952.694366827,"t_decision_unix":1779769817.7083082,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.29063386944181646,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":6144,"estimated_new_tokens":14996,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":21140,"score_linear":62618.0,"score_lmetric":24878},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71854,"ongoing_decode_tokens":0,"pending_prefill_tokens":6830,"num_requests":1,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":21140,"score_linear":71854.0,"score_lmetric":27970},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":21140,"score_linear":79663.0,"score_lmetric":100291},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181114,"ongoing_decode_tokens":181114,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":928,"cache_hit":0,"new_prefill":21140,"score_linear":181114.0,"score_lmetric":42280},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":6144,"new_prefill":14996,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":338954.543147637,"t_first_token_unix":1779769819.557091,"t_done":338956.3923889,"t_done_unix":1779769821.4063323},{"request_id":"1318604:3:1320865:350","session_id":"1318604","input_length":17883,"t_proxy_recv":338956.54953068,"t_decision_unix":1779769821.5634718,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":15360,"affinity_cache_ratio":0.8589162892132193,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2523 < 8000)","cache_hit":15360,"estimated_new_tokens":2523,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":37138,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":17883,"score_linear":54546.0,"score_lmetric":55021},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":17883,"score_linear":62618.0,"score_lmetric":21621},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":17883,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":941,"cache_hit":15360,"new_prefill":2523,"score_linear":-15360.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":17883,"score_linear":79663.0,"score_lmetric":97034},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":17883,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":17883,"score_linear":181414.0,"score_lmetric":35766},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":17883,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":338956.905063221,"t_first_token_unix":1779769821.9190068,"t_done":338957.130067544,"t_done_unix":1779769822.1440108},{"request_id":"1321042:1:1321042:351","session_id":"1321042","input_length":8095,"t_proxy_recv":338957.199815869,"t_decision_unix":1779769822.2137575,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8095,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":37138,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":8095,"score_linear":54546.0,"score_lmetric":45233},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":8095,"score_linear":62618.0,"score_lmetric":11833},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":946,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":8095,"score_linear":79663.0,"score_lmetric":87246},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":8095,"score_linear":181414.0,"score_lmetric":16190},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338957.768053974,"t_first_token_unix":1779769822.7819972,"t_done":338958.080196041,"t_done_unix":1779769823.094139},{"request_id":"1277428:4:1321259:352","session_id":"1277428","input_length":33906,"t_proxy_recv":338957.958640804,"t_decision_unix":1779769822.9725819,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9211349023771603,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2674 < 8000)","cache_hit":31232,"estimated_new_tokens":2674,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":37138,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":33906,"score_linear":54546.0,"score_lmetric":71044},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":4608,"new_prefill":29298,"score_linear":58010.0,"score_lmetric":33036},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":17408,"new_prefill":16498,"score_linear":-17408.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":946,"cache_hit":31232,"new_prefill":2674,"score_linear":-31232.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":33906,"score_linear":79663.0,"score_lmetric":113057},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":8095,"ongoing_decode_tokens":8095,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":33906,"score_linear":8095.0,"score_lmetric":33906},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":33906,"score_linear":181414.0,"score_lmetric":67812},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":33906,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":338958.5685463,"t_first_token_unix":1779769823.58249,"t_done":338959.396845178,"t_done_unix":1779769824.4107883},{"request_id":"1318604:4:1321889:356","session_id":"1318604","input_length":21033,"t_proxy_recv":338960.145070774,"t_decision_unix":1779769825.159012,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":17920,"affinity_cache_ratio":0.8519944848571293,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3113 < 8000)","cache_hit":17920,"estimated_new_tokens":3113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57223,"ongoing_decode_tokens":0,"pending_prefill_tokens":37255,"num_requests":2,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":21033,"score_linear":57223.0,"score_lmetric":116576},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":21033,"score_linear":62618.0,"score_lmetric":24771},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":21033,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72402,"ongoing_decode_tokens":72402,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":951,"cache_hit":17920,"new_prefill":3113,"score_linear":54482.0,"score_lmetric":3113},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":21033,"score_linear":79663.0,"score_lmetric":100184},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":21033,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":21033,"score_linear":181414.0,"score_lmetric":42066},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":21033,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":54482.0,"chosen_score_lmetric":3113,"t_first_token":338960.702382825,"t_first_token_unix":1779769825.7163267,"t_done":338961.133966355,"t_done_unix":1779769826.1479099},{"request_id":"1316848:5:1322192:357","session_id":"1316848","input_length":21471,"t_proxy_recv":338961.319325187,"t_decision_unix":1779769826.3332667,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":20992,"affinity_cache_ratio":0.9776908388058311,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (479 < 8000)","cache_hit":20992,"estimated_new_tokens":479,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57223,"ongoing_decode_tokens":0,"pending_prefill_tokens":37255,"num_requests":2,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":21471,"score_linear":57223.0,"score_lmetric":117452},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":21471,"score_linear":62618.0,"score_lmetric":25209},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":21471,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72402,"ongoing_decode_tokens":72402,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":957,"cache_hit":0,"new_prefill":21471,"score_linear":72402.0,"score_lmetric":21471},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":21471,"score_linear":79663.0,"score_lmetric":100622},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":21471,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":21471,"score_linear":181414.0,"score_lmetric":42942},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":20992,"new_prefill":479,"score_linear":-20992.0,"score_lmetric":0}],"chosen_score_linear":-20992.0,"chosen_score_lmetric":0,"t_first_token":338961.376439417,"t_first_token_unix":1779769826.390383,"t_done":338961.910856189,"t_done_unix":1779769826.9248},{"request_id":"1318036:2:1321282:353","session_id":"1318036","input_length":2677,"t_proxy_recv":338958.011695706,"t_decision_unix":1779769823.0256374,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":2560,"affinity_cache_ratio":0.9562943593574897,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (117 < 8000)","cache_hit":2560,"estimated_new_tokens":117,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":37138,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":2560,"new_prefill":117,"score_linear":51986.0,"score_lmetric":37255},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":2677,"score_linear":62618.0,"score_lmetric":6415},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":2677,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":33906,"ongoing_decode_tokens":0,"pending_prefill_tokens":2674,"num_requests":1,"active_p_offloads":0,"cached_blocks":946,"cache_hit":0,"new_prefill":2677,"score_linear":33906.0,"score_lmetric":5351},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":2677,"score_linear":79663.0,"score_lmetric":81828},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":8095,"ongoing_decode_tokens":8095,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":2677,"score_linear":8095.0,"score_lmetric":2677},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":2677,"score_linear":181414.0,"score_lmetric":5354},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":2677,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":51986.0,"chosen_score_lmetric":37255,"t_first_token":338962.166955877,"t_first_token_unix":1779769827.1809003,"t_done":338962.191811586,"t_done_unix":1779769827.205755},{"request_id":"1279412:15:1321613:355","session_id":"1279412","input_length":72402,"t_proxy_recv":338959.132466343,"t_decision_unix":1779769824.1464074,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":71680,"affinity_cache_ratio":0.990027899781774,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (722 < 8000)","cache_hit":71680,"estimated_new_tokens":722,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57223,"ongoing_decode_tokens":0,"pending_prefill_tokens":37255,"num_requests":2,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":72402,"score_linear":57223.0,"score_lmetric":219314},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":72402,"score_linear":62618.0,"score_lmetric":76140},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":72402,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":33906,"ongoing_decode_tokens":33906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":946,"cache_hit":71680,"new_prefill":722,"score_linear":-37774.0,"score_lmetric":722},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":72402,"score_linear":79663.0,"score_lmetric":151553},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":72402,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":72402,"score_linear":181414.0,"score_lmetric":144804},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":12800,"new_prefill":59602,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-37774.0,"chosen_score_lmetric":722,"t_first_token":338959.369848711,"t_first_token_unix":1779769824.3837922,"t_done":338962.221064103,"t_done_unix":1779769827.235008},{"request_id":"1269373:11:1307381:296","session_id":"1269373","input_length":59446,"t_proxy_recv":338955.382963668,"t_decision_unix":1779769820.3969052,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9904787538270027,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (566 < 8000)","cache_hit":58880,"estimated_new_tokens":566,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":37138,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":7680,"new_prefill":51766,"score_linear":46866.0,"score_lmetric":88904},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":59446,"score_linear":62618.0,"score_lmetric":63184},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":59446,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71854,"ongoing_decode_tokens":71854,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":928,"cache_hit":0,"new_prefill":59446,"score_linear":71854.0,"score_lmetric":59446},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":59446,"score_linear":79663.0,"score_lmetric":138597},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":7680,"new_prefill":51766,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":936,"cache_hit":58880,"new_prefill":566,"score_linear":63088.0,"score_lmetric":566},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21140,"ongoing_decode_tokens":21140,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":59446,"score_linear":21140.0,"score_lmetric":59446}],"chosen_score_linear":63088.0,"chosen_score_lmetric":566,"t_first_token":338955.553554987,"t_first_token_unix":1779769820.5674984,"t_done":338963.52413593,"t_done_unix":1779769828.5380826},{"request_id":"1322540:1:1322540:359","session_id":"1322540","input_length":8939,"t_proxy_recv":338962.602440944,"t_decision_unix":1779769827.616382,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8939,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":8939,"score_linear":54546.0,"score_lmetric":8939},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":8939,"score_linear":62618.0,"score_lmetric":8939},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":8939,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":958,"cache_hit":0,"new_prefill":8939,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":8939,"score_linear":79663.0,"score_lmetric":88090},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":8939,"score_linear":33182.0,"score_lmetric":42121},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":8939,"score_linear":181414.0,"score_lmetric":17878},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":935,"cache_hit":0,"new_prefill":8939,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338963.273837215,"t_first_token_unix":1779769828.2877812,"t_done":338963.53381511,"t_done_unix":1779769828.5477598},{"request_id":"1318031:2:1322552:360","session_id":"1318031","input_length":11145,"t_proxy_recv":338962.658128159,"t_decision_unix":1779769827.6720693,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":3,"affinity_cache_hit":5120,"affinity_cache_ratio":0.45939883355764916,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11145,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":11145,"score_linear":54546.0,"score_lmetric":11145},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":11145,"score_linear":62618.0,"score_lmetric":11145},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":11145,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":958,"cache_hit":0,"new_prefill":11145,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":11145,"score_linear":79663.0,"score_lmetric":90296},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":5120,"new_prefill":6025,"score_linear":28062.0,"score_lmetric":39207},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":11145,"score_linear":181414.0,"score_lmetric":22290},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":8939,"ongoing_decode_tokens":0,"pending_prefill_tokens":8939,"num_requests":1,"active_p_offloads":0,"cached_blocks":935,"cache_hit":0,"new_prefill":11145,"score_linear":8939.0,"score_lmetric":20084}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338963.558376784,"t_first_token_unix":1779769828.57232,"t_done":338963.610524104,"t_done_unix":1779769828.6244671},{"request_id":"1315567:2:1319894:346","session_id":"1315567","input_length":54546,"t_proxy_recv":338952.928651732,"t_decision_unix":1779769817.942593,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":17408,"affinity_cache_ratio":0.31914347523191433,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":17408,"estimated_new_tokens":37138,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":747,"cache_hit":17408,"new_prefill":37138,"score_linear":-17408.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":54546,"score_linear":62618.0,"score_lmetric":58284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":54546,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71854,"ongoing_decode_tokens":0,"pending_prefill_tokens":6830,"num_requests":1,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":54546,"score_linear":71854.0,"score_lmetric":61376},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":54546,"score_linear":79663.0,"score_lmetric":133697},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":7680,"new_prefill":46866,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181114,"ongoing_decode_tokens":181114,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":928,"cache_hit":7680,"new_prefill":46866,"score_linear":173434.0,"score_lmetric":93732},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21140,"ongoing_decode_tokens":0,"pending_prefill_tokens":14996,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":54546,"score_linear":21140.0,"score_lmetric":69542}],"chosen_score_linear":-17408.0,"chosen_score_lmetric":0,"t_first_token":338962.165756865,"t_first_token_unix":1779769827.179701,"t_done":338964.979682225,"t_done_unix":1779769829.9936264},{"request_id":"1322540:2:1323519:362","session_id":"1322540","input_length":9016,"t_proxy_recv":338965.897790691,"t_decision_unix":1779769830.911732,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":8704,"affinity_cache_ratio":0.9653948535936113,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (312 < 8000)","cache_hit":8704,"estimated_new_tokens":312,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":9016,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":9016,"score_linear":62618.0,"score_lmetric":9016},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":9016,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":979,"cache_hit":0,"new_prefill":9016,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":9016,"score_linear":79663.0,"score_lmetric":88167},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46190,"ongoing_decode_tokens":0,"pending_prefill_tokens":38510,"num_requests":2,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":9016,"score_linear":46190.0,"score_lmetric":95052},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":938,"cache_hit":0,"new_prefill":9016,"score_linear":184808.0,"score_lmetric":18032},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":8704,"new_prefill":312,"score_linear":-8704.0,"score_lmetric":0}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":338965.936623382,"t_first_token_unix":1779769830.9505672,"t_done":338966.622868686,"t_done_unix":1779769831.636812},{"request_id":"1316848:6:1323535:363","session_id":"1316848","input_length":24395,"t_proxy_recv":338965.952733486,"t_decision_unix":1779769830.966675,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":21504,"affinity_cache_ratio":0.8814921090387374,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2891 < 8000)","cache_hit":21504,"estimated_new_tokens":2891,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":24395,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":24395,"score_linear":62618.0,"score_lmetric":24395},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":24395,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":979,"cache_hit":0,"new_prefill":24395,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":24395,"score_linear":79663.0,"score_lmetric":103546},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46190,"ongoing_decode_tokens":0,"pending_prefill_tokens":38510,"num_requests":2,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":24395,"score_linear":46190.0,"score_lmetric":125810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":938,"cache_hit":0,"new_prefill":24395,"score_linear":184808.0,"score_lmetric":48790},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":9016,"ongoing_decode_tokens":9016,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":21504,"new_prefill":2891,"score_linear":-12488.0,"score_lmetric":2891}],"chosen_score_linear":-12488.0,"chosen_score_lmetric":2891,"t_first_token":338966.531184363,"t_first_token_unix":1779769831.5451288,"t_done":338967.142122087,"t_done_unix":1779769832.1560652},{"request_id":"1321042:2:1323492:361","session_id":"1321042","input_length":13008,"t_proxy_recv":338965.828933902,"t_decision_unix":1779769830.8428745,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7680,"affinity_cache_ratio":0.5904059040590406,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5328 < 8000)","cache_hit":7680,"estimated_new_tokens":5328,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":13008,"score_linear":62618.0,"score_lmetric":13008},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":979,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":13008,"score_linear":79663.0,"score_lmetric":92159},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":7680,"new_prefill":5328,"score_linear":25502.0,"score_lmetric":38510},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":938,"cache_hit":0,"new_prefill":13008,"score_linear":184808.0,"score_lmetric":26016},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":25502.0,"chosen_score_lmetric":38510,"t_first_token":338966.598552437,"t_first_token_unix":1779769831.6124966,"t_done":338967.16044016,"t_done_unix":1779769832.1743882},{"request_id":"1320630:2:1323748:364","session_id":"1320630","input_length":11298,"t_proxy_recv":338966.670755631,"t_decision_unix":1779769831.6846972,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":2,"affinity_cache_hit":3072,"affinity_cache_ratio":0.27190653212958044,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11298,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":11298,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":11298,"score_linear":62618.0,"score_lmetric":11298},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":11298,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":979,"cache_hit":0,"new_prefill":11298,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":11298,"score_linear":79663.0,"score_lmetric":11298},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46190,"ongoing_decode_tokens":46190,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":932,"cache_hit":3072,"new_prefill":8226,"score_linear":43118.0,"score_lmetric":16452},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":938,"cache_hit":0,"new_prefill":11298,"score_linear":184808.0,"score_lmetric":22596},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24395,"ongoing_decode_tokens":24395,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":11298,"score_linear":24395.0,"score_lmetric":11298}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338967.59894581,"t_first_token_unix":1779769832.6128895,"t_done":338967.880468694,"t_done_unix":1779769832.894412},{"request_id":"1324227:1:1324227:367","session_id":"1324227","input_length":14863,"t_proxy_recv":338968.460459013,"t_decision_unix":1779769833.4744,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-3565,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":14863,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":14863,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":14863,"score_linear":62618.0,"score_lmetric":14863},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1061,"cache_hit":0,"new_prefill":14863,"score_linear":-11298.0,"score_lmetric":-3565},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75624,"ongoing_decode_tokens":75624,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":979,"cache_hit":0,"new_prefill":14863,"score_linear":75624.0,"score_lmetric":14863},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":14863,"score_linear":79663.0,"score_lmetric":14863},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":14863,"score_linear":33182.0,"score_lmetric":14863},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":938,"cache_hit":0,"new_prefill":14863,"score_linear":184808.0,"score_lmetric":29726},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":957,"cache_hit":0,"new_prefill":14863,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-3565,"t_first_token":338969.77744043,"t_first_token_unix":1779769834.7913845,"t_done":338970.027944377,"t_done_unix":1779769835.041888},{"request_id":"1269373:12:1312836:316","session_id":"1269373","input_length":62840,"t_proxy_recv":338963.538443328,"t_decision_unix":1779769828.5523849,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":59904,"affinity_cache_ratio":0.9532781667727562,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2936 < 8000)","cache_hit":59904,"estimated_new_tokens":2936,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":7680,"new_prefill":55160,"score_linear":46866.0,"score_lmetric":55160},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":62840,"score_linear":62618.0,"score_lmetric":62840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":62840,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":11145,"ongoing_decode_tokens":0,"pending_prefill_tokens":11145,"num_requests":1,"active_p_offloads":0,"cached_blocks":958,"cache_hit":0,"new_prefill":62840,"score_linear":11145.0,"score_lmetric":73985},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":62840,"score_linear":79663.0,"score_lmetric":141991},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":7680,"new_prefill":55160,"score_linear":25502.0,"score_lmetric":88342},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":938,"cache_hit":59904,"new_prefill":2936,"score_linear":62064.0,"score_lmetric":2936},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":62840,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":62064.0,"chosen_score_lmetric":2936,"t_first_token":338964.819177351,"t_first_token_unix":1779769829.8331215,"t_done":338970.761322035,"t_done_unix":1779769835.775266},{"request_id":"1279412:16:1323780:365","session_id":"1279412","input_length":75624,"t_proxy_recv":338966.767941504,"t_decision_unix":1779769831.781882,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":72192,"affinity_cache_ratio":0.9546175817200888,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3432 < 8000)","cache_hit":72192,"estimated_new_tokens":3432,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":75624,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":75624,"score_linear":62618.0,"score_lmetric":75624},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":11298,"num_requests":1,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":75624,"score_linear":11298.0,"score_lmetric":86922},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":979,"cache_hit":72192,"new_prefill":3432,"score_linear":-72192.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":75624,"score_linear":79663.0,"score_lmetric":75624},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46190,"ongoing_decode_tokens":46190,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":75624,"score_linear":46190.0,"score_lmetric":151248},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":938,"cache_hit":0,"new_prefill":75624,"score_linear":184808.0,"score_lmetric":151248},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24395,"ongoing_decode_tokens":24395,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":12800,"new_prefill":62824,"score_linear":11595.0,"score_lmetric":62824}],"chosen_score_linear":-72192.0,"chosen_score_lmetric":0,"t_first_token":338968.27515948,"t_first_token_unix":1779769833.289103,"t_done":338970.809859726,"t_done_unix":1779769835.8238034},{"request_id":"1316848:7:1325031:371","session_id":"1316848","input_length":26994,"t_proxy_recv":338971.122908721,"t_decision_unix":1779769836.1368494,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24064,"affinity_cache_ratio":0.8914573608950137,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2930 < 8000)","cache_hit":24064,"estimated_new_tokens":2930,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":26994,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":26994,"score_linear":62618.0,"score_lmetric":26994},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":26994,"score_linear":-11298.0,"score_lmetric":-15696},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":26994,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":26994,"score_linear":79663.0,"score_lmetric":26994},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":26994,"score_linear":33182.0,"score_lmetric":26994},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":121968,"pending_prefill_tokens":3159,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":26994,"score_linear":188103.0,"score_lmetric":60306},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":4312,"ongoing_decode_tokens":0,"pending_prefill_tokens":4312,"num_requests":1,"active_p_offloads":0,"cached_blocks":957,"cache_hit":24064,"new_prefill":2930,"score_linear":-19752.0,"score_lmetric":7242}],"chosen_score_linear":-19752.0,"chosen_score_lmetric":7242,"t_first_token":338971.742924621,"t_first_token_unix":1779769836.7568681,"t_done":338972.472855822,"t_done_unix":1779769837.486799},{"request_id":"1322540:3:1325410:373","session_id":"1322540","input_length":12118,"t_proxy_recv":338972.499143931,"t_decision_unix":1779769837.5130854,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":8704,"affinity_cache_ratio":0.7182703416405347,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3414 < 8000)","cache_hit":8704,"estimated_new_tokens":3414,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":12118,"score_linear":55130.0,"score_lmetric":12118},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":12118,"score_linear":62618.0,"score_lmetric":12118},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":12118,"score_linear":-11298.0,"score_lmetric":-820},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":12118,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":12118,"score_linear":79663.0,"score_lmetric":12118},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":12118,"score_linear":33182.0,"score_lmetric":12118},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":12118,"score_linear":188103.0,"score_lmetric":24236},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":4312,"ongoing_decode_tokens":4312,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":962,"cache_hit":8704,"new_prefill":3414,"score_linear":-4392.0,"score_lmetric":3414}],"chosen_score_linear":-4392.0,"chosen_score_lmetric":3414,"t_first_token":338972.878079269,"t_first_token_unix":1779769837.892024,"t_done":338973.04080607,"t_done_unix":1779769838.05475},{"request_id":"1317598:1:1317598:335","session_id":"1317598","input_length":79663,"t_proxy_recv":338944.975324795,"t_decision_unix":1779769809.9892662,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":512,"estimated_new_tokens":79151,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":53467,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":79663,"score_linear":57051.0,"score_lmetric":133130},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1030,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":512,"new_prefill":79151,"score_linear":-512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":901,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":79663,"score_linear":176419.0,"score_lmetric":159326},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":338966.648874433,"t_first_token_unix":1779769831.6628182,"t_done":338973.900127636,"t_done_unix":1779769838.9140718},{"request_id":"1324964:1:1324964:370","session_id":"1324964","input_length":4312,"t_proxy_recv":338970.877073121,"t_decision_unix":1779769835.8910148,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4312 < 8000)","cache_hit":0,"estimated_new_tokens":4312,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":4312,"score_linear":62618.0,"score_lmetric":4312},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":4312,"score_linear":-11298.0,"score_lmetric":6986},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":4312,"score_linear":79663.0,"score_lmetric":4312},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":4312,"score_linear":33182.0,"score_lmetric":4312},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":121968,"pending_prefill_tokens":3159,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":4312,"score_linear":188103.0,"score_lmetric":14942},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":957,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338971.156967626,"t_first_token_unix":1779769836.1709116,"t_done":338973.960502799,"t_done_unix":1779769838.9744465},{"request_id":"1326192:1:1326192:378","session_id":"1326192","input_length":9513,"t_proxy_recv":338975.473521541,"t_decision_unix":1779769840.4874628,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9513,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":9513,"score_linear":55130.0,"score_lmetric":9513},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":9513,"score_linear":62618.0,"score_lmetric":9513},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":9513,"score_linear":57228.0,"score_lmetric":66741},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":0,"pending_prefill_tokens":5232,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":9513,"score_linear":81008.0,"score_lmetric":14745},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":921,"cache_hit":0,"new_prefill":9513,"score_linear":81220.0,"score_lmetric":9513},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":9513,"score_linear":33182.0,"score_lmetric":9513},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":9513,"score_linear":188103.0,"score_lmetric":19026},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":977,"cache_hit":0,"new_prefill":9513,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338976.21891766,"t_first_token_unix":1779769841.232862,"t_done":338977.241480918,"t_done_unix":1779769842.2554247},{"request_id":"1317598:2:1324364:368","session_id":"1317598","input_length":81220,"t_proxy_recv":338973.914442301,"t_decision_unix":1779769838.928384,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":80384,"affinity_cache_ratio":0.9897069687269145,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (836 < 8000)","cache_hit":80384,"estimated_new_tokens":836,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":81220,"score_linear":55130.0,"score_lmetric":81220},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":81220,"score_linear":62618.0,"score_lmetric":81220},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":55405,"ongoing_decode_tokens":0,"pending_prefill_tokens":55405,"num_requests":0,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":81220,"score_linear":55405.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":81220,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":921,"cache_hit":80384,"new_prefill":836,"score_linear":-80384.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":81220,"score_linear":33182.0,"score_lmetric":81220},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":81220,"score_linear":188103.0,"score_lmetric":162440},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":4312,"ongoing_decode_tokens":4312,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":81220,"score_linear":4312.0,"score_lmetric":81220}],"chosen_score_linear":-80384.0,"chosen_score_lmetric":0,"t_first_token":338974.493044185,"t_first_token_unix":1779769839.506988,"t_done":338977.362683898,"t_done_unix":1779769842.3766274},{"request_id":"1322540:4:1326861:380","session_id":"1322540","input_length":13249,"t_proxy_recv":338977.745795247,"t_decision_unix":1779769842.7597365,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":11776,"affinity_cache_ratio":0.8888217978715375,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1473 < 8000)","cache_hit":11776,"estimated_new_tokens":1473,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":13249,"score_linear":55130.0,"score_lmetric":13249},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":13249,"score_linear":62618.0,"score_lmetric":13249},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":13249,"score_linear":57228.0,"score_lmetric":70477},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":0,"pending_prefill_tokens":5232,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":13249,"score_linear":81008.0,"score_lmetric":18481},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":13249,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":13249,"score_linear":33182.0,"score_lmetric":13249},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":13249,"score_linear":188103.0,"score_lmetric":26498},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28120,"ongoing_decode_tokens":28120,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":995,"cache_hit":11776,"new_prefill":1473,"score_linear":16344.0,"score_lmetric":1473}],"chosen_score_linear":16344.0,"chosen_score_lmetric":1473,"t_first_token":338977.935157222,"t_first_token_unix":1779769842.9491007,"t_done":338978.123878152,"t_done_unix":1779769843.1378217},{"request_id":"1316848:8:1326380:379","session_id":"1316848","input_length":28120,"t_proxy_recv":338976.080590537,"t_decision_unix":1779769841.0945315,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9467994310099573,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1496 < 8000)","cache_hit":26624,"estimated_new_tokens":1496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":28120,"score_linear":55130.0,"score_lmetric":28120},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":28120,"score_linear":62618.0,"score_lmetric":28120},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":28120,"score_linear":57228.0,"score_lmetric":85348},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":0,"pending_prefill_tokens":5232,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":28120,"score_linear":81008.0,"score_lmetric":33352},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":921,"cache_hit":0,"new_prefill":28120,"score_linear":81220.0,"score_lmetric":28120},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":28120,"score_linear":33182.0,"score_lmetric":28120},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":28120,"score_linear":188103.0,"score_lmetric":56240},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":9513,"ongoing_decode_tokens":0,"pending_prefill_tokens":9513,"num_requests":1,"active_p_offloads":0,"cached_blocks":977,"cache_hit":26624,"new_prefill":1496,"score_linear":-17111.0,"score_lmetric":11009}],"chosen_score_linear":-17111.0,"chosen_score_lmetric":11009,"t_first_token":338976.49122971,"t_first_token_unix":1779769841.5051732,"t_done":338978.732669393,"t_done_unix":1779769843.746613},{"request_id":"1269373:13:1317516:334","session_id":"1269373","input_length":66135,"t_proxy_recv":338970.772140514,"t_decision_unix":1779769835.786082,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9522340666817872,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3159 < 8000)","cache_hit":62976,"estimated_new_tokens":3159,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":7680,"new_prefill":58455,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":66135,"score_linear":62618.0,"score_lmetric":66135},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":66135,"score_linear":-11298.0,"score_lmetric":-54837},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75624,"ongoing_decode_tokens":75624,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":979,"cache_hit":0,"new_prefill":66135,"score_linear":75624.0,"score_lmetric":66135},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":66135,"score_linear":79663.0,"score_lmetric":66135},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":7680,"new_prefill":58455,"score_linear":25502.0,"score_lmetric":58455},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":944,"cache_hit":62976,"new_prefill":3159,"score_linear":58992.0,"score_lmetric":3159},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":957,"cache_hit":0,"new_prefill":66135,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":58992.0,"chosen_score_lmetric":3159,"t_first_token":338972.113898323,"t_first_token_unix":1779769837.1278417,"t_done":338979.553877244,"t_done_unix":1779769844.5678213},{"request_id":"1326192:2:1327234:381","session_id":"1326192","input_length":15253,"t_proxy_recv":338979.02816909,"t_decision_unix":1779769844.0421104,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":9216,"affinity_cache_ratio":0.6042090080639874,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6037 < 8000)","cache_hit":9216,"estimated_new_tokens":6037,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":15253,"score_linear":55130.0,"score_lmetric":15253},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":15253,"score_linear":62618.0,"score_lmetric":15253},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":15253,"score_linear":57228.0,"score_lmetric":72481},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":15253,"score_linear":81008.0,"score_lmetric":15253},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":15253,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":15253,"score_linear":33182.0,"score_lmetric":15253},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":15253,"score_linear":188103.0,"score_lmetric":30506},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1000,"cache_hit":9216,"new_prefill":6037,"score_linear":-9216.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":338979.718913808,"t_first_token_unix":1779769844.7328577,"t_done":338980.240024912,"t_done_unix":1779769845.253969},{"request_id":"1324964:2:1327382:382","session_id":"1324964","input_length":5146,"t_proxy_recv":338979.631091406,"t_decision_unix":1779769844.645033,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":4608,"affinity_cache_ratio":0.895452778857365,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (538 < 8000)","cache_hit":4608,"estimated_new_tokens":538,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":5146,"score_linear":55130.0,"score_lmetric":5146},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":5146,"score_linear":62618.0,"score_lmetric":5146},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":5146,"score_linear":57228.0,"score_lmetric":62374},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":5146,"score_linear":81008.0,"score_lmetric":5146},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":5146,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":5146,"score_linear":33182.0,"score_lmetric":5146},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188808,"ongoing_decode_tokens":121968,"pending_prefill_tokens":280,"num_requests":2,"active_p_offloads":0,"cached_blocks":951,"cache_hit":0,"new_prefill":5146,"score_linear":188808.0,"score_lmetric":10852},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":15253,"ongoing_decode_tokens":0,"pending_prefill_tokens":6037,"num_requests":1,"active_p_offloads":0,"cached_blocks":1000,"cache_hit":4608,"new_prefill":538,"score_linear":10645.0,"score_lmetric":6575}],"chosen_score_linear":10645.0,"chosen_score_lmetric":6575,"t_first_token":338979.798586732,"t_first_token_unix":1779769844.8125303,"t_done":338980.349255114,"t_done_unix":1779769845.3631988},{"request_id":"1269373:14:1321512:354","session_id":"1269373","input_length":66840,"t_proxy_recv":338979.564404289,"t_decision_unix":1779769844.578346,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":66560,"affinity_cache_ratio":0.9958108916816277,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (280 < 8000)","cache_hit":66560,"estimated_new_tokens":280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":7680,"new_prefill":59160,"score_linear":47450.0,"score_lmetric":59160},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":66840,"score_linear":62618.0,"score_lmetric":66840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":66840,"score_linear":57228.0,"score_lmetric":124068},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":66840,"score_linear":81008.0,"score_lmetric":66840},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":66840,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":7680,"new_prefill":59160,"score_linear":25502.0,"score_lmetric":59160},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":951,"cache_hit":66560,"new_prefill":280,"score_linear":55408.0,"score_lmetric":280},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":15253,"ongoing_decode_tokens":0,"pending_prefill_tokens":6037,"num_requests":1,"active_p_offloads":0,"cached_blocks":1000,"cache_hit":0,"new_prefill":66840,"score_linear":15253.0,"score_lmetric":72877}],"chosen_score_linear":55408.0,"chosen_score_lmetric":280,"t_first_token":338979.799022258,"t_first_token_unix":1779769844.8129652,"t_done":338983.81983224,"t_done_unix":1779769848.8337758},{"request_id":"1324964:3:1328499:384","session_id":"1324964","input_length":5234,"t_proxy_recv":338983.648669351,"t_decision_unix":1779769848.6626105,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":5120,"affinity_cache_ratio":0.9782193351165457,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (114 < 8000)","cache_hit":5120,"estimated_new_tokens":114,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":5234,"score_linear":55130.0,"score_lmetric":5234},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":5234,"score_linear":62618.0,"score_lmetric":5234},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":5234,"score_linear":57228.0,"score_lmetric":62462},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":5234,"score_linear":81008.0,"score_lmetric":5234},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":5234,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":5234,"score_linear":33182.0,"score_lmetric":5234},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188808,"ongoing_decode_tokens":188808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":951,"cache_hit":0,"new_prefill":5234,"score_linear":188808.0,"score_lmetric":10468},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":5120,"new_prefill":114,"score_linear":-5120.0,"score_lmetric":0}],"chosen_score_linear":-5120.0,"chosen_score_lmetric":0,"t_first_token":338983.678152732,"t_first_token_unix":1779769848.692096,"t_done":338983.891070634,"t_done_unix":1779769848.905014},{"request_id":"1316848:9:1328693:386","session_id":"1316848","input_length":28615,"t_proxy_recv":338984.361997744,"t_decision_unix":1779769849.3759394,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28160,"affinity_cache_ratio":0.9840992486458151,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (455 < 8000)","cache_hit":28160,"estimated_new_tokens":455,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":28615,"score_linear":55130.0,"score_lmetric":28615},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":28615,"score_linear":62618.0,"score_lmetric":28615},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":28615,"score_linear":57228.0,"score_lmetric":85843},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":28615,"score_linear":81008.0,"score_lmetric":28615},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81827,"ongoing_decode_tokens":0,"pending_prefill_tokens":419,"num_requests":1,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":28615,"score_linear":81827.0,"score_lmetric":29034},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":28615,"score_linear":33182.0,"score_lmetric":28615},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":121968,"pending_prefill_tokens":1969,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":28615,"score_linear":191009.0,"score_lmetric":61168},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":28160,"new_prefill":455,"score_linear":-28160.0,"score_lmetric":0}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":338984.478463436,"t_first_token_unix":1779769849.492407,"t_done":338986.168684655,"t_done_unix":1779769851.1826289},{"request_id":"1317598:3:1328678:385","session_id":"1317598","input_length":81827,"t_proxy_recv":338984.324532701,"t_decision_unix":1779769849.338474,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":81408,"affinity_cache_ratio":0.9948794407713836,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (419 < 8000)","cache_hit":81408,"estimated_new_tokens":419,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":81827,"score_linear":55130.0,"score_lmetric":81827},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":81827,"score_linear":62618.0,"score_lmetric":81827},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":81827,"score_linear":57228.0,"score_lmetric":139055},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":81827,"score_linear":81008.0,"score_lmetric":81827},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":923,"cache_hit":81408,"new_prefill":419,"score_linear":-81408.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":81827,"score_linear":33182.0,"score_lmetric":81827},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":121968,"pending_prefill_tokens":1969,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":81827,"score_linear":191009.0,"score_lmetric":167592},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":81827,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-81408.0,"chosen_score_lmetric":0,"t_first_token":338984.734735513,"t_first_token_unix":1779769849.7486794,"t_done":338986.242360593,"t_done_unix":1779769851.256304},{"request_id":"1324964:4:1329201:390","session_id":"1324964","input_length":5414,"t_proxy_recv":338986.171351322,"t_decision_unix":1779769851.1852927,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":5120,"affinity_cache_ratio":0.9456963428149243,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (294 < 8000)","cache_hit":5120,"estimated_new_tokens":294,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":5414,"score_linear":55130.0,"score_lmetric":5414},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99498,"ongoing_decode_tokens":62618,"pending_prefill_tokens":36880,"num_requests":2,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":5414,"score_linear":99498.0,"score_lmetric":84588},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":5414,"score_linear":57228.0,"score_lmetric":62642},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":115272,"ongoing_decode_tokens":81008,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":5414,"score_linear":115272.0,"score_lmetric":26108},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81827,"ongoing_decode_tokens":81827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":5414,"score_linear":81827.0,"score_lmetric":5414},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":42474,"ongoing_decode_tokens":33182,"pending_prefill_tokens":9292,"num_requests":2,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":5414,"score_linear":42474.0,"score_lmetric":29412},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":5414,"score_linear":191009.0,"score_lmetric":10828},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1013,"cache_hit":5120,"new_prefill":294,"score_linear":-5120.0,"score_lmetric":0}],"chosen_score_linear":-5120.0,"chosen_score_lmetric":0,"t_first_token":338986.220077721,"t_first_token_unix":1779769851.234022,"t_done":338986.629165497,"t_done_unix":1779769851.6431088},{"request_id":"1329110:1:1329110:389","session_id":"1329110","input_length":9292,"t_proxy_recv":338985.86097522,"t_decision_unix":1779769850.874916,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":9292,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":9292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":9292,"score_linear":55130.0,"score_lmetric":9292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99498,"ongoing_decode_tokens":62618,"pending_prefill_tokens":36880,"num_requests":2,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":9292,"score_linear":99498.0,"score_lmetric":92344},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":9292,"score_linear":57228.0,"score_lmetric":66520},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":115272,"ongoing_decode_tokens":81008,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":9292,"score_linear":115272.0,"score_lmetric":33864},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81827,"ongoing_decode_tokens":81827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":9292,"score_linear":81827.0,"score_lmetric":9292},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":9292,"score_linear":33182.0,"score_lmetric":9292},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":9292,"score_linear":191009.0,"score_lmetric":18584},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28615,"ongoing_decode_tokens":28615,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":9292,"score_linear":28615.0,"score_lmetric":9292}],"chosen_score_linear":33182.0,"chosen_score_lmetric":9292,"t_first_token":338986.575645571,"t_first_token_unix":1779769851.5895894,"t_done":338987.060988262,"t_done_unix":1779769852.0749314},{"request_id":"1322225:1:1322225:358","session_id":"1322225","input_length":33182,"t_proxy_recv":338961.424640918,"t_decision_unix":1779769826.4385822,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":33182,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":57223,"ongoing_decode_tokens":0,"pending_prefill_tokens":37255,"num_requests":2,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":33182,"score_linear":57223.0,"score_lmetric":140874},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":33182,"score_linear":62618.0,"score_lmetric":36920},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":0,"new_prefill":33182,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72402,"ongoing_decode_tokens":72402,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":957,"cache_hit":0,"new_prefill":33182,"score_linear":72402.0,"score_lmetric":33182},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":33182,"score_linear":79663.0,"score_lmetric":112333},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":33182,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":33182,"score_linear":181414.0,"score_lmetric":66364},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21471,"ongoing_decode_tokens":21471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":33182,"score_linear":21471.0,"score_lmetric":33182}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338966.088771139,"t_first_token_unix":1779769831.1027145,"t_done":338987.522637694,"t_done_unix":1779769852.5365813},{"request_id":"1279412:17:1326166:377","session_id":"1279412","input_length":81008,"t_proxy_recv":338975.317189731,"t_decision_unix":1779769840.331131,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9354137862927119,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5232 < 8000)","cache_hit":75776,"estimated_new_tokens":5232,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":81008,"score_linear":55130.0,"score_lmetric":81008},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":81008,"score_linear":62618.0,"score_lmetric":81008},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":81008,"score_linear":57228.0,"score_lmetric":138236},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":986,"cache_hit":75776,"new_prefill":5232,"score_linear":-75776.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":921,"cache_hit":0,"new_prefill":81008,"score_linear":81220.0,"score_lmetric":81008},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":81008,"score_linear":33182.0,"score_lmetric":81008},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":81008,"score_linear":188103.0,"score_lmetric":162016},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":977,"cache_hit":12800,"new_prefill":68208,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":338977.947822479,"t_first_token_unix":1779769842.9617662,"t_done":338988.313550612,"t_done_unix":1779769853.3274944},{"request_id":"1325541:1:1325541:374","session_id":"1325541","input_length":66703,"t_proxy_recv":338973.083867,"t_decision_unix":1779769838.0978084,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-55405,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":66703,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":66703,"score_linear":55130.0,"score_lmetric":66703},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":66703,"score_linear":62618.0,"score_lmetric":66703},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":66703,"score_linear":-11298.0,"score_lmetric":-55405},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":66703,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":66703,"score_linear":79663.0,"score_lmetric":66703},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":66703,"score_linear":33182.0,"score_lmetric":66703},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":66703,"score_linear":188103.0,"score_lmetric":133406},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":4312,"ongoing_decode_tokens":4312,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":66703,"score_linear":4312.0,"score_lmetric":66703}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-55405,"t_first_token":338988.760320106,"t_first_token_unix":1779769853.7742643,"t_done":338989.074144704,"t_done_unix":1779769854.0880883},{"request_id":"1325987:1:1325987:376","session_id":"1325987","input_length":1823,"t_proxy_recv":338974.587887923,"t_decision_unix":1779769839.6018293,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1823 < 8000)","cache_hit":0,"estimated_new_tokens":1823,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":1823,"score_linear":55130.0,"score_lmetric":1823},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":1823,"score_linear":62618.0,"score_lmetric":1823},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":55405,"ongoing_decode_tokens":0,"pending_prefill_tokens":55405,"num_requests":0,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":1823,"score_linear":55405.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":1823,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":921,"cache_hit":0,"new_prefill":1823,"score_linear":81220.0,"score_lmetric":1823},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":1823,"score_linear":33182.0,"score_lmetric":1823},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":0,"new_prefill":1823,"score_linear":188103.0,"score_lmetric":3646},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":977,"cache_hit":0,"new_prefill":1823,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":55405.0,"chosen_score_lmetric":0,"t_first_token":338988.76105391,"t_first_token_unix":1779769853.7749972,"t_done":338989.075524176,"t_done_unix":1779769854.0894673},{"request_id":"1329470:1:1329470:392","session_id":"1329470","input_length":10847,"t_proxy_recv":338987.193434022,"t_decision_unix":1779769852.2073753,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":10847,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":10847,"score_linear":55130.0,"score_lmetric":10847},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99498,"ongoing_decode_tokens":62618,"pending_prefill_tokens":36880,"num_requests":2,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":10847,"score_linear":99498.0,"score_lmetric":95454},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":10847,"score_linear":57228.0,"score_lmetric":68075},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":115272,"ongoing_decode_tokens":81008,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":10847,"score_linear":115272.0,"score_lmetric":36974},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":10847,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":960,"cache_hit":0,"new_prefill":10847,"score_linear":33182.0,"score_lmetric":10847},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":10847,"score_linear":191009.0,"score_lmetric":21694},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1013,"cache_hit":0,"new_prefill":10847,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338988.059834506,"t_first_token_unix":1779769853.073778,"t_done":338989.573622488,"t_done_unix":1779769854.5875664},{"request_id":"1325987:2:1329336:391","session_id":"1325987","input_length":7423,"t_proxy_recv":338989.081109216,"t_decision_unix":1779769854.0950508,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":5,"affinity_cache_hit":1536,"affinity_cache_ratio":0.20692442408729625,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7423 < 8000)","cache_hit":0,"estimated_new_tokens":7423,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":7423,"score_linear":55130.0,"score_lmetric":7423},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99498,"ongoing_decode_tokens":62618,"pending_prefill_tokens":36880,"num_requests":2,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":7423,"score_linear":99498.0,"score_lmetric":88606},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":1536,"new_prefill":5887,"score_linear":-12834.0,"score_lmetric":5411},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":34264,"ongoing_decode_tokens":34264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":997,"cache_hit":0,"new_prefill":7423,"score_linear":34264.0,"score_lmetric":7423},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":7423,"score_linear":90605.0,"score_lmetric":7916},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":7423,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":7423,"score_linear":191009.0,"score_lmetric":14846},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":10847,"ongoing_decode_tokens":10847,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1013,"cache_hit":0,"new_prefill":7423,"score_linear":10847.0,"score_lmetric":7423}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338989.594108716,"t_first_token_unix":1779769854.6080523,"t_done":338989.79437221,"t_done_unix":1779769854.8083162},{"request_id":"1324964:5:1330169:397","session_id":"1324964","input_length":6691,"t_proxy_recv":338989.565590703,"t_decision_unix":1779769854.5795321,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":5120,"affinity_cache_ratio":0.7652069944701838,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1571 < 8000)","cache_hit":5120,"estimated_new_tokens":1571,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":6691,"score_linear":55130.0,"score_lmetric":6691},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99498,"ongoing_decode_tokens":62618,"pending_prefill_tokens":36880,"num_requests":2,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":6691,"score_linear":99498.0,"score_lmetric":87142},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":427,"ongoing_decode_tokens":0,"pending_prefill_tokens":427,"num_requests":0,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":6691,"score_linear":427.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":34264,"ongoing_decode_tokens":34264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":997,"cache_hit":0,"new_prefill":6691,"score_linear":34264.0,"score_lmetric":6691},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":6691,"score_linear":90605.0,"score_lmetric":7184},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":7423,"ongoing_decode_tokens":0,"pending_prefill_tokens":7423,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":6691,"score_linear":7423.0,"score_lmetric":14114},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":6691,"score_linear":191009.0,"score_lmetric":13382},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":10847,"ongoing_decode_tokens":10847,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1013,"cache_hit":5120,"new_prefill":1571,"score_linear":5727.0,"score_lmetric":1571}],"chosen_score_linear":5727.0,"chosen_score_lmetric":1571,"t_first_token":338989.703848891,"t_first_token_unix":1779769854.7177923,"t_done":338989.911671493,"t_done_unix":1779769854.925615},{"request_id":"1268861:5:1329060:388","session_id":"1268861","input_length":34264,"t_proxy_recv":338985.734417247,"t_decision_unix":1779769850.7483585,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":26624,"affinity_cache_ratio":0.7770254494513191,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7640 < 8000)","cache_hit":26624,"estimated_new_tokens":7640,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":34264,"score_linear":55130.0,"score_lmetric":34264},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99498,"ongoing_decode_tokens":62618,"pending_prefill_tokens":36880,"num_requests":2,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":34264,"score_linear":99498.0,"score_lmetric":142288},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":34264,"score_linear":57228.0,"score_lmetric":91492},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":26624,"new_prefill":7640,"score_linear":54384.0,"score_lmetric":7640},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81827,"ongoing_decode_tokens":81827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":34264,"score_linear":81827.0,"score_lmetric":34264},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":34264,"score_linear":33182.0,"score_lmetric":34264},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":34264,"score_linear":191009.0,"score_lmetric":68528},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28615,"ongoing_decode_tokens":28615,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":34264,"score_linear":28615.0,"score_lmetric":34264}],"chosen_score_linear":54384.0,"chosen_score_lmetric":7640,"t_first_token":338987.359242586,"t_first_token_unix":1779769852.373186,"t_done":338990.570296262,"t_done_unix":1779769855.58424},{"request_id":"1328753:1:1328753:387","session_id":"1328753","input_length":36880,"t_proxy_recv":338984.599440279,"t_decision_unix":1779769849.6133816,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":36880,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":36880,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":36880,"score_linear":55130.0,"score_lmetric":36880},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":36880,"score_linear":62618.0,"score_lmetric":36880},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":36880,"score_linear":57228.0,"score_lmetric":94108},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":36880,"score_linear":81008.0,"score_lmetric":36880},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81827,"ongoing_decode_tokens":0,"pending_prefill_tokens":419,"num_requests":1,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":36880,"score_linear":81827.0,"score_lmetric":37299},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":0,"new_prefill":36880,"score_linear":33182.0,"score_lmetric":36880},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":121968,"pending_prefill_tokens":1969,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":36880,"score_linear":191009.0,"score_lmetric":77698},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28615,"ongoing_decode_tokens":28615,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":36880,"score_linear":28615.0,"score_lmetric":36880}],"chosen_score_linear":62618.0,"chosen_score_lmetric":36880,"t_first_token":338990.180531979,"t_first_token_unix":1779769855.194476,"t_done":338990.801915959,"t_done_unix":1779769855.8158596},{"request_id":"1330130:1:1330130:396","session_id":"1330130","input_length":11725,"t_proxy_recv":338989.463874825,"t_decision_unix":1779769854.4778159,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-427,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11725,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":11725,"score_linear":55130.0,"score_lmetric":11725},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99498,"ongoing_decode_tokens":62618,"pending_prefill_tokens":36880,"num_requests":2,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":11725,"score_linear":99498.0,"score_lmetric":97210},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":11725,"score_linear":-11298.0,"score_lmetric":-427},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":34264,"ongoing_decode_tokens":34264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":997,"cache_hit":0,"new_prefill":11725,"score_linear":34264.0,"score_lmetric":11725},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":11725,"score_linear":90605.0,"score_lmetric":12218},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":7423,"ongoing_decode_tokens":0,"pending_prefill_tokens":7423,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":11725,"score_linear":7423.0,"score_lmetric":19148},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":11725,"score_linear":191009.0,"score_lmetric":23450},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":10847,"ongoing_decode_tokens":10847,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1013,"cache_hit":0,"new_prefill":11725,"score_linear":10847.0,"score_lmetric":11725}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-427,"t_first_token":338990.412876236,"t_first_token_unix":1779769855.4268205,"t_done":338990.870841098,"t_done_unix":1779769855.8847847},{"request_id":"1328753:2:1330033:395","session_id":"1328753","input_length":37007,"t_proxy_recv":338990.809635395,"t_decision_unix":1779769855.8235772,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9961358661874781,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (143 < 8000)","cache_hit":36864,"estimated_new_tokens":143,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":37007,"score_linear":55130.0,"score_lmetric":37007},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":36864,"new_prefill":143,"score_linear":25754.0,"score_lmetric":143},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":427,"ongoing_decode_tokens":11725,"pending_prefill_tokens":-11298,"num_requests":0,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":37007,"score_linear":427.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":37007,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":37007,"score_linear":90605.0,"score_lmetric":37500},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1040,"cache_hit":0,"new_prefill":37007,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":37007,"score_linear":191009.0,"score_lmetric":74014},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":37007,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":25754.0,"chosen_score_lmetric":143,"t_first_token":338990.920590809,"t_first_token_unix":1779769855.934534,"t_done":338991.652315751,"t_done_unix":1779769856.6662595},{"request_id":"1330881:1:1330881:400","session_id":"1330881","input_length":3866,"t_proxy_recv":338992.221715448,"t_decision_unix":1779769857.235657,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3866 < 8000)","cache_hit":0,"estimated_new_tokens":3866,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":3866,"score_linear":55130.0,"score_lmetric":3866},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":3866,"score_linear":62618.0,"score_lmetric":3866},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":3866,"score_linear":-11298.0,"score_lmetric":7432},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":3866,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":3866,"score_linear":90605.0,"score_lmetric":4359},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1040,"cache_hit":0,"new_prefill":3866,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":3866,"score_linear":191009.0,"score_lmetric":7732},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":54665,"ongoing_decode_tokens":28977,"pending_prefill_tokens":10840,"num_requests":2,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":0,"new_prefill":3866,"score_linear":54665.0,"score_lmetric":29412}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338992.463451544,"t_first_token_unix":1779769857.4773953,"t_done":338992.828598287,"t_done_unix":1779769857.8425422},{"request_id":"1326192:3:1330643:399","session_id":"1326192","input_length":25688,"t_proxy_recv":338991.364706108,"t_decision_unix":1779769856.3786476,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":14848,"affinity_cache_ratio":0.5780130800373715,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":14848,"estimated_new_tokens":10840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":25688,"score_linear":55130.0,"score_lmetric":25688},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99625,"ongoing_decode_tokens":99625,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":25688,"score_linear":99625.0,"score_lmetric":51376},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":25688,"score_linear":-11298.0,"score_lmetric":-14390},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":25688,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":25688,"score_linear":90605.0,"score_lmetric":26181},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1040,"cache_hit":0,"new_prefill":25688,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":25688,"score_linear":191009.0,"score_lmetric":51376},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":14848,"new_prefill":10840,"score_linear":14129.0,"score_lmetric":10840}],"chosen_score_linear":14129.0,"chosen_score_lmetric":10840,"t_first_token":338993.110746925,"t_first_token_unix":1779769858.124691,"t_done":338993.493696146,"t_done_unix":1779769858.5076401},{"request_id":"1269373:15:1325630:375","session_id":"1269373","input_length":69041,"t_proxy_recv":338983.832500849,"t_decision_unix":1779769848.8464422,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":67072,"affinity_cache_ratio":0.971480714358135,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1969 < 8000)","cache_hit":67072,"estimated_new_tokens":1969,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":7680,"new_prefill":61361,"score_linear":47450.0,"score_lmetric":61361},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":69041,"score_linear":62618.0,"score_lmetric":69041},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":69041,"score_linear":57228.0,"score_lmetric":126269},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":69041,"score_linear":81008.0,"score_lmetric":69041},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":923,"cache_hit":0,"new_prefill":69041,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":7680,"new_prefill":61361,"score_linear":25502.0,"score_lmetric":61361},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":952,"cache_hit":67072,"new_prefill":1969,"score_linear":54896.0,"score_lmetric":1969},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":5234,"ongoing_decode_tokens":5234,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":69041,"score_linear":5234.0,"score_lmetric":69041}],"chosen_score_linear":54896.0,"chosen_score_lmetric":1969,"t_first_token":338984.845472992,"t_first_token_unix":1779769849.8594167,"t_done":338993.640138327,"t_done_unix":1779769858.6540823},{"request_id":"1326192:4:1331370:401","session_id":"1326192","input_length":27049,"t_proxy_recv":338993.99733571,"t_decision_unix":1779769859.011277,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":25600,"affinity_cache_ratio":0.9464305519612555,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1449 < 8000)","cache_hit":25600,"estimated_new_tokens":1449,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":27049,"score_linear":55130.0,"score_lmetric":27049},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":27049,"score_linear":62618.0,"score_lmetric":27049},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":27049,"score_linear":-11298.0,"score_lmetric":-15751},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":27049,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":27049,"score_linear":90605.0,"score_lmetric":27542},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1040,"cache_hit":0,"new_prefill":27049,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":27049,"score_linear":121968.0,"score_lmetric":27049},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":25600,"new_prefill":1449,"score_linear":3377.0,"score_lmetric":1449}],"chosen_score_linear":3377.0,"chosen_score_lmetric":1449,"t_first_token":338994.347477137,"t_first_token_unix":1779769859.3614209,"t_done":338994.803044206,"t_done_unix":1779769859.8169885},{"request_id":"1329110:2:1331399:402","session_id":"1329110","input_length":12781,"t_proxy_recv":338994.07534056,"t_decision_unix":1779769859.089282,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":9216,"affinity_cache_ratio":0.7210703387841327,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3565 < 8000)","cache_hit":9216,"estimated_new_tokens":3565,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":12781,"score_linear":55130.0,"score_lmetric":12781},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":12781,"score_linear":62618.0,"score_lmetric":12781},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":12781,"score_linear":-11298.0,"score_lmetric":-1483},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":12781,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":12781,"score_linear":90605.0,"score_lmetric":13274},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1040,"cache_hit":9216,"new_prefill":3565,"score_linear":-9216.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":12781,"score_linear":121968.0,"score_lmetric":12781},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":56026,"ongoing_decode_tokens":28977,"pending_prefill_tokens":1449,"num_requests":2,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":12781,"score_linear":56026.0,"score_lmetric":28460}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":338994.486950906,"t_first_token_unix":1779769859.5008945,"t_done":338994.860428069,"t_done_unix":1779769859.8743718},{"request_id":"1316848:10:1330509:398","session_id":"1316848","input_length":28977,"t_proxy_recv":338990.834348056,"t_decision_unix":1779769855.8482895,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9894744107395521,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (305 < 8000)","cache_hit":28672,"estimated_new_tokens":305,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":28977,"score_linear":55130.0,"score_lmetric":28977},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99625,"ongoing_decode_tokens":62618,"pending_prefill_tokens":143,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":28977,"score_linear":99625.0,"score_lmetric":58240},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":427,"ongoing_decode_tokens":11725,"pending_prefill_tokens":-11298,"num_requests":0,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":28977,"score_linear":427.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1012,"cache_hit":0,"new_prefill":28977,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":28977,"score_linear":90605.0,"score_lmetric":29470},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1040,"cache_hit":0,"new_prefill":28977,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":28977,"score_linear":191009.0,"score_lmetric":57954},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1037,"cache_hit":28672,"new_prefill":305,"score_linear":-28672.0,"score_lmetric":0}],"chosen_score_linear":-28672.0,"chosen_score_lmetric":0,"t_first_token":338990.928354573,"t_first_token_unix":1779769855.942298,"t_done":338996.184222218,"t_done_unix":1779769861.1981664},{"request_id":"1329470:2:1332133:409","session_id":"1329470","input_length":11766,"t_proxy_recv":338996.435139592,"t_decision_unix":1779769861.4490814,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9138194798572157,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1014 < 8000)","cache_hit":10752,"estimated_new_tokens":1014,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":11766,"score_linear":55130.0,"score_lmetric":11766},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":11766,"score_linear":62618.0,"score_lmetric":11766},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":76003,"ongoing_decode_tokens":14067,"pending_prefill_tokens":-4624,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":11766,"score_linear":76003.0,"score_lmetric":7142},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12421,"ongoing_decode_tokens":0,"pending_prefill_tokens":12421,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":11766,"score_linear":12421.0,"score_lmetric":24187},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":11766,"score_linear":172747.0,"score_lmetric":24962},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":8880,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":11766,"score_linear":21680.0,"score_lmetric":20646},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":193745,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2657,"num_requests":2,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":11766,"score_linear":193745.0,"score_lmetric":28846},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1061,"cache_hit":10752,"new_prefill":1014,"score_linear":-10752.0,"score_lmetric":0}],"chosen_score_linear":-10752.0,"chosen_score_lmetric":0,"t_first_token":338996.577966372,"t_first_token_unix":1779769861.5919101,"t_done":338996.639175966,"t_done_unix":1779769861.6531198},{"request_id":"1332021:1:1332021:407","session_id":"1332021","input_length":12421,"t_proxy_recv":338996.067316199,"t_decision_unix":1779769861.0812576,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":12421,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":12421,"score_linear":55130.0,"score_lmetric":12421},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":12421,"score_linear":62618.0,"score_lmetric":12421},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":76003,"ongoing_decode_tokens":14067,"pending_prefill_tokens":-4624,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":12421,"score_linear":76003.0,"score_lmetric":7797},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":12421,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":12421,"score_linear":90605.0,"score_lmetric":12914},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":8880,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":12421,"score_linear":21680.0,"score_lmetric":21301},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":193745,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2657,"num_requests":2,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":12421,"score_linear":193745.0,"score_lmetric":30156},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1060,"cache_hit":0,"new_prefill":12421,"score_linear":28977.0,"score_lmetric":12421}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338997.094252157,"t_first_token_unix":1779769862.1081955,"t_done":338997.349323771,"t_done_unix":1779769862.3632674},{"request_id":"1325541:2:1331846:404","session_id":"1325541","input_length":73234,"t_proxy_recv":338995.471719998,"t_decision_unix":1779769860.4856613,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":66560,"affinity_cache_ratio":0.9088674659311249,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6674 < 8000)","cache_hit":66560,"estimated_new_tokens":6674,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":73234,"score_linear":55130.0,"score_lmetric":73234},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":73234,"score_linear":62618.0,"score_lmetric":73234},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":2769,"ongoing_decode_tokens":0,"pending_prefill_tokens":-9007,"num_requests":0,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":66560,"new_prefill":6674,"score_linear":-63791.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":73234,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":73234,"score_linear":90605.0,"score_lmetric":73727},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":73234,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":73234,"score_linear":121968.0,"score_lmetric":73234},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1060,"cache_hit":0,"new_prefill":73234,"score_linear":28977.0,"score_lmetric":73234}],"chosen_score_linear":-63791.0,"chosen_score_lmetric":0,"t_first_token":338998.458696453,"t_first_token_unix":1779769863.47264,"t_done":338998.547240831,"t_done_unix":1779769863.5611844},{"request_id":"1332472:1:1332472:410","session_id":"1332472","input_length":1810,"t_proxy_recv":338997.724037354,"t_decision_unix":1779769862.737979,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1810 < 8000)","cache_hit":0,"estimated_new_tokens":1810,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":1810,"score_linear":55130.0,"score_lmetric":1810},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":1810,"score_linear":62618.0,"score_lmetric":1810},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":76003,"ongoing_decode_tokens":14067,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":1810,"score_linear":76003.0,"score_lmetric":1810},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":0,"new_prefill":1810,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":1810,"score_linear":172747.0,"score_lmetric":5050},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":8880,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":1810,"score_linear":21680.0,"score_lmetric":10690},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":193745,"ongoing_decode_tokens":193745,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":1810,"score_linear":193745.0,"score_lmetric":3620},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1063,"cache_hit":0,"new_prefill":1810,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":338997.845806849,"t_first_token_unix":1779769862.85975,"t_done":338998.547786668,"t_done_unix":1779769863.56173},{"request_id":"1330130:2:1331784:403","session_id":"1330130","input_length":14067,"t_proxy_recv":338995.268325938,"t_decision_unix":1779769860.2822673,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11776,"affinity_cache_ratio":0.8371365607450061,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2291 < 8000)","cache_hit":11776,"estimated_new_tokens":2291,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":14067,"score_linear":55130.0,"score_lmetric":14067},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":14067,"score_linear":62618.0,"score_lmetric":14067},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":11776,"new_prefill":2291,"score_linear":-23074.0,"score_lmetric":9007},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":14067,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":14067,"score_linear":90605.0,"score_lmetric":14560},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":14067,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":14067,"score_linear":121968.0,"score_lmetric":14067},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1060,"cache_hit":0,"new_prefill":14067,"score_linear":28977.0,"score_lmetric":14067}],"chosen_score_linear":-23074.0,"chosen_score_lmetric":9007,"t_first_token":338995.598074496,"t_first_token_unix":1779769860.6120183,"t_done":338998.614392761,"t_done_unix":1779769863.6283362},{"request_id":"1329110:3:1332476:411","session_id":"1329110","input_length":13479,"t_proxy_recv":338997.750918601,"t_decision_unix":1779769862.7648602,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":12800,"affinity_cache_ratio":0.9496253431263447,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (679 < 8000)","cache_hit":12800,"estimated_new_tokens":679,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":13479,"score_linear":55130.0,"score_lmetric":13479},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":13479,"score_linear":62618.0,"score_lmetric":13479},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":76003,"ongoing_decode_tokens":14067,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":13479,"score_linear":76003.0,"score_lmetric":13479},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":0,"new_prefill":13479,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":13479,"score_linear":172747.0,"score_lmetric":28388},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":8880,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":12800,"new_prefill":679,"score_linear":8880.0,"score_lmetric":9559},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":193745,"ongoing_decode_tokens":193745,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":13479,"score_linear":193745.0,"score_lmetric":26958},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":1810,"ongoing_decode_tokens":0,"pending_prefill_tokens":1810,"num_requests":1,"active_p_offloads":0,"cached_blocks":1063,"cache_hit":0,"new_prefill":13479,"score_linear":1810.0,"score_lmetric":15289}],"chosen_score_linear":8880.0,"chosen_score_lmetric":9559,"t_first_token":338998.496614179,"t_first_token_unix":1779769863.5105574,"t_done":338998.995755667,"t_done_unix":1779769864.0096989},{"request_id":"1332013:1:1332013:406","session_id":"1332013","input_length":21680,"t_proxy_recv":338996.052394522,"t_decision_unix":1779769861.0663357,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":12800,"estimated_new_tokens":8880,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":21680,"score_linear":55130.0,"score_lmetric":21680},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":21680,"score_linear":62618.0,"score_lmetric":21680},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":76003,"ongoing_decode_tokens":14067,"pending_prefill_tokens":-4624,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":12800,"new_prefill":8880,"score_linear":63203.0,"score_lmetric":4256},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":21680,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":21680,"score_linear":90605.0,"score_lmetric":22173},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":12800,"new_prefill":8880,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":193745,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2657,"num_requests":2,"active_p_offloads":0,"cached_blocks":956,"cache_hit":12800,"new_prefill":8880,"score_linear":180945.0,"score_lmetric":23074},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1060,"cache_hit":0,"new_prefill":21680,"score_linear":28977.0,"score_lmetric":21680}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":338998.376415644,"t_first_token_unix":1779769863.3903594,"t_done":338999.737205311,"t_done_unix":1779769864.7511487},{"request_id":"1332754:1:1332754:412","session_id":"1332754","input_length":19766,"t_proxy_recv":338998.751674575,"t_decision_unix":1779769863.7656162,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-13092,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":19766,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":19766,"score_linear":55130.0,"score_lmetric":19766},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":19766,"score_linear":62618.0,"score_lmetric":19766},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":19766,"score_linear":-11298.0,"score_lmetric":-13092},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":0,"new_prefill":19766,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":19766,"score_linear":172747.0,"score_lmetric":40962},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":35159,"ongoing_decode_tokens":35159,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":19766,"score_linear":35159.0,"score_lmetric":39532},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":193745,"ongoing_decode_tokens":193745,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":19766,"score_linear":193745.0,"score_lmetric":39532},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1066,"cache_hit":0,"new_prefill":19766,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-13092,"t_first_token":339000.770660007,"t_first_token_unix":1779769865.7846036,"t_done":339001.257779145,"t_done_unix":1779769866.2717228},{"request_id":"1269373:16:1331896:405","session_id":"1269373","input_length":71777,"t_proxy_recv":338995.667071798,"t_decision_unix":1779769860.6810133,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9629825710185714,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2657 < 8000)","cache_hit":69120,"estimated_new_tokens":2657,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":7680,"new_prefill":64097,"score_linear":47450.0,"score_lmetric":64097},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":71777,"score_linear":62618.0,"score_lmetric":71777},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":76003,"ongoing_decode_tokens":14067,"pending_prefill_tokens":-4624,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":71777,"score_linear":76003.0,"score_lmetric":67153},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":71777,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":71777,"score_linear":90605.0,"score_lmetric":72270},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":7680,"new_prefill":64097,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":956,"cache_hit":69120,"new_prefill":2657,"score_linear":52848.0,"score_lmetric":2657},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1060,"cache_hit":0,"new_prefill":71777,"score_linear":28977.0,"score_lmetric":71777}],"chosen_score_linear":52848.0,"chosen_score_lmetric":2657,"t_first_token":338996.846425266,"t_first_token_unix":1779769861.8603687,"t_done":339001.6025266,"t_done_unix":1779769866.6164703},{"request_id":"1279412:18:1332945:413","session_id":"1279412","input_length":82488,"t_proxy_recv":338999.588876197,"t_decision_unix":1779769864.6028175,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":81408,"affinity_cache_ratio":0.9869071864998545,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1080 < 8000)","cache_hit":81408,"estimated_new_tokens":1080,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":82488,"score_linear":55130.0,"score_lmetric":82488},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":82488,"score_linear":62618.0,"score_lmetric":82488},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":8468,"ongoing_decode_tokens":0,"pending_prefill_tokens":13092,"num_requests":0,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":82488,"score_linear":8468.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":81408,"new_prefill":1080,"score_linear":-81408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":82488,"score_linear":172747.0,"score_lmetric":166406},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21680,"ongoing_decode_tokens":21680,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1048,"cache_hit":0,"new_prefill":82488,"score_linear":21680.0,"score_lmetric":82488},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":193745,"ongoing_decode_tokens":193745,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":82488,"score_linear":193745.0,"score_lmetric":164976},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1066,"cache_hit":12800,"new_prefill":69688,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-81408.0,"chosen_score_lmetric":0,"t_first_token":339000.02666114,"t_first_token_unix":1779769865.0406044,"t_done":339002.796915272,"t_done_unix":1779769867.810859},{"request_id":"1316848:11:1333994:416","session_id":"1316848","input_length":29656,"t_proxy_recv":339003.126989028,"t_decision_unix":1779769868.1409304,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9840841650930672,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (472 < 8000)","cache_hit":29184,"estimated_new_tokens":472,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":29656,"score_linear":55130.0,"score_lmetric":29656},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":29656,"score_linear":62618.0,"score_lmetric":29656},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1301,"cache_hit":0,"new_prefill":29656,"score_linear":-11298.0,"score_lmetric":-22982},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":29656,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":29656,"score_linear":172747.0,"score_lmetric":60742},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":29656,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":961,"cache_hit":0,"new_prefill":29656,"score_linear":121968.0,"score_lmetric":29656},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1066,"cache_hit":29184,"new_prefill":472,"score_linear":-29184.0,"score_lmetric":0}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":339003.245432877,"t_first_token_unix":1779769868.259377,"t_done":339003.536059093,"t_done_unix":1779769868.5500026},{"request_id":"1334223:1:1334223:417","session_id":"1334223","input_length":23689,"t_proxy_recv":339003.998319101,"t_decision_unix":1779769869.0122602,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-17015,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":23689,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":23689,"score_linear":55130.0,"score_lmetric":23689},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":23689,"score_linear":62618.0,"score_lmetric":23689},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1301,"cache_hit":0,"new_prefill":23689,"score_linear":-11298.0,"score_lmetric":-17015},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":23689,"score_linear":172747.0,"score_lmetric":48808},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":961,"cache_hit":0,"new_prefill":23689,"score_linear":121968.0,"score_lmetric":23689},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1067,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-17015,"t_first_token":339006.670424719,"t_first_token_unix":1779769871.6843684,"t_done":339007.259331387,"t_done_unix":1779769872.2732747},{"request_id":"1316848:12:1334743:418","session_id":"1316848","input_length":44664,"t_proxy_recv":339005.976662574,"t_decision_unix":1779769870.9906044,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":29696,"affinity_cache_ratio":0.6648755149561167,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":29696,"estimated_new_tokens":14968,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":44664,"score_linear":55130.0,"score_lmetric":44664},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":108219,"ongoing_decode_tokens":62618,"pending_prefill_tokens":1057,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":44664,"score_linear":108219.0,"score_lmetric":91442},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":12391,"ongoing_decode_tokens":0,"pending_prefill_tokens":17015,"num_requests":0,"active_p_offloads":0,"cached_blocks":1301,"cache_hit":0,"new_prefill":44664,"score_linear":12391.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":44664,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":44664,"score_linear":172747.0,"score_lmetric":90758},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":44664,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":961,"cache_hit":0,"new_prefill":44664,"score_linear":121968.0,"score_lmetric":44664},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1067,"cache_hit":29696,"new_prefill":14968,"score_linear":-29696.0,"score_lmetric":0}],"chosen_score_linear":-29696.0,"chosen_score_lmetric":0,"t_first_token":339009.822890152,"t_first_token_unix":1779769874.8368335,"t_done":339010.771318969,"t_done_unix":1779769875.7852623},{"request_id":"1309252:2:1334746:419","session_id":"1309252","input_length":45601,"t_proxy_recv":339005.969031394,"t_decision_unix":1779769870.9829726,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":44544,"affinity_cache_ratio":0.9768206837569352,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1057 < 8000)","cache_hit":44544,"estimated_new_tokens":1057,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":45601,"score_linear":55130.0,"score_lmetric":45601},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":44544,"new_prefill":1057,"score_linear":18074.0,"score_lmetric":1057},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":12391,"ongoing_decode_tokens":0,"pending_prefill_tokens":17015,"num_requests":0,"active_p_offloads":0,"cached_blocks":1301,"cache_hit":0,"new_prefill":45601,"score_linear":12391.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":45601,"score_linear":172747.0,"score_lmetric":92632},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":961,"cache_hit":0,"new_prefill":45601,"score_linear":121968.0,"score_lmetric":45601},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1067,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":18074.0,"chosen_score_lmetric":1057,"t_first_token":339006.327712188,"t_first_token_unix":1779769871.3416557,"t_done":339010.821887775,"t_done_unix":1779769875.8358314},{"request_id":"1330881:2:1336222:422","session_id":"1330881","input_length":11989,"t_proxy_recv":339011.081947325,"t_decision_unix":1779769876.0958886,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":2,"affinity_cache_hit":3584,"affinity_cache_ratio":0.29894069563766784,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":-5315,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11989,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":11989,"score_linear":55130.0,"score_lmetric":11989},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":123847,"ongoing_decode_tokens":123847,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1127,"cache_hit":0,"new_prefill":11989,"score_linear":123847.0,"score_lmetric":23978},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1347,"cache_hit":0,"new_prefill":11989,"score_linear":-11298.0,"score_lmetric":-5315},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":87264,"ongoing_decode_tokens":87264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":3584,"new_prefill":8405,"score_linear":83680.0,"score_lmetric":8405},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":11989,"score_linear":172747.0,"score_lmetric":25408},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":11989,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":961,"cache_hit":0,"new_prefill":11989,"score_linear":121968.0,"score_lmetric":11989},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":11989,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-5315,"t_first_token":339012.066028738,"t_first_token_unix":1779769877.079972,"t_done":339012.227867304,"t_done_unix":1779769877.2418108},{"request_id":"1314357:2:1335184:420","session_id":"1314357","input_length":61229,"t_proxy_recv":339007.331760034,"t_decision_unix":1779769872.3457017,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":56832,"affinity_cache_ratio":0.9281876235117347,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4397 < 8000)","cache_hit":56832,"estimated_new_tokens":4397,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":61229,"score_linear":55130.0,"score_lmetric":61229},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":108219,"ongoing_decode_tokens":108219,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":56832,"new_prefill":4397,"score_linear":51387.0,"score_lmetric":8794},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1347,"cache_hit":0,"new_prefill":61229,"score_linear":-11298.0,"score_lmetric":-54555},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":61229,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":61229,"score_linear":172747.0,"score_lmetric":123888},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":61229,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":961,"cache_hit":0,"new_prefill":61229,"score_linear":121968.0,"score_lmetric":61229},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44664,"ongoing_decode_tokens":0,"pending_prefill_tokens":14968,"num_requests":1,"active_p_offloads":0,"cached_blocks":1067,"cache_hit":0,"new_prefill":61229,"score_linear":44664.0,"score_lmetric":76197}],"chosen_score_linear":51387.0,"chosen_score_lmetric":8794,"t_first_token":339008.927753876,"t_first_token_unix":1779769873.9416978,"t_done":339012.254068298,"t_done_unix":1779769877.2680118},{"request_id":"1279412:19:1335444:421","session_id":"1279412","input_length":87264,"t_proxy_recv":339008.259622694,"t_decision_unix":1779769873.273564,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":82432,"affinity_cache_ratio":0.9446277961129447,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4832 < 8000)","cache_hit":82432,"estimated_new_tokens":4832,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":87264,"score_linear":55130.0,"score_lmetric":87264},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":169448,"ongoing_decode_tokens":108219,"pending_prefill_tokens":4397,"num_requests":3,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":87264,"score_linear":169448.0,"score_lmetric":274983},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1347,"cache_hit":0,"new_prefill":87264,"score_linear":-11298.0,"score_lmetric":-80590},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":82432,"new_prefill":4832,"score_linear":-82432.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":87264,"score_linear":172747.0,"score_lmetric":175958},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":87264,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":961,"cache_hit":0,"new_prefill":87264,"score_linear":121968.0,"score_lmetric":87264},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":44664,"ongoing_decode_tokens":0,"pending_prefill_tokens":14968,"num_requests":1,"active_p_offloads":0,"cached_blocks":1067,"cache_hit":12800,"new_prefill":74464,"score_linear":31864.0,"score_lmetric":89432}],"chosen_score_linear":-82432.0,"chosen_score_lmetric":0,"t_first_token":339010.756329472,"t_first_token_unix":1779769875.7702773,"t_done":339012.745735981,"t_done_unix":1779769877.7596798},{"request_id":"1336620:1:1336620:423","session_id":"1336620","input_length":14666,"t_proxy_recv":339012.384079281,"t_decision_unix":1779769877.398021,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-7992,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":14666,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":7680,"new_prefill":6986,"score_linear":47450.0,"score_lmetric":6986},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":14666,"score_linear":62618.0,"score_lmetric":14666},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":14666,"score_linear":-11298.0,"score_lmetric":-7992},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":87264,"ongoing_decode_tokens":87264,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":14666,"score_linear":87264.0,"score_lmetric":14666},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":14666,"score_linear":172747.0,"score_lmetric":30762},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":7680,"new_prefill":6986,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":961,"cache_hit":7680,"new_prefill":6986,"score_linear":114288.0,"score_lmetric":6986},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":14666,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-7992,"t_first_token":339013.673707266,"t_first_token_unix":1779769878.687651,"t_done":339014.645457277,"t_done_unix":1779769879.6594007},{"request_id":"1337035:1:1337035:425","session_id":"1337035","input_length":12644,"t_proxy_recv":339013.87666544,"t_decision_unix":1779769878.8906069,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":12644,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":12644,"score_linear":55130.0,"score_lmetric":12644},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":12644,"score_linear":62618.0,"score_lmetric":12644},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":3368,"ongoing_decode_tokens":14666,"pending_prefill_tokens":-6674,"num_requests":0,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":12644,"score_linear":3368.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":12644,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":12644,"score_linear":172747.0,"score_lmetric":26718},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":12644,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":141977,"ongoing_decode_tokens":121968,"pending_prefill_tokens":5673,"num_requests":2,"active_p_offloads":0,"cached_blocks":961,"cache_hit":0,"new_prefill":12644,"score_linear":141977.0,"score_lmetric":36634},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":12644,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339014.931306989,"t_first_token_unix":1779769879.9452507,"t_done":339015.105747475,"t_done_unix":1779769880.119691},{"request_id":"1266668:12:1303535:276","session_id":"1266668","input_length":121968,"t_proxy_recv":338920.35609542,"t_decision_unix":1779769785.370037,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":97792,"affinity_cache_ratio":0.8017840745113473,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=7680 chosen=97792)","cache_hit":97792,"estimated_new_tokens":24176,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":659,"cache_hit":7680,"new_prefill":114288,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":24545,"ongoing_decode_tokens":24545,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":121968,"score_linear":24545.0,"score_lmetric":121968},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18089,"ongoing_decode_tokens":0,"pending_prefill_tokens":18089,"num_requests":1,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":121968,"score_linear":18089.0,"score_lmetric":140057},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":64558,"ongoing_decode_tokens":64558,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":854,"cache_hit":0,"new_prefill":121968,"score_linear":64558.0,"score_lmetric":121968},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":121968,"score_linear":43607.0,"score_lmetric":121968},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":865,"cache_hit":7680,"new_prefill":114288,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50680,"ongoing_decode_tokens":50680,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":97792,"new_prefill":24176,"score_linear":-47112.0,"score_lmetric":24176},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98957,"ongoing_decode_tokens":0,"pending_prefill_tokens":84109,"num_requests":2,"active_p_offloads":0,"cached_blocks":704,"cache_hit":0,"new_prefill":121968,"score_linear":98957.0,"score_lmetric":412154}],"chosen_score_linear":-47112.0,"chosen_score_lmetric":24176,"t_first_token":338936.334993279,"t_first_token_unix":1779769801.348937,"t_done":339015.136804184,"t_done_unix":1779769880.150748},{"request_id":"1257323:2:1329541:393","session_id":"1257323","input_length":90605,"t_proxy_recv":338987.640200178,"t_decision_unix":1779769852.6541417,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":90112,"affinity_cache_ratio":0.9945587991832681,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (493 < 8000)","cache_hit":90112,"estimated_new_tokens":493,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":90605,"score_linear":55130.0,"score_lmetric":90605},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99498,"ongoing_decode_tokens":62618,"pending_prefill_tokens":36880,"num_requests":2,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":90605,"score_linear":99498.0,"score_lmetric":254970},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":57228,"ongoing_decode_tokens":0,"pending_prefill_tokens":57228,"num_requests":1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":90605,"score_linear":57228.0,"score_lmetric":147833},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":115272,"ongoing_decode_tokens":115272,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":90605,"score_linear":115272.0,"score_lmetric":181210},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":924,"cache_hit":90112,"new_prefill":493,"score_linear":-90112.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":90605,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":952,"cache_hit":0,"new_prefill":90605,"score_linear":191009.0,"score_lmetric":181210},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":10847,"ongoing_decode_tokens":0,"pending_prefill_tokens":10847,"num_requests":1,"active_p_offloads":0,"cached_blocks":1013,"cache_hit":0,"new_prefill":90605,"score_linear":10847.0,"score_lmetric":101452}],"chosen_score_linear":-90112.0,"chosen_score_lmetric":0,"t_first_token":339015.046027356,"t_first_token_unix":1779769880.0599723,"t_done":339015.730959455,"t_done_unix":1779769880.7449033},{"request_id":"1259178:2:1336768:424","session_id":"1259178","input_length":20009,"t_proxy_recv":339012.849637181,"t_decision_unix":1779769877.8635788,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.716477585086711,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5673 < 8000)","cache_hit":14336,"estimated_new_tokens":5673,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":20009,"score_linear":55130.0,"score_lmetric":20009},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":20009,"score_linear":62618.0,"score_lmetric":20009},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":3368,"ongoing_decode_tokens":0,"pending_prefill_tokens":7992,"num_requests":0,"active_p_offloads":0,"cached_blocks":1370,"cache_hit":0,"new_prefill":20009,"score_linear":3368.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":20009,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":20009,"score_linear":172747.0,"score_lmetric":41448},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":20009,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":961,"cache_hit":14336,"new_prefill":5673,"score_linear":107632.0,"score_lmetric":5673},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":20009,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":107632.0,"chosen_score_lmetric":5673,"t_first_token":339014.968417337,"t_first_token_unix":1779769879.9823608,"t_done":339015.87175733,"t_done_unix":1779769880.8857012},{"request_id":"1316848:13:1337519:427","session_id":"1316848","input_length":45005,"t_proxy_recv":339015.717189547,"t_decision_unix":1779769880.7311306,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":44544,"affinity_cache_ratio":0.9897566937007,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (461 < 8000)","cache_hit":44544,"estimated_new_tokens":461,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91648,"ongoing_decode_tokens":55130,"pending_prefill_tokens":16550,"num_requests":2,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":45005,"score_linear":91648.0,"score_lmetric":123110},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":45005,"score_linear":62618.0,"score_lmetric":45005},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":45005,"score_linear":-11298.0,"score_lmetric":-38331},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":45005,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":172747,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":45005,"score_linear":172747.0,"score_lmetric":90010},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":45005,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20009,"ongoing_decode_tokens":20009,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1010,"cache_hit":0,"new_prefill":45005,"score_linear":20009.0,"score_lmetric":45005},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":44544,"new_prefill":461,"score_linear":-44544.0,"score_lmetric":0}],"chosen_score_linear":-44544.0,"chosen_score_lmetric":0,"t_first_token":339015.895966815,"t_first_token_unix":1779769880.9099102,"t_done":339016.199803098,"t_done_unix":1779769881.2137465},{"request_id":"1259178:3:1337653:429","session_id":"1259178","input_length":22398,"t_proxy_recv":339016.179848619,"t_decision_unix":1779769881.19379,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19968,"affinity_cache_ratio":0.8915081703723546,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2430 < 8000)","cache_hit":19968,"estimated_new_tokens":2430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91648,"ongoing_decode_tokens":55130,"pending_prefill_tokens":16550,"num_requests":2,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":22398,"score_linear":91648.0,"score_lmetric":77896},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":22398,"score_linear":62618.0,"score_lmetric":22398},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":22398,"score_linear":-11298.0,"score_lmetric":-15724},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":22398,"score_linear":95383.0,"score_lmetric":30741},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":925,"cache_hit":0,"new_prefill":22398,"score_linear":82142.0,"score_lmetric":22398},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":22398,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1021,"cache_hit":19968,"new_prefill":2430,"score_linear":-19968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45005,"ongoing_decode_tokens":45005,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":22398,"score_linear":45005.0,"score_lmetric":22398}],"chosen_score_linear":-19968.0,"chosen_score_lmetric":0,"t_first_token":339016.621736643,"t_first_token_unix":1779769881.6356807,"t_done":339017.475030518,"t_done_unix":1779769882.488974},{"request_id":"1311753:2:1337278:426","session_id":"1311753","input_length":36518,"t_proxy_recv":339014.740926936,"t_decision_unix":1779769879.7548683,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.5467988389287475,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":19968,"estimated_new_tokens":16550,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":19968,"new_prefill":16550,"score_linear":35162.0,"score_lmetric":16550},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":36518,"score_linear":62618.0,"score_lmetric":36518},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":36518,"score_linear":-11298.0,"score_lmetric":-29844},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":36518,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":172747,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":2,"active_p_offloads":0,"cached_blocks":924,"cache_hit":0,"new_prefill":36518,"score_linear":172747.0,"score_lmetric":74466},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":12644,"ongoing_decode_tokens":0,"pending_prefill_tokens":12644,"num_requests":1,"active_p_offloads":0,"cached_blocks":1065,"cache_hit":0,"new_prefill":36518,"score_linear":12644.0,"score_lmetric":49162},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":141977,"ongoing_decode_tokens":121968,"pending_prefill_tokens":5673,"num_requests":2,"active_p_offloads":0,"cached_blocks":961,"cache_hit":0,"new_prefill":36518,"score_linear":141977.0,"score_lmetric":84382},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":36518,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":35162.0,"chosen_score_lmetric":16550,"t_first_token":339018.160285086,"t_first_token_unix":1779769883.1742284,"t_done":339018.504855218,"t_done_unix":1779769883.5187986},{"request_id":"1317598:4:1332127:408","session_id":"1317598","input_length":82142,"t_proxy_recv":338996.424107975,"t_decision_unix":1779769861.438049,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":81920,"affinity_cache_ratio":0.9972973631029194,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (222 < 8000)","cache_hit":81920,"estimated_new_tokens":222,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":82142,"score_linear":55130.0,"score_lmetric":82142},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":82142,"score_linear":62618.0,"score_lmetric":82142},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":76003,"ongoing_decode_tokens":14067,"pending_prefill_tokens":-4624,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":82142,"score_linear":76003.0,"score_lmetric":77518},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12421,"ongoing_decode_tokens":0,"pending_prefill_tokens":12421,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":82142,"score_linear":12421.0,"score_lmetric":94563},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90605,"ongoing_decode_tokens":0,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":924,"cache_hit":81920,"new_prefill":222,"score_linear":8685.0,"score_lmetric":715},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":8880,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":82142,"score_linear":21680.0,"score_lmetric":91022},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":193745,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2657,"num_requests":2,"active_p_offloads":0,"cached_blocks":956,"cache_hit":0,"new_prefill":82142,"score_linear":193745.0,"score_lmetric":169598},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1061,"cache_hit":0,"new_prefill":82142,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":8685.0,"chosen_score_lmetric":715,"t_first_token":339015.199168085,"t_first_token_unix":1779769880.2131116,"t_done":339020.04829518,"t_done_unix":1779769885.0622387},{"request_id":"1269373:17:1338104:431","session_id":"1269373","input_length":73073,"t_proxy_recv":339017.825176531,"t_decision_unix":1779769882.8391178,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9809368713478303,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1393 < 8000)","cache_hit":71680,"estimated_new_tokens":1393,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91648,"ongoing_decode_tokens":55130,"pending_prefill_tokens":16550,"num_requests":2,"active_p_offloads":0,"cached_blocks":820,"cache_hit":7680,"new_prefill":65393,"score_linear":83968.0,"score_lmetric":163886},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":73073,"score_linear":62618.0,"score_lmetric":73073},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":7680,"new_prefill":65393,"score_linear":-18978.0,"score_lmetric":-58719},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":73073,"score_linear":95383.0,"score_lmetric":81416},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":925,"cache_hit":0,"new_prefill":73073,"score_linear":82142.0,"score_lmetric":73073},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":7680,"new_prefill":65393,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":71680,"new_prefill":1393,"score_linear":-71680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":73073,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":339018.485840752,"t_first_token_unix":1779769883.499784,"t_done":339020.858601671,"t_done_unix":1779769885.872545},{"request_id":"1317598:5:1337780:430","session_id":"1317598","input_length":83088,"t_proxy_recv":339020.061483667,"t_decision_unix":1779769885.075425,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":82432,"affinity_cache_ratio":0.99210475640285,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (656 < 8000)","cache_hit":82432,"estimated_new_tokens":656,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":83088,"score_linear":55130.0,"score_lmetric":83088},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":83088,"score_linear":62618.0,"score_lmetric":83088},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":83088,"score_linear":63160.0,"score_lmetric":155094},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":83088,"score_linear":95383.0,"score_lmetric":91431},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":82432,"new_prefill":656,"score_linear":-82432.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":83088,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":83088,"score_linear":73073.0,"score_lmetric":83088},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":49688,"ongoing_decode_tokens":0,"pending_prefill_tokens":5144,"num_requests":1,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":83088,"score_linear":49688.0,"score_lmetric":88232}],"chosen_score_linear":-82432.0,"chosen_score_lmetric":0,"t_first_token":339020.482518811,"t_first_token_unix":1779769885.4964626,"t_done":339021.061231995,"t_done_unix":1779769886.0751758},{"request_id":"1279412:20:1337591:428","session_id":"1279412","input_length":95383,"t_proxy_recv":339015.966189673,"t_decision_unix":1779769880.9801311,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":87040,"affinity_cache_ratio":0.912531583196167,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":87040,"estimated_new_tokens":8343,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91648,"ongoing_decode_tokens":55130,"pending_prefill_tokens":16550,"num_requests":2,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":95383,"score_linear":91648.0,"score_lmetric":223866},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":95383,"score_linear":62618.0,"score_lmetric":95383},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":95383,"score_linear":-11298.0,"score_lmetric":-88709},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":87040,"new_prefill":8343,"score_linear":-87040.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":925,"cache_hit":0,"new_prefill":95383,"score_linear":82142.0,"score_lmetric":95383},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":95383,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1021,"cache_hit":0,"new_prefill":95383,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45005,"ongoing_decode_tokens":45005,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":12800,"new_prefill":82583,"score_linear":32205.0,"score_lmetric":82583}],"chosen_score_linear":-87040.0,"chosen_score_lmetric":0,"t_first_token":339020.494527587,"t_first_token_unix":1779769885.508471,"t_done":339021.995945478,"t_done_unix":1779769887.0098891},{"request_id":"1316848:14:1338658:435","session_id":"1316848","input_length":49688,"t_proxy_recv":339019.601089049,"t_decision_unix":1779769884.61503,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":44544,"affinity_cache_ratio":0.8964739977459346,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5144 < 8000)","cache_hit":44544,"estimated_new_tokens":5144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":49688,"score_linear":55130.0,"score_lmetric":49688},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":49688,"score_linear":62618.0,"score_lmetric":49688},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":49688,"score_linear":63160.0,"score_lmetric":88294},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":49688,"score_linear":95383.0,"score_lmetric":58031},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":925,"cache_hit":0,"new_prefill":49688,"score_linear":82142.0,"score_lmetric":49688},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":49688,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":49688,"score_linear":73073.0,"score_lmetric":49688},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":44544,"new_prefill":5144,"score_linear":-44544.0,"score_lmetric":0}],"chosen_score_linear":-44544.0,"chosen_score_lmetric":0,"t_first_token":339021.10975377,"t_first_token_unix":1779769886.1236968,"t_done":339025.330267285,"t_done_unix":1779769890.3442106},{"request_id":"1279412:21:1339581:440","session_id":"1279412","input_length":95543,"t_proxy_recv":339022.931481368,"t_decision_unix":1779769887.945423,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":95232,"affinity_cache_ratio":0.9967449211349864,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (311 < 8000)","cache_hit":95232,"estimated_new_tokens":311,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":95543,"score_linear":55130.0,"score_lmetric":95543},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":95543,"score_linear":128770.0,"score_lmetric":191086},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":95543,"score_linear":63160.0,"score_lmetric":180004},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1070,"cache_hit":95232,"new_prefill":311,"score_linear":-95232.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":95543,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":95543,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":95543,"score_linear":31527.0,"score_lmetric":191086},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":49688,"ongoing_decode_tokens":49688,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":12800,"new_prefill":82743,"score_linear":36888.0,"score_lmetric":82743}],"chosen_score_linear":-95232.0,"chosen_score_lmetric":0,"t_first_token":339023.074520604,"t_first_token_unix":1779769888.0884635,"t_done":339025.384699611,"t_done_unix":1779769890.3986435},{"request_id":"1339977:1:1339977:442","session_id":"1339977","input_length":13816,"t_proxy_recv":339024.419543197,"t_decision_unix":1779769889.4334843,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":13816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":13816,"score_linear":55130.0,"score_lmetric":13816},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174764,"ongoing_decode_tokens":174764,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":13816,"score_linear":174764.0,"score_lmetric":41448},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":13816,"score_linear":63160.0,"score_lmetric":16550},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95543,"ongoing_decode_tokens":95543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1070,"cache_hit":0,"new_prefill":13816,"score_linear":95543.0,"score_lmetric":13816},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":13816,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":13816,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":13816,"score_linear":31527.0,"score_lmetric":27632},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":49688,"ongoing_decode_tokens":49688,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":13816,"score_linear":49688.0,"score_lmetric":13816}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339025.618875668,"t_first_token_unix":1779769890.6328187,"t_done":339025.819598884,"t_done_unix":1779769890.8335423},{"request_id":"1259178:4:1339295:439","session_id":"1259178","input_length":24462,"t_proxy_recv":339021.836925121,"t_decision_unix":1779769886.8508658,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":22528,"affinity_cache_ratio":0.9209385986427929,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1934 < 8000)","cache_hit":22528,"estimated_new_tokens":1934,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":24462,"score_linear":55130.0,"score_lmetric":24462},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":62618,"pending_prefill_tokens":4712,"num_requests":2,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":24462,"score_linear":128770.0,"score_lmetric":58348},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":24462,"score_linear":63160.0,"score_lmetric":37842},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":95383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":24462,"score_linear":95383.0,"score_lmetric":24462},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":24462,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":24462,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":22528,"new_prefill":1934,"score_linear":-15463.0,"score_lmetric":1934},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":49688,"ongoing_decode_tokens":49688,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":24462,"score_linear":49688.0,"score_lmetric":24462}],"chosen_score_linear":-15463.0,"chosen_score_lmetric":1934,"t_first_token":339022.268364676,"t_first_token_unix":1779769887.2823083,"t_done":339027.50371969,"t_done_unix":1779769892.5176632},{"request_id":"1315567:3:1325346:372","session_id":"1315567","input_length":55130,"t_proxy_recv":338972.267749594,"t_decision_unix":1779769837.2816904,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9937239252675494,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (346 < 8000)","cache_hit":54784,"estimated_new_tokens":346,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":54784,"new_prefill":346,"score_linear":-54784.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":55130,"score_linear":62618.0,"score_lmetric":55130},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-11298,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1090,"cache_hit":0,"new_prefill":55130,"score_linear":-11298.0,"score_lmetric":-43832},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":986,"cache_hit":0,"new_prefill":55130,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":55130,"score_linear":79663.0,"score_lmetric":55130},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":942,"cache_hit":7680,"new_prefill":47450,"score_linear":25502.0,"score_lmetric":47450},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":944,"cache_hit":7680,"new_prefill":47450,"score_linear":180423.0,"score_lmetric":94900},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31306,"ongoing_decode_tokens":31306,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":957,"cache_hit":0,"new_prefill":55130,"score_linear":31306.0,"score_lmetric":110260}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":338972.462247605,"t_first_token_unix":1779769837.4761915,"t_done":339028.567172172,"t_done_unix":1779769893.5811162},{"request_id":"1341005:1:1341005:447","session_id":"1341005","input_length":5280,"t_proxy_recv":339027.890169731,"t_decision_unix":1779769892.9041111,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5280 < 8000)","cache_hit":0,"estimated_new_tokens":5280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":5280,"score_linear":55130.0,"score_lmetric":5280},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174764,"ongoing_decode_tokens":174764,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":5280,"score_linear":174764.0,"score_lmetric":15840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":5280,"score_linear":63160.0,"score_lmetric":10560},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":5280,"score_linear":48381.0,"score_lmetric":53661},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":5280,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21221,"ongoing_decode_tokens":21221,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":5280,"score_linear":21221.0,"score_lmetric":5280},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":5280,"score_linear":7065.0,"score_lmetric":5280},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":2279,"ongoing_decode_tokens":2279,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1107,"cache_hit":0,"new_prefill":5280,"score_linear":2279.0,"score_lmetric":5280}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339028.237655343,"t_first_token_unix":1779769893.251599,"t_done":339028.596098319,"t_done_unix":1779769893.610042},{"request_id":"1338292:1:1338292:432","session_id":"1338292","input_length":9325,"t_proxy_recv":339018.364475344,"t_decision_unix":1779769883.3784165,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-2651,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9325,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91648,"ongoing_decode_tokens":91648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":9325,"score_linear":91648.0,"score_lmetric":18650},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":9325,"score_linear":62618.0,"score_lmetric":9325},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":-6674,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":9325,"score_linear":-11298.0,"score_lmetric":-2651},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":9325,"score_linear":95383.0,"score_lmetric":17668},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":925,"cache_hit":0,"new_prefill":9325,"score_linear":82142.0,"score_lmetric":9325},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":9325,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":0,"pending_prefill_tokens":1393,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":9325,"score_linear":73073.0,"score_lmetric":10718},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":9325,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-2651,"t_first_token":339019.070100655,"t_first_token_unix":1779769884.084044,"t_done":339028.627936251,"t_done_unix":1779769893.6418822},{"request_id":"1340903:1:1340903:446","session_id":"1340903","input_length":2279,"t_proxy_recv":339027.549884252,"t_decision_unix":1779769892.5638256,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2279 < 8000)","cache_hit":0,"estimated_new_tokens":2279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":2279,"score_linear":55130.0,"score_lmetric":2279},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174764,"ongoing_decode_tokens":174764,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":2279,"score_linear":174764.0,"score_lmetric":6837},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":2279,"score_linear":63160.0,"score_lmetric":4558},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":2279,"score_linear":48381.0,"score_lmetric":50660},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":2279,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21221,"ongoing_decode_tokens":0,"pending_prefill_tokens":21221,"num_requests":1,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":2279,"score_linear":21221.0,"score_lmetric":23500},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":2279,"score_linear":7065.0,"score_lmetric":2279},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1107,"cache_hit":0,"new_prefill":2279,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339027.699098843,"t_first_token_unix":1779769892.7130425,"t_done":339028.71895163,"t_done_unix":1779769893.732895},{"request_id":"1340290:1:1340290:445","session_id":"1340290","input_length":21221,"t_proxy_recv":339025.540309442,"t_decision_unix":1779769890.554251,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":21221,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":21221,"score_linear":55130.0,"score_lmetric":21221},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174764,"ongoing_decode_tokens":174764,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":21221,"score_linear":174764.0,"score_lmetric":63663},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":21221,"score_linear":63160.0,"score_lmetric":31360},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":21221,"score_linear":48381.0,"score_lmetric":69602},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13816,"ongoing_decode_tokens":0,"pending_prefill_tokens":13816,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":21221,"score_linear":13816.0,"score_lmetric":35037},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":21221,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":1024,"new_prefill":20197,"score_linear":30503.0,"score_lmetric":40394},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1107,"cache_hit":0,"new_prefill":21221,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339027.788809382,"t_first_token_unix":1779769892.8027527,"t_done":339029.589426305,"t_done_unix":1779769894.6033697},{"request_id":"1336620:2:1338440:434","session_id":"1336620","input_length":14918,"t_proxy_recv":339018.777111064,"t_decision_unix":1779769883.7910526,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":14336,"affinity_cache_ratio":0.960986727443357,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (582 < 8000)","cache_hit":14336,"estimated_new_tokens":582,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":7680,"new_prefill":7238,"score_linear":47450.0,"score_lmetric":7238},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":14918,"score_linear":62618.0,"score_lmetric":14918},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":48242,"ongoing_decode_tokens":0,"pending_prefill_tokens":3202,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":14336,"new_prefill":582,"score_linear":33906.0,"score_lmetric":3784},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":14918,"score_linear":95383.0,"score_lmetric":23261},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":925,"cache_hit":0,"new_prefill":14918,"score_linear":82142.0,"score_lmetric":14918},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":7680,"new_prefill":7238,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":7680,"new_prefill":7238,"score_linear":65393.0,"score_lmetric":7238},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":14918,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":33906.0,"chosen_score_lmetric":3784,"t_first_token":339028.49279255,"t_first_token_unix":1779769893.5067356,"t_done":339029.969077545,"t_done_unix":1779769894.983021},{"request_id":"1309252:3:1339771:441","session_id":"1309252","input_length":45994,"t_proxy_recv":339023.659666537,"t_decision_unix":1779769888.6736078,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":45568,"affinity_cache_ratio":0.9907379223376962,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (426 < 8000)","cache_hit":45568,"estimated_new_tokens":426,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":45994,"score_linear":55130.0,"score_lmetric":45994},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":45568,"new_prefill":426,"score_linear":83202.0,"score_lmetric":852},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":45994,"score_linear":63160.0,"score_lmetric":80906},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95543,"ongoing_decode_tokens":95543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1070,"cache_hit":0,"new_prefill":45994,"score_linear":95543.0,"score_lmetric":45994},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":45994,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":45994,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":45994,"score_linear":31527.0,"score_lmetric":91988},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":49688,"ongoing_decode_tokens":49688,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":45994,"score_linear":49688.0,"score_lmetric":45994}],"chosen_score_linear":83202.0,"chosen_score_lmetric":852,"t_first_token":339023.827131323,"t_first_token_unix":1779769888.8410747,"t_done":339031.090327216,"t_done_unix":1779769896.1042707},{"request_id":"1341912:1:1341912:449","session_id":"1341912","input_length":9909,"t_proxy_recv":339030.92982358,"t_decision_unix":1779769895.9437647,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9909,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":9909,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174764,"ongoing_decode_tokens":174764,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":9909,"score_linear":174764.0,"score_lmetric":29727},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":50215,"pending_prefill_tokens":19365,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":9909,"score_linear":97815.0,"score_lmetric":58548},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":9909,"score_linear":48381.0,"score_lmetric":58290},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":9909,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":9909,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":9909,"score_linear":7065.0,"score_lmetric":9909},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1111,"cache_hit":0,"new_prefill":9909,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339031.697728565,"t_first_token_unix":1779769896.7116718,"t_done":339032.198378767,"t_done_unix":1779769897.2123225},{"request_id":"1342327:1:1342327:452","session_id":"1342327","input_length":6180,"t_proxy_recv":339032.349196491,"t_decision_unix":1779769897.363138,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6180 < 8000)","cache_hit":0,"estimated_new_tokens":6180,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":872,"cache_hit":0,"new_prefill":6180,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":6180,"score_linear":128770.0,"score_lmetric":12360},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":71586,"pending_prefill_tokens":12842,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":6180,"score_linear":97815.0,"score_lmetric":38044},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":6180,"score_linear":48381.0,"score_lmetric":54561},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":6180,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41267,"ongoing_decode_tokens":0,"pending_prefill_tokens":20275,"num_requests":2,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":6180,"score_linear":41267.0,"score_lmetric":52910},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":6180,"score_linear":7065.0,"score_lmetric":6180},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1111,"cache_hit":0,"new_prefill":6180,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339032.756428093,"t_first_token_unix":1779769897.7703717,"t_done":339033.057326899,"t_done_unix":1779769898.0712702},{"request_id":"1341948:1:1341948:450","session_id":"1341948","input_length":19887,"t_proxy_recv":339031.04695218,"t_decision_unix":1779769896.0608938,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":19887,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9909,"ongoing_decode_tokens":0,"pending_prefill_tokens":9909,"num_requests":1,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":19887,"score_linear":9909.0,"score_lmetric":29796},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174764,"ongoing_decode_tokens":174764,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":19887,"score_linear":174764.0,"score_lmetric":59661},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":71586,"pending_prefill_tokens":12842,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":19887,"score_linear":97815.0,"score_lmetric":65458},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":19887,"score_linear":48381.0,"score_lmetric":68268},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":19887,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":19887,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":19887,"score_linear":7065.0,"score_lmetric":19887},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1111,"cache_hit":0,"new_prefill":19887,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339033.131360164,"t_first_token_unix":1779769898.1453035,"t_done":339033.453666697,"t_done_unix":1779769898.4676101},{"request_id":"1340903:2:1342632:453","session_id":"1340903","input_length":2129,"t_proxy_recv":339033.386846448,"t_decision_unix":1779769898.400788,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":2048,"affinity_cache_ratio":0.9619539689995303,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (81 < 8000)","cache_hit":2048,"estimated_new_tokens":81,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":872,"cache_hit":0,"new_prefill":2129,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":2129,"score_linear":128770.0,"score_lmetric":4258},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":71586,"pending_prefill_tokens":12842,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":2129,"score_linear":97815.0,"score_lmetric":29942},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":2129,"score_linear":48381.0,"score_lmetric":50510},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":2129,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41267,"ongoing_decode_tokens":41267,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":2129,"score_linear":41267.0,"score_lmetric":4258},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":2129,"score_linear":7065.0,"score_lmetric":2129},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":2048,"new_prefill":81,"score_linear":-2048.0,"score_lmetric":0}],"chosen_score_linear":-2048.0,"chosen_score_lmetric":0,"t_first_token":339033.404965814,"t_first_token_unix":1779769898.4189088,"t_done":339033.478860792,"t_done_unix":1779769898.4928048},{"request_id":"1340290:2:1342119:451","session_id":"1340290","input_length":21380,"t_proxy_recv":339031.584563987,"t_decision_unix":1779769896.5985048,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":20992,"affinity_cache_ratio":0.9818521983161833,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (388 < 8000)","cache_hit":20992,"estimated_new_tokens":388,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9909,"ongoing_decode_tokens":0,"pending_prefill_tokens":9909,"num_requests":1,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":21380,"score_linear":9909.0,"score_lmetric":31289},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":21380,"score_linear":128770.0,"score_lmetric":42760},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":71586,"pending_prefill_tokens":12842,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":21380,"score_linear":97815.0,"score_lmetric":68444},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":21380,"score_linear":48381.0,"score_lmetric":69761},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":21380,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":19887,"ongoing_decode_tokens":0,"pending_prefill_tokens":19887,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":20992,"new_prefill":388,"score_linear":-1105.0,"score_lmetric":20275},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":1024,"new_prefill":20356,"score_linear":6041.0,"score_lmetric":20356},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1111,"cache_hit":0,"new_prefill":21380,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1105.0,"chosen_score_lmetric":20275,"t_first_token":339033.133054181,"t_first_token_unix":1779769898.1469972,"t_done":339033.881527232,"t_done_unix":1779769898.8954709},{"request_id":"1340278:1:1340278:444","session_id":"1340278","input_length":48381,"t_proxy_recv":339025.517855191,"t_decision_unix":1779769890.5317965,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":48381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":48381,"score_linear":55130.0,"score_lmetric":48381},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174764,"ongoing_decode_tokens":174764,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":48381,"score_linear":174764.0,"score_lmetric":145143},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":48381,"score_linear":63160.0,"score_lmetric":85680},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":48381,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13816,"ongoing_decode_tokens":0,"pending_prefill_tokens":13816,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":48381,"score_linear":13816.0,"score_lmetric":62197},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":48381,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":48381,"score_linear":31527.0,"score_lmetric":96762},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1107,"cache_hit":0,"new_prefill":48381,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339034.272723111,"t_first_token_unix":1779769899.2866673,"t_done":339034.863039268,"t_done_unix":1779769899.8769834},{"request_id":"1343191:1:1343191:457","session_id":"1343191","input_length":11912,"t_proxy_recv":339035.459621371,"t_decision_unix":1779769900.4735625,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11912,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":872,"cache_hit":0,"new_prefill":11912,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":11912,"score_linear":128770.0,"score_lmetric":23824},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":109113,"pending_prefill_tokens":-1133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":11912,"score_linear":97815.0,"score_lmetric":21558},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1165,"cache_hit":0,"new_prefill":11912,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":11912,"score_linear":68579.0,"score_lmetric":150742},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":11912,"score_linear":65289.0,"score_lmetric":77201},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":11912,"score_linear":7065.0,"score_lmetric":11912},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":11912,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339036.421552733,"t_first_token_unix":1779769901.4354987,"t_done":339036.472547417,"t_done_unix":1779769901.4864914},{"request_id":"1341912:2:1343220:458","session_id":"1341912","input_length":10232,"t_proxy_recv":339035.534828778,"t_decision_unix":1779769900.54877,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":9728,"affinity_cache_ratio":0.9507427677873339,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (504 < 8000)","cache_hit":9728,"estimated_new_tokens":504,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":872,"cache_hit":9728,"new_prefill":504,"score_linear":-9728.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":10232,"score_linear":128770.0,"score_lmetric":20464},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":109113,"pending_prefill_tokens":-1133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":10232,"score_linear":97815.0,"score_lmetric":18198},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1165,"cache_hit":0,"new_prefill":10232,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":10232,"score_linear":68579.0,"score_lmetric":147382},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":10232,"score_linear":65289.0,"score_lmetric":75521},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":10232,"score_linear":7065.0,"score_lmetric":10232},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11912,"ongoing_decode_tokens":0,"pending_prefill_tokens":11912,"num_requests":1,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":10232,"score_linear":11912.0,"score_lmetric":22144}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":339035.603386695,"t_first_token_unix":1779769900.61733,"t_done":339036.474842615,"t_done_unix":1779769901.4887862},{"request_id":"1338292:2:1343599:460","session_id":"1338292","input_length":14103,"t_proxy_recv":339036.66432827,"t_decision_unix":1779769901.6782694,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":9216,"affinity_cache_ratio":0.6534779834077856,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4887 < 8000)","cache_hit":9216,"estimated_new_tokens":4887,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":873,"cache_hit":0,"new_prefill":14103,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":14103,"score_linear":128770.0,"score_lmetric":28206},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":109113,"pending_prefill_tokens":-1133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":9216,"new_prefill":4887,"score_linear":88599.0,"score_lmetric":7508},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21402,"ongoing_decode_tokens":0,"pending_prefill_tokens":21402,"num_requests":1,"active_p_offloads":0,"cached_blocks":1165,"cache_hit":0,"new_prefill":14103,"score_linear":21402.0,"score_lmetric":35505},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":14103,"score_linear":68579.0,"score_lmetric":155124},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":14103,"score_linear":65289.0,"score_lmetric":79392},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":14103,"score_linear":7065.0,"score_lmetric":14103},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1146,"cache_hit":0,"new_prefill":14103,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":88599.0,"chosen_score_lmetric":7508,"t_first_token":339037.254637067,"t_first_token_unix":1779769902.268581,"t_done":339037.465023207,"t_done_unix":1779769902.4789667},{"request_id":"1342327:2:1343857:461","session_id":"1342327","input_length":9820,"t_proxy_recv":339037.486263616,"t_decision_unix":1779769902.5002053,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.6256619144602852,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3676 < 8000)","cache_hit":6144,"estimated_new_tokens":3676,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":873,"cache_hit":0,"new_prefill":9820,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":9820,"score_linear":128770.0,"score_lmetric":19640},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":109113,"pending_prefill_tokens":-1133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1426,"cache_hit":0,"new_prefill":9820,"score_linear":97815.0,"score_lmetric":17374},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21402,"ongoing_decode_tokens":0,"pending_prefill_tokens":21402,"num_requests":1,"active_p_offloads":0,"cached_blocks":1165,"cache_hit":0,"new_prefill":9820,"score_linear":21402.0,"score_lmetric":31222},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":9820,"score_linear":68579.0,"score_lmetric":146558},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":9820,"score_linear":65289.0,"score_lmetric":75109},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":9820,"score_linear":7065.0,"score_lmetric":9820},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1146,"cache_hit":6144,"new_prefill":3676,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":339037.845961178,"t_first_token_unix":1779769902.8599052,"t_done":339038.080561904,"t_done_unix":1779769903.0945053},{"request_id":"1253743:5:1338406:433","session_id":"1253743","input_length":50215,"t_proxy_recv":339018.686367991,"t_decision_unix":1779769883.7003095,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9890271831126157,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (551 < 8000)","cache_hit":49664,"estimated_new_tokens":551,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":50215,"score_linear":55130.0,"score_lmetric":50215},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":4608,"new_prefill":45607,"score_linear":58010.0,"score_lmetric":45607},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-1973,"ongoing_decode_tokens":0,"pending_prefill_tokens":2651,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":49664,"new_prefill":551,"score_linear":-51637.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":0,"pending_prefill_tokens":8343,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":49664,"new_prefill":551,"score_linear":45719.0,"score_lmetric":8894},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":925,"cache_hit":0,"new_prefill":50215,"score_linear":82142.0,"score_lmetric":50215},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":50215,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":50215,"score_linear":73073.0,"score_lmetric":50215},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":50215,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-51637.0,"chosen_score_lmetric":0,"t_first_token":339028.491900653,"t_first_token_unix":1779769893.5058446,"t_done":339038.153601738,"t_done_unix":1779769903.1675553},{"request_id":"1343312:1:1343312:459","session_id":"1343312","input_length":21402,"t_proxy_recv":339035.822419203,"t_decision_unix":1779769900.8363602,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":21402,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":10232,"ongoing_decode_tokens":10232,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":872,"cache_hit":0,"new_prefill":21402,"score_linear":10232.0,"score_lmetric":21402},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":21402,"score_linear":128770.0,"score_lmetric":42804},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":109113,"pending_prefill_tokens":-1133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":21402,"score_linear":97815.0,"score_lmetric":40538},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1165,"cache_hit":0,"new_prefill":21402,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":21402,"score_linear":68579.0,"score_lmetric":169722},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":21402,"score_linear":65289.0,"score_lmetric":86691},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":21402,"score_linear":7065.0,"score_lmetric":21402},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11912,"ongoing_decode_tokens":0,"pending_prefill_tokens":11912,"num_requests":1,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":21402,"score_linear":11912.0,"score_lmetric":33314}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339038.11794268,"t_first_token_unix":1779769903.1318882,"t_done":339038.176707463,"t_done_unix":1779769903.1906512},{"request_id":"1334223:2:1341796:448","session_id":"1334223","input_length":37527,"t_proxy_recv":339030.497191751,"t_decision_unix":1779769895.5111322,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":23552,"affinity_cache_ratio":0.6276014602819303,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":23552,"estimated_new_tokens":13975,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":853,"cache_hit":0,"new_prefill":37527,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174764,"ongoing_decode_tokens":174764,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":37527,"score_linear":174764.0,"score_lmetric":112581},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":60288,"ongoing_decode_tokens":50215,"pending_prefill_tokens":5390,"num_requests":1,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":23552,"new_prefill":13975,"score_linear":36736.0,"score_lmetric":19365},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":37527,"score_linear":48381.0,"score_lmetric":85908},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":37527,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":37527,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":37527,"score_linear":7065.0,"score_lmetric":37527},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1111,"cache_hit":0,"new_prefill":37527,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":36736.0,"chosen_score_lmetric":19365,"t_first_token":339033.958579351,"t_first_token_unix":1779769898.9725227,"t_done":339038.494931227,"t_done_unix":1779769903.508875},{"request_id":"1316848:15:1344261:463","session_id":"1316848","input_length":50447,"t_proxy_recv":339038.89794864,"t_decision_unix":1779769903.9118896,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":50176,"affinity_cache_ratio":0.9946280254524551,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (271 < 8000)","cache_hit":50176,"estimated_new_tokens":271,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":873,"cache_hit":0,"new_prefill":50447,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":50447,"score_linear":128770.0,"score_lmetric":100894},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":50447,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95781,"ongoing_decode_tokens":0,"pending_prefill_tokens":37,"num_requests":1,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":50447,"score_linear":95781.0,"score_lmetric":50484},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":50447,"score_linear":68579.0,"score_lmetric":227812},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":50447,"score_linear":65289.0,"score_lmetric":115736},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":50447,"score_linear":7065.0,"score_lmetric":50447},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1153,"cache_hit":50176,"new_prefill":271,"score_linear":-50176.0,"score_lmetric":0}],"chosen_score_linear":-50176.0,"chosen_score_lmetric":0,"t_first_token":339039.082667299,"t_first_token_unix":1779769904.0966113,"t_done":339039.403072679,"t_done_unix":1779769904.417019},{"request_id":"1279412:22:1344210:462","session_id":"1279412","input_length":95781,"t_proxy_recv":339038.755473307,"t_decision_unix":1779769903.7694144,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9996137020912289,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (37 < 8000)","cache_hit":95744,"estimated_new_tokens":37,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":873,"cache_hit":0,"new_prefill":95781,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":95781,"score_linear":128770.0,"score_lmetric":191562},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":95781,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":95744,"new_prefill":37,"score_linear":-95744.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":95781,"score_linear":68579.0,"score_lmetric":318480},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":95781,"score_linear":65289.0,"score_lmetric":161070},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":95781,"score_linear":7065.0,"score_lmetric":95781},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1153,"cache_hit":12800,"new_prefill":82981,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-95744.0,"chosen_score_lmetric":0,"t_first_token":339038.903809482,"t_first_token_unix":1779769903.9177525,"t_done":339041.019925608,"t_done_unix":1779769906.0338697},{"request_id":"1342327:3:1344788:466","session_id":"1342327","input_length":12398,"t_proxy_recv":339040.738399372,"t_decision_unix":1779769905.7523408,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":9728,"affinity_cache_ratio":0.78464268430392,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2670 < 8000)","cache_hit":9728,"estimated_new_tokens":2670,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":873,"cache_hit":0,"new_prefill":12398,"score_linear":32816.0,"score_lmetric":45214},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":12398,"score_linear":128770.0,"score_lmetric":24796},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":12398,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95781,"ongoing_decode_tokens":95781,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":12398,"score_linear":95781.0,"score_lmetric":12398},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":12398,"score_linear":68579.0,"score_lmetric":151714},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":12398,"score_linear":65289.0,"score_lmetric":77687},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":12398,"score_linear":7065.0,"score_lmetric":12398},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1153,"cache_hit":9728,"new_prefill":2670,"score_linear":-9728.0,"score_lmetric":0}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":339041.067715861,"t_first_token_unix":1779769906.0816593,"t_done":339041.209678278,"t_done_unix":1779769906.2236218},{"request_id":"1314357:3:1338999:437","session_id":"1314357","input_length":66152,"t_proxy_recv":339020.801577126,"t_decision_unix":1779769885.8155181,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9287701052122385,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4712 < 8000)","cache_hit":61440,"estimated_new_tokens":4712,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":66152,"score_linear":55130.0,"score_lmetric":66152},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":61440,"new_prefill":4712,"score_linear":1178.0,"score_lmetric":4712},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":66152,"score_linear":63160.0,"score_lmetric":121222},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":95383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":66152,"score_linear":95383.0,"score_lmetric":66152},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83088,"ongoing_decode_tokens":83088,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":66152,"score_linear":83088.0,"score_lmetric":66152},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":66152,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":66152,"score_linear":73073.0,"score_lmetric":66152},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":49688,"ongoing_decode_tokens":0,"pending_prefill_tokens":5144,"num_requests":1,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":66152,"score_linear":49688.0,"score_lmetric":71296}],"chosen_score_linear":1178.0,"chosen_score_lmetric":4712,"t_first_token":339022.862871032,"t_first_token_unix":1779769887.8768146,"t_done":339042.867135735,"t_done_unix":1779769907.8810792},{"request_id":"1345921:1:1345921:469","session_id":"1345921","input_length":505,"t_proxy_recv":339044.588238136,"t_decision_unix":1779769909.6021793,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (505 < 8000)","cache_hit":0,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":873,"cache_hit":0,"new_prefill":505,"score_linear":32816.0,"score_lmetric":33321},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":505,"score_linear":62618.0,"score_lmetric":505},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":505,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":505,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":505,"score_linear":68579.0,"score_lmetric":127928},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":505,"score_linear":65289.0,"score_lmetric":65794},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81011,"ongoing_decode_tokens":81011,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":505,"score_linear":81011.0,"score_lmetric":1010},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58815,"ongoing_decode_tokens":0,"pending_prefill_tokens":8639,"num_requests":1,"active_p_offloads":0,"cached_blocks":1158,"cache_hit":0,"new_prefill":505,"score_linear":58815.0,"score_lmetric":9144}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339044.645139372,"t_first_token_unix":1779769909.6590824,"t_done":339044.97294086,"t_done_unix":1779769909.986884},{"request_id":"1344773:1:1344773:465","session_id":"1344773","input_length":32816,"t_proxy_recv":339040.690154895,"t_decision_unix":1779769905.704096,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":32816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":873,"cache_hit":0,"new_prefill":32816,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":32816,"score_linear":128770.0,"score_lmetric":65632},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":32816,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95781,"ongoing_decode_tokens":95781,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":32816,"score_linear":95781.0,"score_lmetric":32816},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":32816,"score_linear":68579.0,"score_lmetric":192550},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":32816,"score_linear":65289.0,"score_lmetric":98105},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":32816,"score_linear":7065.0,"score_lmetric":32816},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1153,"cache_hit":0,"new_prefill":32816,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339045.23602452,"t_first_token_unix":1779769910.2499683,"t_done":339045.380114447,"t_done_unix":1779769910.394058},{"request_id":"1344773:2:1346419:473","session_id":"1344773","input_length":33008,"t_proxy_recv":339046.462889939,"t_decision_unix":1779769911.4768314,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":32768,"affinity_cache_ratio":0.9927290353853612,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (240 < 8000)","cache_hit":32768,"estimated_new_tokens":240,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":937,"cache_hit":32768,"new_prefill":240,"score_linear":-32768.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":33008,"score_linear":62618.0,"score_lmetric":33008},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":33008,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37019,"ongoing_decode_tokens":0,"pending_prefill_tokens":2715,"num_requests":1,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":33008,"score_linear":37019.0,"score_lmetric":35723},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":68579,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":33008,"score_linear":68579.0,"score_lmetric":66016},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":33008,"score_linear":65289.0,"score_lmetric":98297},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81011,"ongoing_decode_tokens":81011,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":33008,"score_linear":81011.0,"score_lmetric":66016},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":73657,"ongoing_decode_tokens":0,"pending_prefill_tokens":11193,"num_requests":2,"active_p_offloads":0,"cached_blocks":1158,"cache_hit":0,"new_prefill":33008,"score_linear":73657.0,"score_lmetric":88402}],"chosen_score_linear":-32768.0,"chosen_score_lmetric":0,"t_first_token":339046.59019928,"t_first_token_unix":1779769911.604143,"t_done":339046.768851842,"t_done_unix":1779769911.782795},{"request_id":"1341005:2:1342961:456","session_id":"1341005","input_length":9360,"t_proxy_recv":339034.55407851,"t_decision_unix":1779769899.5680199,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":5120,"affinity_cache_ratio":0.5470085470085471,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4240 < 8000)","cache_hit":5120,"estimated_new_tokens":4240,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":872,"cache_hit":0,"new_prefill":9360,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":9360,"score_linear":128770.0,"score_lmetric":18720},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":109113,"pending_prefill_tokens":-1133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":9360,"score_linear":97815.0,"score_lmetric":16454},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":48381,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":9360,"score_linear":48381.0,"score_lmetric":9360},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":964,"cache_hit":5120,"new_prefill":4240,"score_linear":54099.0,"score_lmetric":63459},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":9360,"score_linear":65289.0,"score_lmetric":74649},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":9360,"score_linear":7065.0,"score_lmetric":9360},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":9360,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":54099.0,"chosen_score_lmetric":63459,"t_first_token":339046.413081285,"t_first_token_unix":1779769911.4270246,"t_done":339046.904098757,"t_done_unix":1779769911.9180424},{"request_id":"1342327:4:1346218:471","session_id":"1342327","input_length":14842,"t_proxy_recv":339045.689438081,"t_decision_unix":1779769910.7033794,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":12288,"affinity_cache_ratio":0.8279207653954993,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2554 < 8000)","cache_hit":12288,"estimated_new_tokens":2554,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":937,"cache_hit":0,"new_prefill":14842,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":14842,"score_linear":62618.0,"score_lmetric":14842},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":14842,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":0,"new_prefill":14842,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":14842,"score_linear":68579.0,"score_lmetric":156602},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":14842,"score_linear":65289.0,"score_lmetric":80131},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81011,"ongoing_decode_tokens":81011,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":14842,"score_linear":81011.0,"score_lmetric":29684},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58815,"ongoing_decode_tokens":0,"pending_prefill_tokens":8639,"num_requests":1,"active_p_offloads":0,"cached_blocks":1158,"cache_hit":12288,"new_prefill":2554,"score_linear":46527.0,"score_lmetric":11193}],"chosen_score_linear":46527.0,"chosen_score_lmetric":11193,"t_first_token":339047.043150153,"t_first_token_unix":1779769912.0570943,"t_done":339047.109918053,"t_done_unix":1779769912.1238616},{"request_id":"1316848:16:1345678:468","session_id":"1316848","input_length":58815,"t_proxy_recv":339043.710278185,"t_decision_unix":1779769908.724219,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":50176,"affinity_cache_ratio":0.8531157017767577,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":50176,"estimated_new_tokens":8639,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":873,"cache_hit":0,"new_prefill":58815,"score_linear":32816.0,"score_lmetric":91631},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":58815,"score_linear":62618.0,"score_lmetric":58815},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":58815,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":58815,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":58815,"score_linear":68579.0,"score_lmetric":244548},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":58815,"score_linear":65289.0,"score_lmetric":124104},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81011,"ongoing_decode_tokens":81011,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":58815,"score_linear":81011.0,"score_lmetric":117630},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1158,"cache_hit":50176,"new_prefill":8639,"score_linear":-50176.0,"score_lmetric":0}],"chosen_score_linear":-50176.0,"chosen_score_lmetric":0,"t_first_token":339046.741931075,"t_first_token_unix":1779769911.7558753,"t_done":339047.175721863,"t_done_unix":1779769912.1896656},{"request_id":"1269373:18:1345570:467","session_id":"1269373","input_length":73946,"t_proxy_recv":339043.266501773,"t_decision_unix":1779769908.2804432,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9901279311930328,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (730 < 8000)","cache_hit":73216,"estimated_new_tokens":730,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":873,"cache_hit":7680,"new_prefill":66266,"score_linear":25136.0,"score_lmetric":99082},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":73946,"score_linear":62618.0,"score_lmetric":73946},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":7680,"new_prefill":66266,"score_linear":2393.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":73946,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":73946,"score_linear":68579.0,"score_lmetric":274810},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":7680,"new_prefill":66266,"score_linear":57609.0,"score_lmetric":131555},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":73216,"new_prefill":730,"score_linear":-66151.0,"score_lmetric":730},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1158,"cache_hit":0,"new_prefill":73946,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-66151.0,"chosen_score_lmetric":730,"t_first_token":339043.685488665,"t_first_token_unix":1779769908.6994326,"t_done":339047.425662819,"t_done_unix":1779769912.4396067},{"request_id":"1342634:1:1342634:454","session_id":"1342634","input_length":59219,"t_proxy_recv":339033.420094578,"t_decision_unix":1779769898.4340363,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":59219,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":872,"cache_hit":0,"new_prefill":59219,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":59219,"score_linear":128770.0,"score_lmetric":118438},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":71586,"pending_prefill_tokens":12842,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":59219,"score_linear":97815.0,"score_lmetric":144122},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":59219,"score_linear":48381.0,"score_lmetric":107600},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":59219,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":41267,"ongoing_decode_tokens":41267,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":59219,"score_linear":41267.0,"score_lmetric":118438},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":59219,"score_linear":7065.0,"score_lmetric":59219},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":2129,"ongoing_decode_tokens":2129,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":59219,"score_linear":2129.0,"score_lmetric":59219}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339046.411044582,"t_first_token_unix":1779769911.4249885,"t_done":339047.453966144,"t_done_unix":1779769912.4679096},{"request_id":"1268861:6:1346340:472","session_id":"1268861","input_length":37019,"t_proxy_recv":339046.136111457,"t_decision_unix":1779769911.1500528,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9266592830708555,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2715 < 8000)","cache_hit":34304,"estimated_new_tokens":2715,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":937,"cache_hit":0,"new_prefill":37019,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":37019,"score_linear":62618.0,"score_lmetric":37019},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":37019,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1207,"cache_hit":34304,"new_prefill":2715,"score_linear":-34304.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":68579,"ongoing_decode_tokens":0,"pending_prefill_tokens":63459,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":37019,"score_linear":68579.0,"score_lmetric":200956},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":37019,"score_linear":65289.0,"score_lmetric":102308},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81011,"ongoing_decode_tokens":81011,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":37019,"score_linear":81011.0,"score_lmetric":74038},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":73657,"ongoing_decode_tokens":0,"pending_prefill_tokens":11193,"num_requests":2,"active_p_offloads":0,"cached_blocks":1158,"cache_hit":0,"new_prefill":37019,"score_linear":73657.0,"score_lmetric":96424}],"chosen_score_linear":-34304.0,"chosen_score_lmetric":0,"t_first_token":339046.811361057,"t_first_token_unix":1779769911.8253047,"t_done":339048.649539389,"t_done_unix":1779769913.6634858},{"request_id":"1345921:2:1347089:474","session_id":"1345921","input_length":773,"t_proxy_recv":339048.7747089,"t_decision_unix":1779769913.7886503,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":512,"affinity_cache_ratio":0.6623544631306598,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (261 < 8000)","cache_hit":512,"estimated_new_tokens":261,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":937,"cache_hit":0,"new_prefill":773,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":773,"score_linear":62618.0,"score_lmetric":773},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":773,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":512,"new_prefill":261,"score_linear":-512.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1087,"cache_hit":0,"new_prefill":773,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":773,"score_linear":65289.0,"score_lmetric":66062},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1036,"cache_hit":0,"new_prefill":773,"score_linear":7065.0,"score_lmetric":773},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1179,"cache_hit":0,"new_prefill":773,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":339048.813490695,"t_first_token_unix":1779769913.8274412,"t_done":339049.066673556,"t_done_unix":1779769914.0806172},{"request_id":"1341948:2:1347296:475","session_id":"1341948","input_length":25661,"t_proxy_recv":339049.49732397,"t_decision_unix":1779769914.5112653,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":19456,"affinity_cache_ratio":0.7581933673668212,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6205 < 8000)","cache_hit":19456,"estimated_new_tokens":6205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":937,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":25661,"score_linear":62618.0,"score_lmetric":25661},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":25661,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1087,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":19456,"new_prefill":6205,"score_linear":45833.0,"score_lmetric":6205},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1036,"cache_hit":0,"new_prefill":25661,"score_linear":7065.0,"score_lmetric":25661},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1179,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":45833.0,"chosen_score_lmetric":6205,"t_first_token":339050.546385036,"t_first_token_unix":1779769915.5603287,"t_done":339051.07153515,"t_done_unix":1779769916.0854793},{"request_id":"1347763:1:1347763:476","session_id":"1347763","input_length":5465,"t_proxy_recv":339051.099376579,"t_decision_unix":1779769916.1133184,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5465 < 8000)","cache_hit":0,"estimated_new_tokens":5465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":937,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":5465,"score_linear":62618.0,"score_lmetric":5465},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":5465,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1087,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":5465,"score_linear":65289.0,"score_lmetric":5465},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1036,"cache_hit":0,"new_prefill":5465,"score_linear":7065.0,"score_lmetric":5465},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1179,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339051.454025373,"t_first_token_unix":1779769916.4679694,"t_done":339051.581182821,"t_done_unix":1779769916.5951264},{"request_id":"1339070:1:1339070:438","session_id":"1339070","input_length":7065,"t_proxy_recv":339021.009935333,"t_decision_unix":1779769886.023877,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7065 < 8000)","cache_hit":0,"estimated_new_tokens":7065,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":7065,"score_linear":55130.0,"score_lmetric":7065},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":62618,"pending_prefill_tokens":4712,"num_requests":2,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":7065,"score_linear":128770.0,"score_lmetric":23554},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63160,"ongoing_decode_tokens":9325,"pending_prefill_tokens":-5541,"num_requests":2,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":7065,"score_linear":63160.0,"score_lmetric":3048},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":95383,"ongoing_decode_tokens":95383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1054,"cache_hit":0,"new_prefill":7065,"score_linear":95383.0,"score_lmetric":7065},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83088,"ongoing_decode_tokens":83088,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":7065,"score_linear":83088.0,"score_lmetric":7065},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1089,"cache_hit":0,"new_prefill":7065,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":7065,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":49688,"ongoing_decode_tokens":0,"pending_prefill_tokens":5144,"num_requests":1,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":7065,"score_linear":49688.0,"score_lmetric":12209}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339021.490509499,"t_first_token_unix":1779769886.504454,"t_done":339052.59678089,"t_done_unix":1779769917.6107244},{"request_id":"1342921:1:1342921:455","session_id":"1342921","input_length":65289,"t_proxy_recv":339034.465694278,"t_decision_unix":1779769899.4796355,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":65289,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":872,"cache_hit":0,"new_prefill":65289,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128770,"ongoing_decode_tokens":128770,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1137,"cache_hit":0,"new_prefill":65289,"score_linear":128770.0,"score_lmetric":130578},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97815,"ongoing_decode_tokens":109113,"pending_prefill_tokens":-1133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":0,"new_prefill":65289,"score_linear":97815.0,"score_lmetric":128312},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":48381,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":65289,"score_linear":48381.0,"score_lmetric":65289},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":65289,"score_linear":59219.0,"score_lmetric":124508},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":65289,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":0,"new_prefill":65289,"score_linear":7065.0,"score_lmetric":65289},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":65289,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339049.410153302,"t_first_token_unix":1779769914.4240985,"t_done":339053.91651946,"t_done_unix":1779769918.9304638},{"request_id":"1311753:3:1348389:480","session_id":"1311753","input_length":39636,"t_proxy_recv":339053.580641318,"t_decision_unix":1779769918.5945823,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":36352,"affinity_cache_ratio":0.9171460288626501,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3284 < 8000)","cache_hit":36352,"estimated_new_tokens":3284,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":36352,"new_prefill":3284,"score_linear":-36352.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":39636,"score_linear":62618.0,"score_lmetric":39636},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":21371,"pending_prefill_tokens":2693,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":39636,"score_linear":51275.0,"score_lmetric":42329},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":39636,"score_linear":96026.0,"score_lmetric":39636},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1087,"cache_hit":0,"new_prefill":39636,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":39636,"score_linear":65289.0,"score_lmetric":39636},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":39636,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1179,"cache_hit":0,"new_prefill":39636,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-36352.0,"chosen_score_lmetric":0,"t_first_token":339054.440878452,"t_first_token_unix":1779769919.454822,"t_done":339054.72840498,"t_done_unix":1779769919.7423491},{"request_id":"1345921:3:1348784:483","session_id":"1345921","input_length":1397,"t_proxy_recv":339054.772669158,"t_decision_unix":1779769919.7866106,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":7,"affinity_cache_hit":512,"affinity_cache_ratio":0.36649964209019326,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1397 < 8000)","cache_hit":0,"estimated_new_tokens":1397,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":953,"cache_hit":0,"new_prefill":1397,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":1397,"score_linear":62618.0,"score_lmetric":1397},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":62573,"pending_prefill_tokens":-1133,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":1397,"score_linear":51275.0,"score_lmetric":264},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":512,"new_prefill":885,"score_linear":95514.0,"score_lmetric":885},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17217,"ongoing_decode_tokens":0,"pending_prefill_tokens":17217,"num_requests":1,"active_p_offloads":0,"cached_blocks":1087,"cache_hit":0,"new_prefill":1397,"score_linear":17217.0,"score_lmetric":18614},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":1397,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12701,"ongoing_decode_tokens":0,"pending_prefill_tokens":12701,"num_requests":1,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":1397,"score_linear":12701.0,"score_lmetric":14098},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1179,"cache_hit":0,"new_prefill":1397,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339054.877742036,"t_first_token_unix":1779769919.8916855,"t_done":339054.996415017,"t_done_unix":1779769920.0103583},{"request_id":"1348394:1:1348394:481","session_id":"1348394","input_length":17217,"t_proxy_recv":339053.61167952,"t_decision_unix":1779769918.6256213,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":17217,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":39636,"ongoing_decode_tokens":0,"pending_prefill_tokens":3284,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":17217,"score_linear":39636.0,"score_lmetric":20501},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":17217,"score_linear":62618.0,"score_lmetric":17217},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":21371,"pending_prefill_tokens":2693,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":17217,"score_linear":51275.0,"score_lmetric":19910},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":17217,"score_linear":96026.0,"score_lmetric":17217},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1087,"cache_hit":0,"new_prefill":17217,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":17217,"score_linear":65289.0,"score_lmetric":17217},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":17217,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1179,"cache_hit":0,"new_prefill":17217,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339055.328074424,"t_first_token_unix":1779769920.3420184,"t_done":339055.71030319,"t_done_unix":1779769920.7242467},{"request_id":"1348691:1:1348691:482","session_id":"1348691","input_length":12701,"t_proxy_recv":339054.472482703,"t_decision_unix":1779769919.4864242,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":12701,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":39636,"ongoing_decode_tokens":39636,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":12701,"score_linear":39636.0,"score_lmetric":12701},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":12701,"score_linear":62618.0,"score_lmetric":12701},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":62573,"pending_prefill_tokens":-1133,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":12701,"score_linear":51275.0,"score_lmetric":11568},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":12701,"score_linear":96026.0,"score_lmetric":12701},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17217,"ongoing_decode_tokens":0,"pending_prefill_tokens":17217,"num_requests":1,"active_p_offloads":0,"cached_blocks":1087,"cache_hit":0,"new_prefill":12701,"score_linear":17217.0,"score_lmetric":29918},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":12701,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":12701,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1179,"cache_hit":0,"new_prefill":12701,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339055.534867404,"t_first_token_unix":1779769920.5488122,"t_done":339055.890332515,"t_done_unix":1779769920.9042764},{"request_id":"1344773:3:1349525:486","session_id":"1344773","input_length":33334,"t_proxy_recv":339057.40642104,"t_decision_unix":1779769922.4203618,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":32768,"affinity_cache_ratio":0.9830203395932081,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (566 < 8000)","cache_hit":32768,"estimated_new_tokens":566,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":953,"cache_hit":32768,"new_prefill":566,"score_linear":-32768.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":33334,"score_linear":62618.0,"score_lmetric":33334},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":62573,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":33334,"score_linear":51275.0,"score_lmetric":33334},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":33334,"score_linear":96026.0,"score_lmetric":33334},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1120,"cache_hit":0,"new_prefill":33334,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":33334,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":76493,"ongoing_decode_tokens":0,"pending_prefill_tokens":2253,"num_requests":1,"active_p_offloads":0,"cached_blocks":1075,"cache_hit":0,"new_prefill":33334,"score_linear":76493.0,"score_lmetric":35587},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":33334,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-32768.0,"chosen_score_lmetric":0,"t_first_token":339057.60625262,"t_first_token_unix":1779769922.6201966,"t_done":339059.058415407,"t_done_unix":1779769924.0723588},{"request_id":"1349975:1:1349975:487","session_id":"1349975","input_length":4988,"t_proxy_recv":339058.996162203,"t_decision_unix":1779769924.0101035,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4988 < 8000)","cache_hit":0,"estimated_new_tokens":4988,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":33334,"ongoing_decode_tokens":33334,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":953,"cache_hit":0,"new_prefill":4988,"score_linear":33334.0,"score_lmetric":4988},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":4988,"score_linear":62618.0,"score_lmetric":4988},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":62573,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":4988,"score_linear":51275.0,"score_lmetric":4988},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":4988,"score_linear":96026.0,"score_lmetric":4988},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1120,"cache_hit":0,"new_prefill":4988,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":4988,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":76493,"ongoing_decode_tokens":76493,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1075,"cache_hit":0,"new_prefill":4988,"score_linear":76493.0,"score_lmetric":4988},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":4988,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339059.31746619,"t_first_token_unix":1779769924.3314095,"t_done":339059.642111717,"t_done_unix":1779769924.656055},{"request_id":"1350162:1:1350162:488","session_id":"1350162","input_length":4594,"t_proxy_recv":339059.683957751,"t_decision_unix":1779769924.6978993,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4594 < 8000)","cache_hit":0,"estimated_new_tokens":4594,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":4594,"score_linear":62618.0,"score_lmetric":4594},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":62573,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":4594,"score_linear":51275.0,"score_lmetric":4594},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":4594,"score_linear":96026.0,"score_lmetric":4594},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":76493,"ongoing_decode_tokens":76493,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1075,"cache_hit":0,"new_prefill":4594,"score_linear":76493.0,"score_lmetric":4594},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339059.975084616,"t_first_token_unix":1779769924.989028,"t_done":339060.092477682,"t_done_unix":1779769925.1064208},{"request_id":"1269373:19:1349406:485","session_id":"1269373","input_length":76493,"t_proxy_recv":339057.060125167,"t_decision_unix":1779769922.0740666,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":74240,"affinity_cache_ratio":0.9705463245002811,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2253 < 8000)","cache_hit":74240,"estimated_new_tokens":2253,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":953,"cache_hit":7680,"new_prefill":68813,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":76493,"score_linear":62618.0,"score_lmetric":76493},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":62573,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":7680,"new_prefill":68813,"score_linear":43595.0,"score_lmetric":68813},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":76493,"score_linear":96026.0,"score_lmetric":76493},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1120,"cache_hit":0,"new_prefill":76493,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":7680,"new_prefill":68813,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1075,"cache_hit":74240,"new_prefill":2253,"score_linear":-74240.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":76493,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74240.0,"chosen_score_lmetric":0,"t_first_token":339058.298926962,"t_first_token_unix":1779769923.3128705,"t_done":339061.766168011,"t_done_unix":1779769926.7801116},{"request_id":"1350896:1:1350896:489","session_id":"1350896","input_length":1471,"t_proxy_recv":339062.128094197,"t_decision_unix":1779769927.1420355,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1471 < 8000)","cache_hit":0,"estimated_new_tokens":1471,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":1471,"score_linear":62618.0,"score_lmetric":1471},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":62573,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":1471,"score_linear":51275.0,"score_lmetric":1471},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":1471,"score_linear":96026.0,"score_lmetric":1471},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339062.237239955,"t_first_token_unix":1779769927.251183,"t_done":339062.604811423,"t_done_unix":1779769927.6187546},{"request_id":"1350980:1:1350980:490","session_id":"1350980","input_length":15313,"t_proxy_recv":339062.389251419,"t_decision_unix":1779769927.4031928,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":15313,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":15313,"score_linear":62618.0,"score_lmetric":15313},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51275,"ongoing_decode_tokens":62573,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":15313,"score_linear":51275.0,"score_lmetric":15313},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":15313,"score_linear":96026.0,"score_lmetric":15313},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":1471,"ongoing_decode_tokens":1471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":15313,"score_linear":1471.0,"score_lmetric":15313},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339063.759052205,"t_first_token_unix":1779769928.772996,"t_done":339063.796871819,"t_done_unix":1779769928.8108149},{"request_id":"1334223:3:1348241:479","session_id":"1334223","input_length":41202,"t_proxy_recv":339053.012696689,"t_decision_unix":1779769918.0266378,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37376,"affinity_cache_ratio":0.9071404300762099,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3826 < 8000)","cache_hit":37376,"estimated_new_tokens":3826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":41202,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":41202,"score_linear":62618.0,"score_lmetric":41202},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":37376,"new_prefill":3826,"score_linear":-27303.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":41202,"score_linear":96026.0,"score_lmetric":41202},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1087,"cache_hit":0,"new_prefill":41202,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":41202,"score_linear":65289.0,"score_lmetric":41202},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":41202,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1179,"cache_hit":0,"new_prefill":41202,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-27303.0,"chosen_score_lmetric":0,"t_first_token":339054.0207773,"t_first_token_unix":1779769919.0347211,"t_done":339064.640463421,"t_done_unix":1779769929.654407},{"request_id":"1253743:5:1351907:492","session_id":"1253743","input_length":51340,"t_proxy_recv":339065.911559327,"t_decision_unix":1779769930.9255002,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":50176,"affinity_cache_ratio":0.9773276197896377,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1164 < 8000)","cache_hit":50176,"estimated_new_tokens":1164,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":4608,"new_prefill":46732,"score_linear":58010.0,"score_lmetric":46732},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1463,"cache_hit":50176,"new_prefill":1164,"score_linear":-40103.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":49664,"new_prefill":1676,"score_linear":46362.0,"score_lmetric":1676},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1141,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40103.0,"chosen_score_lmetric":0,"t_first_token":339066.292646679,"t_first_token_unix":1779769931.3065906,"t_done":339066.869549276,"t_done_unix":1779769931.883493},{"request_id":"1344773:4:1352462:494","session_id":"1344773","input_length":33579,"t_proxy_recv":339067.707230488,"t_decision_unix":1779769932.7211716,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":33280,"affinity_cache_ratio":0.9910956252419667,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (299 < 8000)","cache_hit":33280,"estimated_new_tokens":299,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":954,"cache_hit":33280,"new_prefill":299,"score_linear":-33280.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":33579,"score_linear":62618.0,"score_lmetric":33579},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":33579,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":33579,"score_linear":96026.0,"score_lmetric":33579},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1141,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21436,"ongoing_decode_tokens":0,"pending_prefill_tokens":9148,"num_requests":1,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":33579,"score_linear":21436.0,"score_lmetric":42727},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-33280.0,"chosen_score_lmetric":0,"t_first_token":339067.784976226,"t_first_token_unix":1779769932.7989194,"t_done":339067.955095688,"t_done_unix":1779769932.9690394},{"request_id":"1350162:2:1352468:495","session_id":"1350162","input_length":10575,"t_proxy_recv":339067.720522422,"t_decision_unix":1779769932.734464,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":4608,"affinity_cache_ratio":0.4357446808510638,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5967 < 8000)","cache_hit":4608,"estimated_new_tokens":5967,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":33579,"ongoing_decode_tokens":0,"pending_prefill_tokens":299,"num_requests":1,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":10575,"score_linear":33579.0,"score_lmetric":10874},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":10575,"score_linear":62618.0,"score_lmetric":10575},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":10575,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":10575,"score_linear":96026.0,"score_lmetric":10575},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1141,"cache_hit":4608,"new_prefill":5967,"score_linear":-4608.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":10575,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21436,"ongoing_decode_tokens":0,"pending_prefill_tokens":9148,"num_requests":1,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":10575,"score_linear":21436.0,"score_lmetric":19723},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":10575,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":339068.286783268,"t_first_token_unix":1779769933.300727,"t_done":339068.411057676,"t_done_unix":1779769933.425001},{"request_id":"1348691:2:1352277:493","session_id":"1348691","input_length":21436,"t_proxy_recv":339067.09641492,"t_decision_unix":1779769932.1103563,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":12288,"affinity_cache_ratio":0.5732412763575294,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":12288,"estimated_new_tokens":9148,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":21436,"score_linear":62618.0,"score_lmetric":21436},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":21436,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":21436,"score_linear":96026.0,"score_lmetric":21436},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1141,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":12288,"new_prefill":9148,"score_linear":-12288.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-12288.0,"chosen_score_lmetric":0,"t_first_token":339068.392518551,"t_first_token_unix":1779769933.4064617,"t_done":339068.615120663,"t_done_unix":1779769933.6290638},{"request_id":"1345921:4:1353010:501","session_id":"1345921","input_length":6935,"t_proxy_recv":339069.700081205,"t_decision_unix":1779769934.7140226,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":2,"affinity_cache_hit":1024,"affinity_cache_ratio":0.14765681326604183,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6935 < 8000)","cache_hit":0,"estimated_new_tokens":6935,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":6935,"score_linear":18840.0,"score_lmetric":25775},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":6935,"score_linear":130877.0,"score_lmetric":13870},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":6935,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":512,"new_prefill":6423,"score_linear":95514.0,"score_lmetric":6423},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83326,"ongoing_decode_tokens":0,"pending_prefill_tokens":382,"num_requests":1,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":6935,"score_linear":83326.0,"score_lmetric":7317},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":6935,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":77790,"ongoing_decode_tokens":77790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1097,"cache_hit":0,"new_prefill":6935,"score_linear":77790.0,"score_lmetric":6935},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":14239,"ongoing_decode_tokens":0,"pending_prefill_tokens":14239,"num_requests":1,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":1024,"new_prefill":5911,"score_linear":13215.0,"score_lmetric":20150}],"chosen_score_linear":10073.0,"chosen_score_lmetric":0,"t_first_token":339070.173741486,"t_first_token_unix":1779769935.187685,"t_done":339070.178202383,"t_done_unix":1779769935.1921458},{"request_id":"1352606:1:1352606:496","session_id":"1352606","input_length":18840,"t_proxy_recv":339068.290379456,"t_decision_unix":1779769933.304321,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":18840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":18840,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":18840,"score_linear":62618.0,"score_lmetric":18840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":18840,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":18840,"score_linear":96026.0,"score_lmetric":18840},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":10575,"ongoing_decode_tokens":10575,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1141,"cache_hit":0,"new_prefill":18840,"score_linear":10575.0,"score_lmetric":18840},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":18840,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21436,"ongoing_decode_tokens":0,"pending_prefill_tokens":9148,"num_requests":1,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":18840,"score_linear":21436.0,"score_lmetric":27988},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":18840,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339070.218488031,"t_first_token_unix":1779769935.232432,"t_done":339070.667632932,"t_done_unix":1779769935.6815774},{"request_id":"1317598:6:1352957:499","session_id":"1317598","input_length":83326,"t_proxy_recv":339069.560042184,"t_decision_unix":1779769934.5739834,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":82944,"affinity_cache_ratio":0.9954155965724983,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (382 < 8000)","cache_hit":82944,"estimated_new_tokens":382,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":83326,"score_linear":18840.0,"score_lmetric":102166},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":83326,"score_linear":130877.0,"score_lmetric":166652},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":83326,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":83326,"score_linear":96026.0,"score_lmetric":83326},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":82944,"new_prefill":382,"score_linear":-82944.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":83326,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":77790,"ongoing_decode_tokens":77790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1097,"cache_hit":0,"new_prefill":83326,"score_linear":77790.0,"score_lmetric":83326},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":83326,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-82944.0,"chosen_score_lmetric":0,"t_first_token":339069.831720774,"t_first_token_unix":1779769934.8456645,"t_done":339070.891875713,"t_done_unix":1779769935.90582},{"request_id":"1349975:2:1352986:500","session_id":"1349975","input_length":14239,"t_proxy_recv":339069.659918121,"t_decision_unix":1779769934.6738596,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":7,"affinity_cache_hit":4608,"affinity_cache_ratio":0.32361823161738884,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":14239,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":14239,"score_linear":18840.0,"score_lmetric":33079},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":14239,"score_linear":130877.0,"score_lmetric":28478},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":14239,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":14239,"score_linear":96026.0,"score_lmetric":14239},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83326,"ongoing_decode_tokens":0,"pending_prefill_tokens":382,"num_requests":1,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":4608,"new_prefill":9631,"score_linear":78718.0,"score_lmetric":10013},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":14239,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":77790,"ongoing_decode_tokens":77790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1097,"cache_hit":0,"new_prefill":14239,"score_linear":77790.0,"score_lmetric":14239},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":14239,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339070.896988722,"t_first_token_unix":1779769935.9109318,"t_done":339071.087338472,"t_done_unix":1779769936.1012821},{"request_id":"1269373:20:1352797:498","session_id":"1269373","input_length":77790,"t_proxy_recv":339068.961445434,"t_decision_unix":1779769933.9753866,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":76800,"affinity_cache_ratio":0.9872734284612418,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (990 < 8000)","cache_hit":76800,"estimated_new_tokens":990,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":954,"cache_hit":7680,"new_prefill":70110,"score_linear":11160.0,"score_lmetric":88950},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":77790,"score_linear":130877.0,"score_lmetric":155580},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":7680,"new_prefill":70110,"score_linear":2393.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":77790,"score_linear":96026.0,"score_lmetric":77790},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":77790,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":7680,"new_prefill":70110,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1097,"cache_hit":76800,"new_prefill":990,"score_linear":-76800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":77790,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":339069.515194939,"t_first_token_unix":1779769934.529138,"t_done":339071.615293216,"t_done_unix":1779769936.629237},{"request_id":"1279412:23:1348089:477","session_id":"1279412","input_length":96026,"t_proxy_recv":339052.397795899,"t_decision_unix":1779769917.4117372,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9970632953575074,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (282 < 8000)","cache_hit":95744,"estimated_new_tokens":282,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":96026,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":96026,"score_linear":62618.0,"score_lmetric":96026},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1454,"cache_hit":0,"new_prefill":96026,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":95744,"new_prefill":282,"score_linear":-95744.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1087,"cache_hit":0,"new_prefill":96026,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1181,"cache_hit":0,"new_prefill":96026,"score_linear":65289.0,"score_lmetric":96026},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1036,"cache_hit":0,"new_prefill":96026,"score_linear":7065.0,"score_lmetric":96026},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1179,"cache_hit":12800,"new_prefill":83226,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-95744.0,"chosen_score_lmetric":0,"t_first_token":339052.550785544,"t_first_token_unix":1779769917.5647295,"t_done":339072.761073563,"t_done_unix":1779769937.775018},{"request_id":"1344773:5:1353365:502","session_id":"1344773","input_length":33875,"t_proxy_recv":339071.167315475,"t_decision_unix":1779769936.181257,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":33280,"affinity_cache_ratio":0.9824354243542436,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (595 < 8000)","cache_hit":33280,"estimated_new_tokens":595,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":990,"cache_hit":33280,"new_prefill":595,"score_linear":-33280.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":33875,"score_linear":130877.0,"score_lmetric":67750},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":33875,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":33875,"score_linear":96026.0,"score_lmetric":33875},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":33875,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":33875,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":77790,"ongoing_decode_tokens":77790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1097,"cache_hit":0,"new_prefill":33875,"score_linear":77790.0,"score_lmetric":33875},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":33875,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-33280.0,"chosen_score_lmetric":0,"t_first_token":339071.302560893,"t_first_token_unix":1779769936.3165045,"t_done":339073.160464611,"t_done_unix":1779769938.1744087},{"request_id":"1253743:6:1354346:505","session_id":"1253743","input_length":51436,"t_proxy_recv":339074.412747275,"t_decision_unix":1779769939.4266882,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9954117738548877,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (236 < 8000)","cache_hit":51200,"estimated_new_tokens":236,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":51436,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":4608,"new_prefill":46828,"score_linear":126269.0,"score_lmetric":93656},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":51200,"new_prefill":236,"score_linear":-41127.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":49664,"new_prefill":1772,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27651,"ongoing_decode_tokens":0,"pending_prefill_tokens":27651,"num_requests":1,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":51436,"score_linear":27651.0,"score_lmetric":79087},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":51436,"score_linear":83526.0,"score_lmetric":134962},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":51436,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":51436,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41127.0,"chosen_score_lmetric":0,"t_first_token":339074.528760427,"t_first_token_unix":1779769939.5427039,"t_done":339075.098403006,"t_done_unix":1779769940.1123471},{"request_id":"1353821:1:1353821:503","session_id":"1353821","input_length":27651,"t_proxy_recv":339072.58281969,"t_decision_unix":1779769937.596761,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":27651,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":33875,"ongoing_decode_tokens":33875,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":990,"cache_hit":0,"new_prefill":27651,"score_linear":33875.0,"score_lmetric":27651},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":27651,"score_linear":130877.0,"score_lmetric":55302},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":27651,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":27651,"score_linear":96026.0,"score_lmetric":27651},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339076.051376011,"t_first_token_unix":1779769941.0653198,"t_done":339078.351312642,"t_done_unix":1779769943.3652556},{"request_id":"1286804:3:1354910:506","session_id":"1286804","input_length":15519,"t_proxy_recv":339076.385117965,"t_decision_unix":1779769941.3990595,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":13824,"affinity_cache_ratio":0.8907790450415619,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1695 < 8000)","cache_hit":13824,"estimated_new_tokens":1695,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":15519,"score_linear":130877.0,"score_lmetric":31038},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":15519,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27651,"ongoing_decode_tokens":27651,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":13824,"new_prefill":1695,"score_linear":13827.0,"score_lmetric":1695},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":15519,"score_linear":83526.0,"score_lmetric":99045},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":13827.0,"chosen_score_lmetric":1695,"t_first_token":339077.793640244,"t_first_token_unix":1779769942.8075843,"t_done":339079.57221917,"t_done_unix":1779769944.5861623},{"request_id":"1355484:1:1355484:508","session_id":"1355484","input_length":19366,"t_proxy_recv":339078.293275223,"t_decision_unix":1779769943.3072166,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6566 < 8000)","cache_hit":12800,"estimated_new_tokens":6566,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":19366,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":19366,"score_linear":130877.0,"score_lmetric":38732},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":19366,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":12800,"new_prefill":6566,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43170,"ongoing_decode_tokens":43170,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":19366,"score_linear":43170.0,"score_lmetric":38732},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":19366,"score_linear":83526.0,"score_lmetric":102892},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24503,"ongoing_decode_tokens":0,"pending_prefill_tokens":24503,"num_requests":1,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":19366,"score_linear":24503.0,"score_lmetric":43869},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":12800,"new_prefill":6566,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":339079.228104054,"t_first_token_unix":1779769944.2420475,"t_done":339079.828315271,"t_done_unix":1779769944.8422585},{"request_id":"1344773:6:1356097:513","session_id":"1344773","input_length":34198,"t_proxy_recv":339080.544689979,"t_decision_unix":1779769945.5586317,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9881279606994561,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (406 < 8000)","cache_hit":33792,"estimated_new_tokens":406,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":33792,"new_prefill":406,"score_linear":-33792.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":34198,"score_linear":130877.0,"score_lmetric":68396},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":21371,"pending_prefill_tokens":3137,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":34198,"score_linear":58266.0,"score_lmetric":37335},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48977,"ongoing_decode_tokens":0,"pending_prefill_tokens":849,"num_requests":1,"active_p_offloads":0,"cached_blocks":1227,"cache_hit":0,"new_prefill":34198,"score_linear":48977.0,"score_lmetric":35047},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1209,"cache_hit":0,"new_prefill":34198,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":34198,"score_linear":83526.0,"score_lmetric":117724},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102592,"ongoing_decode_tokens":0,"pending_prefill_tokens":24768,"num_requests":2,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":34198,"score_linear":102592.0,"score_lmetric":117932},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":34198,"score_linear":35308.0,"score_lmetric":69506}],"chosen_score_linear":-33792.0,"chosen_score_lmetric":0,"t_first_token":339080.632782315,"t_first_token_unix":1779769945.6467254,"t_done":339080.793399316,"t_done_unix":1779769945.8073428},{"request_id":"1355325:1:1355325:507","session_id":"1355325","input_length":24503,"t_proxy_recv":339077.858985671,"t_decision_unix":1779769942.8729272,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":24503,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":24503,"score_linear":130877.0,"score_lmetric":49006},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":24503,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43170,"ongoing_decode_tokens":43170,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":24503,"score_linear":43170.0,"score_lmetric":49006},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":24503,"score_linear":83526.0,"score_lmetric":108029},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339080.686075926,"t_first_token_unix":1779769945.7000191,"t_done":339080.850387695,"t_done_unix":1779769945.864331},{"request_id":"1340278:2:1356079:512","session_id":"1340278","input_length":48977,"t_proxy_recv":339080.506691096,"t_decision_unix":1779769945.5206325,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9826653327071891,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (849 < 8000)","cache_hit":48128,"estimated_new_tokens":849,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":48977,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":48977,"score_linear":130877.0,"score_lmetric":97954},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":21371,"pending_prefill_tokens":3137,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":48977,"score_linear":58266.0,"score_lmetric":52114},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1227,"cache_hit":48128,"new_prefill":849,"score_linear":-48128.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1209,"cache_hit":0,"new_prefill":48977,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":48977,"score_linear":83526.0,"score_lmetric":132503},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102592,"ongoing_decode_tokens":0,"pending_prefill_tokens":24768,"num_requests":2,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":48977,"score_linear":102592.0,"score_lmetric":147490},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":48977,"score_linear":35308.0,"score_lmetric":84285}],"chosen_score_linear":-48128.0,"chosen_score_lmetric":0,"t_first_token":339080.786566532,"t_first_token_unix":1779769945.80051,"t_done":339081.459792205,"t_done_unix":1779769946.4737363},{"request_id":"1356948:1:1356948:515","session_id":"1356948","input_length":6711,"t_proxy_recv":339083.574174131,"t_decision_unix":1779769948.5881155,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6711 < 8000)","cache_hit":0,"estimated_new_tokens":6711,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":122597,"num_requests":1,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":6711,"score_linear":122597.0,"score_lmetric":129308},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":6711,"score_linear":130877.0,"score_lmetric":13422},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":21371,"pending_prefill_tokens":3137,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":6711,"score_linear":58266.0,"score_lmetric":9848},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1228,"cache_hit":0,"new_prefill":6711,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1209,"cache_hit":0,"new_prefill":6711,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":6711,"score_linear":83526.0,"score_lmetric":90237},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78089,"ongoing_decode_tokens":78089,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1146,"cache_hit":0,"new_prefill":6711,"score_linear":78089.0,"score_lmetric":6711},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":6711,"score_linear":35308.0,"score_lmetric":42019}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339084.029245045,"t_first_token_unix":1779769949.0431888,"t_done":339084.307304589,"t_done_unix":1779769949.3212485},{"request_id":"1269373:21:1355702:509","session_id":"1269373","input_length":78089,"t_proxy_recv":339078.973690247,"t_decision_unix":1779769943.9876316,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":77824,"affinity_cache_ratio":0.9966064362458221,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (265 < 8000)","cache_hit":77824,"estimated_new_tokens":265,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":7680,"new_prefill":70409,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":78089,"score_linear":130877.0,"score_lmetric":156178},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":7680,"new_prefill":70409,"score_linear":2393.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":19366,"ongoing_decode_tokens":0,"pending_prefill_tokens":6566,"num_requests":1,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":78089,"score_linear":19366.0,"score_lmetric":84655},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":15519,"ongoing_decode_tokens":15519,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":78089,"score_linear":15519.0,"score_lmetric":78089},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":7680,"new_prefill":70409,"score_linear":75846.0,"score_lmetric":153935},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24503,"ongoing_decode_tokens":0,"pending_prefill_tokens":24503,"num_requests":1,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":77824,"new_prefill":265,"score_linear":-53321.0,"score_lmetric":24768},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":78089,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53321.0,"chosen_score_lmetric":24768,"t_first_token":339080.801433756,"t_first_token_unix":1779769945.8153772,"t_done":339084.557446903,"t_done_unix":1779769949.5713909},{"request_id":"1355951:1:1355951:511","session_id":"1355951","input_length":35308,"t_proxy_recv":339079.88275924,"t_decision_unix":1779769944.8967006,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":35308,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":35308,"score_linear":130877.0,"score_lmetric":70616},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":21371,"pending_prefill_tokens":3137,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":35308,"score_linear":58266.0,"score_lmetric":38445},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1227,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1209,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":35308,"score_linear":83526.0,"score_lmetric":118834},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102592,"ongoing_decode_tokens":0,"pending_prefill_tokens":24768,"num_requests":2,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":35308,"score_linear":102592.0,"score_lmetric":120152},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339085.060394064,"t_first_token_unix":1779769950.0743372,"t_done":339085.44139377,"t_done_unix":1779769950.4553375},{"request_id":"1353821:2:1357420:517","session_id":"1353821","input_length":27805,"t_proxy_recv":339084.966119685,"t_decision_unix":1779769949.9800608,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":27648,"affinity_cache_ratio":0.9943535335371336,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (157 < 8000)","cache_hit":27648,"estimated_new_tokens":157,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":122597,"num_requests":1,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":27805,"score_linear":122597.0,"score_lmetric":150402},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":27805,"score_linear":130877.0,"score_lmetric":55610},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":21371,"pending_prefill_tokens":3137,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":27805,"score_linear":58266.0,"score_lmetric":30942},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13520,"ongoing_decode_tokens":0,"pending_prefill_tokens":13520,"num_requests":1,"active_p_offloads":0,"cached_blocks":1228,"cache_hit":0,"new_prefill":27805,"score_linear":13520.0,"score_lmetric":41325},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":27648,"new_prefill":157,"score_linear":-27648.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":27805,"score_linear":83526.0,"score_lmetric":111331},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":27805,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":27805,"score_linear":35308.0,"score_lmetric":63113}],"chosen_score_linear":-27648.0,"chosen_score_lmetric":0,"t_first_token":339085.027522952,"t_first_token_unix":1779769950.0414665,"t_done":339085.789674954,"t_done_unix":1779769950.8036187},{"request_id":"1357132:1:1357132:516","session_id":"1357132","input_length":13520,"t_proxy_recv":339084.119632896,"t_decision_unix":1779769949.1335742,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":13520,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":122597,"num_requests":1,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":13520,"score_linear":122597.0,"score_lmetric":136117},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":13520,"score_linear":130877.0,"score_lmetric":27040},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":21371,"pending_prefill_tokens":3137,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":13520,"score_linear":58266.0,"score_lmetric":16657},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1228,"cache_hit":0,"new_prefill":13520,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":6711,"ongoing_decode_tokens":6711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1209,"cache_hit":0,"new_prefill":13520,"score_linear":6711.0,"score_lmetric":13520},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":13520,"score_linear":83526.0,"score_lmetric":97046},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78089,"ongoing_decode_tokens":78089,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1146,"cache_hit":0,"new_prefill":13520,"score_linear":78089.0,"score_lmetric":13520},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":13520,"score_linear":35308.0,"score_lmetric":48828}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339085.275348833,"t_first_token_unix":1779769950.289292,"t_done":339085.886577839,"t_done_unix":1779769950.900521},{"request_id":"1253743:7:1359251:524","session_id":"1253743","input_length":51509,"t_proxy_recv":339091.610921812,"t_decision_unix":1779769956.6248631,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9940010483604806,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (309 < 8000)","cache_hit":51200,"estimated_new_tokens":309,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":51509,"score_linear":156948.0,"score_lmetric":349330},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":4608,"new_prefill":46901,"score_linear":126269.0,"score_lmetric":93802},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":69564,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":51200,"new_prefill":309,"score_linear":7066.0,"score_lmetric":309},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":49664,"new_prefill":1845,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":51509,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":51509,"score_linear":83526.0,"score_lmetric":135035},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":51509,"score_linear":80246.0,"score_lmetric":51509},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":34694,"num_requests":1,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":51509,"score_linear":58758.0,"score_lmetric":86203}],"chosen_score_linear":7066.0,"chosen_score_lmetric":309,"t_first_token":339091.717809273,"t_first_token_unix":1779769956.7317526,"t_done":339092.047179934,"t_done_unix":1779769957.0611236},{"request_id":"1355484:2:1359370:525","session_id":"1355484","input_length":19493,"t_proxy_recv":339092.066242225,"t_decision_unix":1779769957.0801835,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":19456,"affinity_cache_ratio":0.9981018827271328,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (37 < 8000)","cache_hit":19456,"estimated_new_tokens":37,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":19493,"score_linear":156948.0,"score_lmetric":285298},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":19493,"score_linear":130877.0,"score_lmetric":38986},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":69564,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":19493,"score_linear":58266.0,"score_lmetric":19493},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":19456,"new_prefill":37,"score_linear":-19456.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":19493,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":19493,"score_linear":83526.0,"score_lmetric":103019},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":19493,"score_linear":80246.0,"score_lmetric":19493},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":34694,"num_requests":1,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":12800,"new_prefill":6693,"score_linear":45958.0,"score_lmetric":41387}],"chosen_score_linear":-19456.0,"chosen_score_lmetric":0,"t_first_token":339092.124623152,"t_first_token_unix":1779769957.1385665,"t_done":339093.250727592,"t_done_unix":1779769958.2646713},{"request_id":"1294611:11:1355750:510","session_id":"1294611","input_length":48193,"t_proxy_recv":339079.070641453,"t_decision_unix":1779769944.0845828,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9349075591890939,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3137 < 8000)","cache_hit":45056,"estimated_new_tokens":3137,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":48193,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":48193,"score_linear":130877.0,"score_lmetric":96386},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":45056,"new_prefill":3137,"score_linear":-34983.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":19366,"ongoing_decode_tokens":0,"pending_prefill_tokens":6566,"num_requests":1,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":48193,"score_linear":19366.0,"score_lmetric":54759},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":15519,"ongoing_decode_tokens":15519,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1206,"cache_hit":0,"new_prefill":48193,"score_linear":15519.0,"score_lmetric":48193},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":12800,"new_prefill":35393,"score_linear":70726.0,"score_lmetric":118919},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102592,"ongoing_decode_tokens":0,"pending_prefill_tokens":24768,"num_requests":2,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":12800,"new_prefill":35393,"score_linear":89792.0,"score_lmetric":120322},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":48193,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-34983.0,"chosen_score_lmetric":0,"t_first_token":339087.824694667,"t_first_token_unix":1779769952.8386383,"t_done":339097.659670134,"t_done_unix":1779769962.6736135},{"request_id":"1353821:3:1360898:529","session_id":"1353821","input_length":27933,"t_proxy_recv":339097.3827402,"t_decision_unix":1779769962.3966818,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":27648,"affinity_cache_ratio":0.98979701428418,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (285 < 8000)","cache_hit":27648,"estimated_new_tokens":285,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":27933,"score_linear":156948.0,"score_lmetric":302178},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":27933,"score_linear":130877.0,"score_lmetric":55866},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":69564,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":27933,"score_linear":58266.0,"score_lmetric":27933},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":27933,"score_linear":117888.0,"score_lmetric":145821},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":27648,"new_prefill":285,"score_linear":-27648.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":83526,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":27933,"score_linear":83526.0,"score_lmetric":27933},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":27933,"score_linear":80246.0,"score_lmetric":27933},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":92236,"ongoing_decode_tokens":58758,"pending_prefill_tokens":5318,"num_requests":2,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":27933,"score_linear":92236.0,"score_lmetric":66502}],"chosen_score_linear":-27648.0,"chosen_score_lmetric":0,"t_first_token":339097.437683527,"t_first_token_unix":1779769962.4516265,"t_done":339097.836733222,"t_done_unix":1779769962.8506765},{"request_id":"1353951:1:1353951:504","session_id":"1353951","input_length":83526,"t_proxy_recv":339073.069101028,"t_decision_unix":1779769938.0830424,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":83526,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":33875,"ongoing_decode_tokens":33875,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":990,"cache_hit":0,"new_prefill":83526,"score_linear":33875.0,"score_lmetric":83526},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":83526,"score_linear":130877.0,"score_lmetric":167052},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":83526,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27651,"ongoing_decode_tokens":0,"pending_prefill_tokens":27651,"num_requests":1,"active_p_offloads":0,"cached_blocks":1152,"cache_hit":0,"new_prefill":83526,"score_linear":27651.0,"score_lmetric":111177},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1099,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339096.612165607,"t_first_token_unix":1779769961.626111,"t_done":339098.29526339,"t_done_unix":1779769963.309207},{"request_id":"1353951:2:1358300:520","session_id":"1353951","input_length":83905,"t_proxy_recv":339098.311638324,"t_decision_unix":1779769963.32558,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":83456,"affinity_cache_ratio":0.9946487098504261,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (449 < 8000)","cache_hit":83456,"estimated_new_tokens":449,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":83905,"score_linear":156948.0,"score_lmetric":414122},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":83905,"score_linear":130877.0,"score_lmetric":167810},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":83905,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":83905,"score_linear":137602.0,"score_lmetric":404102},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":83905,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66032,"ongoing_decode_tokens":66032,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1472,"cache_hit":83456,"new_prefill":449,"score_linear":-17424.0,"score_lmetric":449},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":83905,"score_linear":80246.0,"score_lmetric":83905},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":92236,"ongoing_decode_tokens":58758,"pending_prefill_tokens":5318,"num_requests":2,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":83905,"score_linear":92236.0,"score_lmetric":178446}],"chosen_score_linear":-17424.0,"chosen_score_lmetric":449,"t_first_token":339098.581764658,"t_first_token_unix":1779769963.5957081,"t_done":339098.827235059,"t_done_unix":1779769963.841179},{"request_id":"1342921:2:1361022:530","session_id":"1342921","input_length":66032,"t_proxy_recv":339097.849007843,"t_decision_unix":1779769962.862949,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":65536,"affinity_cache_ratio":0.992488490428883,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (496 < 8000)","cache_hit":65536,"estimated_new_tokens":496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":66032,"score_linear":156948.0,"score_lmetric":378376},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":66032,"score_linear":130877.0,"score_lmetric":132064},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":66032,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":66032,"score_linear":117888.0,"score_lmetric":183920},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":66032,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":83526,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":65536,"new_prefill":496,"score_linear":17990.0,"score_lmetric":496},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":66032,"score_linear":80246.0,"score_lmetric":66032},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":92236,"ongoing_decode_tokens":58758,"pending_prefill_tokens":5318,"num_requests":2,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":66032,"score_linear":92236.0,"score_lmetric":142700}],"chosen_score_linear":17990.0,"chosen_score_lmetric":496,"t_first_token":339098.190622824,"t_first_token_unix":1779769963.2045662,"t_done":339098.895072912,"t_done_unix":1779769963.9090166},{"request_id":"1356948:2:1361262:532","session_id":"1356948","input_length":16480,"t_proxy_recv":339098.861898529,"t_decision_unix":1779769963.87584,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":6656,"affinity_cache_ratio":0.40388349514563104,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":6656,"estimated_new_tokens":9824,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":16480,"score_linear":156948.0,"score_lmetric":279272},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":16480,"score_linear":130877.0,"score_lmetric":32960},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":16480,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":16480,"score_linear":137602.0,"score_lmetric":269252},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":6656,"new_prefill":9824,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66032,"ongoing_decode_tokens":66032,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1472,"cache_hit":0,"new_prefill":16480,"score_linear":66032.0,"score_lmetric":16480},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":16480,"score_linear":80246.0,"score_lmetric":16480},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":92236,"ongoing_decode_tokens":58758,"pending_prefill_tokens":5318,"num_requests":2,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":16480,"score_linear":92236.0,"score_lmetric":43596}],"chosen_score_linear":-6656.0,"chosen_score_lmetric":0,"t_first_token":339100.013349136,"t_first_token_unix":1779769965.0272927,"t_done":339100.180518528,"t_done_unix":1779769965.1944623},{"request_id":"1361992:1:1361992:535","session_id":"1361992","input_length":8113,"t_proxy_recv":339101.437673959,"t_decision_unix":1779769966.4516153,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":8113,"score_linear":156948.0,"score_lmetric":262538},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":8113,"score_linear":130877.0,"score_lmetric":16226},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":8113,"score_linear":87042.0,"score_lmetric":85082},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":8113,"score_linear":137602.0,"score_lmetric":252518},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1241,"cache_hit":0,"new_prefill":8113,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1473,"cache_hit":0,"new_prefill":8113,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":8113,"score_linear":80246.0,"score_lmetric":8113},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":92236,"ongoing_decode_tokens":58758,"pending_prefill_tokens":5318,"num_requests":2,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":8113,"score_linear":92236.0,"score_lmetric":26862}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339102.009896743,"t_first_token_unix":1779769967.0238404,"t_done":339102.150197245,"t_done_unix":1779769967.1641402},{"request_id":"1270340:3:1360716:528","session_id":"1270340","input_length":33478,"t_proxy_recv":339096.724705596,"t_decision_unix":1779769961.738647,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28160,"affinity_cache_ratio":0.8411494115538563,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5318 < 8000)","cache_hit":28160,"estimated_new_tokens":5318,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":33478,"score_linear":156948.0,"score_lmetric":313268},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":33478,"score_linear":130877.0,"score_lmetric":66956},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":69564,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":33478,"score_linear":58266.0,"score_lmetric":33478},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":33478,"score_linear":117888.0,"score_lmetric":151366},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":1536,"new_prefill":31942,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":83526,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":33478,"score_linear":83526.0,"score_lmetric":33478},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":33478,"score_linear":80246.0,"score_lmetric":33478},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":34694,"num_requests":1,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":28160,"new_prefill":5318,"score_linear":30598.0,"score_lmetric":40012}],"chosen_score_linear":30598.0,"chosen_score_lmetric":40012,"t_first_token":339101.705452243,"t_first_token_unix":1779769966.7193959,"t_done":339102.479640962,"t_done_unix":1779769967.4935842},{"request_id":"1362265:1:1362265:536","session_id":"1362265","input_length":8825,"t_proxy_recv":339102.283690696,"t_decision_unix":1779769967.297632,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8825,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":8825,"score_linear":156948.0,"score_lmetric":263962},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":8825,"score_linear":130877.0,"score_lmetric":17650},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":8825,"score_linear":87042.0,"score_lmetric":85794},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":8825,"score_linear":137602.0,"score_lmetric":253942},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":8825,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1473,"cache_hit":0,"new_prefill":8825,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":8825,"score_linear":80246.0,"score_lmetric":8825},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":92236,"ongoing_decode_tokens":92236,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":8825,"score_linear":92236.0,"score_lmetric":17650}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339102.947335322,"t_first_token_unix":1779769967.9612787,"t_done":339104.016972072,"t_done_unix":1779769969.0309165},{"request_id":"1361992:2:1362904:537","session_id":"1361992","input_length":10347,"t_proxy_recv":339104.529475095,"t_decision_unix":1779769969.5434163,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":7680,"affinity_cache_ratio":0.7422441287329661,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2667 < 8000)","cache_hit":7680,"estimated_new_tokens":2667,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":10347,"score_linear":156948.0,"score_lmetric":267006},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":10347,"score_linear":130877.0,"score_lmetric":20694},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":10347,"score_linear":87042.0,"score_lmetric":87316},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":10347,"score_linear":137602.0,"score_lmetric":256986},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":7680,"new_prefill":2667,"score_linear":-7680.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":0,"new_prefill":10347,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":10347,"score_linear":80246.0,"score_lmetric":10347},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1316,"cache_hit":0,"new_prefill":10347,"score_linear":58758.0,"score_lmetric":10347}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":339104.793699723,"t_first_token_unix":1779769969.8076441,"t_done":339104.941046601,"t_done_unix":1779769969.954991},{"request_id":"1363440:1:1363440:542","session_id":"1363440","input_length":543,"t_proxy_recv":339106.47831978,"t_decision_unix":1779769971.4922612,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":543,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (543 < 8000)","cache_hit":0,"estimated_new_tokens":543,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":543,"score_linear":156948.0,"score_lmetric":247398},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":543,"score_linear":130877.0,"score_lmetric":1086},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":543,"score_linear":87042.0,"score_lmetric":77512},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":543,"score_linear":137602.0,"score_lmetric":237378},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":543,"score_linear":90778.0,"score_lmetric":127346},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":16057,"ongoing_decode_tokens":0,"pending_prefill_tokens":2233,"num_requests":1,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":0,"new_prefill":543,"score_linear":16057.0,"score_lmetric":2776},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":543,"score_linear":80246.0,"score_lmetric":543},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95990,"ongoing_decode_tokens":58758,"pending_prefill_tokens":1904,"num_requests":2,"active_p_offloads":0,"cached_blocks":1316,"cache_hit":0,"new_prefill":543,"score_linear":95990.0,"score_lmetric":4894}],"chosen_score_linear":80246.0,"chosen_score_lmetric":543,"t_first_token":339106.690303087,"t_first_token_unix":1779769971.7042465,"t_done":339107.071324636,"t_done_unix":1779769972.0852687},{"request_id":"1355951:2:1363365:540","session_id":"1355951","input_length":37232,"t_proxy_recv":339106.186712317,"t_decision_unix":1779769971.2006536,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9488611946712505,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1904 < 8000)","cache_hit":35328,"estimated_new_tokens":1904,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":37232,"score_linear":156948.0,"score_lmetric":320776},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":37232,"score_linear":130877.0,"score_lmetric":74464},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":37232,"score_linear":87042.0,"score_lmetric":114201},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":37232,"score_linear":137602.0,"score_lmetric":310756},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":37232,"score_linear":62685.0,"score_lmetric":99917},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":16057,"ongoing_decode_tokens":0,"pending_prefill_tokens":2233,"num_requests":1,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":0,"new_prefill":37232,"score_linear":16057.0,"score_lmetric":39465},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":37232,"score_linear":80246.0,"score_lmetric":37232},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1316,"cache_hit":35328,"new_prefill":1904,"score_linear":23430.0,"score_lmetric":1904}],"chosen_score_linear":23430.0,"chosen_score_lmetric":1904,"t_first_token":339106.787833596,"t_first_token_unix":1779769971.801777,"t_done":339107.810327079,"t_done_unix":1779769972.8242712},{"request_id":"1313181:2:1358046:519","session_id":"1313181","input_length":58758,"t_proxy_recv":339087.255683634,"t_decision_unix":1779769952.2696252,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24064,"affinity_cache_ratio":0.40954423227475406,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":24064,"estimated_new_tokens":34694,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":58758,"score_linear":156948.0,"score_lmetric":363828},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":58758,"score_linear":130877.0,"score_lmetric":117516},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":21371,"pending_prefill_tokens":3137,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":58758,"score_linear":58266.0,"score_lmetric":61895},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":12800,"new_prefill":45958,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":58758,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":58758,"score_linear":83526.0,"score_lmetric":142284},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":58758,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":24064,"new_prefill":34694,"score_linear":-24064.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":339096.986321307,"t_first_token_unix":1779769962.0002651,"t_done":339108.563854367,"t_done_unix":1779769973.5777984},{"request_id":"1313181:3:1316742:330","session_id":"1313181","input_length":24451,"t_proxy_recv":339108.567863349,"t_decision_unix":1779769973.581805,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9841724264856243,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (387 < 8000)","cache_hit":24064,"estimated_new_tokens":387,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":24451,"score_linear":156948.0,"score_lmetric":295214},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":24451,"score_linear":130877.0,"score_lmetric":48902},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":24451,"score_linear":87042.0,"score_lmetric":101420},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":12800,"new_prefill":11651,"score_linear":124802.0,"score_lmetric":259594},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":24451,"score_linear":90778.0,"score_lmetric":175162},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":29954,"ongoing_decode_tokens":16057,"pending_prefill_tokens":6217,"num_requests":2,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":0,"new_prefill":24451,"score_linear":29954.0,"score_lmetric":61336},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":92882,"ongoing_decode_tokens":80246,"pending_prefill_tokens":8540,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":24451,"score_linear":92882.0,"score_lmetric":65982},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":24064,"new_prefill":387,"score_linear":-24064.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":339108.616651674,"t_first_token_unix":1779769973.6305957,"t_done":339109.209625712,"t_done_unix":1779769974.2235694},{"request_id":"1363943:1:1363943:543","session_id":"1363943","input_length":12636,"t_proxy_recv":339108.16049025,"t_decision_unix":1779769973.1744316,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":8540,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"local_cost 1.07s <= migrate_cost 0.99s + margin 0.20s","v2_chosen_cache_hit":4096,"v2_src_idx":0,"v2_src_cache_hit":8704,"v2_new_local":8540,"v2_new_src":3932,"v2_cost_local_s":1.0675,"v2_cost_src_interf_s":0.3932,"v2_cost_xfer_s":0.5951388888888889,"v2_cost_migrate_s":0.9883388888888889,"cache_hit":4096,"estimated_new_tokens":8540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":8704,"new_prefill":3932,"score_linear":148244.0,"score_lmetric":254176},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":12636,"score_linear":130877.0,"score_lmetric":25272},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":4096,"new_prefill":8540,"score_linear":82946.0,"score_lmetric":85509},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":12636,"score_linear":137602.0,"score_lmetric":261564},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":12636,"score_linear":90778.0,"score_lmetric":151532},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":16057,"ongoing_decode_tokens":16057,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":4096,"new_prefill":8540,"score_linear":11961.0,"score_lmetric":8540},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":4096,"new_prefill":8540,"score_linear":76150.0,"score_lmetric":8540},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1319,"cache_hit":0,"new_prefill":12636,"score_linear":58758.0,"score_lmetric":12636}],"chosen_score_linear":76150.0,"chosen_score_lmetric":8540,"t_first_token":339109.01734832,"t_first_token_unix":1779769974.031292,"t_done":339111.847817239,"t_done_unix":1779769976.8617601},{"request_id":"1364960:1:1364960:549","session_id":"1364960","input_length":2019,"t_proxy_recv":339111.570407845,"t_decision_unix":1779769976.5843492,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":4038,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2019 < 8000)","cache_hit":0,"estimated_new_tokens":2019,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":2019,"score_linear":156948.0,"score_lmetric":250350},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":2019,"score_linear":130877.0,"score_lmetric":4038},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":2019,"score_linear":87042.0,"score_lmetric":78988},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":235640,"ongoing_decode_tokens":0,"pending_prefill_tokens":118392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":2019,"score_linear":235640.0,"score_lmetric":361233},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":2019,"score_linear":90778.0,"score_lmetric":130298},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":29954,"ongoing_decode_tokens":29954,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1494,"cache_hit":0,"new_prefill":2019,"score_linear":29954.0,"score_lmetric":4038},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":92882,"ongoing_decode_tokens":92882,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":2019,"score_linear":92882.0,"score_lmetric":4038},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":68220,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":2019,"score_linear":68220.0,"score_lmetric":74938}],"chosen_score_linear":92882.0,"chosen_score_lmetric":4038,"t_first_token":339111.836847013,"t_first_token_unix":1779769976.8507905,"t_done":339112.517426275,"t_done_unix":1779769977.5313697},{"request_id":"1363440:2:1365095:551","session_id":"1363440","input_length":3866,"t_proxy_recv":339112.153057024,"t_decision_unix":1779769977.1669984,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":512,"affinity_cache_ratio":0.13243662700465597,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":6708,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3354 < 8000)","cache_hit":512,"estimated_new_tokens":3354,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":3866,"score_linear":156948.0,"score_lmetric":254044},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":3866,"score_linear":130877.0,"score_lmetric":7732},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":3866,"score_linear":87042.0,"score_lmetric":80835},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":250361,"ongoing_decode_tokens":0,"pending_prefill_tokens":119801,"num_requests":4,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":3866,"score_linear":250361.0,"score_lmetric":494668},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":3866,"score_linear":90778.0,"score_lmetric":133992},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":29954,"ongoing_decode_tokens":29954,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1494,"cache_hit":0,"new_prefill":3866,"score_linear":29954.0,"score_lmetric":7732},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":82265,"ongoing_decode_tokens":82265,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1165,"cache_hit":512,"new_prefill":3354,"score_linear":81753.0,"score_lmetric":6708},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":68220,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":3866,"score_linear":68220.0,"score_lmetric":78632}],"chosen_score_linear":81753.0,"chosen_score_lmetric":6708,"t_first_token":339112.474166284,"t_first_token_unix":1779769977.4881096,"t_done":339113.002755807,"t_done_unix":1779769978.0166993},{"request_id":"1305906:2:1363233:539","session_id":"1305906","input_length":16057,"t_proxy_recv":339105.646476886,"t_decision_unix":1779769970.6604176,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":13824,"affinity_cache_ratio":0.8609329264495236,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2233 < 8000)","cache_hit":13824,"estimated_new_tokens":2233,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":16057,"score_linear":156948.0,"score_lmetric":278426},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":16057,"score_linear":130877.0,"score_lmetric":32114},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":16057,"score_linear":87042.0,"score_lmetric":93026},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":16057,"score_linear":137602.0,"score_lmetric":268406},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":16057,"score_linear":62685.0,"score_lmetric":78742},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":13824,"new_prefill":2233,"score_linear":-13824.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":16057,"score_linear":80246.0,"score_lmetric":16057},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1316,"cache_hit":0,"new_prefill":16057,"score_linear":58758.0,"score_lmetric":16057}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":339107.112124131,"t_first_token_unix":1779769972.1260674,"t_done":339113.072533023,"t_done_unix":1779769978.0864768},{"request_id":"1364090:1:1364090:544","session_id":"1364090","input_length":13897,"t_proxy_recv":339108.538998721,"t_decision_unix":1779769973.5529401,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":6217,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6217 < 8000)","cache_hit":7680,"estimated_new_tokens":6217,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":7680,"new_prefill":6217,"score_linear":149268.0,"score_lmetric":258746},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":13897,"score_linear":130877.0,"score_lmetric":27794},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":7680,"new_prefill":6217,"score_linear":79362.0,"score_lmetric":83186},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":13897,"score_linear":137602.0,"score_lmetric":264086},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":13897,"score_linear":90778.0,"score_lmetric":154054},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":16057,"ongoing_decode_tokens":16057,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":7680,"new_prefill":6217,"score_linear":8377.0,"score_lmetric":6217},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":92882,"ongoing_decode_tokens":80246,"pending_prefill_tokens":8540,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":11264,"new_prefill":2633,"score_linear":81618.0,"score_lmetric":22346},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1319,"cache_hit":0,"new_prefill":13897,"score_linear":58758.0,"score_lmetric":13897}],"chosen_score_linear":8377.0,"chosen_score_lmetric":6217,"t_first_token":339109.778994565,"t_first_token_unix":1779769974.792938,"t_done":339114.051552018,"t_done_unix":1779769979.0654957},{"request_id":"1365440:1:1365440:552","session_id":"1365440","input_length":9913,"t_proxy_recv":339113.372404926,"t_decision_unix":1779769978.3863463,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":9913,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":9913,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":9913,"score_linear":156948.0,"score_lmetric":266138},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":9913,"score_linear":130877.0,"score_lmetric":19826},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":9913,"score_linear":87042.0,"score_lmetric":86882},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":250361,"ongoing_decode_tokens":0,"pending_prefill_tokens":119801,"num_requests":4,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":9913,"score_linear":250361.0,"score_lmetric":518856},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":9913,"score_linear":90778.0,"score_lmetric":146086},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13897,"ongoing_decode_tokens":13897,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1499,"cache_hit":0,"new_prefill":9913,"score_linear":13897.0,"score_lmetric":9913},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1175,"cache_hit":0,"new_prefill":9913,"score_linear":80246.0,"score_lmetric":9913},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":68220,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":9913,"score_linear":68220.0,"score_lmetric":90726}],"chosen_score_linear":80246.0,"chosen_score_lmetric":9913,"t_first_token":339114.196797055,"t_first_token_unix":1779769979.210741,"t_done":339114.427551629,"t_done_unix":1779769979.441495},{"request_id":"1366128:1:1366128:553","session_id":"1366128","input_length":11154,"t_proxy_recv":339115.668306554,"t_decision_unix":1779769980.6822479,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11154,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":11154,"score_linear":156948.0,"score_lmetric":268620},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":11154,"score_linear":130877.0,"score_lmetric":22308},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":11154,"score_linear":87042.0,"score_lmetric":88123},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":250361,"ongoing_decode_tokens":0,"pending_prefill_tokens":119801,"num_requests":4,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":11154,"score_linear":250361.0,"score_lmetric":523820},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":11154,"score_linear":90778.0,"score_lmetric":148568},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1512,"cache_hit":0,"new_prefill":11154,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":0,"new_prefill":11154,"score_linear":80246.0,"score_lmetric":11154},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":68220,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":11154,"score_linear":68220.0,"score_lmetric":93208}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339116.55684863,"t_first_token_unix":1779769981.5707924,"t_done":339117.067656902,"t_done_unix":1779769982.0816007},{"request_id":"1353821:4:1363399:541","session_id":"1353821","input_length":28093,"t_proxy_recv":339106.306942279,"t_decision_unix":1779769971.320884,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":27648,"affinity_cache_ratio":0.984159755099135,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (445 < 8000)","cache_hit":27648,"estimated_new_tokens":445,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":28093,"score_linear":156948.0,"score_lmetric":302498},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":28093,"score_linear":130877.0,"score_lmetric":56186},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":28093,"score_linear":87042.0,"score_lmetric":105062},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":28093,"score_linear":137602.0,"score_lmetric":292478},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":1,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":27648,"new_prefill":445,"score_linear":35037.0,"score_lmetric":63130},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":16057,"ongoing_decode_tokens":0,"pending_prefill_tokens":2233,"num_requests":1,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":0,"new_prefill":28093,"score_linear":16057.0,"score_lmetric":30326},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":28093,"score_linear":80246.0,"score_lmetric":28093},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95990,"ongoing_decode_tokens":58758,"pending_prefill_tokens":1904,"num_requests":2,"active_p_offloads":0,"cached_blocks":1316,"cache_hit":0,"new_prefill":28093,"score_linear":95990.0,"score_lmetric":59994}],"chosen_score_linear":35037.0,"chosen_score_lmetric":63130,"t_first_token":339119.231981232,"t_first_token_unix":1779769984.245925,"t_done":339119.477395385,"t_done_unix":1779769984.491339},{"request_id":"1363093:1:1363093:538","session_id":"1363093","input_length":62685,"t_proxy_recv":339105.164955942,"t_decision_unix":1779769970.1788971,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":62685,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":62685,"score_linear":156948.0,"score_lmetric":371682},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":62685,"score_linear":130877.0,"score_lmetric":125370},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":62685,"score_linear":87042.0,"score_lmetric":139654},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":62685,"score_linear":137602.0,"score_lmetric":361662},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":62685,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":0,"new_prefill":62685,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":62685,"score_linear":80246.0,"score_lmetric":62685},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1316,"cache_hit":0,"new_prefill":62685,"score_linear":58758.0,"score_lmetric":62685}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339119.230700618,"t_first_token_unix":1779769984.24465,"t_done":339119.820462161,"t_done_unix":1779769984.834406},{"request_id":"1363943:2:1366428:556","session_id":"1363943","input_length":12984,"t_proxy_recv":339116.77755957,"t_decision_unix":1779769981.7915006,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":12800,"affinity_cache_ratio":0.9858287122612446,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (184 < 8000)","cache_hit":12800,"estimated_new_tokens":184,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":8704,"new_prefill":4280,"score_linear":148244.0,"score_lmetric":254872},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":12984,"score_linear":130877.0,"score_lmetric":25968},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":4096,"new_prefill":8888,"score_linear":82946.0,"score_lmetric":85857},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":250361,"ongoing_decode_tokens":0,"pending_prefill_tokens":119801,"num_requests":4,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":12984,"score_linear":250361.0,"score_lmetric":531140},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":12984,"score_linear":90778.0,"score_lmetric":152228},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":11154,"ongoing_decode_tokens":11154,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1512,"cache_hit":4096,"new_prefill":8888,"score_linear":7058.0,"score_lmetric":8888},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":12800,"new_prefill":184,"score_linear":67446.0,"score_lmetric":184},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":68220,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":12984,"score_linear":68220.0,"score_lmetric":96868}],"chosen_score_linear":67446.0,"chosen_score_lmetric":184,"t_first_token":339116.857766129,"t_first_token_unix":1779769981.87171,"t_done":339120.338058336,"t_done_unix":1779769985.352002},{"request_id":"1367424:1:1367424:569","session_id":"1367424","input_length":3392,"t_proxy_recv":339120.403255677,"t_decision_unix":1779769985.4171972,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.625,"fallback_score":3392,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3392 < 8000)","cache_hit":0,"estimated_new_tokens":3392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":3392,"score_linear":156948.0,"score_lmetric":253096},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157714,"ongoing_decode_tokens":130877,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":3392,"score_linear":157714.0,"score_lmetric":90687},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":3392,"score_linear":87042.0,"score_lmetric":80361},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":3392,"score_linear":336145.0,"score_lmetric":754212},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":63232,"ongoing_decode_tokens":0,"pending_prefill_tokens":13749,"num_requests":2,"active_p_offloads":0,"cached_blocks":1383,"cache_hit":0,"new_prefill":3392,"score_linear":63232.0,"score_lmetric":34282},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44360,"ongoing_decode_tokens":0,"pending_prefill_tokens":24904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":3392,"score_linear":44360.0,"score_lmetric":84888},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":0,"new_prefill":3392,"score_linear":80246.0,"score_lmetric":3392},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":59514,"pending_prefill_tokens":7737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":3392,"score_linear":109237.0,"score_lmetric":33387}],"chosen_score_linear":80246.0,"chosen_score_lmetric":3392,"t_first_token":339120.730553463,"t_first_token_unix":1779769985.744497,"t_done":339120.855407683,"t_done_unix":1779769985.8693511},{"request_id":"1367312:1:1367312:566","session_id":"1367312","input_length":13749,"t_proxy_recv":339119.999403588,"t_decision_unix":1779769985.0133443,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":13749,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":13749,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":13749,"score_linear":156948.0,"score_lmetric":273810},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":13749,"score_linear":130877.0,"score_lmetric":27498},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":13749,"score_linear":87042.0,"score_lmetric":90718},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":13749,"score_linear":336145.0,"score_lmetric":816354},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1383,"cache_hit":0,"new_prefill":13749,"score_linear":49483.0,"score_lmetric":13749},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":13749,"score_linear":34226.0,"score_lmetric":74446},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93230,"ongoing_decode_tokens":93230,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":0,"new_prefill":13749,"score_linear":93230.0,"score_lmetric":27498},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":59514,"pending_prefill_tokens":7737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":13749,"score_linear":109237.0,"score_lmetric":64458}],"chosen_score_linear":49483.0,"chosen_score_lmetric":13749,"t_first_token":339121.181323925,"t_first_token_unix":1779769986.1952674,"t_done":339121.516666437,"t_done_unix":1779769986.5306098},{"request_id":"1361694:1:1361694:534","session_id":"1361694","input_length":76969,"t_proxy_recv":339100.413984822,"t_decision_unix":1779769965.4279263,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (3584 < 4000)","cache_hit":0,"estimated_new_tokens":76969,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":76969,"score_linear":156948.0,"score_lmetric":400250},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":76969,"score_linear":130877.0,"score_lmetric":153938},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":76969,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":76969,"score_linear":137602.0,"score_lmetric":390230},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1241,"cache_hit":0,"new_prefill":76969,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1473,"cache_hit":0,"new_prefill":76969,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":3584,"new_prefill":73385,"score_linear":76662.0,"score_lmetric":73385},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":92236,"ongoing_decode_tokens":58758,"pending_prefill_tokens":5318,"num_requests":2,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":76969,"score_linear":92236.0,"score_lmetric":164574}],"chosen_score_linear":10073.0,"chosen_score_lmetric":0,"t_first_token":339120.681341972,"t_first_token_unix":1779769985.695286,"t_done":339122.04673523,"t_done_unix":1779769987.06068},{"request_id":"1362265:2:1367373:568","session_id":"1362265","input_length":10134,"t_proxy_recv":339120.204452433,"t_decision_unix":1779769985.2183938,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":8704,"affinity_cache_ratio":0.8588908624432603,"affinity_num_requests":2,"avg_num_requests":2.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1430 < 8000)","cache_hit":8704,"estimated_new_tokens":1430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":10134,"score_linear":156948.0,"score_lmetric":266580},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157714,"ongoing_decode_tokens":130877,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":10134,"score_linear":157714.0,"score_lmetric":110913},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":10134,"score_linear":87042.0,"score_lmetric":87103},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":10134,"score_linear":336145.0,"score_lmetric":794664},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":63232,"ongoing_decode_tokens":0,"pending_prefill_tokens":13749,"num_requests":2,"active_p_offloads":0,"cached_blocks":1383,"cache_hit":0,"new_prefill":10134,"score_linear":63232.0,"score_lmetric":47766},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":8704,"new_prefill":1430,"score_linear":25522.0,"score_lmetric":49808},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93230,"ongoing_decode_tokens":93230,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":0,"new_prefill":10134,"score_linear":93230.0,"score_lmetric":20268},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":59514,"pending_prefill_tokens":7737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":10134,"score_linear":109237.0,"score_lmetric":53613}],"chosen_score_linear":25522.0,"chosen_score_lmetric":49808,"t_first_token":339122.644551755,"t_first_token_unix":1779769987.6584954,"t_done":339122.94018754,"t_done_unix":1779769987.9541314},{"request_id":"1364128:1:1364128:545","session_id":"1364128","input_length":8706,"t_proxy_recv":339108.646974471,"t_decision_unix":1779769973.6609156,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":8706,"tie_break_used":false,"v2_pd_sep":true,"v2_decision":"pd_sep","v2_reason":"local_cost 1.09s > migrate_cost 0.73s + margin 0.20s","v2_chosen_cache_hit":0,"v2_src_idx":5,"v2_src_cache_hit":6656,"v2_new_local":8706,"v2_new_src":2050,"v2_cost_local_s":1.08825,"v2_cost_src_interf_s":0.205,"v2_cost_xfer_s":0.5256944444444445,"v2_cost_migrate_s":0.7306944444444444,"cache_hit":0,"estimated_new_tokens":8706,"route_class":"PD_SEP_V2","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":8706,"score_linear":156948.0,"score_lmetric":263724},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":8706,"score_linear":130877.0,"score_lmetric":17412},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":8706,"score_linear":87042.0,"score_lmetric":85675},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":8706,"score_linear":137602.0,"score_lmetric":253704},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":8706,"score_linear":90778.0,"score_lmetric":143672},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":29954,"ongoing_decode_tokens":16057,"pending_prefill_tokens":6217,"num_requests":2,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":6656,"new_prefill":2050,"score_linear":23298.0,"score_lmetric":16534},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":92882,"ongoing_decode_tokens":80246,"pending_prefill_tokens":8540,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":8706,"score_linear":92882.0,"score_lmetric":34492},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24451,"ongoing_decode_tokens":24451,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":8706,"score_linear":24451.0,"score_lmetric":8706}],"chosen_score_linear":24451.0,"chosen_score_lmetric":8706,"v2_src_url":"http://127.0.0.1:8005","t_prefill_sent":339108.647356163,"t_prefill_sent_unix":1779769973.6612992,"t_prefill_done":339109.949812003,"t_prefill_done_unix":1779769974.9637556,"t_decode_sent":339109.949974676,"t_decode_sent_unix":1779769974.9639173,"t_first_token":339121.127371618,"t_first_token_unix":1779769986.1413164,"t_done":339122.957554437,"t_done_unix":1779769987.9714985},{"request_id":"1270340:4:1366608:558","session_id":"1270340","input_length":41017,"t_proxy_recv":339117.395817504,"t_decision_unix":1779769982.4097588,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33280,"affinity_cache_ratio":0.8113708949947582,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7737 < 8000)","cache_hit":33280,"estimated_new_tokens":7737,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":41017,"score_linear":156948.0,"score_lmetric":328346},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":41017,"score_linear":130877.0,"score_lmetric":82034},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":41017,"score_linear":87042.0,"score_lmetric":117986},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":250361,"ongoing_decode_tokens":0,"pending_prefill_tokens":119801,"num_requests":4,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":41017,"score_linear":250361.0,"score_lmetric":643272},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":1536,"new_prefill":39481,"score_linear":89242.0,"score_lmetric":205222},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":41017,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93230,"ongoing_decode_tokens":93230,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":0,"new_prefill":41017,"score_linear":93230.0,"score_lmetric":82034},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":68220,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":33280,"new_prefill":7737,"score_linear":34940.0,"score_lmetric":86374}],"chosen_score_linear":34940.0,"chosen_score_lmetric":86374,"t_first_token":339121.100134875,"t_first_token_unix":1779769986.1140788,"t_done":339123.148556025,"t_done_unix":1779769988.1625},{"request_id":"1367312:2:1368064:574","session_id":"1367312","input_length":15501,"t_proxy_recv":339122.695351085,"t_decision_unix":1779769987.7092927,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":13312,"affinity_cache_ratio":0.8587833043029482,"affinity_num_requests":1,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2189 < 8000)","cache_hit":13312,"estimated_new_tokens":2189,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":15501,"score_linear":156948.0,"score_lmetric":277314},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157714,"ongoing_decode_tokens":130877,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":15501,"score_linear":157714.0,"score_lmetric":127014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":15501,"score_linear":82992.0,"score_lmetric":69476},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":15501,"score_linear":336145.0,"score_lmetric":826866},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1409,"cache_hit":13312,"new_prefill":2189,"score_linear":36171.0,"score_lmetric":2189},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44360,"ongoing_decode_tokens":44360,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":15501,"score_linear":44360.0,"score_lmetric":46503},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":15501,"score_linear":80246.0,"score_lmetric":15501},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":100531,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":15501,"score_linear":109237.0,"score_lmetric":46503}],"chosen_score_linear":36171.0,"chosen_score_lmetric":2189,"t_first_token":339122.97303241,"t_first_token_unix":1779769987.9869754,"t_done":339123.292034357,"t_done_unix":1779769988.3059776},{"request_id":"1366128:2:1367220:563","session_id":"1366128","input_length":15411,"t_proxy_recv":339119.702986554,"t_decision_unix":1779769984.716928,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":10752,"affinity_cache_ratio":0.6976834728440724,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4659 < 8000)","cache_hit":10752,"estimated_new_tokens":4659,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":15411,"score_linear":156948.0,"score_lmetric":277134},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":15411,"score_linear":130877.0,"score_lmetric":30822},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":15411,"score_linear":87042.0,"score_lmetric":92380},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":286662,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":5,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":15411,"score_linear":286662.0,"score_lmetric":688605},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":62685,"ongoing_decode_tokens":62685,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":15411,"score_linear":62685.0,"score_lmetric":15411},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":18815,"ongoing_decode_tokens":0,"pending_prefill_tokens":18815,"num_requests":1,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":10752,"new_prefill":4659,"score_linear":8063.0,"score_lmetric":23474},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93230,"ongoing_decode_tokens":93230,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":0,"new_prefill":15411,"score_linear":93230.0,"score_lmetric":30822},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":59514,"pending_prefill_tokens":7737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":15411,"score_linear":109237.0,"score_lmetric":69444}],"chosen_score_linear":8063.0,"chosen_score_lmetric":23474,"t_first_token":339121.895820917,"t_first_token_unix":1779769986.909764,"t_done":339123.625136121,"t_done_unix":1779769988.63908},{"request_id":"1364128:2:1367249:565","session_id":"1364128","input_length":9095,"t_proxy_recv":339122.960844298,"t_decision_unix":1779769987.9747858,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":8704,"affinity_cache_ratio":0.9570093457943926,"affinity_num_requests":2,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (391 < 8000)","cache_hit":8704,"estimated_new_tokens":391,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":9095,"score_linear":156948.0,"score_lmetric":264502},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157714,"ongoing_decode_tokens":130877,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":9095,"score_linear":157714.0,"score_lmetric":107796},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":9095,"score_linear":82992.0,"score_lmetric":63070},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":9095,"score_linear":336145.0,"score_lmetric":788430},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64984,"ongoing_decode_tokens":0,"pending_prefill_tokens":2189,"num_requests":2,"active_p_offloads":0,"cached_blocks":1409,"cache_hit":0,"new_prefill":9095,"score_linear":64984.0,"score_lmetric":22568},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34226,"ongoing_decode_tokens":34226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":8704,"new_prefill":391,"score_linear":25522.0,"score_lmetric":782},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102019,"ongoing_decode_tokens":80246,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":9095,"score_linear":102019.0,"score_lmetric":61736},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":100531,"ongoing_decode_tokens":100531,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1405,"cache_hit":8704,"new_prefill":391,"score_linear":91827.0,"score_lmetric":782}],"chosen_score_linear":91827.0,"chosen_score_lmetric":782,"t_first_token":339123.050032891,"t_first_token_unix":1779769988.0639765,"t_done":339123.681919498,"t_done_unix":1779769988.6958628},{"request_id":"1364960:2:1368239:578","session_id":"1364960","input_length":7048,"t_proxy_recv":339123.296088758,"t_decision_unix":1779769988.3100307,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":4,"affinity_cache_hit":2048,"affinity_cache_ratio":0.2905788876276958,"affinity_num_requests":2,"avg_num_requests":2.5,"fallback_score":7048,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7048 < 8000)","cache_hit":0,"estimated_new_tokens":7048,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":7048,"score_linear":156948.0,"score_lmetric":260408},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157714,"ongoing_decode_tokens":130877,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":7048,"score_linear":157714.0,"score_lmetric":101655},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":7048,"score_linear":82992.0,"score_lmetric":61023},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":7048,"score_linear":336145.0,"score_lmetric":776148},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1413,"cache_hit":0,"new_prefill":7048,"score_linear":49483.0,"score_lmetric":7048},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43071,"ongoing_decode_tokens":43071,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":7048,"score_linear":43071.0,"score_lmetric":21144},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102019,"ongoing_decode_tokens":80246,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":2048,"new_prefill":5000,"score_linear":99971.0,"score_lmetric":53546},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":68609,"ongoing_decode_tokens":68609,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1420,"cache_hit":0,"new_prefill":7048,"score_linear":68609.0,"score_lmetric":14096}],"chosen_score_linear":49483.0,"chosen_score_lmetric":7048,"t_first_token":339123.777071712,"t_first_token_unix":1779769988.7910151,"t_done":339124.096482346,"t_done_unix":1779769989.1104262},{"request_id":"1367169:1:1367169:562","session_id":"1367169","input_length":18815,"t_proxy_recv":339119.515675769,"t_decision_unix":1779769984.5296173,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":18815,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":11776,"new_prefill":7039,"score_linear":145172.0,"score_lmetric":260390},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":18815,"score_linear":130877.0,"score_lmetric":37630},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":18815,"score_linear":87042.0,"score_lmetric":95784},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":286662,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":5,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":18815,"score_linear":286662.0,"score_lmetric":705625},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":62685,"ongoing_decode_tokens":62685,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":18815,"score_linear":62685.0,"score_lmetric":18815},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":18815,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93230,"ongoing_decode_tokens":93230,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":11776,"new_prefill":7039,"score_linear":81454.0,"score_lmetric":14078},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":59514,"pending_prefill_tokens":7737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":18815,"score_linear":109237.0,"score_lmetric":79656}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339121.894040949,"t_first_token_unix":1779769986.9079847,"t_done":339124.848881998,"t_done_unix":1779769989.8628259},{"request_id":"1367329:1:1367329:567","session_id":"1367329","input_length":26837,"t_proxy_recv":339120.063550787,"t_decision_unix":1779769985.0774918,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":53674,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":26837,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":26837,"score_linear":156948.0,"score_lmetric":299986},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":26837,"score_linear":130877.0,"score_lmetric":53674},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":26837,"score_linear":87042.0,"score_lmetric":103806},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":26837,"score_linear":336145.0,"score_lmetric":894882},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":63232,"ongoing_decode_tokens":0,"pending_prefill_tokens":13749,"num_requests":2,"active_p_offloads":0,"cached_blocks":1383,"cache_hit":0,"new_prefill":26837,"score_linear":63232.0,"score_lmetric":81172},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":26837,"score_linear":34226.0,"score_lmetric":100622},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93230,"ongoing_decode_tokens":93230,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":0,"new_prefill":26837,"score_linear":93230.0,"score_lmetric":53674},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":59514,"pending_prefill_tokens":7737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":26837,"score_linear":109237.0,"score_lmetric":103722}],"chosen_score_linear":130877.0,"chosen_score_lmetric":53674,"t_first_token":339123.404855882,"t_first_token_unix":1779769988.4187994,"t_done":339124.928786309,"t_done_unix":1779769989.9427297},{"request_id":"1368070:1:1368070:575","session_id":"1368070","input_length":21773,"t_proxy_recv":339122.72150811,"t_decision_unix":1779769987.7354496,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.625,"fallback_score":21773,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":21773,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":21773,"score_linear":156948.0,"score_lmetric":289858},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157714,"ongoing_decode_tokens":130877,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":21773,"score_linear":157714.0,"score_lmetric":145830},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":21773,"score_linear":82992.0,"score_lmetric":75748},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":21773,"score_linear":336145.0,"score_lmetric":864498},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64984,"ongoing_decode_tokens":0,"pending_prefill_tokens":2189,"num_requests":2,"active_p_offloads":0,"cached_blocks":1409,"cache_hit":0,"new_prefill":21773,"score_linear":64984.0,"score_lmetric":47924},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44360,"ongoing_decode_tokens":44360,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":21773,"score_linear":44360.0,"score_lmetric":65319},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":21773,"score_linear":80246.0,"score_lmetric":21773},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":100531,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":21773,"score_linear":109237.0,"score_lmetric":65319}],"chosen_score_linear":80246.0,"chosen_score_lmetric":21773,"t_first_token":339125.100955128,"t_first_token_unix":1779769990.1148987,"t_done":339125.375458467,"t_done_unix":1779769990.3894022},{"request_id":"1368132:1:1368132:577","session_id":"1368132","input_length":8845,"t_proxy_recv":339122.971260082,"t_decision_unix":1779769987.9852016,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.625,"fallback_score":4378,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2189 < 8000)","cache_hit":6656,"estimated_new_tokens":2189,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":8845,"score_linear":156948.0,"score_lmetric":264002},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157714,"ongoing_decode_tokens":130877,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":8845,"score_linear":157714.0,"score_lmetric":107046},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":8845,"score_linear":82992.0,"score_lmetric":62820},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":8845,"score_linear":336145.0,"score_lmetric":786930},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64984,"ongoing_decode_tokens":0,"pending_prefill_tokens":2189,"num_requests":2,"active_p_offloads":0,"cached_blocks":1409,"cache_hit":0,"new_prefill":8845,"score_linear":64984.0,"score_lmetric":22068},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34226,"ongoing_decode_tokens":34226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":6656,"new_prefill":2189,"score_linear":27570.0,"score_lmetric":4378},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102019,"ongoing_decode_tokens":80246,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":8845,"score_linear":102019.0,"score_lmetric":61236},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109626,"ongoing_decode_tokens":100531,"pending_prefill_tokens":391,"num_requests":3,"active_p_offloads":0,"cached_blocks":1405,"cache_hit":6656,"new_prefill":2189,"score_linear":102970.0,"score_lmetric":7740}],"chosen_score_linear":27570.0,"chosen_score_lmetric":4378,"t_first_token":339123.216155397,"t_first_token_unix":1779769988.230099,"t_done":339125.525953509,"t_done_unix":1779769990.5398967},{"request_id":"1313181:3:1364311:547","session_id":"1313181","input_length":59514,"t_proxy_recv":339109.257454511,"t_decision_unix":1779769974.271396,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24064,"affinity_cache_ratio":0.4043418355344961,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":35450,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":24064,"estimated_new_tokens":35450,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":59514,"score_linear":156948.0,"score_lmetric":365340},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":59514,"score_linear":130877.0,"score_lmetric":119028},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":59514,"score_linear":87042.0,"score_lmetric":136483},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":235640,"ongoing_decode_tokens":0,"pending_prefill_tokens":118392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":12800,"new_prefill":46714,"score_linear":222840.0,"score_lmetric":495318},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":59514,"score_linear":90778.0,"score_lmetric":245288},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":38660,"ongoing_decode_tokens":16057,"pending_prefill_tokens":6217,"num_requests":3,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":0,"new_prefill":59514,"score_linear":38660.0,"score_lmetric":197193},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":92882,"ongoing_decode_tokens":92882,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":59514,"score_linear":92882.0,"score_lmetric":119028},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":8706,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":24064,"new_prefill":35450,"score_linear":-15358.0,"score_lmetric":35450}],"chosen_score_linear":-15358.0,"chosen_score_lmetric":35450,"t_first_token":339119.24037221,"t_first_token_unix":1779769984.2543159,"t_done":339125.67767882,"t_done_unix":1779769990.6916225},{"request_id":"1313181:4:1318843:342","session_id":"1313181","input_length":42704,"t_proxy_recv":339125.69232528,"t_decision_unix":1779769990.7062669,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":42496,"affinity_cache_ratio":0.9951292618958412,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (208 < 8000)","cache_hit":42496,"estimated_new_tokens":208,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":42704,"score_linear":156948.0,"score_lmetric":331720},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":42704,"score_linear":130877.0,"score_lmetric":85408},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":42704,"score_linear":82992.0,"score_lmetric":96679},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":12800,"new_prefill":29904,"score_linear":323345.0,"score_lmetric":913284},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":112718,"ongoing_decode_tokens":63235,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1426,"cache_hit":0,"new_prefill":42704,"score_linear":112718.0,"score_lmetric":85408},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":42704,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":0,"new_prefill":42704,"score_linear":80246.0,"score_lmetric":42704},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":42496,"new_prefill":208,"score_linear":-42496.0,"score_lmetric":0}],"chosen_score_linear":-42496.0,"chosen_score_lmetric":0,"t_first_token":339125.76233822,"t_first_token_unix":1779769990.7762816,"t_done":339126.448267098,"t_done_unix":1779769991.4622104},{"request_id":"1363093:2:1368700:580","session_id":"1363093","input_length":63235,"t_proxy_recv":339124.883150766,"t_decision_unix":1779769989.897092,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":62464,"affinity_cache_ratio":0.9878073851506286,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (771 < 8000)","cache_hit":62464,"estimated_new_tokens":771,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":63235,"score_linear":156948.0,"score_lmetric":372782},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157714,"ongoing_decode_tokens":157714,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":63235,"score_linear":157714.0,"score_lmetric":189705},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":63235,"score_linear":82992.0,"score_lmetric":117210},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":63235,"score_linear":336145.0,"score_lmetric":1113270},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1426,"cache_hit":62464,"new_prefill":771,"score_linear":-12981.0,"score_lmetric":771},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":8845,"ongoing_decode_tokens":8845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1581,"cache_hit":0,"new_prefill":63235,"score_linear":8845.0,"score_lmetric":63235},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":102019,"ongoing_decode_tokens":80246,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":63235,"score_linear":102019.0,"score_lmetric":170016},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":59514,"ongoing_decode_tokens":59514,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1420,"cache_hit":0,"new_prefill":63235,"score_linear":59514.0,"score_lmetric":63235}],"chosen_score_linear":-12981.0,"chosen_score_lmetric":771,"t_first_token":339125.237453611,"t_first_token_unix":1779769990.2513974,"t_done":339126.576062369,"t_done_unix":1779769991.5900059},{"request_id":"1369172:1:1369172:581","session_id":"1369172","input_length":6345,"t_proxy_recv":339126.562198167,"t_decision_unix":1779769991.5761392,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6345 < 8000)","cache_hit":0,"estimated_new_tokens":6345,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":6345,"score_linear":156948.0,"score_lmetric":259002},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":6345,"score_linear":130877.0,"score_lmetric":12690},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":6345,"score_linear":82992.0,"score_lmetric":60320},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":6345,"score_linear":336145.0,"score_lmetric":771930},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":112718,"ongoing_decode_tokens":63235,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1426,"cache_hit":0,"new_prefill":6345,"score_linear":112718.0,"score_lmetric":12690},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":6345,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":0,"new_prefill":6345,"score_linear":80246.0,"score_lmetric":6345},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":0,"pending_prefill_tokens":17508,"num_requests":1,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":6345,"score_linear":60004.0,"score_lmetric":23853}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339126.983181091,"t_first_token_unix":1779769991.9971247,"t_done":339127.227130932,"t_done_unix":1779769992.2410743},{"request_id":"1253804:6:1318560:340","session_id":"1253804","input_length":62618,"t_proxy_recv":338948.205333029,"t_decision_unix":1779769813.2192743,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9403047047174934,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3738 < 8000)","cache_hit":58880,"estimated_new_tokens":3738,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":58880,"new_prefill":3738,"score_linear":-58880.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1039,"cache_hit":3584,"new_prefill":59034,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":3584,"new_prefill":59034,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79151,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":62618,"score_linear":79663.0,"score_lmetric":141769},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":920,"cache_hit":0,"new_prefill":62618,"score_linear":176419.0,"score_lmetric":125236},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":338962.113751753,"t_first_token_unix":1779769827.1276956,"t_done":339128.870090855,"t_done_unix":1779769993.8840349},{"request_id":"1369768:1:1369768:584","session_id":"1369768","input_length":7712,"t_proxy_recv":339128.566772772,"t_decision_unix":1779769993.5807137,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7712 < 8000)","cache_hit":0,"estimated_new_tokens":7712,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":7712,"score_linear":156948.0,"score_lmetric":261736},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":7712,"score_linear":130877.0,"score_lmetric":15424},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":7712,"score_linear":82992.0,"score_lmetric":61687},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":7712,"score_linear":336145.0,"score_lmetric":780132},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":7712,"score_linear":49483.0,"score_lmetric":7712},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":7712,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98406,"ongoing_decode_tokens":98406,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":0,"new_prefill":7712,"score_linear":98406.0,"score_lmetric":15424},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":0,"pending_prefill_tokens":17508,"num_requests":1,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":7712,"score_linear":60004.0,"score_lmetric":25220}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339129.103469455,"t_first_token_unix":1779769994.1174128,"t_done":339130.464568192,"t_done_unix":1779769995.4785116},{"request_id":"1366128:3:1369902:585","session_id":"1366128","input_length":23307,"t_proxy_recv":339129.033330734,"t_decision_unix":1779769994.047272,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":15360,"affinity_cache_ratio":0.6590294761230532,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7947 < 8000)","cache_hit":15360,"estimated_new_tokens":7947,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":23307,"score_linear":156948.0,"score_lmetric":292926},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":23307,"score_linear":68259.0,"score_lmetric":23307},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":23307,"score_linear":82992.0,"score_lmetric":77282},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":23307,"score_linear":336145.0,"score_lmetric":873702},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":23307,"score_linear":49483.0,"score_lmetric":23307},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":7712,"ongoing_decode_tokens":0,"pending_prefill_tokens":7712,"num_requests":1,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":15360,"new_prefill":7947,"score_linear":-7648.0,"score_lmetric":15659},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98406,"ongoing_decode_tokens":98406,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":0,"new_prefill":23307,"score_linear":98406.0,"score_lmetric":46614},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":0,"pending_prefill_tokens":17508,"num_requests":1,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":23307,"score_linear":60004.0,"score_lmetric":40815}],"chosen_score_linear":-7648.0,"chosen_score_lmetric":15659,"t_first_token":339130.324868966,"t_first_token_unix":1779769995.3388126,"t_done":339130.591123169,"t_done_unix":1779769995.605067},{"request_id":"1367424:2:1370211:588","session_id":"1367424","input_length":11133,"t_proxy_recv":339130.042714387,"t_decision_unix":1779769995.0566556,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":1,"affinity_cache_hit":3072,"affinity_cache_ratio":0.27593640528159524,"affinity_num_requests":3,"avg_num_requests":2.25,"fallback_score":11133,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (3072 < 4000)","cache_hit":0,"estimated_new_tokens":11133,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":11133,"score_linear":156948.0,"score_lmetric":268578},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":11133,"score_linear":68259.0,"score_lmetric":11133},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":11133,"score_linear":82992.0,"score_lmetric":65108},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":11133,"score_linear":336145.0,"score_lmetric":800658},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":67784,"ongoing_decode_tokens":0,"pending_prefill_tokens":18301,"num_requests":2,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":11133,"score_linear":67784.0,"score_lmetric":58868},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":31019,"ongoing_decode_tokens":7712,"pending_prefill_tokens":7947,"num_requests":2,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":11133,"score_linear":31019.0,"score_lmetric":38160},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127907,"ongoing_decode_tokens":98406,"pending_prefill_tokens":7997,"num_requests":3,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":3072,"new_prefill":8061,"score_linear":124835.0,"score_lmetric":48174},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":0,"pending_prefill_tokens":17508,"num_requests":1,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":11133,"score_linear":60004.0,"score_lmetric":28641}],"chosen_score_linear":68259.0,"chosen_score_lmetric":11133,"t_first_token":339130.942993195,"t_first_token_unix":1779769995.9569376,"t_done":339131.074752782,"t_done_unix":1779769996.0886962},{"request_id":"1368070:2:1370108:586","session_id":"1368070","input_length":29501,"t_proxy_recv":339129.755526683,"t_decision_unix":1779769994.7694678,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":21504,"affinity_cache_ratio":0.7289244432392122,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7997 < 8000)","cache_hit":21504,"estimated_new_tokens":7997,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":29501,"score_linear":156948.0,"score_lmetric":305314},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":29501,"score_linear":68259.0,"score_lmetric":29501},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":29501,"score_linear":82992.0,"score_lmetric":83476},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":29501,"score_linear":336145.0,"score_lmetric":910866},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":29501,"score_linear":49483.0,"score_lmetric":29501},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":31019,"ongoing_decode_tokens":7712,"pending_prefill_tokens":7947,"num_requests":2,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":29501,"score_linear":31019.0,"score_lmetric":74896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98406,"ongoing_decode_tokens":98406,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":21504,"new_prefill":7997,"score_linear":76902.0,"score_lmetric":15994},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":0,"pending_prefill_tokens":17508,"num_requests":1,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":29501,"score_linear":60004.0,"score_lmetric":47009}],"chosen_score_linear":76902.0,"chosen_score_lmetric":15994,"t_first_token":339131.269588679,"t_first_token_unix":1779769996.2835324,"t_done":339131.593813394,"t_done_unix":1779769996.6077569},{"request_id":"1370466:1:1370466:592","session_id":"1370466","input_length":7710,"t_proxy_recv":339131.089476698,"t_decision_unix":1779769996.1034184,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7710 < 8000)","cache_hit":0,"estimated_new_tokens":7710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":7710,"score_linear":156948.0,"score_lmetric":261732},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":7710,"score_linear":68259.0,"score_lmetric":7710},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":7710,"score_linear":82992.0,"score_lmetric":61685},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":7710,"score_linear":336145.0,"score_lmetric":780120},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":132315,"ongoing_decode_tokens":0,"pending_prefill_tokens":23952,"num_requests":3,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":7710,"score_linear":132315.0,"score_lmetric":94986},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1627,"cache_hit":0,"new_prefill":7710,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127907,"ongoing_decode_tokens":98406,"pending_prefill_tokens":7997,"num_requests":3,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":0,"new_prefill":7710,"score_linear":127907.0,"score_lmetric":47121},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62385,"ongoing_decode_tokens":0,"pending_prefill_tokens":19889,"num_requests":2,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":7710,"score_linear":62385.0,"score_lmetric":55198}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339131.622400541,"t_first_token_unix":1779769996.6363437,"t_done":339131.740171225,"t_done_unix":1779769996.7541144},{"request_id":"1344773:7:1357714:518","session_id":"1344773","input_length":34351,"t_proxy_recv":339086.050136585,"t_decision_unix":1779769951.0640783,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9837268201799074,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (559 < 8000)","cache_hit":33792,"estimated_new_tokens":559,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":122597,"num_requests":1,"active_p_offloads":0,"cached_blocks":991,"cache_hit":33792,"new_prefill":559,"score_linear":88805.0,"score_lmetric":123156},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":34351,"score_linear":130877.0,"score_lmetric":68702},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":21371,"pending_prefill_tokens":3137,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":34351,"score_linear":58266.0,"score_lmetric":37488},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":34351,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":34351,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":34351,"score_linear":83526.0,"score_lmetric":117877},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":34351,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":34351,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":88805.0,"chosen_score_lmetric":123156,"t_first_token":339131.264785426,"t_first_token_unix":1779769996.2787287,"t_done":339132.040939418,"t_done_unix":1779769997.0548828},{"request_id":"1363943:3:1369535:582","session_id":"1363943","input_length":18160,"t_proxy_recv":339127.744634207,"t_decision_unix":1779769992.7585757,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":12800,"affinity_cache_ratio":0.7048458149779736,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5360 < 8000)","cache_hit":12800,"estimated_new_tokens":5360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":8704,"new_prefill":9456,"score_linear":148244.0,"score_lmetric":265224},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":18160,"score_linear":130877.0,"score_lmetric":36320},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":4096,"new_prefill":14064,"score_linear":78896.0,"score_lmetric":68039},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":18160,"score_linear":336145.0,"score_lmetric":842820},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":18160,"score_linear":49483.0,"score_lmetric":18160},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":4096,"new_prefill":14064,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":12800,"new_prefill":5360,"score_linear":67446.0,"score_lmetric":5360},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":0,"pending_prefill_tokens":17508,"num_requests":1,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":18160,"score_linear":60004.0,"score_lmetric":35668}],"chosen_score_linear":67446.0,"chosen_score_lmetric":5360,"t_first_token":339128.442547441,"t_first_token_unix":1779769993.4564912,"t_done":339132.538131999,"t_done_unix":1779769997.5520751},{"request_id":"1344773:8:1358989:523","session_id":"1344773","input_length":34552,"t_proxy_recv":339132.047545108,"t_decision_unix":1779769997.0614865,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9928224125955082,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (248 < 8000)","cache_hit":34304,"estimated_new_tokens":248,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":122597,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":34304,"new_prefill":248,"score_linear":88293.0,"score_lmetric":248},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":34552,"score_linear":68259.0,"score_lmetric":34552},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":34552,"score_linear":82992.0,"score_lmetric":88527},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":34552,"score_linear":336145.0,"score_lmetric":941172},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":132315,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":34552,"score_linear":132315.0,"score_lmetric":120609},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":34552,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98406,"ongoing_decode_tokens":98406,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":34552,"score_linear":98406.0,"score_lmetric":69104},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62385,"ongoing_decode_tokens":0,"pending_prefill_tokens":19889,"num_requests":2,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":34552,"score_linear":62385.0,"score_lmetric":108882}],"chosen_score_linear":88293.0,"chosen_score_lmetric":248,"t_first_token":339132.179792305,"t_first_token_unix":1779769997.1937354,"t_done":339132.736852567,"t_done_unix":1779769997.7507963},{"request_id":"1370269:1:1370269:591","session_id":"1370269","input_length":2381,"t_proxy_recv":339130.307372355,"t_decision_unix":1779769995.3213134,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":19889,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2381 < 8000)","cache_hit":0,"estimated_new_tokens":2381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":2381,"score_linear":156948.0,"score_lmetric":251074},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":79392,"ongoing_decode_tokens":68259,"pending_prefill_tokens":11133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":2381,"score_linear":79392.0,"score_lmetric":27028},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":2381,"score_linear":82992.0,"score_lmetric":56356},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":2381,"score_linear":336145.0,"score_lmetric":748146},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":67784,"ongoing_decode_tokens":0,"pending_prefill_tokens":18301,"num_requests":2,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":2381,"score_linear":67784.0,"score_lmetric":41364},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":31019,"ongoing_decode_tokens":7712,"pending_prefill_tokens":7947,"num_requests":2,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":2381,"score_linear":31019.0,"score_lmetric":20656},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127907,"ongoing_decode_tokens":98406,"pending_prefill_tokens":7997,"num_requests":3,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":0,"new_prefill":2381,"score_linear":127907.0,"score_lmetric":31134},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":0,"pending_prefill_tokens":17508,"num_requests":1,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":2381,"score_linear":60004.0,"score_lmetric":19889}],"chosen_score_linear":60004.0,"chosen_score_lmetric":19889,"t_first_token":339132.510156389,"t_first_token_unix":1779769997.5241,"t_done":339133.309282777,"t_done_unix":1779769998.3232265},{"request_id":"1344773:9:1360064:526","session_id":"1344773","input_length":34611,"t_proxy_recv":339132.743622913,"t_decision_unix":1779769997.7575643,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9911299875762041,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (307 < 8000)","cache_hit":34304,"estimated_new_tokens":307,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":122597,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":34304,"new_prefill":307,"score_linear":88293.0,"score_lmetric":307},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":34611,"score_linear":68259.0,"score_lmetric":34611},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":34611,"score_linear":82992.0,"score_lmetric":88586},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":34611,"score_linear":336145.0,"score_lmetric":941526},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":132315,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":34611,"score_linear":132315.0,"score_lmetric":120786},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":34611,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":0,"new_prefill":34611,"score_linear":80246.0,"score_lmetric":34611},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62385,"ongoing_decode_tokens":62385,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":34611,"score_linear":62385.0,"score_lmetric":69222}],"chosen_score_linear":88293.0,"chosen_score_lmetric":307,"t_first_token":339132.834063126,"t_first_token_unix":1779769997.8480065,"t_done":339133.679111158,"t_done_unix":1779769998.6930542},{"request_id":"1356885:1:1356885:514","session_id":"1356885","input_length":122597,"t_proxy_recv":339083.324194916,"t_decision_unix":1779769948.338136,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":122597,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":122597,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":122597,"score_linear":130877.0,"score_lmetric":245194},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":21371,"pending_prefill_tokens":3137,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":122597,"score_linear":58266.0,"score_lmetric":125734},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1228,"cache_hit":0,"new_prefill":122597,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1209,"cache_hit":0,"new_prefill":122597,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":122597,"score_linear":83526.0,"score_lmetric":206123},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78089,"ongoing_decode_tokens":78089,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1146,"cache_hit":0,"new_prefill":122597,"score_linear":78089.0,"score_lmetric":122597},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1237,"cache_hit":6656,"new_prefill":115941,"score_linear":28652.0,"score_lmetric":151249}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339131.262984195,"t_first_token_unix":1779769996.2769275,"t_done":339134.004395491,"t_done_unix":1779769999.0183394},{"request_id":"1367329:2:1371226:595","session_id":"1367329","input_length":32814,"t_proxy_recv":339133.530275643,"t_decision_unix":1779769998.544217,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":26624,"affinity_cache_ratio":0.8113610044493205,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6190 < 8000)","cache_hit":26624,"estimated_new_tokens":6190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":157208,"ongoing_decode_tokens":157208,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":32814,"score_linear":157208.0,"score_lmetric":65628},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":26624,"new_prefill":6190,"score_linear":41635.0,"score_lmetric":6190},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":32814,"score_linear":82992.0,"score_lmetric":86789},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":32814,"score_linear":336145.0,"score_lmetric":930744},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":132315,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":32814,"score_linear":132315.0,"score_lmetric":115395},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":32814,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":0,"new_prefill":32814,"score_linear":80246.0,"score_lmetric":32814},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":32814,"score_linear":60004.0,"score_lmetric":32814}],"chosen_score_linear":41635.0,"chosen_score_lmetric":6190,"t_first_token":339134.842745727,"t_first_token_unix":1779769999.8566897,"t_done":339136.080137268,"t_done_unix":1779770001.0940804},{"request_id":"1344773:10:1361326:533","session_id":"1344773","input_length":34679,"t_proxy_recv":339133.685599971,"t_decision_unix":1779769998.6995416,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9891865394042504,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (375 < 8000)","cache_hit":34304,"estimated_new_tokens":375,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":122597,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":34304,"new_prefill":375,"score_linear":88293.0,"score_lmetric":375},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101073,"ongoing_decode_tokens":68259,"pending_prefill_tokens":6190,"num_requests":2,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":34679,"score_linear":101073.0,"score_lmetric":81738},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":34679,"score_linear":82992.0,"score_lmetric":88654},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":34679,"score_linear":336145.0,"score_lmetric":941934},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":132315,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":34679,"score_linear":132315.0,"score_lmetric":120990},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":34679,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":0,"new_prefill":34679,"score_linear":80246.0,"score_lmetric":34679},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":34679,"score_linear":60004.0,"score_lmetric":34679}],"chosen_score_linear":88293.0,"chosen_score_lmetric":375,"t_first_token":339133.773431,"t_first_token_unix":1779769998.7873743,"t_done":339136.367600463,"t_done_unix":1779770001.381544},{"request_id":"1344773:11:1366387:555","session_id":"1344773","input_length":35071,"t_proxy_recv":339136.37547291,"t_decision_unix":1779770001.389414,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9927290353853612,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (255 < 8000)","cache_hit":34816,"estimated_new_tokens":255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1232,"cache_hit":34816,"new_prefill":255,"score_linear":-21200.0,"score_lmetric":255},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1243,"cache_hit":0,"new_prefill":35071,"score_linear":68259.0,"score_lmetric":35071},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":35071,"score_linear":82992.0,"score_lmetric":89046},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":35071,"score_linear":336145.0,"score_lmetric":944286},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":35071,"score_linear":145931.0,"score_lmetric":162888},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":71141,"ongoing_decode_tokens":9399,"pending_prefill_tokens":24366,"num_requests":3,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":35071,"score_linear":71141.0,"score_lmetric":178311},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":0,"new_prefill":35071,"score_linear":80246.0,"score_lmetric":35071},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77411,"ongoing_decode_tokens":60004,"pending_prefill_tokens":17407,"num_requests":2,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":35071,"score_linear":77411.0,"score_lmetric":104956}],"chosen_score_linear":-21200.0,"chosen_score_lmetric":255,"t_first_token":339136.449179213,"t_first_token_unix":1779770001.4631226,"t_done":339137.043891041,"t_done_unix":1779770002.0578344},{"request_id":"1363943:4:1372250:603","session_id":"1363943","input_length":18396,"t_proxy_recv":339137.30597025,"t_decision_unix":1779770002.3199117,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17920,"affinity_cache_ratio":0.974124809741248,"affinity_num_requests":1,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (476 < 8000)","cache_hit":17920,"estimated_new_tokens":476,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":48810,"ongoing_decode_tokens":35194,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1232,"cache_hit":8704,"new_prefill":9692,"score_linear":40106.0,"score_lmetric":19384},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1243,"cache_hit":0,"new_prefill":18396,"score_linear":68259.0,"score_lmetric":18396},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":4096,"new_prefill":14300,"score_linear":78896.0,"score_lmetric":68275},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":18396,"score_linear":336145.0,"score_lmetric":844236},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":18396,"score_linear":145931.0,"score_lmetric":96188},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":71141,"ongoing_decode_tokens":9399,"pending_prefill_tokens":24366,"num_requests":3,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":4096,"new_prefill":14300,"score_linear":67045.0,"score_lmetric":115998},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":17920,"new_prefill":476,"score_linear":62326.0,"score_lmetric":476},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77411,"ongoing_decode_tokens":60004,"pending_prefill_tokens":17407,"num_requests":2,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":18396,"score_linear":77411.0,"score_lmetric":71606}],"chosen_score_linear":62326.0,"chosen_score_lmetric":476,"t_first_token":339137.365936343,"t_first_token_unix":1779770002.3798795,"t_done":339138.334628201,"t_done_unix":1779770003.348572},{"request_id":"1344773:12:1367864:571","session_id":"1344773","input_length":35194,"t_proxy_recv":339137.050947271,"t_decision_unix":1779770002.0648887,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9892595328749219,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (378 < 8000)","cache_hit":34816,"estimated_new_tokens":378,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1232,"cache_hit":34816,"new_prefill":378,"score_linear":-21200.0,"score_lmetric":378},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1243,"cache_hit":0,"new_prefill":35194,"score_linear":68259.0,"score_lmetric":35194},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":35194,"score_linear":82992.0,"score_lmetric":89169},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":35194,"score_linear":336145.0,"score_lmetric":945024},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":35194,"score_linear":145931.0,"score_lmetric":163380},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":71141,"ongoing_decode_tokens":9399,"pending_prefill_tokens":24366,"num_requests":3,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":35194,"score_linear":71141.0,"score_lmetric":178680},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":0,"new_prefill":35194,"score_linear":80246.0,"score_lmetric":35194},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77411,"ongoing_decode_tokens":60004,"pending_prefill_tokens":17407,"num_requests":2,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":35194,"score_linear":77411.0,"score_lmetric":105202}],"chosen_score_linear":-21200.0,"chosen_score_lmetric":378,"t_first_token":339137.12203223,"t_first_token_unix":1779770002.1359754,"t_done":339138.435199228,"t_done_unix":1779770003.4491425},{"request_id":"1314357:4:1352612:497","session_id":"1314357","input_length":68259,"t_proxy_recv":339068.324081333,"t_decision_unix":1779769933.3380227,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":67072,"affinity_cache_ratio":0.9826103517484874,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1187 < 8000)","cache_hit":67072,"estimated_new_tokens":1187,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":954,"cache_hit":0,"new_prefill":68259,"score_linear":18840.0,"score_lmetric":87099},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":67072,"new_prefill":1187,"score_linear":-4454.0,"score_lmetric":1187},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":68259,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1212,"cache_hit":0,"new_prefill":68259,"score_linear":96026.0,"score_lmetric":68259},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":10575,"ongoing_decode_tokens":10575,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1141,"cache_hit":0,"new_prefill":68259,"score_linear":10575.0,"score_lmetric":68259},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":68259,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21436,"ongoing_decode_tokens":0,"pending_prefill_tokens":9148,"num_requests":1,"active_p_offloads":0,"cached_blocks":1080,"cache_hit":0,"new_prefill":68259,"score_linear":21436.0,"score_lmetric":77407},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":68259,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-4454.0,"chosen_score_lmetric":1187,"t_first_token":339068.81390747,"t_first_token_unix":1779769933.827851,"t_done":339138.442894906,"t_done_unix":1779770003.4568384},{"request_id":"1344773:13:1370257:589","session_id":"1344773","input_length":35413,"t_proxy_recv":339138.445523529,"t_decision_unix":1779770003.4594653,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9975997515036851,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (85 < 8000)","cache_hit":35328,"estimated_new_tokens":85,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":35328,"new_prefill":85,"score_linear":-21712.0,"score_lmetric":85},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":35413,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":35413,"score_linear":82992.0,"score_lmetric":89388},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":35413,"score_linear":336145.0,"score_lmetric":946338},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":35413,"score_linear":145931.0,"score_lmetric":164256},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":71141,"ongoing_decode_tokens":44436,"pending_prefill_tokens":12369,"num_requests":3,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":35413,"score_linear":71141.0,"score_lmetric":143346},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":35413,"score_linear":80246.0,"score_lmetric":35413},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":77411,"ongoing_decode_tokens":77411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":35413,"score_linear":77411.0,"score_lmetric":70826}],"chosen_score_linear":-21712.0,"chosen_score_lmetric":85,"t_first_token":339138.51872392,"t_first_token_unix":1779770003.5326672,"t_done":339138.537168293,"t_done_unix":1779770003.5511112},{"request_id":"1371929:1:1371929:601","session_id":"1371929","input_length":17407,"t_proxy_recv":339136.068809561,"t_decision_unix":1779770001.0827506,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":17407,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":17407,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":48295,"ongoing_decode_tokens":34679,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":17407,"score_linear":48295.0,"score_lmetric":34814},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101073,"ongoing_decode_tokens":101073,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":17407,"score_linear":101073.0,"score_lmetric":34814},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":17407,"score_linear":82992.0,"score_lmetric":71382},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":17407,"score_linear":336145.0,"score_lmetric":838302},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":17407,"score_linear":145931.0,"score_lmetric":92232},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":71141,"ongoing_decode_tokens":9399,"pending_prefill_tokens":24366,"num_requests":3,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":17407,"score_linear":71141.0,"score_lmetric":125319},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":0,"new_prefill":17407,"score_linear":80246.0,"score_lmetric":17407},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":17407,"score_linear":60004.0,"score_lmetric":17407}],"chosen_score_linear":60004.0,"chosen_score_lmetric":17407,"t_first_token":339137.770898549,"t_first_token_unix":1779770002.7848427,"t_done":339138.597929348,"t_done_unix":1779770003.6118724},{"request_id":"1372791:1:1372791:604","session_id":"1372791","input_length":4811,"t_proxy_recv":339139.074160919,"t_decision_unix":1779770004.088102,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4811 < 8000)","cache_hit":0,"estimated_new_tokens":4811,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":4811,"score_linear":13616.0,"score_lmetric":4811},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1247,"cache_hit":0,"new_prefill":4811,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":4811,"score_linear":82992.0,"score_lmetric":58786},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":4811,"score_linear":336145.0,"score_lmetric":762726},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":4811,"score_linear":145931.0,"score_lmetric":41848},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":71141,"ongoing_decode_tokens":44436,"pending_prefill_tokens":12369,"num_requests":3,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":4811,"score_linear":71141.0,"score_lmetric":51540},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":4811,"score_linear":80246.0,"score_lmetric":4811},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1462,"cache_hit":0,"new_prefill":4811,"score_linear":60004.0,"score_lmetric":4811}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339139.383399996,"t_first_token_unix":1779770004.3973434,"t_done":339139.563354968,"t_done_unix":1779770004.5772989},{"request_id":"1371337:1:1371337:597","session_id":"1371337","input_length":9399,"t_proxy_recv":339133.959430722,"t_decision_unix":1779769998.973372,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":9399,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":157276,"ongoing_decode_tokens":157276,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":9399,"score_linear":157276.0,"score_lmetric":18798},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101073,"ongoing_decode_tokens":68259,"pending_prefill_tokens":6190,"num_requests":2,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":9399,"score_linear":101073.0,"score_lmetric":31178},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":9399,"score_linear":82992.0,"score_lmetric":63374},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":9399,"score_linear":336145.0,"score_lmetric":790254},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":132315,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":9399,"score_linear":132315.0,"score_lmetric":45150},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":9399,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":0,"new_prefill":9399,"score_linear":80246.0,"score_lmetric":9399},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":9399,"score_linear":60004.0,"score_lmetric":9399}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339134.670166692,"t_first_token_unix":1779769999.6841104,"t_done":339139.808017511,"t_done_unix":1779770004.8219614},{"request_id":"1366128:4:1371629:599","session_id":"1366128","input_length":35037,"t_proxy_recv":339134.989189098,"t_decision_unix":1779770000.0031304,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":23040,"affinity_cache_ratio":0.6575905471358849,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":23040,"estimated_new_tokens":11997,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":48295,"ongoing_decode_tokens":34679,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":35037,"score_linear":48295.0,"score_lmetric":70074},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101073,"ongoing_decode_tokens":101073,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":35037,"score_linear":101073.0,"score_lmetric":70074},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":35037,"score_linear":82992.0,"score_lmetric":89012},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":35037,"score_linear":336145.0,"score_lmetric":944082},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":35037,"score_linear":145931.0,"score_lmetric":162752},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":9399,"ongoing_decode_tokens":9399,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":23040,"new_prefill":11997,"score_linear":-13641.0,"score_lmetric":11997},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":0,"new_prefill":35037,"score_linear":80246.0,"score_lmetric":35037},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":35037,"score_linear":60004.0,"score_lmetric":35037}],"chosen_score_linear":-13641.0,"chosen_score_lmetric":11997,"t_first_token":339137.517904532,"t_first_token_unix":1779770002.531848,"t_done":339139.88092186,"t_done_unix":1779770004.8948655},{"request_id":"1313181:4:1368020:573","session_id":"1313181","input_length":60004,"t_proxy_recv":339126.460720556,"t_decision_unix":1779769991.474662,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":42496,"affinity_cache_ratio":0.7082194520365309,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":42496,"estimated_new_tokens":17508,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":60004,"score_linear":156948.0,"score_lmetric":366320},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":60004,"score_linear":130877.0,"score_lmetric":120008},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":60004,"score_linear":82992.0,"score_lmetric":113979},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":12800,"new_prefill":47204,"score_linear":323345.0,"score_lmetric":1017084},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":112718,"ongoing_decode_tokens":63235,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1426,"cache_hit":0,"new_prefill":60004,"score_linear":112718.0,"score_lmetric":120008},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1585,"cache_hit":0,"new_prefill":60004,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":0,"new_prefill":60004,"score_linear":80246.0,"score_lmetric":60004},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":42496,"new_prefill":17508,"score_linear":-42496.0,"score_lmetric":0}],"chosen_score_linear":-42496.0,"chosen_score_lmetric":0,"t_first_token":339132.322064031,"t_first_token_unix":1779769997.336007,"t_done":339141.255145996,"t_done_unix":1779770006.26909},{"request_id":"1362265:3:1373357:605","session_id":"1362265","input_length":10284,"t_proxy_recv":339141.076324106,"t_decision_unix":1779770006.090265,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":9728,"affinity_cache_ratio":0.9459354336833917,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (556 < 8000)","cache_hit":9728,"estimated_new_tokens":556,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":10284,"score_linear":13616.0,"score_lmetric":10284},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":10284,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":94290,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":10284,"score_linear":82992.0,"score_lmetric":10284},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":10284,"score_linear":336145.0,"score_lmetric":795564},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":10284,"score_linear":145931.0,"score_lmetric":63740},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26705,"ongoing_decode_tokens":26705,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1683,"cache_hit":9728,"new_prefill":556,"score_linear":16977.0,"score_lmetric":556},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":10284,"score_linear":80246.0,"score_lmetric":10284},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1462,"cache_hit":6656,"new_prefill":3628,"score_linear":53348.0,"score_lmetric":3628}],"chosen_score_linear":16977.0,"chosen_score_lmetric":556,"t_first_token":339141.135626548,"t_first_token_unix":1779770006.14957,"t_done":339141.283449654,"t_done_unix":1779770006.297393},{"request_id":"1367957:1:1367957:572","session_id":"1367957","input_length":72919,"t_proxy_recv":339122.283863111,"t_decision_unix":1779769987.2978046,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":18944,"estimated_new_tokens":53975,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":72919,"score_linear":156948.0,"score_lmetric":392150},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":157714,"ongoing_decode_tokens":130877,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":72919,"score_linear":157714.0,"score_lmetric":299268},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":18944,"new_prefill":53975,"score_linear":-8871.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":72919,"score_linear":336145.0,"score_lmetric":1171374},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1409,"cache_hit":0,"new_prefill":72919,"score_linear":49483.0,"score_lmetric":72919},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44360,"ongoing_decode_tokens":34226,"pending_prefill_tokens":1430,"num_requests":3,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":72919,"score_linear":44360.0,"score_lmetric":223047},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1200,"cache_hit":0,"new_prefill":72919,"score_linear":80246.0,"score_lmetric":72919},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":100531,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":72919,"score_linear":109237.0,"score_lmetric":218757}],"chosen_score_linear":-8871.0,"chosen_score_lmetric":0,"t_first_token":339140.575315374,"t_first_token_unix":1779770005.5892596,"t_done":339141.461057628,"t_done_unix":1779770006.4750016},{"request_id":"1363943:5:1373637:609","session_id":"1363943","input_length":18676,"t_proxy_recv":339142.143164381,"t_decision_unix":1779770007.157106,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9869351038766331,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (244 < 8000)","cache_hit":18432,"estimated_new_tokens":244,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":8704,"new_prefill":9972,"score_linear":4912.0,"score_lmetric":9972},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":18676,"score_linear":126340.0,"score_lmetric":145016},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":4096,"new_prefill":14580,"score_linear":85472.0,"score_lmetric":94075},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":18676,"score_linear":336145.0,"score_lmetric":845916},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":18676,"score_linear":145931.0,"score_lmetric":97308},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26705,"ongoing_decode_tokens":26705,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":4096,"new_prefill":14580,"score_linear":22609.0,"score_lmetric":14580},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":18432,"new_prefill":244,"score_linear":61814.0,"score_lmetric":244},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":18676,"score_linear":105902.0,"score_lmetric":64162}],"chosen_score_linear":61814.0,"chosen_score_lmetric":244,"t_first_token":339142.242599556,"t_first_token_unix":1779770007.2565432,"t_done":339144.258025273,"t_done_unix":1779770009.2719688},{"request_id":"1366128:5:1373829:610","session_id":"1366128","input_length":46764,"t_proxy_recv":339142.798882548,"t_decision_unix":1779770007.8128238,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":34816,"affinity_cache_ratio":0.7445043195620563,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":34816,"estimated_new_tokens":11948,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":46764,"score_linear":13616.0,"score_lmetric":46764},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":46764,"score_linear":126340.0,"score_lmetric":173104},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":46764,"score_linear":89568.0,"score_lmetric":126259},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":46764,"score_linear":336145.0,"score_lmetric":1014444},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":46764,"score_linear":145931.0,"score_lmetric":209660},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26705,"ongoing_decode_tokens":26705,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":34816,"new_prefill":11948,"score_linear":-8111.0,"score_lmetric":11948},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98922,"ongoing_decode_tokens":98922,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":46764,"score_linear":98922.0,"score_lmetric":93528},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":46764,"score_linear":105902.0,"score_lmetric":92250}],"chosen_score_linear":-8111.0,"chosen_score_lmetric":11948,"t_first_token":339146.115965824,"t_first_token_unix":1779770011.12991,"t_done":339146.393075692,"t_done_unix":1779770011.4070196},{"request_id":"1374465:1:1374465:613","session_id":"1374465","input_length":18497,"t_proxy_recv":339145.027564543,"t_decision_unix":1779770010.0415058,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":18497,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":18497,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":18497,"score_linear":13616.0,"score_lmetric":18497},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":18497,"score_linear":126340.0,"score_lmetric":144837},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":18497,"score_linear":89568.0,"score_lmetric":97992},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":250361,"pending_prefill_tokens":2509,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":18497,"score_linear":336145.0,"score_lmetric":126036},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":209467,"ongoing_decode_tokens":18301,"pending_prefill_tokens":6211,"num_requests":5,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":18497,"score_linear":209467.0,"score_lmetric":123540},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":73469,"ongoing_decode_tokens":26705,"pending_prefill_tokens":11948,"num_requests":2,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":18497,"score_linear":73469.0,"score_lmetric":60890},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":100108,"ongoing_decode_tokens":80246,"pending_prefill_tokens":19862,"num_requests":2,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":18497,"score_linear":100108.0,"score_lmetric":76718},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":18497,"score_linear":105902.0,"score_lmetric":63983}],"chosen_score_linear":13616.0,"chosen_score_lmetric":18497,"t_first_token":339146.867486228,"t_first_token_unix":1779770011.8814297,"t_done":339147.080858124,"t_done_unix":1779770012.094802},{"request_id":"1374347:1:1374347:611","session_id":"1374347","input_length":19862,"t_proxy_recv":339144.585440476,"t_decision_unix":1779770009.5993817,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":19862,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":19862,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":19862,"score_linear":13616.0,"score_lmetric":19862},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":19862,"score_linear":126340.0,"score_lmetric":146202},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":19862,"score_linear":89568.0,"score_lmetric":99357},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":250361,"pending_prefill_tokens":2509,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":19862,"score_linear":336145.0,"score_lmetric":134226},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":19862,"score_linear":145931.0,"score_lmetric":102052},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":73469,"ongoing_decode_tokens":26705,"pending_prefill_tokens":11948,"num_requests":2,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":19862,"score_linear":73469.0,"score_lmetric":63620},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":19862,"score_linear":80246.0,"score_lmetric":19862},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":19862,"score_linear":105902.0,"score_lmetric":65348}],"chosen_score_linear":80246.0,"chosen_score_lmetric":19862,"t_first_token":339146.665740409,"t_first_token_unix":1779770011.6796842,"t_done":339147.399715405,"t_done_unix":1779770012.4136584},{"request_id":"1342634:2:1370263:590","session_id":"1342634","input_length":64531,"t_proxy_recv":339130.314694754,"t_decision_unix":1779769995.3286362,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9124296849576172,"affinity_num_requests":2,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5651 < 8000)","cache_hit":58880,"estimated_new_tokens":5651,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":64531,"score_linear":156948.0,"score_lmetric":375374},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":79392,"ongoing_decode_tokens":68259,"pending_prefill_tokens":11133,"num_requests":2,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":64531,"score_linear":79392.0,"score_lmetric":151328},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":64531,"score_linear":82992.0,"score_lmetric":118506},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":64531,"score_linear":336145.0,"score_lmetric":1121046},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":67784,"ongoing_decode_tokens":0,"pending_prefill_tokens":18301,"num_requests":2,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":58880,"new_prefill":5651,"score_linear":8904.0,"score_lmetric":47904},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":31019,"ongoing_decode_tokens":7712,"pending_prefill_tokens":7947,"num_requests":2,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":0,"new_prefill":64531,"score_linear":31019.0,"score_lmetric":144956},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127907,"ongoing_decode_tokens":98406,"pending_prefill_tokens":7997,"num_requests":3,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":0,"new_prefill":64531,"score_linear":127907.0,"score_lmetric":217584},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62385,"ongoing_decode_tokens":0,"pending_prefill_tokens":19889,"num_requests":2,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":64531,"score_linear":62385.0,"score_lmetric":168840}],"chosen_score_linear":8904.0,"chosen_score_lmetric":47904,"t_first_token":339146.845640965,"t_first_token_unix":1779770011.8595848,"t_done":339148.962229965,"t_done_unix":1779770013.9761739},{"request_id":"1363943:6:1375416:618","session_id":"1363943","input_length":19116,"t_proxy_recv":339148.335827203,"t_decision_unix":1779770013.3497684,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9642184557438794,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (684 < 8000)","cache_hit":18432,"estimated_new_tokens":684,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49504,"ongoing_decode_tokens":0,"pending_prefill_tokens":48,"num_requests":2,"active_p_offloads":0,"cached_blocks":1269,"cache_hit":8704,"new_prefill":10412,"score_linear":40800.0,"score_lmetric":20920},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":19116,"score_linear":126340.0,"score_lmetric":145456},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":4096,"new_prefill":15020,"score_linear":85472.0,"score_lmetric":94515},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":377653,"ongoing_decode_tokens":250361,"pending_prefill_tokens":7153,"num_requests":7,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":19116,"score_linear":377653.0,"score_lmetric":183883},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":195851,"ongoing_decode_tokens":146368,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1431,"cache_hit":0,"new_prefill":19116,"score_linear":195851.0,"score_lmetric":76464},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":85216,"ongoing_decode_tokens":26705,"pending_prefill_tokens":11919,"num_requests":2,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":4096,"new_prefill":15020,"score_linear":81120.0,"score_lmetric":53878},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":18432,"new_prefill":684,"score_linear":61814.0,"score_lmetric":684},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":19116,"score_linear":105902.0,"score_lmetric":64602}],"chosen_score_linear":61814.0,"chosen_score_lmetric":684,"t_first_token":339148.473573281,"t_first_token_unix":1779770013.4875185,"t_done":339150.266168979,"t_done_unix":1779770015.2801127},{"request_id":"1355484:3:1361065:531","session_id":"1355484","input_length":19714,"t_proxy_recv":339098.014353074,"t_decision_unix":1779769963.0282943,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":19456,"affinity_cache_ratio":0.9869128538094755,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (258 < 8000)","cache_hit":19456,"estimated_new_tokens":258,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":19714,"score_linear":156948.0,"score_lmetric":285740},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":19714,"score_linear":130877.0,"score_lmetric":39428},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":19714,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":19456,"new_prefill":258,"score_linear":98432.0,"score_lmetric":118146},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":19714,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":149558,"ongoing_decode_tokens":83526,"pending_prefill_tokens":496,"num_requests":2,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":19714,"score_linear":149558.0,"score_lmetric":40420},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":19714,"score_linear":80246.0,"score_lmetric":19714},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":92236,"ongoing_decode_tokens":58758,"pending_prefill_tokens":5318,"num_requests":2,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":12800,"new_prefill":6914,"score_linear":79436.0,"score_lmetric":24464}],"chosen_score_linear":98432.0,"chosen_score_lmetric":118146,"t_first_token":339143.704888216,"t_first_token_unix":1779770008.7188313,"t_done":339150.79274228,"t_done_unix":1779770015.806686},{"request_id":"1277428:5:1366630:559","session_id":"1277428","input_length":36301,"t_proxy_recv":339117.46577688,"t_decision_unix":1779769982.4797184,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9308834467370045,"affinity_num_requests":4,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2509 < 8000)","cache_hit":33792,"estimated_new_tokens":2509,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":36301,"score_linear":156948.0,"score_lmetric":318914},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":4608,"new_prefill":31693,"score_linear":126269.0,"score_lmetric":63386},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":17408,"new_prefill":18893,"score_linear":69634.0,"score_lmetric":95862},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":250361,"ongoing_decode_tokens":0,"pending_prefill_tokens":119801,"num_requests":4,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":33792,"new_prefill":2509,"score_linear":216569.0,"score_lmetric":489240},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":36301,"score_linear":90778.0,"score_lmetric":198862},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":36301,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93230,"ongoing_decode_tokens":93230,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":0,"new_prefill":36301,"score_linear":93230.0,"score_lmetric":72602},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":0,"pending_prefill_tokens":43187,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":36301,"score_linear":109237.0,"score_lmetric":238464}],"chosen_score_linear":216569.0,"chosen_score_lmetric":489240,"t_first_token":339149.018784182,"t_first_token_unix":1779770014.0327275,"t_done":339152.22719893,"t_done_unix":1779770017.2411425},{"request_id":"1370129:1:1370129:587","session_id":"1370129","input_length":18301,"t_proxy_recv":339129.820323223,"t_decision_unix":1779769994.8342645,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":18301,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":18301,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":7680,"new_prefill":10621,"score_linear":149268.0,"score_lmetric":267554},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":18301,"score_linear":68259.0,"score_lmetric":18301},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":7680,"new_prefill":10621,"score_linear":75312.0,"score_lmetric":64596},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":18301,"score_linear":336145.0,"score_lmetric":843666},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":18301,"score_linear":49483.0,"score_lmetric":18301},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":31019,"ongoing_decode_tokens":7712,"pending_prefill_tokens":7947,"num_requests":2,"active_p_offloads":0,"cached_blocks":1597,"cache_hit":7680,"new_prefill":10621,"score_linear":23339.0,"score_lmetric":37136},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127907,"ongoing_decode_tokens":98406,"pending_prefill_tokens":7997,"num_requests":3,"active_p_offloads":0,"cached_blocks":1242,"cache_hit":7680,"new_prefill":10621,"score_linear":120227.0,"score_lmetric":55854},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":0,"pending_prefill_tokens":17508,"num_requests":1,"active_p_offloads":0,"cached_blocks":1424,"cache_hit":0,"new_prefill":18301,"score_linear":60004.0,"score_lmetric":35809}],"chosen_score_linear":49483.0,"chosen_score_lmetric":18301,"t_first_token":339132.012693257,"t_first_token_unix":1779769997.0266373,"t_done":339153.005125011,"t_done_unix":1779770018.0190685},{"request_id":"1269373:22:1358956:522","session_id":"1269373","input_length":80246,"t_proxy_recv":339090.48145788,"t_decision_unix":1779769955.4953992,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9761981905640157,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1910 < 8000)","cache_hit":78336,"estimated_new_tokens":1910,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":7680,"new_prefill":72566,"score_linear":149268.0,"score_lmetric":391444},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":80246,"score_linear":130877.0,"score_lmetric":160492},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":69564,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":7680,"new_prefill":72566,"score_linear":50586.0,"score_lmetric":72566},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":80246,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":80246,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":7680,"new_prefill":72566,"score_linear":75846.0,"score_lmetric":156092},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":78336,"new_prefill":1910,"score_linear":-78336.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":34694,"num_requests":1,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":80246,"score_linear":58758.0,"score_lmetric":114940}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":339091.53478153,"t_first_token_unix":1779769956.548725,"t_done":339153.246107574,"t_done_unix":1779770018.2600515},{"request_id":"1374465:2:1376358:620","session_id":"1374465","input_length":21588,"t_proxy_recv":339151.429970237,"t_decision_unix":1779770016.4439113,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":18432,"affinity_cache_ratio":0.8538076709282935,"affinity_num_requests":2,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3156 < 8000)","cache_hit":18432,"estimated_new_tokens":3156,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49504,"ongoing_decode_tokens":0,"pending_prefill_tokens":48,"num_requests":2,"active_p_offloads":0,"cached_blocks":1269,"cache_hit":18432,"new_prefill":3156,"score_linear":31072.0,"score_lmetric":6408},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":21588,"score_linear":126340.0,"score_lmetric":147928},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":21588,"score_linear":89568.0,"score_lmetric":101083},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":377745,"ongoing_decode_tokens":266948,"pending_prefill_tokens":4644,"num_requests":7,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":21588,"score_linear":377745.0,"score_lmetric":183624},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131320,"ongoing_decode_tokens":81837,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1442,"cache_hit":0,"new_prefill":21588,"score_linear":131320.0,"score_lmetric":64764},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":85216,"ongoing_decode_tokens":85216,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":21588,"score_linear":85216.0,"score_lmetric":43176},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":100052,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1307,"cache_hit":0,"new_prefill":21588,"score_linear":100052.0,"score_lmetric":43176},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":21588,"score_linear":143943.0,"score_lmetric":136502}],"chosen_score_linear":31072.0,"chosen_score_lmetric":6408,"t_first_token":339153.264979987,"t_first_token_unix":1779770018.2789233,"t_done":339153.811053607,"t_done_unix":1779770018.8249972},{"request_id":"1371622:1:1371622:598","session_id":"1371622","input_length":13616,"t_proxy_recv":339134.962978707,"t_decision_unix":1779769999.97692,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":13616,"tie_break_used":true,"v2_pd_sep":true,"v2_decision":"pd_sep","v2_reason":"local_cost 1.70s > migrate_cost 1.03s + margin 0.20s","v2_chosen_cache_hit":0,"v2_src_idx":4,"v2_src_cache_hit":11264,"v2_new_local":13616,"v2_new_src":2352,"v2_cost_local_s":1.702,"v2_cost_src_interf_s":0.3528,"v2_cost_xfer_s":0.6819444444444445,"v2_cost_migrate_s":1.0347444444444445,"cache_hit":0,"estimated_new_tokens":13616,"route_class":"PD_SEP_V2","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34679,"ongoing_decode_tokens":34679,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":13616,"score_linear":34679.0,"score_lmetric":13616},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101073,"ongoing_decode_tokens":101073,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":13616,"score_linear":101073.0,"score_lmetric":27232},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":13616,"score_linear":82992.0,"score_lmetric":67591},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":13616,"score_linear":336145.0,"score_lmetric":815556},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":132315,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":11264,"new_prefill":2352,"score_linear":121051.0,"score_lmetric":24009},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":9399,"ongoing_decode_tokens":9399,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":0,"new_prefill":13616,"score_linear":9399.0,"score_lmetric":13616},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":0,"new_prefill":13616,"score_linear":80246.0,"score_lmetric":13616},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":13616,"score_linear":60004.0,"score_lmetric":13616}],"chosen_score_linear":34679.0,"chosen_score_lmetric":13616,"v2_src_url":"http://127.0.0.1:8004","t_prefill_sent":339134.96352682,"t_prefill_sent_unix":1779769999.9774694,"t_prefill_done":339147.58589712,"t_prefill_done_unix":1779770012.5998404,"t_decode_sent":339147.586554103,"t_decode_sent_unix":1779770012.6004972,"t_first_token":339153.292160317,"t_first_token_unix":1779770018.3061037,"t_done":339154.467557075,"t_done_unix":1779770019.4815004},{"request_id":"1371337:2:1376534:621","session_id":"1371337","input_length":24323,"t_proxy_recv":339152.097040543,"t_decision_unix":1779770017.1109815,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":9216,"affinity_cache_ratio":0.3789006290342474,"affinity_num_requests":2,"avg_num_requests":2.625,"fallback_score":30214,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":9216,"estimated_new_tokens":15107,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":71092,"ongoing_decode_tokens":0,"pending_prefill_tokens":3204,"num_requests":3,"active_p_offloads":0,"cached_blocks":1269,"cache_hit":0,"new_prefill":24323,"score_linear":71092.0,"score_lmetric":82581},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":24323,"score_linear":126340.0,"score_lmetric":150663},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":24323,"score_linear":89568.0,"score_lmetric":103818},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":377745,"ongoing_decode_tokens":266948,"pending_prefill_tokens":4644,"num_requests":7,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":24323,"score_linear":377745.0,"score_lmetric":202769},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131320,"ongoing_decode_tokens":81837,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1442,"cache_hit":0,"new_prefill":24323,"score_linear":131320.0,"score_lmetric":72969},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":85216,"ongoing_decode_tokens":85216,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":9216,"new_prefill":15107,"score_linear":76000.0,"score_lmetric":30214},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":100052,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1307,"cache_hit":0,"new_prefill":24323,"score_linear":100052.0,"score_lmetric":48646},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":24323,"score_linear":143943.0,"score_lmetric":141972}],"chosen_score_linear":76000.0,"chosen_score_lmetric":30214,"t_first_token":339154.228077385,"t_first_token_unix":1779770019.242021,"t_done":339155.463938057,"t_done_unix":1779770020.4778812},{"request_id":"1366128:6:1375094:616","session_id":"1366128","input_length":58511,"t_proxy_recv":339147.291524505,"t_decision_unix":1779770012.3054662,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":46592,"affinity_cache_ratio":0.7962947138144965,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":46592,"estimated_new_tokens":11919,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1269,"cache_hit":0,"new_prefill":58511,"score_linear":13616.0,"score_lmetric":58511},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":58511,"score_linear":126340.0,"score_lmetric":184851},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":58511,"score_linear":89568.0,"score_lmetric":138006},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":377653,"ongoing_decode_tokens":250361,"pending_prefill_tokens":7153,"num_requests":7,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":58511,"score_linear":377653.0,"score_lmetric":459648},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":209467,"ongoing_decode_tokens":82832,"pending_prefill_tokens":560,"num_requests":5,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":58511,"score_linear":209467.0,"score_lmetric":295355},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26705,"ongoing_decode_tokens":26705,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":46592,"new_prefill":11919,"score_linear":-19887.0,"score_lmetric":11919},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":100108,"ongoing_decode_tokens":100108,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":58511,"score_linear":100108.0,"score_lmetric":117022},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":58511,"score_linear":105902.0,"score_lmetric":103997}],"chosen_score_linear":-19887.0,"chosen_score_lmetric":11919,"t_first_token":339151.42302471,"t_first_token_unix":1779770016.436968,"t_done":339155.866811893,"t_done_unix":1779770020.8807557},{"request_id":"1377061:1:1377061:622","session_id":"1377061","input_length":16241,"t_proxy_recv":339154.064169056,"t_decision_unix":1779770019.0781105,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":16241,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":16241,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49504,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1275,"cache_hit":0,"new_prefill":16241,"score_linear":49504.0,"score_lmetric":32482},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":16241,"score_linear":126340.0,"score_lmetric":142581},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":16241,"score_linear":89568.0,"score_lmetric":95736},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":16241,"score_linear":378027.0,"score_lmetric":147812},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169305,"ongoing_decode_tokens":63536,"pending_prefill_tokens":37854,"num_requests":3,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":16241,"score_linear":169305.0,"score_lmetric":162285},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":109539,"ongoing_decode_tokens":85216,"pending_prefill_tokens":15107,"num_requests":3,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":16241,"score_linear":109539.0,"score_lmetric":94044},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1312,"cache_hit":0,"new_prefill":16241,"score_linear":19806.0,"score_lmetric":16241},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":16241,"score_linear":143943.0,"score_lmetric":125808}],"chosen_score_linear":19806.0,"chosen_score_lmetric":16241,"t_first_token":339155.560170655,"t_first_token_unix":1779770020.5741143,"t_done":339155.999253405,"t_done_unix":1779770021.0131974},{"request_id":"1364090:2:1371798:600","session_id":"1364090","input_length":26705,"t_proxy_recv":339135.608171469,"t_decision_unix":1779770000.6221128,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":14336,"affinity_cache_ratio":0.5368283093053735,"affinity_num_requests":2,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=11264 chosen=14336)","cache_hit":14336,"estimated_new_tokens":12369,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":48295,"ongoing_decode_tokens":34679,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":7680,"new_prefill":19025,"score_linear":40615.0,"score_lmetric":38050},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101073,"ongoing_decode_tokens":101073,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":26705,"score_linear":101073.0,"score_lmetric":53410},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":21371,"pending_prefill_tokens":53975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":7680,"new_prefill":19025,"score_linear":75312.0,"score_lmetric":73000},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":26705,"score_linear":336145.0,"score_lmetric":894090},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":26705,"score_linear":145931.0,"score_lmetric":129424},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44436,"ongoing_decode_tokens":9399,"pending_prefill_tokens":11997,"num_requests":2,"active_p_offloads":0,"cached_blocks":1642,"cache_hit":14336,"new_prefill":12369,"score_linear":30100.0,"score_lmetric":48732},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1267,"cache_hit":11264,"new_prefill":15441,"score_linear":68982.0,"score_lmetric":15441},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1428,"cache_hit":0,"new_prefill":26705,"score_linear":60004.0,"score_lmetric":26705}],"chosen_score_linear":30100.0,"chosen_score_lmetric":48732,"t_first_token":339139.518764267,"t_first_token_unix":1779770004.5327082,"t_done":339156.413524628,"t_done_unix":1779770021.4274678},{"request_id":"1363943:7:1377305:623","session_id":"1363943","input_length":20727,"t_proxy_recv":339154.981182471,"t_decision_unix":1779770019.9951236,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18944,"affinity_cache_ratio":0.9139769382930477,"affinity_num_requests":2,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1783 < 8000)","cache_hit":18944,"estimated_new_tokens":1783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1301,"cache_hit":8704,"new_prefill":12023,"score_linear":27184.0,"score_lmetric":12023},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":20727,"score_linear":126340.0,"score_lmetric":147067},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":4096,"new_prefill":16631,"score_linear":85472.0,"score_lmetric":96126},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":20727,"score_linear":378027.0,"score_lmetric":179214},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169305,"ongoing_decode_tokens":63536,"pending_prefill_tokens":37854,"num_requests":3,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":4096,"new_prefill":16631,"score_linear":165209.0,"score_lmetric":163455},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":109539,"ongoing_decode_tokens":109539,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":4096,"new_prefill":16631,"score_linear":105443.0,"score_lmetric":49893},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":36047,"ongoing_decode_tokens":0,"pending_prefill_tokens":16241,"num_requests":2,"active_p_offloads":0,"cached_blocks":1312,"cache_hit":18944,"new_prefill":1783,"score_linear":17103.0,"score_lmetric":36048},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":20727,"score_linear":143943.0,"score_lmetric":134780}],"chosen_score_linear":17103.0,"chosen_score_lmetric":36048,"t_first_token":339155.82833339,"t_first_token_unix":1779770020.8422773,"t_done":339157.003538858,"t_done_unix":1779770022.0174825},{"request_id":"1372791:2:1377698:624","session_id":"1372791","input_length":14851,"t_proxy_recv":339156.296457147,"t_decision_unix":1779770021.310398,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":0,"affinity_cache_hit":4608,"affinity_cache_ratio":0.3102821358831055,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":14851,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"local_cost 1.86s <= migrate_cost 1.74s + margin 0.20s","v2_chosen_cache_hit":0,"v2_src_idx":1,"v2_src_cache_hit":4608,"v2_new_local":14851,"v2_new_src":10243,"v2_cost_local_s":1.856375,"v2_cost_src_interf_s":1.280375,"v2_cost_xfer_s":0.45625,"v2_cost_migrate_s":1.736625,"cache_hit":0,"estimated_new_tokens":14851,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1301,"cache_hit":0,"new_prefill":14851,"score_linear":35888.0,"score_lmetric":14851},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":4608,"new_prefill":10243,"score_linear":121732.0,"score_lmetric":136583},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":14851,"score_linear":89568.0,"score_lmetric":94346},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":14851,"score_linear":378027.0,"score_lmetric":138082},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169305,"ongoing_decode_tokens":63536,"pending_prefill_tokens":37854,"num_requests":3,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":14851,"score_linear":169305.0,"score_lmetric":158115},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26705,"ongoing_decode_tokens":26705,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1759,"cache_hit":0,"new_prefill":14851,"score_linear":26705.0,"score_lmetric":14851},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40533,"ongoing_decode_tokens":20727,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":14851,"score_linear":40533.0,"score_lmetric":29702},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":14851,"score_linear":143943.0,"score_lmetric":123028}],"chosen_score_linear":35888.0,"chosen_score_lmetric":14851,"t_first_token":339157.621800075,"t_first_token_unix":1779770022.6357439,"t_done":339157.914658084,"t_done_unix":1779770022.9286013},{"request_id":"1366128:7:1378071:626","session_id":"1366128","input_length":59007,"t_proxy_recv":339157.54742328,"t_decision_unix":1779770022.5613644,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9891707763485688,"affinity_num_requests":0,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (639 < 8000)","cache_hit":58368,"estimated_new_tokens":639,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50739,"ongoing_decode_tokens":35888,"pending_prefill_tokens":14851,"num_requests":2,"active_p_offloads":0,"cached_blocks":1301,"cache_hit":0,"new_prefill":59007,"score_linear":50739.0,"score_lmetric":147716},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":59007,"score_linear":126340.0,"score_lmetric":185347},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":59007,"score_linear":89568.0,"score_lmetric":138502},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":59007,"score_linear":378027.0,"score_lmetric":447174},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169305,"ongoing_decode_tokens":63536,"pending_prefill_tokens":37854,"num_requests":3,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":59007,"score_linear":169305.0,"score_lmetric":290583},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1785,"cache_hit":58368,"new_prefill":639,"score_linear":-58368.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1346,"cache_hit":0,"new_prefill":59007,"score_linear":19806.0,"score_lmetric":59007},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":59007,"score_linear":143943.0,"score_lmetric":211340}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":339157.75053063,"t_first_token_unix":1779770022.7644742,"t_done":339158.527691165,"t_done_unix":1779770023.5416346},{"request_id":"1377061:2:1378590:631","session_id":"1377061","input_length":16515,"t_proxy_recv":339159.325997898,"t_decision_unix":1779770024.3399394,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":15872,"affinity_cache_ratio":0.961065697850439,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (643 < 8000)","cache_hit":15872,"estimated_new_tokens":643,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1330,"cache_hit":0,"new_prefill":16515,"score_linear":35888.0,"score_lmetric":16515},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":16515,"score_linear":126340.0,"score_lmetric":142855},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":16515,"score_linear":89568.0,"score_lmetric":96010},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":16515,"score_linear":378027.0,"score_lmetric":149730},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":16515,"score_linear":191437.0,"score_lmetric":244564},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":5458,"ongoing_decode_tokens":0,"pending_prefill_tokens":5458,"num_requests":1,"active_p_offloads":0,"cached_blocks":1786,"cache_hit":0,"new_prefill":16515,"score_linear":5458.0,"score_lmetric":21973},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1346,"cache_hit":15872,"new_prefill":643,"score_linear":3934.0,"score_lmetric":643},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":16515,"score_linear":143943.0,"score_lmetric":126356}],"chosen_score_linear":3934.0,"chosen_score_lmetric":643,"t_first_token":339159.41369312,"t_first_token_unix":1779770024.4276364,"t_done":339159.482954531,"t_done_unix":1779770024.496898},{"request_id":"1378684:1:1378684:632","session_id":"1378684","input_length":7810,"t_proxy_recv":339159.661334172,"t_decision_unix":1779770024.6752753,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":7810,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7810 < 8000)","cache_hit":0,"estimated_new_tokens":7810,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1330,"cache_hit":0,"new_prefill":7810,"score_linear":35888.0,"score_lmetric":7810},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":7810,"score_linear":126340.0,"score_lmetric":134150},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":7810,"score_linear":89568.0,"score_lmetric":87305},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":7810,"score_linear":378027.0,"score_lmetric":88795},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":7810,"score_linear":191437.0,"score_lmetric":209744},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":5458,"ongoing_decode_tokens":5458,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1786,"cache_hit":0,"new_prefill":7810,"score_linear":5458.0,"score_lmetric":7810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1347,"cache_hit":0,"new_prefill":7810,"score_linear":19806.0,"score_lmetric":7810},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":7810,"score_linear":143943.0,"score_lmetric":108946}],"chosen_score_linear":19806.0,"chosen_score_lmetric":7810,"t_first_token":339160.204635222,"t_first_token_unix":1779770025.2185795,"t_done":339160.470773535,"t_done_unix":1779770025.4847171},{"request_id":"1378543:1:1378543:630","session_id":"1378543","input_length":5458,"t_proxy_recv":339159.169159247,"t_decision_unix":1779770024.1831007,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5458 < 8000)","cache_hit":0,"estimated_new_tokens":5458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1330,"cache_hit":0,"new_prefill":5458,"score_linear":35888.0,"score_lmetric":5458},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":5458,"score_linear":126340.0,"score_lmetric":131798},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":5458,"score_linear":89568.0,"score_lmetric":84953},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":5458,"score_linear":378027.0,"score_lmetric":72331},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":5458,"score_linear":191437.0,"score_lmetric":200336},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1786,"cache_hit":0,"new_prefill":5458,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1346,"cache_hit":0,"new_prefill":5458,"score_linear":19806.0,"score_lmetric":5458},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":5458,"score_linear":143943.0,"score_lmetric":104242}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339159.524380584,"t_first_token_unix":1779770024.5383234,"t_done":339160.608025615,"t_done_unix":1779770025.6219692},{"request_id":"1357132:2:1364990:550","session_id":"1357132","input_length":14721,"t_proxy_recv":339111.64555352,"t_decision_unix":1779769976.659495,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":13312,"affinity_cache_ratio":0.9042863935873922,"affinity_num_requests":3,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1409 < 8000)","cache_hit":13312,"estimated_new_tokens":1409,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":14721,"score_linear":156948.0,"score_lmetric":275754},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":14721,"score_linear":130877.0,"score_lmetric":29442},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":14721,"score_linear":87042.0,"score_lmetric":91690},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":235640,"ongoing_decode_tokens":0,"pending_prefill_tokens":118392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":13312,"new_prefill":1409,"score_linear":222328.0,"score_lmetric":359403},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":14721,"score_linear":90778.0,"score_lmetric":155702},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":29954,"ongoing_decode_tokens":29954,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1494,"cache_hit":0,"new_prefill":14721,"score_linear":29954.0,"score_lmetric":29442},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":94901,"ongoing_decode_tokens":92882,"pending_prefill_tokens":2019,"num_requests":3,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":14721,"score_linear":94901.0,"score_lmetric":50220},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":68220,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":14721,"score_linear":68220.0,"score_lmetric":100342}],"chosen_score_linear":222328.0,"chosen_score_lmetric":359403,"t_first_token":339144.200037937,"t_first_token_unix":1779770009.2139816,"t_done":339161.412408542,"t_done_unix":1779770026.4263518},{"request_id":"1379185:1:1379185:634","session_id":"1379185","input_length":1422,"t_proxy_recv":339161.394540858,"t_decision_unix":1779770026.408482,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":1422,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1422 < 8000)","cache_hit":0,"estimated_new_tokens":1422,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1330,"cache_hit":0,"new_prefill":1422,"score_linear":35888.0,"score_lmetric":1422},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":1422,"score_linear":126340.0,"score_lmetric":127762},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":1422,"score_linear":89568.0,"score_lmetric":80917},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":328544,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":6,"active_p_offloads":0,"cached_blocks":1260,"cache_hit":0,"new_prefill":1422,"score_linear":328544.0,"score_lmetric":37782},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":1422,"score_linear":191437.0,"score_lmetric":184192},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":1422,"score_linear":15283.0,"score_lmetric":16705},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":1422,"score_linear":19806.0,"score_lmetric":1422},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":1422,"score_linear":143943.0,"score_lmetric":96170}],"chosen_score_linear":35888.0,"chosen_score_lmetric":1422,"t_first_token":339161.560689128,"t_first_token_unix":1779770026.5746326,"t_done":339161.869770395,"t_done_unix":1779770026.8837135},{"request_id":"1279412:24:1364169:546","session_id":"1279412","input_length":98038,"t_proxy_recv":339108.798546072,"t_decision_unix":1779769973.812487,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9974907688855342,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (246 < 8000)","cache_hit":97792,"estimated_new_tokens":246,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":98038,"score_linear":156948.0,"score_lmetric":442388},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":98038,"score_linear":130877.0,"score_lmetric":196076},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":98038,"score_linear":87042.0,"score_lmetric":175007},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":137602,"ongoing_decode_tokens":0,"pending_prefill_tokens":118146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":97792,"new_prefill":246,"score_linear":39810.0,"score_lmetric":236784},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":2,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":98038,"score_linear":90778.0,"score_lmetric":322336},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":38660,"ongoing_decode_tokens":16057,"pending_prefill_tokens":6217,"num_requests":3,"active_p_offloads":0,"cached_blocks":1490,"cache_hit":0,"new_prefill":98038,"score_linear":38660.0,"score_lmetric":312765},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":92882,"ongoing_decode_tokens":80246,"pending_prefill_tokens":8540,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":98038,"score_linear":92882.0,"score_lmetric":213156},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33157,"ongoing_decode_tokens":24451,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":12800,"new_prefill":85238,"score_linear":20357.0,"score_lmetric":170476}],"chosen_score_linear":39810.0,"chosen_score_lmetric":236784,"t_first_token":339143.709815503,"t_first_token_unix":1779770008.7237587,"t_done":339162.716906851,"t_done_unix":1779770027.7308502},{"request_id":"1357132:3:1366582:557","session_id":"1357132","input_length":14816,"t_proxy_recv":339161.416707557,"t_decision_unix":1779770026.4306493,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.9676025917926566,"affinity_num_requests":5,"avg_num_requests":2.125,"fallback_score":14816,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":14816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":37310,"ongoing_decode_tokens":35888,"pending_prefill_tokens":1422,"num_requests":2,"active_p_offloads":0,"cached_blocks":1330,"cache_hit":0,"new_prefill":14816,"score_linear":37310.0,"score_lmetric":32476},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":14816,"score_linear":126340.0,"score_lmetric":141156},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":14816,"score_linear":89568.0,"score_lmetric":94311},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":313823,"ongoing_decode_tokens":215926,"pending_prefill_tokens":4875,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":14336,"new_prefill":480,"score_linear":299487.0,"score_lmetric":26775},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":14816,"score_linear":191437.0,"score_lmetric":237768},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":14816,"score_linear":15283.0,"score_lmetric":30099},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":14816,"score_linear":19806.0,"score_lmetric":14816},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":14816,"score_linear":143943.0,"score_lmetric":122958}],"chosen_score_linear":19806.0,"chosen_score_lmetric":14816,"t_first_token":339162.743874665,"t_first_token_unix":1779770027.7578182,"t_done":339162.920912286,"t_done_unix":1779770027.934858},{"request_id":"1357132:4:1367654:570","session_id":"1357132","input_length":14949,"t_proxy_recv":339162.924638363,"t_decision_unix":1779770027.93858,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.9589939126362967,"affinity_num_requests":2,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (613 < 8000)","cache_hit":14336,"estimated_new_tokens":613,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1332,"cache_hit":0,"new_prefill":14949,"score_linear":35888.0,"score_lmetric":14949},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":14949,"score_linear":126340.0,"score_lmetric":141289},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":14949,"score_linear":89568.0,"score_lmetric":94444},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":14336,"new_prefill":613,"score_linear":357372.0,"score_lmetric":43794},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":14949,"score_linear":191437.0,"score_lmetric":238300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":17915,"ongoing_decode_tokens":17915,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":14949,"score_linear":17915.0,"score_lmetric":29898},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":14336,"new_prefill":613,"score_linear":26418.0,"score_lmetric":1226},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":14949,"score_linear":143943.0,"score_lmetric":123224}],"chosen_score_linear":26418.0,"chosen_score_lmetric":1226,"t_first_token":339162.989837342,"t_first_token_unix":1779770028.0037808,"t_done":339163.213415899,"t_done_unix":1779770028.2273593},{"request_id":"1379569:1:1379569:638","session_id":"1379569","input_length":2632,"t_proxy_recv":339162.60917338,"t_decision_unix":1779770027.6231148,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":2632,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2632 < 8000)","cache_hit":0,"estimated_new_tokens":2632,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1332,"cache_hit":0,"new_prefill":2632,"score_linear":35888.0,"score_lmetric":2632},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":2632,"score_linear":126340.0,"score_lmetric":128972},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":2632,"score_linear":89568.0,"score_lmetric":82127},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371479,"ongoing_decode_tokens":215926,"pending_prefill_tokens":6211,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":2632,"score_linear":371479.0,"score_lmetric":53058},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":2632,"score_linear":191437.0,"score_lmetric":189032},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":2632,"score_linear":15283.0,"score_lmetric":2632},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":55570,"ongoing_decode_tokens":0,"pending_prefill_tokens":15284,"num_requests":3,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":2632,"score_linear":55570.0,"score_lmetric":53748},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":2632,"score_linear":143943.0,"score_lmetric":98590}],"chosen_score_linear":15283.0,"chosen_score_lmetric":2632,"t_first_token":339162.786788969,"t_first_token_unix":1779770027.8007321,"t_done":339163.384820039,"t_done_unix":1779770028.3987641},{"request_id":"1357132:5:1368445:579","session_id":"1357132","input_length":15871,"t_proxy_recv":339163.218123965,"t_decision_unix":1779770028.2320657,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14848,"affinity_cache_ratio":0.93554281393737,"affinity_num_requests":2,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1023 < 8000)","cache_hit":14848,"estimated_new_tokens":1023,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1332,"cache_hit":0,"new_prefill":15871,"score_linear":35888.0,"score_lmetric":15871},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":15871,"score_linear":126340.0,"score_lmetric":142211},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":15871,"score_linear":89568.0,"score_lmetric":95366},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":14336,"new_prefill":1535,"score_linear":357372.0,"score_lmetric":49326},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":119822,"pending_prefill_tokens":6772,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":15871,"score_linear":191437.0,"score_lmetric":90572},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":17915,"ongoing_decode_tokens":17915,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":15871,"score_linear":17915.0,"score_lmetric":31742},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1391,"cache_hit":14848,"new_prefill":1023,"score_linear":25906.0,"score_lmetric":2046},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":15871,"score_linear":143943.0,"score_lmetric":125068}],"chosen_score_linear":25906.0,"chosen_score_lmetric":2046,"t_first_token":339163.392930126,"t_first_token_unix":1779770028.4068732,"t_done":339163.617645771,"t_done_unix":1779770028.6315897},{"request_id":"1377061:3:1380174:641","session_id":"1377061","input_length":17368,"t_proxy_recv":339164.732636258,"t_decision_unix":1779770029.7465775,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":16384,"affinity_cache_ratio":0.943344081068632,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (984 < 8000)","cache_hit":16384,"estimated_new_tokens":984,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1332,"cache_hit":0,"new_prefill":17368,"score_linear":35888.0,"score_lmetric":17368},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":17368,"score_linear":126340.0,"score_lmetric":143708},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":100866,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":17368,"score_linear":89568.0,"score_lmetric":17368},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":17368,"score_linear":371708.0,"score_lmetric":144324},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":119822,"pending_prefill_tokens":6772,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":17368,"score_linear":191437.0,"score_lmetric":96560},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":17368,"score_linear":15283.0,"score_lmetric":17368},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":16384,"new_prefill":984,"score_linear":24370.0,"score_lmetric":1968},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":17368,"score_linear":143943.0,"score_lmetric":128062}],"chosen_score_linear":24370.0,"chosen_score_lmetric":1968,"t_first_token":339164.911863233,"t_first_token_unix":1779770029.9258068,"t_done":339165.044471783,"t_done_unix":1779770030.0584154},{"request_id":"1355951:3:1375866:619","session_id":"1355951","input_length":38041,"t_proxy_recv":339149.969416012,"t_decision_unix":1779770014.9833574,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9690596987460898,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1177 < 8000)","cache_hit":36864,"estimated_new_tokens":1177,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49504,"ongoing_decode_tokens":0,"pending_prefill_tokens":48,"num_requests":2,"active_p_offloads":0,"cached_blocks":1269,"cache_hit":0,"new_prefill":38041,"score_linear":49504.0,"score_lmetric":76178},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":38041,"score_linear":126340.0,"score_lmetric":164381},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":38041,"score_linear":89568.0,"score_lmetric":117536},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":377653,"ongoing_decode_tokens":286662,"pending_prefill_tokens":4644,"num_requests":7,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":38041,"score_linear":377653.0,"score_lmetric":298795},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131320,"ongoing_decode_tokens":81837,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1442,"cache_hit":0,"new_prefill":38041,"score_linear":131320.0,"score_lmetric":114123},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":85216,"ongoing_decode_tokens":26705,"pending_prefill_tokens":11919,"num_requests":2,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":38041,"score_linear":85216.0,"score_lmetric":99920},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":99362,"ongoing_decode_tokens":99362,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":38041,"score_linear":99362.0,"score_lmetric":76082},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":36864,"new_prefill":1177,"score_linear":69038.0,"score_lmetric":46663}],"chosen_score_linear":69038.0,"chosen_score_lmetric":46663,"t_first_token":339164.775858654,"t_first_token_unix":1779770029.789802,"t_done":339165.428966127,"t_done_unix":1779770030.4429097},{"request_id":"1378543:2:1380447:644","session_id":"1378543","input_length":5314,"t_proxy_recv":339165.647786589,"t_decision_unix":1779770030.661728,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":5120,"affinity_cache_ratio":0.963492660895747,"affinity_num_requests":2,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (194 < 8000)","cache_hit":5120,"estimated_new_tokens":194,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66795,"ongoing_decode_tokens":35888,"pending_prefill_tokens":9403,"num_requests":2,"active_p_offloads":0,"cached_blocks":1332,"cache_hit":0,"new_prefill":5314,"score_linear":66795.0,"score_lmetric":29434},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":5314,"score_linear":126340.0,"score_lmetric":131654},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":100866,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":5314,"score_linear":89568.0,"score_lmetric":5314},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":5314,"score_linear":371708.0,"score_lmetric":72000},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":5314,"score_linear":191437.0,"score_lmetric":21256},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74513,"ongoing_decode_tokens":15283,"pending_prefill_tokens":350,"num_requests":2,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":5120,"new_prefill":194,"score_linear":69393.0,"score_lmetric":1088},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1394,"cache_hit":0,"new_prefill":5314,"score_linear":40754.0,"score_lmetric":10628},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":105902,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":5314,"score_linear":105902.0,"score_lmetric":5314}],"chosen_score_linear":69393.0,"chosen_score_lmetric":1088,"t_first_token":339165.780815518,"t_first_token_unix":1779770030.7947586,"t_done":339165.781369743,"t_done_unix":1779770030.795313},{"request_id":"1366128:8:1380438:643","session_id":"1366128","input_length":59230,"t_proxy_recv":339165.614188312,"t_decision_unix":1779770030.6281295,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9940908323484721,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (350 < 8000)","cache_hit":58880,"estimated_new_tokens":350,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66795,"ongoing_decode_tokens":35888,"pending_prefill_tokens":9403,"num_requests":2,"active_p_offloads":0,"cached_blocks":1332,"cache_hit":0,"new_prefill":59230,"score_linear":66795.0,"score_lmetric":137266},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":59230,"score_linear":126340.0,"score_lmetric":185570},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":100866,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":59230,"score_linear":89568.0,"score_lmetric":59230},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":59230,"score_linear":371708.0,"score_lmetric":395496},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":59230,"score_linear":191437.0,"score_lmetric":236920},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":58880,"new_prefill":350,"score_linear":-43597.0,"score_lmetric":350},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1394,"cache_hit":0,"new_prefill":59230,"score_linear":40754.0,"score_lmetric":118460},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":105902,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":59230,"score_linear":105902.0,"score_lmetric":59230}],"chosen_score_linear":-43597.0,"chosen_score_lmetric":350,"t_first_token":339165.775132146,"t_first_token_unix":1779770030.7890756,"t_done":339166.57231979,"t_done_unix":1779770031.586263},{"request_id":"1379569:2:1380473:645","session_id":"1379569","input_length":6767,"t_proxy_recv":339165.79583305,"t_decision_unix":1779770030.8097744,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":2,"affinity_cache_hit":2560,"affinity_cache_ratio":0.37830648736515443,"affinity_num_requests":2,"avg_num_requests":2.375,"fallback_score":6767,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6767 < 8000)","cache_hit":0,"estimated_new_tokens":6767,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":66795,"ongoing_decode_tokens":35888,"pending_prefill_tokens":9403,"num_requests":2,"active_p_offloads":0,"cached_blocks":1332,"cache_hit":0,"new_prefill":6767,"score_linear":66795.0,"score_lmetric":32340},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":6767,"score_linear":126340.0,"score_lmetric":133107},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":100866,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":6767,"score_linear":89568.0,"score_lmetric":6767},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":6767,"score_linear":371708.0,"score_lmetric":80718},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":6767,"score_linear":191437.0,"score_lmetric":27068},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74513,"ongoing_decode_tokens":74513,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":2560,"new_prefill":4207,"score_linear":71953.0,"score_lmetric":8414},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1394,"cache_hit":0,"new_prefill":6767,"score_linear":40754.0,"score_lmetric":13534},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":105902,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":6767,"score_linear":105902.0,"score_lmetric":6767}],"chosen_score_linear":89568.0,"chosen_score_lmetric":6767,"t_first_token":339166.266907182,"t_first_token_unix":1779770031.2808506,"t_done":339167.847964719,"t_done_unix":1779770032.861908},{"request_id":"1374465:3:1380413:642","session_id":"1374465","input_length":30907,"t_proxy_recv":339165.524247202,"t_decision_unix":1779770030.5381885,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":21504,"affinity_cache_ratio":0.6957647134953246,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":21504,"estimated_new_tokens":9403,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1332,"cache_hit":21504,"new_prefill":9403,"score_linear":14384.0,"score_lmetric":9403},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":30907,"score_linear":126340.0,"score_lmetric":157247},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":100866,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":30907,"score_linear":89568.0,"score_lmetric":30907},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":30907,"score_linear":371708.0,"score_lmetric":225558},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":30907,"score_linear":191437.0,"score_lmetric":123628},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":30907,"score_linear":15283.0,"score_lmetric":30907},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1394,"cache_hit":0,"new_prefill":30907,"score_linear":40754.0,"score_lmetric":61814},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":105902,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":30907,"score_linear":105902.0,"score_lmetric":30907}],"chosen_score_linear":14384.0,"chosen_score_lmetric":9403,"t_first_token":339167.403796359,"t_first_token_unix":1779770032.4177399,"t_done":339168.349316022,"t_done_unix":1779770033.3632596},{"request_id":"1373577:1:1373577:608","session_id":"1373577","input_length":79495,"t_proxy_recv":339142.018153045,"t_decision_unix":1779770007.0320942,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (2560 < 4000)","cache_hit":0,"estimated_new_tokens":79495,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":79495,"score_linear":13616.0,"score_lmetric":79495},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":79495,"score_linear":126340.0,"score_lmetric":205835},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":79495,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":79495,"score_linear":336145.0,"score_lmetric":1210830},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":2560,"new_prefill":76935,"score_linear":143371.0,"score_lmetric":330344},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26705,"ongoing_decode_tokens":26705,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":79495,"score_linear":26705.0,"score_lmetric":79495},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":79495,"score_linear":80246.0,"score_lmetric":79495},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":79495,"score_linear":105902.0,"score_lmetric":124981}],"chosen_score_linear":10073.0,"chosen_score_lmetric":0,"t_first_token":339163.449979276,"t_first_token_unix":1779770028.4639232,"t_done":339168.668371033,"t_done_unix":1779770033.6823144},{"request_id":"1313181:5:1320817:349","session_id":"1313181","input_length":105902,"t_proxy_recv":339141.297906523,"t_decision_unix":1779770006.3118482,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":60416,"affinity_cache_ratio":0.5704896980227002,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":60416,"estimated_new_tokens":45486,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":105902,"score_linear":13616.0,"score_lmetric":105902},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":105902,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":82992,"ongoing_decode_tokens":94290,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":105902,"score_linear":82992.0,"score_lmetric":105902},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":12800,"new_prefill":93102,"score_linear":323345.0,"score_lmetric":1292472},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":105902,"score_linear":145931.0,"score_lmetric":446212},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26705,"ongoing_decode_tokens":26705,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":105902,"score_linear":26705.0,"score_lmetric":105902},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":105902,"score_linear":80246.0,"score_lmetric":105902},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":60416,"new_prefill":45486,"score_linear":-60416.0,"score_lmetric":0}],"chosen_score_linear":-60416.0,"chosen_score_lmetric":0,"t_first_token":339164.77390382,"t_first_token_unix":1779770029.7878475,"t_done":339168.883530324,"t_done_unix":1779770033.8974738},{"request_id":"1363943:8:1379215:635","session_id":"1363943","input_length":20948,"t_proxy_recv":339161.533913909,"t_decision_unix":1779770026.5478551,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":20480,"affinity_cache_ratio":0.97765896505633,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (468 < 8000)","cache_hit":20480,"estimated_new_tokens":468,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":37310,"ongoing_decode_tokens":35888,"pending_prefill_tokens":1422,"num_requests":2,"active_p_offloads":0,"cached_blocks":1330,"cache_hit":8704,"new_prefill":12244,"score_linear":28606.0,"score_lmetric":27332},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":20948,"score_linear":126340.0,"score_lmetric":147288},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":4096,"new_prefill":16852,"score_linear":85472.0,"score_lmetric":96347},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":313823,"ongoing_decode_tokens":215926,"pending_prefill_tokens":4875,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":20948,"score_linear":313823.0,"score_lmetric":129115},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":4096,"new_prefill":16852,"score_linear":187341.0,"score_lmetric":245912},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":4096,"new_prefill":16852,"score_linear":11187.0,"score_lmetric":32135},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34622,"ongoing_decode_tokens":0,"pending_prefill_tokens":14816,"num_requests":2,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":20480,"new_prefill":468,"score_linear":14142.0,"score_lmetric":30568},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":20948,"score_linear":143943.0,"score_lmetric":135222}],"chosen_score_linear":14142.0,"chosen_score_lmetric":30568,"t_first_token":339162.757468697,"t_first_token_unix":1779770027.7714117,"t_done":339169.680224017,"t_done_unix":1779770034.6941674},{"request_id":"1362265:4:1381673:650","session_id":"1362265","input_length":10645,"t_proxy_recv":339170.106516098,"t_decision_unix":1779770035.1204574,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":10240,"affinity_cache_ratio":0.9619539689995303,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (405 < 8000)","cache_hit":10240,"estimated_new_tokens":405,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49877,"ongoing_decode_tokens":35888,"pending_prefill_tokens":13989,"num_requests":2,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":10645,"score_linear":49877.0,"score_lmetric":49268},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":10645,"score_linear":126340.0,"score_lmetric":136985},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98840,"ongoing_decode_tokens":110138,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1910,"cache_hit":0,"new_prefill":10645,"score_linear":98840.0,"score_lmetric":21290},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":10645,"score_linear":351902.0,"score_lmetric":63435},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":10645,"score_linear":191437.0,"score_lmetric":42580},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":10240,"new_prefill":405,"score_linear":5043.0,"score_lmetric":405},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1395,"cache_hit":0,"new_prefill":10645,"score_linear":19806.0,"score_lmetric":10645},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":6656,"new_prefill":3989,"score_linear":54202.0,"score_lmetric":3989}],"chosen_score_linear":5043.0,"chosen_score_lmetric":405,"t_first_token":339170.199587059,"t_first_token_unix":1779770035.2135315,"t_done":339170.488446751,"t_done_unix":1779770035.5023904},{"request_id":"1381728:1:1381728:651","session_id":"1381728","input_length":1391,"t_proxy_recv":339170.33405096,"t_decision_unix":1779770035.3479922,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":1391,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1391 < 8000)","cache_hit":0,"estimated_new_tokens":1391,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49877,"ongoing_decode_tokens":49877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":1391,"score_linear":49877.0,"score_lmetric":2782},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":1391,"score_linear":126340.0,"score_lmetric":127731},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98840,"ongoing_decode_tokens":110138,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1910,"cache_hit":0,"new_prefill":1391,"score_linear":98840.0,"score_lmetric":2782},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":1391,"score_linear":351902.0,"score_lmetric":17165},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":1391,"score_linear":191437.0,"score_lmetric":5564},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":25928,"ongoing_decode_tokens":25928,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":1391,"score_linear":25928.0,"score_lmetric":2782},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1395,"cache_hit":0,"new_prefill":1391,"score_linear":19806.0,"score_lmetric":1391},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":0,"new_prefill":1391,"score_linear":60858.0,"score_lmetric":1391}],"chosen_score_linear":19806.0,"chosen_score_lmetric":1391,"t_first_token":339170.436408605,"t_first_token_unix":1779770035.450352,"t_done":339170.582623199,"t_done_unix":1779770035.5965664},{"request_id":"1379569:3:1381510:649","session_id":"1379569","input_length":7620,"t_proxy_recv":339169.422574228,"t_decision_unix":1779770034.4365156,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":6656,"affinity_cache_ratio":0.873490813648294,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (964 < 8000)","cache_hit":6656,"estimated_new_tokens":964,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49877,"ongoing_decode_tokens":35888,"pending_prefill_tokens":13989,"num_requests":2,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":7620,"score_linear":49877.0,"score_lmetric":43218},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":7620,"score_linear":126340.0,"score_lmetric":133960},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":21371,"pending_prefill_tokens":1275,"num_requests":1,"active_p_offloads":0,"cached_blocks":1910,"cache_hit":6656,"new_prefill":964,"score_linear":84564.0,"score_lmetric":2239},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":7620,"score_linear":371708.0,"score_lmetric":85836},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":7620,"score_linear":191437.0,"score_lmetric":30480},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":2560,"new_prefill":5060,"score_linear":12723.0,"score_lmetric":5060},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1394,"cache_hit":0,"new_prefill":7620,"score_linear":40754.0,"score_lmetric":15240},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":0,"new_prefill":7620,"score_linear":60858.0,"score_lmetric":7620}],"chosen_score_linear":84564.0,"chosen_score_lmetric":2239,"t_first_token":339169.717784419,"t_first_token_unix":1779770034.7317276,"t_done":339171.002479429,"t_done_unix":1779770036.016423},{"request_id":"1286804:4:1378156:627","session_id":"1286804","input_length":22132,"t_proxy_recv":339157.800655446,"t_decision_unix":1779770022.8145971,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":15360,"affinity_cache_ratio":0.694017711910356,"affinity_num_requests":3,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6772 < 8000)","cache_hit":15360,"estimated_new_tokens":6772,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50739,"ongoing_decode_tokens":50739,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1301,"cache_hit":0,"new_prefill":22132,"score_linear":50739.0,"score_lmetric":44264},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":22132,"score_linear":126340.0,"score_lmetric":148472},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":22132,"score_linear":89568.0,"score_lmetric":101627},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":22132,"score_linear":378027.0,"score_lmetric":189049},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169305,"ongoing_decode_tokens":63536,"pending_prefill_tokens":37854,"num_requests":3,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":15360,"new_prefill":6772,"score_linear":153945.0,"score_lmetric":133878},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":59007,"ongoing_decode_tokens":59007,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1785,"cache_hit":0,"new_prefill":22132,"score_linear":59007.0,"score_lmetric":22132},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1346,"cache_hit":0,"new_prefill":22132,"score_linear":19806.0,"score_lmetric":22132},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":22132,"score_linear":143943.0,"score_lmetric":137590}],"chosen_score_linear":153945.0,"chosen_score_lmetric":133878,"t_first_token":339164.83765571,"t_first_token_unix":1779770029.8515997,"t_done":339172.090739807,"t_done_unix":1779770037.1046832},{"request_id":"1381387:1:1381387:648","session_id":"1381387","input_length":13989,"t_proxy_recv":339169.013123408,"t_decision_unix":1779770034.0270648,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":13989,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":13989,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":13989,"score_linear":35888.0,"score_lmetric":13989},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":13989,"score_linear":126340.0,"score_lmetric":140329},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":21371,"pending_prefill_tokens":1275,"num_requests":1,"active_p_offloads":0,"cached_blocks":1910,"cache_hit":0,"new_prefill":13989,"score_linear":91220.0,"score_lmetric":15264},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":13989,"score_linear":371708.0,"score_lmetric":124050},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":13989,"score_linear":191437.0,"score_lmetric":55956},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":13989,"score_linear":15283.0,"score_lmetric":13989},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1394,"cache_hit":0,"new_prefill":13989,"score_linear":40754.0,"score_lmetric":27978},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":0,"new_prefill":13989,"score_linear":60858.0,"score_lmetric":13989}],"chosen_score_linear":35888.0,"chosen_score_lmetric":13989,"t_first_token":339170.236083039,"t_first_token_unix":1779770035.2500265,"t_done":339172.568417019,"t_done_unix":1779770037.5823603},{"request_id":"1382089:1:1382089:655","session_id":"1382089","input_length":38111,"t_proxy_recv":339171.602853459,"t_decision_unix":1779770036.6167946,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":3295,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3295 < 8000)","cache_hit":34816,"estimated_new_tokens":3295,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49877,"ongoing_decode_tokens":49877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":38111,"score_linear":49877.0,"score_lmetric":76222},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":38111,"score_linear":126340.0,"score_lmetric":164451},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":102518,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":38111,"score_linear":91220.0,"score_lmetric":38111},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":38111,"score_linear":351902.0,"score_lmetric":200765},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":38111,"score_linear":191437.0,"score_lmetric":152444},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":38111,"score_linear":122058.0,"score_lmetric":173592},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":38111,"score_linear":19806.0,"score_lmetric":38111},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":34816,"new_prefill":3295,"score_linear":26042.0,"score_lmetric":3295}],"chosen_score_linear":26042.0,"chosen_score_lmetric":3295,"t_first_token":339172.43528114,"t_first_token_unix":1779770037.4492245,"t_done":339172.684725078,"t_done_unix":1779770037.6986685},{"request_id":"1382294:1:1382294:656","session_id":"1382294","input_length":3034,"t_proxy_recv":339172.333117572,"t_decision_unix":1779770037.3470588,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":3034,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3034 < 8000)","cache_hit":0,"estimated_new_tokens":3034,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49877,"ongoing_decode_tokens":49877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":3034,"score_linear":49877.0,"score_lmetric":6068},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":3034,"score_linear":126340.0,"score_lmetric":129374},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":102518,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":3034,"score_linear":91220.0,"score_lmetric":3034},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":3034,"score_linear":351902.0,"score_lmetric":25380},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169305,"ongoing_decode_tokens":119822,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":3034,"score_linear":169305.0,"score_lmetric":9102},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":3034,"score_linear":122058.0,"score_lmetric":68361},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":3034,"score_linear":73818.0,"score_lmetric":114092},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98969,"ongoing_decode_tokens":60858,"pending_prefill_tokens":3295,"num_requests":2,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":0,"new_prefill":3034,"score_linear":98969.0,"score_lmetric":12658}],"chosen_score_linear":91220.0,"chosen_score_lmetric":3034,"t_first_token":339172.64726715,"t_first_token_unix":1779770037.6612105,"t_done":339173.061416359,"t_done_unix":1779770038.0753603},{"request_id":"1340278:2:1367225:564","session_id":"1340278","input_length":49483,"t_proxy_recv":339119.749968339,"t_decision_unix":1779769984.7639096,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":4,"affinity_cache_hit":48640,"affinity_cache_ratio":0.9829638461693915,"affinity_num_requests":5,"avg_num_requests":2.25,"fallback_score":49483,"tie_break_used":false,"v2_pd_sep":true,"v2_decision":"pd_sep","v2_reason":"local_cost 11.75s > migrate_cost 2.16s + margin 0.20s","v2_chosen_cache_hit":0,"v2_src_idx":3,"v2_src_cache_hit":48640,"v2_new_local":49483,"v2_new_src":843,"v2_cost_local_s":11.752212499999999,"v2_cost_src_interf_s":0.21075,"v2_cost_xfer_s":1.9493055555555554,"v2_cost_migrate_s":2.160055555555555,"cache_hit":0,"estimated_new_tokens":49483,"route_class":"PD_SEP_V2","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":49483,"score_linear":156948.0,"score_lmetric":345278},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":49483,"score_linear":130877.0,"score_lmetric":98966},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":87042,"ongoing_decode_tokens":21371,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1486,"cache_hit":0,"new_prefill":49483,"score_linear":87042.0,"score_lmetric":126452},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":286662,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":5,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":48640,"new_prefill":843,"score_linear":238022.0,"score_lmetric":615765},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":62685,"ongoing_decode_tokens":62685,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1261,"cache_hit":0,"new_prefill":49483,"score_linear":62685.0,"score_lmetric":49483},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":49483,"score_linear":34226.0,"score_lmetric":145914},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93230,"ongoing_decode_tokens":93230,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1194,"cache_hit":0,"new_prefill":49483,"score_linear":93230.0,"score_lmetric":98966},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":109237,"ongoing_decode_tokens":59514,"pending_prefill_tokens":7737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1388,"cache_hit":0,"new_prefill":49483,"score_linear":109237.0,"score_lmetric":171660}],"chosen_score_linear":62685.0,"chosen_score_lmetric":49483,"v2_src_url":"http://127.0.0.1:8003","t_prefill_sent":339119.752739074,"t_prefill_sent_unix":1779769984.7666826,"t_prefill_done":339161.055389393,"t_prefill_done_unix":1779770026.0693333,"t_decode_sent":339161.056423834,"t_decode_sent_unix":1779770026.0703673,"t_first_token":339164.876933938,"t_first_token_unix":1779770029.890878,"t_done":339173.30176901,"t_done_unix":1779770038.3157127},{"request_id":"1301929:2:1375154:617","session_id":"1301929","input_length":35888,"t_proxy_recv":339147.477323547,"t_decision_unix":1779770012.491265,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":35840,"affinity_cache_ratio":0.9986625055728935,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (48 < 8000)","cache_hit":35840,"estimated_new_tokens":48,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1269,"cache_hit":35840,"new_prefill":48,"score_linear":-22224.0,"score_lmetric":48},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":35888,"score_linear":126340.0,"score_lmetric":162228},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":35888,"score_linear":89568.0,"score_lmetric":115383},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":377653,"ongoing_decode_tokens":250361,"pending_prefill_tokens":7153,"num_requests":7,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":35888,"score_linear":377653.0,"score_lmetric":301287},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":209467,"ongoing_decode_tokens":82832,"pending_prefill_tokens":560,"num_requests":5,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":35888,"score_linear":209467.0,"score_lmetric":182240},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":85216,"ongoing_decode_tokens":26705,"pending_prefill_tokens":11919,"num_requests":2,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":35888,"score_linear":85216.0,"score_lmetric":95614},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":35888,"score_linear":80246.0,"score_lmetric":35888},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":35888,"score_linear":105902.0,"score_lmetric":81374}],"chosen_score_linear":-22224.0,"chosen_score_lmetric":48,"t_first_token":339152.75091849,"t_first_token_unix":1779770017.7648623,"t_done":339173.583732963,"t_done_unix":1779770038.597677},{"request_id":"1340278:3:1368072:576","session_id":"1340278","input_length":50124,"t_proxy_recv":339173.328087278,"t_decision_unix":1779770038.3420289,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9908227595563004,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (460 < 8000)","cache_hit":49664,"estimated_new_tokens":460,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1377,"cache_hit":0,"new_prefill":50124,"score_linear":35888.0,"score_lmetric":50124},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":50124,"score_linear":126340.0,"score_lmetric":176464},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":102518,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":0,"new_prefill":50124,"score_linear":91220.0,"score_lmetric":50124},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":49152,"new_prefill":972,"score_linear":302750.0,"score_lmetric":15070},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119822,"ongoing_decode_tokens":119822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":49664,"new_prefill":460,"score_linear":70158.0,"score_lmetric":920},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":50124,"score_linear":122058.0,"score_lmetric":209631},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":50124,"score_linear":73818.0,"score_lmetric":208272},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":50124,"score_linear":60858.0,"score_lmetric":50124}],"chosen_score_linear":70158.0,"chosen_score_lmetric":920,"t_first_token":339173.565117508,"t_first_token_unix":1779770038.579061,"t_done":339173.602680046,"t_done_unix":1779770038.6166236},{"request_id":"1382667:1:1382667:658","session_id":"1382667","input_length":1955,"t_proxy_recv":339173.74735923,"t_decision_unix":1779770038.7613006,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1955 < 8000)","cache_hit":0,"estimated_new_tokens":1955,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1377,"cache_hit":0,"new_prefill":1955,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":1955,"score_linear":126340.0,"score_lmetric":128295},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":102518,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":0,"new_prefill":1955,"score_linear":91220.0,"score_lmetric":1955},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":1955,"score_linear":351902.0,"score_lmetric":19985},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119822,"ongoing_decode_tokens":119822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":1955,"score_linear":119822.0,"score_lmetric":3910},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":1955,"score_linear":122058.0,"score_lmetric":65124},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":1955,"score_linear":73818.0,"score_lmetric":111934},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":1955,"score_linear":60858.0,"score_lmetric":1955}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339173.87635735,"t_first_token_unix":1779770038.8903008,"t_done":339174.109284346,"t_done_unix":1779770039.1232278},{"request_id":"1382710:1:1382710:659","session_id":"1382710","input_length":3190,"t_proxy_recv":339173.845652502,"t_decision_unix":1779770038.8595936,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":3190,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3190 < 8000)","cache_hit":0,"estimated_new_tokens":3190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":1955,"ongoing_decode_tokens":0,"pending_prefill_tokens":1955,"num_requests":1,"active_p_offloads":0,"cached_blocks":1377,"cache_hit":0,"new_prefill":3190,"score_linear":1955.0,"score_lmetric":5145},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":3190,"score_linear":126340.0,"score_lmetric":129530},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":102518,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":0,"new_prefill":3190,"score_linear":91220.0,"score_lmetric":3190},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":3190,"score_linear":351902.0,"score_lmetric":26160},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119822,"ongoing_decode_tokens":119822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":3190,"score_linear":119822.0,"score_lmetric":6380},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":3190,"score_linear":122058.0,"score_lmetric":68829},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":3190,"score_linear":73818.0,"score_lmetric":114404},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":3190,"score_linear":60858.0,"score_lmetric":3190}],"chosen_score_linear":91220.0,"chosen_score_lmetric":3190,"t_first_token":339174.163193052,"t_first_token_unix":1779770039.1771367,"t_done":339174.500753825,"t_done_unix":1779770039.5146973},{"request_id":"1373577:2:1380740:647","session_id":"1373577","input_length":81147,"t_proxy_recv":339168.681170743,"t_decision_unix":1779770033.6951125,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":79872,"affinity_cache_ratio":0.9842877740397057,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1275 < 8000)","cache_hit":79872,"estimated_new_tokens":1275,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":81147,"score_linear":35888.0,"score_lmetric":81147},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":81147,"score_linear":126340.0,"score_lmetric":207487},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1910,"cache_hit":79872,"new_prefill":1275,"score_linear":-69799.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":81147,"score_linear":371708.0,"score_lmetric":526998},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":2560,"new_prefill":78587,"score_linear":188877.0,"score_lmetric":314348},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":81147,"score_linear":15283.0,"score_lmetric":81147},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1394,"cache_hit":0,"new_prefill":81147,"score_linear":40754.0,"score_lmetric":162294},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":105902,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":81147,"score_linear":105902.0,"score_lmetric":81147}],"chosen_score_linear":-69799.0,"chosen_score_lmetric":0,"t_first_token":339169.541729427,"t_first_token_unix":1779770034.5556726,"t_done":339174.565473214,"t_done_unix":1779770039.5794165},{"request_id":"1382804:1:1382804:660","session_id":"1382804","input_length":627,"t_proxy_recv":339174.075548985,"t_decision_unix":1779770039.0894902,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":627,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (627 < 8000)","cache_hit":0,"estimated_new_tokens":627,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":1955,"ongoing_decode_tokens":1955,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1377,"cache_hit":0,"new_prefill":627,"score_linear":1955.0,"score_lmetric":627},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":627,"score_linear":126340.0,"score_lmetric":126967},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":94410,"ongoing_decode_tokens":102518,"pending_prefill_tokens":3190,"num_requests":2,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":0,"new_prefill":627,"score_linear":94410.0,"score_lmetric":7634},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":627,"score_linear":351902.0,"score_lmetric":13345},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119822,"ongoing_decode_tokens":119822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":627,"score_linear":119822.0,"score_lmetric":1254},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":627,"score_linear":122058.0,"score_lmetric":61140},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":627,"score_linear":73818.0,"score_lmetric":109278},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1558,"cache_hit":0,"new_prefill":627,"score_linear":60858.0,"score_lmetric":627}],"chosen_score_linear":60858.0,"chosen_score_lmetric":627,"t_first_token":339174.246149691,"t_first_token_unix":1779770039.2600935,"t_done":339174.566891973,"t_done_unix":1779770039.580835},{"request_id":"1383009:1:1383009:663","session_id":"1383009","input_length":5135,"t_proxy_recv":339174.721867255,"t_decision_unix":1779770039.7358084,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5135 < 8000)","cache_hit":0,"estimated_new_tokens":5135,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1380,"cache_hit":0,"new_prefill":5135,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":5135,"score_linear":126340.0,"score_lmetric":131475},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":0,"new_prefill":5135,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":5135,"score_linear":351902.0,"score_lmetric":35885},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119822,"ongoing_decode_tokens":119822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":5135,"score_linear":119822.0,"score_lmetric":10270},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":5135,"score_linear":122058.0,"score_lmetric":74664},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":5135,"score_linear":73818.0,"score_lmetric":118294},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":5135,"score_linear":60858.0,"score_lmetric":5135}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339175.055715962,"t_first_token_unix":1779770040.0696597,"t_done":339175.326078101,"t_done_unix":1779770040.3400211},{"request_id":"1370129:2:1373467:607","session_id":"1370129","input_length":56286,"t_proxy_recv":339153.028166894,"t_decision_unix":1779770018.0421083,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":18432,"affinity_cache_ratio":0.3274704189318836,"affinity_num_requests":2,"avg_num_requests":2.625,"fallback_score":75708,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=7680 chosen=18432)","cache_hit":18432,"estimated_new_tokens":37854,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":71092,"ongoing_decode_tokens":35888,"pending_prefill_tokens":3156,"num_requests":3,"active_p_offloads":0,"cached_blocks":1269,"cache_hit":7680,"new_prefill":48606,"score_linear":63412.0,"score_lmetric":155286},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":56286,"score_linear":126340.0,"score_lmetric":182626},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":7680,"new_prefill":48606,"score_linear":81888.0,"score_lmetric":128101},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":56286,"score_linear":378027.0,"score_lmetric":428127},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113019,"ongoing_decode_tokens":63536,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":18432,"new_prefill":37854,"score_linear":94587.0,"score_lmetric":75708},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":109539,"ongoing_decode_tokens":85216,"pending_prefill_tokens":15107,"num_requests":3,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":7680,"new_prefill":48606,"score_linear":101859.0,"score_lmetric":191139},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":100052,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1307,"cache_hit":7680,"new_prefill":48606,"score_linear":92372.0,"score_lmetric":97212},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":56286,"score_linear":143943.0,"score_lmetric":205898}],"chosen_score_linear":94587.0,"chosen_score_lmetric":75708,"t_first_token":339163.026981891,"t_first_token_unix":1779770028.0409255,"t_done":339175.804817339,"t_done_unix":1779770040.818761},{"request_id":"1382294:2:1383073:664","session_id":"1382294","input_length":5164,"t_proxy_recv":339174.944707744,"t_decision_unix":1779770039.9586487,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":2560,"affinity_cache_ratio":0.4957397366382649,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2604 < 8000)","cache_hit":2560,"estimated_new_tokens":2604,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5135,"ongoing_decode_tokens":0,"pending_prefill_tokens":5135,"num_requests":1,"active_p_offloads":0,"cached_blocks":1380,"cache_hit":0,"new_prefill":5164,"score_linear":5135.0,"score_lmetric":10299},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":5164,"score_linear":126340.0,"score_lmetric":131504},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":2560,"new_prefill":2604,"score_linear":7513.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":5164,"score_linear":351902.0,"score_lmetric":36030},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119822,"ongoing_decode_tokens":119822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":5164,"score_linear":119822.0,"score_lmetric":10328},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":5164,"score_linear":122058.0,"score_lmetric":74751},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":5164,"score_linear":73818.0,"score_lmetric":118352},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":5164,"score_linear":60858.0,"score_lmetric":5164}],"chosen_score_linear":7513.0,"chosen_score_lmetric":0,"t_first_token":339175.133363612,"t_first_token_unix":1779770040.1473067,"t_done":339175.852112893,"t_done_unix":1779770040.8660562},{"request_id":"1379569:4:1383116:666","session_id":"1379569","input_length":10217,"t_proxy_recv":339175.078989356,"t_decision_unix":1779770040.092931,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":7680,"affinity_cache_ratio":0.7516883625330332,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2537 < 8000)","cache_hit":7680,"estimated_new_tokens":2537,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5135,"ongoing_decode_tokens":5135,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1380,"cache_hit":0,"new_prefill":10217,"score_linear":5135.0,"score_lmetric":10217},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":10217,"score_linear":126340.0,"score_lmetric":136557},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":15237,"ongoing_decode_tokens":21371,"pending_prefill_tokens":2604,"num_requests":1,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":7680,"new_prefill":2537,"score_linear":7557.0,"score_lmetric":5141},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":10217,"score_linear":351902.0,"score_lmetric":61295},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119822,"ongoing_decode_tokens":119822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":10217,"score_linear":119822.0,"score_lmetric":20434},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":2560,"new_prefill":7657,"score_linear":119498.0,"score_lmetric":82230},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":10217,"score_linear":73818.0,"score_lmetric":128458},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":70163,"ongoing_decode_tokens":60858,"pending_prefill_tokens":9305,"num_requests":2,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":10217,"score_linear":70163.0,"score_lmetric":39044}],"chosen_score_linear":7557.0,"chosen_score_lmetric":5141,"t_first_token":339175.411511846,"t_first_token_unix":1779770040.4254556,"t_done":339175.942340873,"t_done_unix":1779770040.9562845},{"request_id":"1383090:1:1383090:665","session_id":"1383090","input_length":9305,"t_proxy_recv":339174.995190718,"t_decision_unix":1779770040.0091324,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":9305,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":9305,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5135,"ongoing_decode_tokens":0,"pending_prefill_tokens":5135,"num_requests":1,"active_p_offloads":0,"cached_blocks":1380,"cache_hit":0,"new_prefill":9305,"score_linear":5135.0,"score_lmetric":14440},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":9305,"score_linear":126340.0,"score_lmetric":135645},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":15237,"ongoing_decode_tokens":21371,"pending_prefill_tokens":2604,"num_requests":1,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":0,"new_prefill":9305,"score_linear":15237.0,"score_lmetric":11909},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":9305,"score_linear":351902.0,"score_lmetric":56735},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119822,"ongoing_decode_tokens":119822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":9305,"score_linear":119822.0,"score_lmetric":18610},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":9305,"score_linear":122058.0,"score_lmetric":87174},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":9305,"score_linear":73818.0,"score_lmetric":126634},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":9305,"score_linear":60858.0,"score_lmetric":9305}],"chosen_score_linear":60858.0,"chosen_score_lmetric":9305,"t_first_token":339175.749636894,"t_first_token_unix":1779770040.763581,"t_done":339176.18378503,"t_done_unix":1779770041.197729},{"request_id":"1313181:5:1374956:615","session_id":"1313181","input_length":60858,"t_proxy_recv":339168.892307161,"t_decision_unix":1779770033.9062488,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":60416,"affinity_cache_ratio":0.9927371914949554,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (442 < 8000)","cache_hit":60416,"estimated_new_tokens":442,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":60858,"score_linear":35888.0,"score_lmetric":60858},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":60858,"score_linear":126340.0,"score_lmetric":187198},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":21371,"pending_prefill_tokens":1275,"num_requests":1,"active_p_offloads":0,"cached_blocks":1910,"cache_hit":0,"new_prefill":60858,"score_linear":91220.0,"score_lmetric":62133},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":371708,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6686,"num_requests":6,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":12800,"new_prefill":48058,"score_linear":358908.0,"score_lmetric":328464},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":60858,"score_linear":191437.0,"score_lmetric":243432},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":60858,"score_linear":15283.0,"score_lmetric":60858},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":40754,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1394,"cache_hit":0,"new_prefill":60858,"score_linear":40754.0,"score_lmetric":121716},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":60416,"new_prefill":442,"score_linear":-60416.0,"score_lmetric":0}],"chosen_score_linear":-60416.0,"chosen_score_lmetric":0,"t_first_token":339168.989396223,"t_first_token_unix":1779770034.00334,"t_done":339176.211544082,"t_done_unix":1779770041.2254877},{"request_id":"1366128:9:1381799:652","session_id":"1366128","input_length":59374,"t_proxy_recv":339170.587345978,"t_decision_unix":1779770035.6012874,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9916798598713241,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (494 < 8000)","cache_hit":58880,"estimated_new_tokens":494,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49877,"ongoing_decode_tokens":49877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":59374,"score_linear":49877.0,"score_lmetric":118748},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":59374,"score_linear":126340.0,"score_lmetric":185714},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98840,"ongoing_decode_tokens":110138,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1910,"cache_hit":0,"new_prefill":59374,"score_linear":98840.0,"score_lmetric":118748},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":59374,"score_linear":351902.0,"score_lmetric":307080},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":59374,"score_linear":191437.0,"score_lmetric":237496},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":58880,"new_prefill":494,"score_linear":-43597.0,"score_lmetric":494},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":59374,"score_linear":19806.0,"score_lmetric":59374},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":0,"new_prefill":59374,"score_linear":60858.0,"score_lmetric":59374}],"chosen_score_linear":-43597.0,"chosen_score_lmetric":494,"t_first_token":339170.730232852,"t_first_token_unix":1779770035.7441764,"t_done":339176.287945536,"t_done_unix":1779770041.3018887},{"request_id":"1366128:10:1382843:661","session_id":"1366128","input_length":59506,"t_proxy_recv":339176.298363284,"t_decision_unix":1779770041.312305,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":59392,"affinity_cache_ratio":0.9980842268006588,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (114 < 8000)","cache_hit":59392,"estimated_new_tokens":114,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":0,"new_prefill":59506,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":59506,"score_linear":126340.0,"score_lmetric":185846},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":0,"new_prefill":59506,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":59506,"score_linear":351902.0,"score_lmetric":306585},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":63536,"pending_prefill_tokens":4625,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":59506,"score_linear":124993.0,"score_lmetric":128262},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62684,"ongoing_decode_tokens":62684,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":59392,"new_prefill":114,"score_linear":3292.0,"score_lmetric":228},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":59506,"score_linear":73818.0,"score_lmetric":227036},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":59506,"score_linear":106305.0,"score_lmetric":104371}],"chosen_score_linear":3292.0,"chosen_score_lmetric":228,"t_first_token":339176.476122649,"t_first_token_unix":1779770041.4900658,"t_done":339177.284849757,"t_done_unix":1779770042.2987933},{"request_id":"1383625:1:1383625:668","session_id":"1383625","input_length":3186,"t_proxy_recv":339176.917423338,"t_decision_unix":1779770041.9313643,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3186 < 8000)","cache_hit":0,"estimated_new_tokens":3186,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":0,"new_prefill":3186,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":3186,"score_linear":126340.0,"score_lmetric":129526},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":0,"new_prefill":3186,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":3186,"score_linear":351902.0,"score_lmetric":24985},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":63536,"pending_prefill_tokens":4625,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":3186,"score_linear":124993.0,"score_lmetric":15622},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122190,"ongoing_decode_tokens":122190,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":3186,"score_linear":122190.0,"score_lmetric":9558},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":3186,"score_linear":73818.0,"score_lmetric":114396},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":3186,"score_linear":106305.0,"score_lmetric":48051}],"chosen_score_linear":10073.0,"chosen_score_lmetric":0,"t_first_token":339177.139440868,"t_first_token_unix":1779770042.153385,"t_done":339177.33969814,"t_done_unix":1779770042.3536415},{"request_id":"1383625:2:1384279:673","session_id":"1383625","input_length":4150,"t_proxy_recv":339179.335008153,"t_decision_unix":1779770044.3489492,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":3072,"affinity_cache_ratio":0.7402409638554217,"affinity_num_requests":0,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1078 < 8000)","cache_hit":3072,"estimated_new_tokens":1078,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28383,"ongoing_decode_tokens":0,"pending_prefill_tokens":14559,"num_requests":1,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":0,"new_prefill":4150,"score_linear":28383.0,"score_lmetric":18709},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":4150,"score_linear":126340.0,"score_lmetric":130490},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1942,"cache_hit":3072,"new_prefill":1078,"score_linear":7001.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":4150,"score_linear":351902.0,"score_lmetric":29805},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":4150,"score_linear":124993.0,"score_lmetric":8300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":4150,"score_linear":224624.0,"score_lmetric":98290},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":4150,"score_linear":73818.0,"score_lmetric":116324},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":4150,"score_linear":106305.0,"score_lmetric":49015}],"chosen_score_linear":7001.0,"chosen_score_lmetric":0,"t_first_token":339179.472829389,"t_first_token_unix":1779770044.4867728,"t_done":339180.069816177,"t_done_unix":1779770045.0837595},{"request_id":"1382667:2:1384299:674","session_id":"1382667","input_length":5989,"t_proxy_recv":339179.399682119,"t_decision_unix":1779770044.4136236,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":2,"affinity_cache_hit":1536,"affinity_cache_ratio":0.2564701953581566,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":7067,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5989 < 8000)","cache_hit":0,"estimated_new_tokens":5989,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28383,"ongoing_decode_tokens":0,"pending_prefill_tokens":14559,"num_requests":1,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":1536,"new_prefill":4453,"score_linear":26847.0,"score_lmetric":19012},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":5989,"score_linear":126340.0,"score_lmetric":132329},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14223,"ongoing_decode_tokens":21371,"pending_prefill_tokens":1078,"num_requests":1,"active_p_offloads":0,"cached_blocks":1942,"cache_hit":0,"new_prefill":5989,"score_linear":14223.0,"score_lmetric":7067},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":5989,"score_linear":351902.0,"score_lmetric":39000},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":5989,"score_linear":124993.0,"score_lmetric":11978},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":5989,"score_linear":224624.0,"score_lmetric":107485},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":5989,"score_linear":73818.0,"score_lmetric":120002},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":5989,"score_linear":106305.0,"score_lmetric":50854}],"chosen_score_linear":14223.0,"chosen_score_lmetric":7067,"t_first_token":339179.858793409,"t_first_token_unix":1779770044.8727367,"t_done":339180.350062112,"t_done_unix":1779770045.3640053},{"request_id":"1336620:3:1339999:443","session_id":"1336620","input_length":21371,"t_proxy_recv":339029.977187322,"t_decision_unix":1779769894.991129,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":14848,"affinity_cache_ratio":0.6947732909082401,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6523 < 8000)","cache_hit":14848,"estimated_new_tokens":6523,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":853,"cache_hit":7680,"new_prefill":13691,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":174764,"ongoing_decode_tokens":174764,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":21371,"score_linear":174764.0,"score_lmetric":64113},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38917,"ongoing_decode_tokens":50215,"pending_prefill_tokens":-1133,"num_requests":0,"active_p_offloads":0,"cached_blocks":1417,"cache_hit":14848,"new_prefill":6523,"score_linear":24069.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":48381,"ongoing_decode_tokens":0,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1071,"cache_hit":0,"new_prefill":21371,"score_linear":48381.0,"score_lmetric":69752},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":21371,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":7680,"new_prefill":13691,"score_linear":-7680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1034,"cache_hit":7680,"new_prefill":13691,"score_linear":-615.0,"score_lmetric":13691},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1111,"cache_hit":0,"new_prefill":21371,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":24069.0,"chosen_score_lmetric":0,"t_first_token":339030.961951627,"t_first_token_unix":1779769895.975895,"t_done":339180.451363316,"t_done_unix":1779770045.4653065},{"request_id":"1384772:1:1384772:677","session_id":"1384772","input_length":4690,"t_proxy_recv":339180.862462613,"t_decision_unix":1779770045.876404,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":-4690,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4690 < 8000)","cache_hit":0,"estimated_new_tokens":4690,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28383,"ongoing_decode_tokens":28383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":0,"new_prefill":4690,"score_linear":28383.0,"score_lmetric":4690},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":4690,"score_linear":126340.0,"score_lmetric":131030},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1973,"cache_hit":0,"new_prefill":4690,"score_linear":-11298.0,"score_lmetric":-4690},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":4690,"score_linear":351902.0,"score_lmetric":32505},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":174858,"ongoing_decode_tokens":174858,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":4690,"score_linear":174858.0,"score_lmetric":14070},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":4690,"score_linear":224624.0,"score_lmetric":100990},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":4690,"score_linear":73818.0,"score_lmetric":117404},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":4690,"score_linear":106305.0,"score_lmetric":49555}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-4690,"t_first_token":339181.160046384,"t_first_token_unix":1779770046.1739905,"t_done":339181.292278283,"t_done_unix":1779770046.3062212},{"request_id":"1381387:2:1383735:669","session_id":"1381387","input_length":28383,"t_proxy_recv":339177.228951057,"t_decision_unix":1779770042.2428925,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":13824,"affinity_cache_ratio":0.4870521086565902,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":13824,"estimated_new_tokens":14559,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":13824,"new_prefill":14559,"score_linear":-13824.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":28383,"score_linear":126340.0,"score_lmetric":154723},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13259,"ongoing_decode_tokens":24557,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":0,"new_prefill":28383,"score_linear":13259.0,"score_lmetric":28383},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":28383,"score_linear":351902.0,"score_lmetric":150970},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":63536,"pending_prefill_tokens":4625,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":28383,"score_linear":124993.0,"score_lmetric":66016},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122190,"ongoing_decode_tokens":122190,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":28383,"score_linear":122190.0,"score_lmetric":85149},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":28383,"score_linear":73818.0,"score_lmetric":164790},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":28383,"score_linear":106305.0,"score_lmetric":73248}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":339179.614063298,"t_first_token_unix":1779770044.628007,"t_done":339181.44041741,"t_done_unix":1779770046.4543605},{"request_id":"1383009:2:1384930:679","session_id":"1383009","input_length":11113,"t_proxy_recv":339181.371534236,"t_decision_unix":1779770046.3854759,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":2,"affinity_cache_hit":5120,"affinity_cache_ratio":0.46072167731485647,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":-11113,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28383,"ongoing_decode_tokens":28383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":5120,"new_prefill":5993,"score_linear":23263.0,"score_lmetric":5993},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":11113,"score_linear":126340.0,"score_lmetric":137453},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1982,"cache_hit":0,"new_prefill":11113,"score_linear":-11298.0,"score_lmetric":-11113},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":11113,"score_linear":351902.0,"score_lmetric":64620},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":174858,"ongoing_decode_tokens":174858,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":11113,"score_linear":174858.0,"score_lmetric":33339},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":11113,"score_linear":224624.0,"score_lmetric":133105},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":11113,"score_linear":73818.0,"score_lmetric":130250},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":11113,"score_linear":106305.0,"score_lmetric":55978}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-11113,"t_first_token":339182.257358854,"t_first_token_unix":1779770047.2713027,"t_done":339182.419936329,"t_done_unix":1779770047.4338799},{"request_id":"1355484:4:1364784:548","session_id":"1355484","input_length":19806,"t_proxy_recv":339150.79748907,"t_decision_unix":1779770015.8114305,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":6,"affinity_cache_hit":19456,"affinity_cache_ratio":0.9823285872967787,"affinity_num_requests":6,"avg_num_requests":2.25,"fallback_score":19806,"tie_break_used":false,"v2_pd_sep":true,"v2_decision":"pd_sep","v2_reason":"local_cost 3.96s > migrate_cost 1.06s + margin 0.20s","v2_chosen_cache_hit":0,"v2_src_idx":3,"v2_src_cache_hit":19456,"v2_new_local":19806,"v2_new_src":350,"v2_cost_local_s":3.9612000000000003,"v2_cost_src_interf_s":0.10499999999999998,"v2_cost_xfer_s":0.9597222222222221,"v2_cost_migrate_s":1.0647222222222221,"cache_hit":0,"estimated_new_tokens":19806,"route_class":"PD_SEP_V2","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49504,"ongoing_decode_tokens":0,"pending_prefill_tokens":48,"num_requests":2,"active_p_offloads":0,"cached_blocks":1269,"cache_hit":0,"new_prefill":19806,"score_linear":49504.0,"score_lmetric":39708},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":19806,"score_linear":126340.0,"score_lmetric":146146},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":19806,"score_linear":89568.0,"score_lmetric":99301},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":357939,"ongoing_decode_tokens":266948,"pending_prefill_tokens":4644,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":19456,"new_prefill":350,"score_linear":338483.0,"score_lmetric":29964},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131320,"ongoing_decode_tokens":81837,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1442,"cache_hit":0,"new_prefill":19806,"score_linear":131320.0,"score_lmetric":59418},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":85216,"ongoing_decode_tokens":26705,"pending_prefill_tokens":11919,"num_requests":2,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":19806,"score_linear":85216.0,"score_lmetric":63450},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1307,"cache_hit":0,"new_prefill":19806,"score_linear":80246.0,"score_lmetric":19806},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":12800,"new_prefill":7006,"score_linear":131143.0,"score_lmetric":107338}],"chosen_score_linear":80246.0,"chosen_score_lmetric":19806,"v2_src_url":"http://127.0.0.1:8003","t_prefill_sent":339150.798572173,"t_prefill_sent_unix":1779770015.8125148,"t_prefill_done":339169.917819197,"t_prefill_done_unix":1779770034.9317627,"t_decode_sent":339169.918110713,"t_decode_sent_unix":1779770034.9320536,"t_first_token":339170.860349749,"t_first_token_unix":1779770035.874294,"t_done":339182.777817974,"t_done_unix":1779770047.7917614},{"request_id":"1340278:3:1384549:676","session_id":"1340278","input_length":49865,"t_proxy_recv":339180.152970874,"t_decision_unix":1779770045.166912,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9959691166148601,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (201 < 8000)","cache_hit":49664,"estimated_new_tokens":201,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28383,"ongoing_decode_tokens":28383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":0,"new_prefill":49865,"score_linear":28383.0,"score_lmetric":49865},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":49865,"score_linear":126340.0,"score_lmetric":176205},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16062,"ongoing_decode_tokens":27360,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1944,"cache_hit":0,"new_prefill":49865,"score_linear":16062.0,"score_lmetric":49865},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":49152,"new_prefill":713,"score_linear":302750.0,"score_lmetric":12620},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":49664,"new_prefill":201,"score_linear":75329.0,"score_lmetric":402},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":49865,"score_linear":224624.0,"score_lmetric":326865},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":49865,"score_linear":73818.0,"score_lmetric":207754},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":49865,"score_linear":106305.0,"score_lmetric":94730}],"chosen_score_linear":75329.0,"chosen_score_lmetric":402,"t_first_token":339180.244644487,"t_first_token_unix":1779770045.2585883,"t_done":339183.092692791,"t_done_unix":1779770048.1066365},{"request_id":"1355484:5:1366753:560","session_id":"1355484","input_length":20026,"t_proxy_recv":339182.782808459,"t_decision_unix":1779770047.79675,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19968,"affinity_cache_ratio":0.997103765105363,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (58 < 8000)","cache_hit":19968,"estimated_new_tokens":58,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":19034,"ongoing_decode_tokens":0,"pending_prefill_tokens":19034,"num_requests":1,"active_p_offloads":0,"cached_blocks":1418,"cache_hit":0,"new_prefill":20026,"score_linear":19034.0,"score_lmetric":39060},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":20026,"score_linear":126340.0,"score_lmetric":146366},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":20026,"score_linear":-11298.0,"score_lmetric":-20026},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":19456,"new_prefill":570,"score_linear":332446.0,"score_lmetric":11905},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":174858,"ongoing_decode_tokens":174858,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":20026,"score_linear":174858.0,"score_lmetric":60078},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":20026,"score_linear":224624.0,"score_lmetric":177670},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":54012,"ongoing_decode_tokens":54012,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1436,"cache_hit":19968,"new_prefill":58,"score_linear":34044.0,"score_lmetric":58},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":12800,"new_prefill":7226,"score_linear":110239.0,"score_lmetric":119218}],"chosen_score_linear":34044.0,"chosen_score_lmetric":58,"t_first_token":339182.856174164,"t_first_token_unix":1779770047.8701177,"t_done":339183.428467208,"t_done_unix":1779770048.4424105},{"request_id":"1340278:3:1385417:683","session_id":"1340278","input_length":50272,"t_proxy_recv":339183.183506558,"t_decision_unix":1779770048.197448,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9879057924888606,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (608 < 8000)","cache_hit":49664,"estimated_new_tokens":608,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":19034,"ongoing_decode_tokens":0,"pending_prefill_tokens":19034,"num_requests":1,"active_p_offloads":0,"cached_blocks":1418,"cache_hit":0,"new_prefill":50272,"score_linear":19034.0,"score_lmetric":69306},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":50272,"score_linear":126340.0,"score_lmetric":176612},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":50272,"score_linear":-11298.0,"score_lmetric":-50272},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":49152,"new_prefill":1120,"score_linear":302750.0,"score_lmetric":14655},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":49664,"new_prefill":608,"score_linear":75329.0,"score_lmetric":1216},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":50272,"score_linear":224624.0,"score_lmetric":328900},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":74038,"ongoing_decode_tokens":74038,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1436,"cache_hit":0,"new_prefill":50272,"score_linear":74038.0,"score_lmetric":100544},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":50272,"score_linear":123039.0,"score_lmetric":205310}],"chosen_score_linear":75329.0,"chosen_score_lmetric":1216,"t_first_token":339183.438263368,"t_first_token_unix":1779770048.4522064,"t_done":339183.461257365,"t_done_unix":1779770048.4752004},{"request_id":"1384947:1:1384947:680","session_id":"1384947","input_length":19034,"t_proxy_recv":339181.463527714,"t_decision_unix":1779770046.4774692,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":19034,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1418,"cache_hit":0,"new_prefill":19034,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":19034,"score_linear":126340.0,"score_lmetric":145374},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-185,"ongoing_decode_tokens":0,"pending_prefill_tokens":11113,"num_requests":0,"active_p_offloads":0,"cached_blocks":1982,"cache_hit":0,"new_prefill":19034,"score_linear":-185.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":19034,"score_linear":351902.0,"score_lmetric":104225},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":174858,"ongoing_decode_tokens":174858,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":19034,"score_linear":174858.0,"score_lmetric":57102},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":19034,"score_linear":224624.0,"score_lmetric":172710},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":19034,"score_linear":73818.0,"score_lmetric":146092},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":19034,"score_linear":106305.0,"score_lmetric":63899}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339183.395436736,"t_first_token_unix":1779770048.4093802,"t_done":339183.662074012,"t_done_unix":1779770048.6760175},{"request_id":"1382086:1:1382086:654","session_id":"1382086","input_length":54012,"t_proxy_recv":339171.612525275,"t_decision_unix":1779770036.6264665,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":54012,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":54012,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49877,"ongoing_decode_tokens":49877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":54012,"score_linear":49877.0,"score_lmetric":108024},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":54012,"score_linear":126340.0,"score_lmetric":180352},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":102518,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":54012,"score_linear":91220.0,"score_lmetric":54012},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":54012,"score_linear":351902.0,"score_lmetric":280270},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":54012,"score_linear":191437.0,"score_lmetric":216048},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":54012,"score_linear":122058.0,"score_lmetric":221295},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":54012,"score_linear":19806.0,"score_lmetric":54012},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98969,"ongoing_decode_tokens":60858,"pending_prefill_tokens":3295,"num_requests":2,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":0,"new_prefill":54012,"score_linear":98969.0,"score_lmetric":114614}],"chosen_score_linear":19806.0,"chosen_score_lmetric":54012,"t_first_token":339182.323245485,"t_first_token_unix":1779770047.3371892,"t_done":339183.820331125,"t_done_unix":1779770048.8342745},{"request_id":"1355484:6:1369697:583","session_id":"1355484","input_length":20158,"t_proxy_recv":339183.435201347,"t_decision_unix":1779770048.449143,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9905744617521579,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (190 < 8000)","cache_hit":19968,"estimated_new_tokens":190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":19034,"ongoing_decode_tokens":19034,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1418,"cache_hit":0,"new_prefill":20158,"score_linear":19034.0,"score_lmetric":20158},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":20158,"score_linear":126340.0,"score_lmetric":146498},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":20158,"score_linear":-11298.0,"score_lmetric":-20158},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":19456,"new_prefill":702,"score_linear":332446.0,"score_lmetric":12565},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":175265,"ongoing_decode_tokens":124993,"pending_prefill_tokens":608,"num_requests":3,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":20158,"score_linear":175265.0,"score_lmetric":62298},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":20158,"score_linear":224624.0,"score_lmetric":178330},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":54012,"ongoing_decode_tokens":54012,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1436,"cache_hit":19968,"new_prefill":190,"score_linear":34044.0,"score_lmetric":190},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":12800,"new_prefill":7358,"score_linear":110239.0,"score_lmetric":119482}],"chosen_score_linear":34044.0,"chosen_score_lmetric":190,"t_first_token":339183.513874264,"t_first_token_unix":1779770048.5278177,"t_done":339184.174433031,"t_done_unix":1779770049.1883767},{"request_id":"1355484:7:1371284:596","session_id":"1355484","input_length":22992,"t_proxy_recv":339184.181180945,"t_decision_unix":1779770049.1951225,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19968,"affinity_cache_ratio":0.8684759916492694,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3024 < 8000)","cache_hit":19968,"estimated_new_tokens":3024,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":22992,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":22992,"score_linear":126340.0,"score_lmetric":149332},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":22992,"score_linear":-11298.0,"score_lmetric":-22992},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":19456,"new_prefill":3536,"score_linear":332446.0,"score_lmetric":26735},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1660,"cache_hit":0,"new_prefill":22992,"score_linear":124993.0,"score_lmetric":45984},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":22992,"score_linear":224624.0,"score_lmetric":192500},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1541,"cache_hit":19968,"new_prefill":3024,"score_linear":-19968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":12800,"new_prefill":10192,"score_linear":110239.0,"score_lmetric":125150}],"chosen_score_linear":-19968.0,"chosen_score_lmetric":0,"t_first_token":339184.694651693,"t_first_token_unix":1779770049.7085953,"t_done":339186.483531896,"t_done_unix":1779770051.4974751},{"request_id":"1370129:3:1379538:637","session_id":"1370129","input_length":61457,"t_proxy_recv":339175.818735842,"t_decision_unix":1779770040.8326774,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":56832,"affinity_cache_ratio":0.9247441300421433,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4625 < 8000)","cache_hit":56832,"estimated_new_tokens":4625,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":7680,"new_prefill":53777,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":61457,"score_linear":126340.0,"score_lmetric":187797},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":25454,"ongoing_decode_tokens":36752,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":7680,"new_prefill":53777,"score_linear":17774.0,"score_lmetric":107554},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":61457,"score_linear":351902.0,"score_lmetric":316340},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":63536,"ongoing_decode_tokens":63536,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":56832,"new_prefill":4625,"score_linear":6704.0,"score_lmetric":4625},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":7680,"new_prefill":53777,"score_linear":114378.0,"score_lmetric":220590},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":7680,"new_prefill":53777,"score_linear":66138.0,"score_lmetric":215578},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":70163,"ongoing_decode_tokens":70163,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1559,"cache_hit":0,"new_prefill":61457,"score_linear":70163.0,"score_lmetric":122914}],"chosen_score_linear":6704.0,"chosen_score_lmetric":4625,"t_first_token":339177.651874558,"t_first_token_unix":1779770042.6658185,"t_done":339187.039868871,"t_done_unix":1779770052.0538127},{"request_id":"1355484:8:1377993:625","session_id":"1355484","input_length":23457,"t_proxy_recv":339186.48962443,"t_decision_unix":1779770051.503566,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9822227906381891,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (417 < 8000)","cache_hit":23040,"estimated_new_tokens":417,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":23457,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":23457,"score_linear":126340.0,"score_lmetric":149797},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":23457,"score_linear":-11298.0,"score_lmetric":-23457},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":19456,"new_prefill":4001,"score_linear":332446.0,"score_lmetric":29060},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1660,"cache_hit":0,"new_prefill":23457,"score_linear":124993.0,"score_lmetric":46914},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":23457,"score_linear":224624.0,"score_lmetric":194825},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1547,"cache_hit":23040,"new_prefill":417,"score_linear":-23040.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":12800,"new_prefill":10657,"score_linear":110239.0,"score_lmetric":126080}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":339186.590596294,"t_first_token_unix":1779770051.6045396,"t_done":339188.005125197,"t_done_unix":1779770053.0190685},{"request_id":"1355484:9:1379722:640","session_id":"1355484","input_length":23717,"t_proxy_recv":339188.009970903,"t_decision_unix":1779770053.0239124,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9930429649618417,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (165 < 8000)","cache_hit":23552,"estimated_new_tokens":165,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":23717,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":23717,"score_linear":126340.0,"score_lmetric":150057},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":23717,"score_linear":-11298.0,"score_lmetric":-23717},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":253635,"pending_prefill_tokens":475,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":19456,"new_prefill":4261,"score_linear":332446.0,"score_lmetric":23680},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":23717,"score_linear":148720.0,"score_lmetric":93898},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":23717,"score_linear":224624.0,"score_lmetric":196125},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":23552,"new_prefill":165,"score_linear":-23552.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":12800,"new_prefill":10917,"score_linear":110239.0,"score_lmetric":126600}],"chosen_score_linear":-23552.0,"chosen_score_lmetric":0,"t_first_token":339188.067995774,"t_first_token_unix":1779770053.081939,"t_done":339188.873273752,"t_done_unix":1779770053.8872168},{"request_id":"1381728:2:1387040:690","session_id":"1381728","input_length":12399,"t_proxy_recv":339188.657746847,"t_decision_unix":1779770053.671688,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":2,"affinity_cache_hit":1024,"affinity_cache_ratio":0.08258730542785708,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":-12399,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":12399,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":12399,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":12399,"score_linear":126340.0,"score_lmetric":138739},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":12399,"score_linear":-11298.0,"score_lmetric":-12399},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":253635,"pending_prefill_tokens":475,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":12399,"score_linear":351902.0,"score_lmetric":64370},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":12399,"score_linear":148720.0,"score_lmetric":71262},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":12399,"score_linear":224624.0,"score_lmetric":139535},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":23717,"ongoing_decode_tokens":23717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":1024,"new_prefill":11375,"score_linear":22693.0,"score_lmetric":11375},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":12399,"score_linear":123039.0,"score_lmetric":129564}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-12399,"t_first_token":339189.68191287,"t_first_token_unix":1779770054.695857,"t_done":339189.770653036,"t_done_unix":1779770054.7845964},{"request_id":"1384772:2:1387335:693","session_id":"1384772","input_length":11530,"t_proxy_recv":339189.75612363,"t_decision_unix":1779770054.7700646,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.39965307892454466,"affinity_num_requests":0,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6922 < 8000)","cache_hit":4608,"estimated_new_tokens":6922,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":11530,"score_linear":50793.0,"score_lmetric":34163},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":11530,"score_linear":126340.0,"score_lmetric":137870},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1101,"ongoing_decode_tokens":12399,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":4608,"new_prefill":6922,"score_linear":-3507.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":253635,"pending_prefill_tokens":475,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":11530,"score_linear":351902.0,"score_lmetric":60025},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":11530,"score_linear":148720.0,"score_lmetric":69524},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":11530,"score_linear":224624.0,"score_lmetric":135190},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":25226,"ongoing_decode_tokens":25226,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":0,"new_prefill":11530,"score_linear":25226.0,"score_lmetric":11530},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":11530,"score_linear":123039.0,"score_lmetric":127826}],"chosen_score_linear":-3507.0,"chosen_score_lmetric":0,"t_first_token":339190.400283246,"t_first_token_unix":1779770055.4142275,"t_done":339190.458994716,"t_done_unix":1779770055.472938},{"request_id":"1277428:6:1371101:593","session_id":"1277428","input_length":36583,"t_proxy_recv":339152.234786339,"t_decision_unix":1779770017.2487278,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":36352,"affinity_cache_ratio":0.9936855916682612,"affinity_num_requests":6,"avg_num_requests":2.625,"fallback_score":29250,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (231 < 8000)","cache_hit":36352,"estimated_new_tokens":231,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":71092,"ongoing_decode_tokens":0,"pending_prefill_tokens":3204,"num_requests":3,"active_p_offloads":0,"cached_blocks":1269,"cache_hit":0,"new_prefill":36583,"score_linear":71092.0,"score_lmetric":119361},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":4608,"new_prefill":31975,"score_linear":121732.0,"score_lmetric":158315},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":17408,"new_prefill":19175,"score_linear":72160.0,"score_lmetric":98670},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":341444,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4644,"num_requests":6,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":36352,"new_prefill":231,"score_linear":305092.0,"score_lmetric":29250},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131320,"ongoing_decode_tokens":81837,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1442,"cache_hit":0,"new_prefill":36583,"score_linear":131320.0,"score_lmetric":109749},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":109539,"ongoing_decode_tokens":85216,"pending_prefill_tokens":15107,"num_requests":3,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":36583,"score_linear":109539.0,"score_lmetric":155070},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":100052,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1307,"cache_hit":0,"new_prefill":36583,"score_linear":100052.0,"score_lmetric":73166},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":36583,"score_linear":143943.0,"score_lmetric":166492}],"chosen_score_linear":305092.0,"chosen_score_lmetric":29250,"t_first_token":339175.590848693,"t_first_token_unix":1779770040.6047924,"t_done":339190.477898343,"t_done_unix":1779770055.491842},{"request_id":"1355484:10:1384484:675","session_id":"1355484","input_length":25226,"t_proxy_recv":339188.879310802,"t_decision_unix":1779770053.8932524,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9336398953460715,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1674 < 8000)","cache_hit":23552,"estimated_new_tokens":1674,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":25226,"score_linear":50793.0,"score_lmetric":47859},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":25226,"score_linear":126340.0,"score_lmetric":151566},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1101,"ongoing_decode_tokens":0,"pending_prefill_tokens":12399,"num_requests":0,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":25226,"score_linear":1101.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":253635,"pending_prefill_tokens":475,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":19456,"new_prefill":5770,"score_linear":332446.0,"score_lmetric":31225},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":25226,"score_linear":148720.0,"score_lmetric":96916},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":25226,"score_linear":224624.0,"score_lmetric":203670},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":23552,"new_prefill":1674,"score_linear":-23552.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":12800,"new_prefill":12426,"score_linear":110239.0,"score_lmetric":129618}],"chosen_score_linear":-23552.0,"chosen_score_lmetric":0,"t_first_token":339189.210242898,"t_first_token_unix":1779770054.2241864,"t_done":339190.676621736,"t_done_unix":1779770055.690565},{"request_id":"1355484:11:1388276:696","session_id":"1355484","input_length":26837,"t_proxy_recv":339192.989156986,"t_decision_unix":1779770058.0030973,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9348287811603383,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1749 < 8000)","cache_hit":25088,"estimated_new_tokens":1749,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":26837,"score_linear":50793.0,"score_lmetric":49470},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":126340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":26837,"score_linear":126340.0,"score_lmetric":26837},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26954,"ongoing_decode_tokens":0,"pending_prefill_tokens":20844,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":26837,"score_linear":26954.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":19456,"new_prefill":7381,"score_linear":295863.0,"score_lmetric":31424},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":26837,"score_linear":148720.0,"score_lmetric":100138},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":132149,"pending_prefill_tokens":12091,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":26837,"score_linear":224624.0,"score_lmetric":194640},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":25088,"new_prefill":1749,"score_linear":-25088.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":12800,"new_prefill":14037,"score_linear":110239.0,"score_lmetric":132840}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":0,"t_first_token":339193.340801748,"t_first_token_unix":1779770058.3547456,"t_done":339194.42184724,"t_done_unix":1779770059.4357905},{"request_id":"1382804:2:1388620:698","session_id":"1382804","input_length":3678,"t_proxy_recv":339194.361820476,"t_decision_unix":1779770059.3757615,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":6,"affinity_cache_hit":512,"affinity_cache_ratio":0.13920609026644915,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":3678,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3678 < 8000)","cache_hit":0,"estimated_new_tokens":3678,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":3678,"score_linear":50793.0,"score_lmetric":26311},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":126340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":3678,"score_linear":126340.0,"score_lmetric":3678},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":0,"pending_prefill_tokens":92743,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":3678,"score_linear":98853.0,"score_lmetric":96421},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":3678,"score_linear":315319.0,"score_lmetric":16612},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":3678,"score_linear":148720.0,"score_lmetric":53820},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":203403,"pending_prefill_tokens":229,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":3678,"score_linear":224624.0,"score_lmetric":19535},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26837,"ongoing_decode_tokens":26837,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":3678,"score_linear":26837.0,"score_lmetric":3678},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":512,"new_prefill":3166,"score_linear":122527.0,"score_lmetric":111098}],"chosen_score_linear":26837.0,"chosen_score_lmetric":3678,"t_first_token":339194.598539439,"t_first_token_unix":1779770059.612483,"t_done":339194.650525186,"t_done_unix":1779770059.6644683},{"request_id":"1384947:2:1388842:700","session_id":"1384947","input_length":20183,"t_proxy_recv":339195.126326787,"t_decision_unix":1779770060.1402676,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":18944,"affinity_cache_ratio":0.9386117029182975,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1239 < 8000)","cache_hit":18944,"estimated_new_tokens":1239,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50793,"ongoing_decode_tokens":50793,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":18944,"new_prefill":1239,"score_linear":31849.0,"score_lmetric":1239},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":126340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":20183,"score_linear":126340.0,"score_lmetric":20183},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":0,"pending_prefill_tokens":92743,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":20183,"score_linear":98853.0,"score_lmetric":112926},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":20183,"score_linear":315319.0,"score_lmetric":82632},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":20183,"score_linear":148720.0,"score_lmetric":86830},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":203403,"pending_prefill_tokens":229,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":20183,"score_linear":224624.0,"score_lmetric":102060},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":20183,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":20183,"score_linear":123039.0,"score_lmetric":145132}],"chosen_score_linear":31849.0,"chosen_score_lmetric":1239,"t_first_token":339195.369499948,"t_first_token_unix":1779770060.383444,"t_done":339197.117283945,"t_done_unix":1779770062.1312275},{"request_id":"1381387:3:1387093:691","session_id":"1381387","input_length":50793,"t_proxy_recv":339188.871924577,"t_decision_unix":1779770053.8858662,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":28160,"affinity_cache_ratio":0.5544071033410115,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":28160,"estimated_new_tokens":22633,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":28160,"new_prefill":22633,"score_linear":-28160.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":50793,"score_linear":126340.0,"score_lmetric":177133},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1101,"ongoing_decode_tokens":0,"pending_prefill_tokens":12399,"num_requests":0,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":50793,"score_linear":1101.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":253635,"pending_prefill_tokens":475,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":50793,"score_linear":351902.0,"score_lmetric":256340},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":50793,"score_linear":148720.0,"score_lmetric":148050},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":50793,"score_linear":224624.0,"score_lmetric":331505},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":23717,"ongoing_decode_tokens":23717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":0,"new_prefill":50793,"score_linear":23717.0,"score_lmetric":50793},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":50793,"score_linear":123039.0,"score_lmetric":206352}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":339194.893544016,"t_first_token_unix":1779770059.9074886,"t_done":339197.499568862,"t_done_unix":1779770062.5135128},{"request_id":"1342921:3:1384020:670","session_id":"1342921","input_length":69465,"t_proxy_recv":339178.292535499,"t_decision_unix":1779770043.3064766,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":66048,"affinity_cache_ratio":0.950809760310948,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3417 < 8000)","cache_hit":66048,"estimated_new_tokens":3417,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28383,"ongoing_decode_tokens":0,"pending_prefill_tokens":14559,"num_requests":1,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":0,"new_prefill":69465,"score_linear":28383.0,"score_lmetric":84024},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":69465,"score_linear":126340.0,"score_lmetric":195805},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1942,"cache_hit":0,"new_prefill":69465,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":69465,"score_linear":351902.0,"score_lmetric":356380},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":69465,"score_linear":124993.0,"score_lmetric":138930},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62684,"ongoing_decode_tokens":62684,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":66048,"new_prefill":3417,"score_linear":-3364.0,"score_lmetric":6834},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":69465,"score_linear":73818.0,"score_lmetric":246954},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":69465,"score_linear":106305.0,"score_lmetric":114330}],"chosen_score_linear":-3364.0,"chosen_score_lmetric":6834,"t_first_token":339190.343207033,"t_first_token_unix":1779770055.3571506,"t_done":339198.101976181,"t_done_unix":1779770063.1159198},{"request_id":"1373431:1:1373431:606","session_id":"1373431","input_length":126340,"t_proxy_recv":339141.47802052,"t_decision_unix":1779770006.4919622,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":126340,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":126340,"score_linear":13616.0,"score_lmetric":126340},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":126340,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":126340,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":0,"pending_prefill_tokens":122310,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":126340,"score_linear":336145.0,"score_lmetric":1491900},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":126340,"score_linear":145931.0,"score_lmetric":527964},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":26705,"ongoing_decode_tokens":26705,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":1024,"new_prefill":125316,"score_linear":25681.0,"score_lmetric":125316},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":126340,"score_linear":80246.0,"score_lmetric":126340},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":126340,"score_linear":105902.0,"score_lmetric":171826}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339192.234105987,"t_first_token_unix":1779770057.2480502,"t_done":339198.497929507,"t_done_unix":1779770063.511873},{"request_id":"1366128:11:1384044:671","session_id":"1366128","input_length":71254,"t_proxy_recv":339178.41022084,"t_decision_unix":1779770043.424162,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":59392,"affinity_cache_ratio":0.8335251354309934,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":59392,"estimated_new_tokens":11862,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28383,"ongoing_decode_tokens":0,"pending_prefill_tokens":14559,"num_requests":1,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":0,"new_prefill":71254,"score_linear":28383.0,"score_lmetric":85813},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":71254,"score_linear":126340.0,"score_lmetric":197594},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1942,"cache_hit":0,"new_prefill":71254,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":71254,"score_linear":351902.0,"score_lmetric":365325},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":71254,"score_linear":124993.0,"score_lmetric":142508},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":132149,"ongoing_decode_tokens":62684,"pending_prefill_tokens":3417,"num_requests":3,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":59392,"new_prefill":11862,"score_linear":72757.0,"score_lmetric":45837},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":71254,"score_linear":73818.0,"score_lmetric":250532},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":71254,"score_linear":106305.0,"score_lmetric":116119}],"chosen_score_linear":72757.0,"chosen_score_lmetric":45837,"t_first_token":339193.011977611,"t_first_token_unix":1779770058.025921,"t_done":339198.916151825,"t_done_unix":1779770063.9300954},{"request_id":"1340290:3:1384149:672","session_id":"1340290","input_length":21221,"t_proxy_recv":339178.858692955,"t_decision_unix":1779770043.8726344,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":20992,"affinity_cache_ratio":0.9892088026011969,"affinity_num_requests":4,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (229 < 8000)","cache_hit":20992,"estimated_new_tokens":229,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28383,"ongoing_decode_tokens":0,"pending_prefill_tokens":14559,"num_requests":1,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":0,"new_prefill":21221,"score_linear":28383.0,"score_lmetric":35780},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":21221,"score_linear":126340.0,"score_lmetric":147561},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1942,"cache_hit":0,"new_prefill":21221,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":21221,"score_linear":351902.0,"score_lmetric":115160},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":21221,"score_linear":124993.0,"score_lmetric":42442},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203403,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15279,"num_requests":4,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":20992,"new_prefill":229,"score_linear":182411.0,"score_lmetric":62032},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":1024,"new_prefill":20197,"score_linear":72794.0,"score_lmetric":148418},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":21221,"score_linear":106305.0,"score_lmetric":66086}],"chosen_score_linear":182411.0,"chosen_score_lmetric":62032,"t_first_token":339195.13687568,"t_first_token_unix":1779770060.150819,"t_done":339199.527299385,"t_done_unix":1779770064.5412428},{"request_id":"1363093:3:1374394:612","session_id":"1363093","input_length":63536,"t_proxy_recv":339144.739840855,"t_decision_unix":1779770009.753782,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9911860992193402,"affinity_num_requests":4,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (560 < 8000)","cache_hit":62976,"estimated_new_tokens":560,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":63536,"score_linear":13616.0,"score_lmetric":63536},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":63536,"score_linear":126340.0,"score_lmetric":189876},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":63536,"score_linear":89568.0,"score_lmetric":143031},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":250361,"pending_prefill_tokens":2509,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":63536,"score_linear":336145.0,"score_lmetric":396270},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145931,"ongoing_decode_tokens":18301,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":62976,"new_prefill":560,"score_linear":82955.0,"score_lmetric":24844},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":73469,"ongoing_decode_tokens":26705,"pending_prefill_tokens":11948,"num_requests":2,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":63536,"score_linear":73469.0,"score_lmetric":150968},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":100108,"ongoing_decode_tokens":80246,"pending_prefill_tokens":19862,"num_requests":2,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":63536,"score_linear":100108.0,"score_lmetric":166796},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":63536,"score_linear":105902.0,"score_lmetric":109022}],"chosen_score_linear":82955.0,"chosen_score_lmetric":24844,"t_first_token":339147.578603026,"t_first_token_unix":1779770012.5925543,"t_done":339199.666296436,"t_done_unix":1779770064.68024},{"request_id":"1373431:2:1378320:629","session_id":"1373431","input_length":127267,"t_proxy_recv":339198.518624922,"t_decision_unix":1779770063.5325665,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":126464,"affinity_cache_ratio":0.9936904303550803,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (803 < 8000)","cache_hit":126464,"estimated_new_tokens":803,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":127267,"score_linear":70990.0,"score_lmetric":198257},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":126464,"new_prefill":803,"score_linear":-126464.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":127267,"score_linear":98853.0,"score_lmetric":199166},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":127267,"score_linear":315319.0,"score_lmetric":510968},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":148720,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":127267,"score_linear":148720.0,"score_lmetric":254534},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":155159,"ongoing_decode_tokens":155159,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1809,"cache_hit":1024,"new_prefill":126243,"score_linear":154135.0,"score_lmetric":504972},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":127267,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":127267,"score_linear":123039.0,"score_lmetric":359300}],"chosen_score_linear":-126464.0,"chosen_score_lmetric":0,"t_first_token":339199.134789364,"t_first_token_unix":1779770064.148733,"t_done":339200.746625366,"t_done_unix":1779770065.760569},{"request_id":"1366128:12:1386045:687","session_id":"1366128","input_length":71578,"t_proxy_recv":339198.929990472,"t_decision_unix":1779770063.943932,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9942719830115399,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (410 < 8000)","cache_hit":71168,"estimated_new_tokens":410,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":71578,"score_linear":70990.0,"score_lmetric":142568},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":127267,"ongoing_decode_tokens":0,"pending_prefill_tokens":803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":71578,"score_linear":127267.0,"score_lmetric":72381},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":71578,"score_linear":98853.0,"score_lmetric":143477},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":71578,"score_linear":315319.0,"score_lmetric":288212},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":148720,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":71578,"score_linear":148720.0,"score_lmetric":143156},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83905,"ongoing_decode_tokens":83905,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":71168,"new_prefill":410,"score_linear":12737.0,"score_lmetric":1230},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":71578,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161281,"ongoing_decode_tokens":0,"pending_prefill_tokens":52737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":71578,"score_linear":161281.0,"score_lmetric":372945}],"chosen_score_linear":12737.0,"chosen_score_lmetric":1230,"t_first_token":339199.161599072,"t_first_token_unix":1779770064.1755428,"t_done":339200.875744453,"t_done_unix":1779770065.8896892},{"request_id":"1390240:1:1390240:705","session_id":"1390240","input_length":5110,"t_proxy_recv":339200.310481984,"t_decision_unix":1779770065.3244233,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5110 < 8000)","cache_hit":0,"estimated_new_tokens":5110,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":5110,"score_linear":70990.0,"score_lmetric":76100},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":127267,"ongoing_decode_tokens":127267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":5110,"score_linear":127267.0,"score_lmetric":5110},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":5110,"score_linear":98853.0,"score_lmetric":77009},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":5110,"score_linear":315319.0,"score_lmetric":22340},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":85184,"pending_prefill_tokens":3163,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":5110,"score_linear":153371.0,"score_lmetric":16546},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160408,"ongoing_decode_tokens":134262,"pending_prefill_tokens":5154,"num_requests":4,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":5110,"score_linear":160408.0,"score_lmetric":41056},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":5110,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161281,"ongoing_decode_tokens":123039,"pending_prefill_tokens":354,"num_requests":3,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":5110,"score_linear":161281.0,"score_lmetric":16392}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339200.641217289,"t_first_token_unix":1779770065.6551607,"t_done":339201.031705341,"t_done_unix":1779770066.0456488},{"request_id":"1383090:2:1385262:682","session_id":"1383090","input_length":16734,"t_proxy_recv":339182.618757005,"t_decision_unix":1779770047.6326983,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":9216,"affinity_cache_ratio":0.5507350304768734,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7518 < 8000)","cache_hit":9216,"estimated_new_tokens":7518,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":19034,"ongoing_decode_tokens":0,"pending_prefill_tokens":19034,"num_requests":1,"active_p_offloads":0,"cached_blocks":1418,"cache_hit":0,"new_prefill":16734,"score_linear":19034.0,"score_lmetric":35768},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":16734,"score_linear":126340.0,"score_lmetric":143074},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":16734,"score_linear":-11298.0,"score_lmetric":-16734},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":16734,"score_linear":351902.0,"score_lmetric":92725},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":174858,"ongoing_decode_tokens":174858,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":16734,"score_linear":174858.0,"score_lmetric":50202},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":16734,"score_linear":224624.0,"score_lmetric":161210},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":54012,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":16734,"score_linear":73818.0,"score_lmetric":33468},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":0,"pending_prefill_tokens":44865,"num_requests":1,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":9216,"new_prefill":7518,"score_linear":97089.0,"score_lmetric":52383}],"chosen_score_linear":97089.0,"chosen_score_lmetric":52383,"t_first_token":339200.29480439,"t_first_token_unix":1779770065.3087509,"t_done":339201.055972179,"t_done_unix":1779770066.0699162},{"request_id":"1355951:4:1389858:703","session_id":"1355951","input_length":38242,"t_proxy_recv":339198.909513758,"t_decision_unix":1779770063.923455,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9907431619685163,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (354 < 8000)","cache_hit":37888,"estimated_new_tokens":354,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":38242,"score_linear":70990.0,"score_lmetric":109232},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":127267,"ongoing_decode_tokens":0,"pending_prefill_tokens":803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":38242,"score_linear":127267.0,"score_lmetric":39045},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":38242,"score_linear":98853.0,"score_lmetric":110141},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":38242,"score_linear":315319.0,"score_lmetric":154868},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":148720,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":38242,"score_linear":148720.0,"score_lmetric":76484},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":155159,"ongoing_decode_tokens":155159,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1809,"cache_hit":0,"new_prefill":38242,"score_linear":155159.0,"score_lmetric":152968},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":38242,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":37888,"new_prefill":354,"score_linear":85151.0,"score_lmetric":105474}],"chosen_score_linear":85151.0,"chosen_score_lmetric":105474,"t_first_token":339200.393589652,"t_first_token_unix":1779770065.407534,"t_done":339201.069804306,"t_done_unix":1779770066.0837483},{"request_id":"1340290:4:1385568:684","session_id":"1340290","input_length":26146,"t_proxy_recv":339199.536362331,"t_decision_unix":1779770064.5503037,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":20992,"affinity_cache_ratio":0.8028761569647365,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5154 < 8000)","cache_hit":20992,"estimated_new_tokens":5154,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":26146,"score_linear":70990.0,"score_lmetric":97136},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":127267,"ongoing_decode_tokens":127267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":26146,"score_linear":127267.0,"score_lmetric":26146},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":26146,"score_linear":98853.0,"score_lmetric":98045},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":26146,"score_linear":315319.0,"score_lmetric":106484},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":148720,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":26146,"score_linear":148720.0,"score_lmetric":52292},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":134262,"ongoing_decode_tokens":134262,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":20992,"new_prefill":5154,"score_linear":113270.0,"score_lmetric":15462},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":1024,"new_prefill":25122,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161281,"ongoing_decode_tokens":0,"pending_prefill_tokens":52737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":26146,"score_linear":161281.0,"score_lmetric":236649}],"chosen_score_linear":113270.0,"chosen_score_lmetric":15462,"t_first_token":339200.46317155,"t_first_token_unix":1779770065.4771156,"t_done":339201.319173695,"t_done_unix":1779770066.3331172},{"request_id":"1366128:13:1387308:692","session_id":"1366128","input_length":71708,"t_proxy_recv":339200.889985505,"t_decision_unix":1779770065.9039268,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9924694594745356,"affinity_num_requests":3,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (540 < 8000)","cache_hit":71168,"estimated_new_tokens":540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":71708,"score_linear":70990.0,"score_lmetric":142698},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128201,"ongoing_decode_tokens":0,"pending_prefill_tokens":1225,"num_requests":1,"active_p_offloads":0,"cached_blocks":1504,"cache_hit":0,"new_prefill":71708,"score_linear":128201.0,"score_lmetric":72933},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":71708,"score_linear":98853.0,"score_lmetric":143607},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":71708,"score_linear":315319.0,"score_lmetric":288732},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":85184,"pending_prefill_tokens":3163,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":71708,"score_linear":153371.0,"score_lmetric":149742},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":88830,"ongoing_decode_tokens":88830,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":71168,"new_prefill":540,"score_linear":17662.0,"score_lmetric":1620},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":6459,"ongoing_decode_tokens":6459,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":71708,"score_linear":6459.0,"score_lmetric":143416},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161281,"ongoing_decode_tokens":161281,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":71708,"score_linear":161281.0,"score_lmetric":215124}],"chosen_score_linear":17662.0,"chosen_score_lmetric":1620,"t_first_token":339201.056945123,"t_first_token_unix":1779770066.0708885,"t_done":339201.970456594,"t_done_unix":1779770066.9843998},{"request_id":"1263749:2:1379302:636","session_id":"1263749","input_length":57656,"t_proxy_recv":339161.859528555,"t_decision_unix":1779770026.87347,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":56320,"affinity_cache_ratio":0.9768280838074095,"affinity_num_requests":5,"avg_num_requests":2.375,"fallback_score":31055,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1336 < 8000)","cache_hit":56320,"estimated_new_tokens":1336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":37310,"ongoing_decode_tokens":37310,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1330,"cache_hit":0,"new_prefill":57656,"score_linear":37310.0,"score_lmetric":115312},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":57656,"score_linear":126340.0,"score_lmetric":183996},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":57656,"score_linear":89568.0,"score_lmetric":137151},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":313823,"ongoing_decode_tokens":215926,"pending_prefill_tokens":4875,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":56320,"new_prefill":1336,"score_linear":257503.0,"score_lmetric":31055},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":57656,"score_linear":191437.0,"score_lmetric":409128},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":57656,"score_linear":15283.0,"score_lmetric":72939},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":55570,"ongoing_decode_tokens":0,"pending_prefill_tokens":15284,"num_requests":3,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":57656,"score_linear":55570.0,"score_lmetric":218820},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":57656,"score_linear":143943.0,"score_lmetric":208638}],"chosen_score_linear":257503.0,"chosen_score_lmetric":31055,"t_first_token":339187.557913823,"t_first_token_unix":1779770052.5718584,"t_done":339202.061435089,"t_done_unix":1779770067.0753787},{"request_id":"1313181:6:1324908:369","session_id":"1313181","input_length":106305,"t_proxy_recv":339176.230354811,"t_decision_unix":1779770041.2442966,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61440,"affinity_cache_ratio":0.5779596444193594,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":61440,"estimated_new_tokens":44865,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1390,"cache_hit":0,"new_prefill":106305,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":106305,"score_linear":126340.0,"score_lmetric":232645},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10073,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":0,"new_prefill":106305,"score_linear":10073.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":12800,"new_prefill":93505,"score_linear":339102.0,"score_lmetric":476580},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":63536,"pending_prefill_tokens":4625,"num_requests":2,"active_p_offloads":0,"cached_blocks":1659,"cache_hit":0,"new_prefill":106305,"score_linear":124993.0,"score_lmetric":221860},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":122058,"ongoing_decode_tokens":74657,"pending_prefill_tokens":19753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":0,"new_prefill":106305,"score_linear":122058.0,"score_lmetric":378174},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":73818,"ongoing_decode_tokens":0,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":0,"new_prefill":106305,"score_linear":73818.0,"score_lmetric":320634},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":61440,"new_prefill":44865,"score_linear":-61440.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":339199.955712938,"t_first_token_unix":1779770064.969657,"t_done":339202.574989216,"t_done_unix":1779770067.5889328},{"request_id":"1373431:3:1383388:667","session_id":"1373431","input_length":128201,"t_proxy_recv":339200.767404932,"t_decision_unix":1779770065.7813466,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":126976,"affinity_cache_ratio":0.9904446923190927,"affinity_num_requests":0,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1225 < 8000)","cache_hit":126976,"estimated_new_tokens":1225,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":128201,"score_linear":70990.0,"score_lmetric":199191},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1504,"cache_hit":126976,"new_prefill":1225,"score_linear":-126976.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":128201,"score_linear":98853.0,"score_lmetric":200100},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":128201,"score_linear":315319.0,"score_lmetric":514704},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":85184,"pending_prefill_tokens":3163,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":128201,"score_linear":153371.0,"score_lmetric":262728},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160408,"ongoing_decode_tokens":160408,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":1024,"new_prefill":127177,"score_linear":159384.0,"score_lmetric":508708},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":6459,"ongoing_decode_tokens":5110,"pending_prefill_tokens":1349,"num_requests":2,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":128201,"score_linear":6459.0,"score_lmetric":259100},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161281,"ongoing_decode_tokens":161281,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":128201,"score_linear":161281.0,"score_lmetric":384603}],"chosen_score_linear":-126976.0,"chosen_score_lmetric":0,"t_first_token":339201.624357997,"t_first_token_unix":1779770066.6383014,"t_done":339202.626986957,"t_done_unix":1779770067.6409307},{"request_id":"1390356:1:1390356:706","session_id":"1390356","input_length":1349,"t_proxy_recv":339200.738079488,"t_decision_unix":1779770065.752021,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":1349,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1349 < 8000)","cache_hit":0,"estimated_new_tokens":1349,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":1349,"score_linear":70990.0,"score_lmetric":72339},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":127267,"ongoing_decode_tokens":127267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":1349,"score_linear":127267.0,"score_lmetric":1349},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":1349,"score_linear":98853.0,"score_lmetric":73248},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":1349,"score_linear":315319.0,"score_lmetric":7296},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":85184,"pending_prefill_tokens":3163,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":1349,"score_linear":153371.0,"score_lmetric":9024},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160408,"ongoing_decode_tokens":160408,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":1349,"score_linear":160408.0,"score_lmetric":5396},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":5110,"ongoing_decode_tokens":5110,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":1349,"score_linear":5110.0,"score_lmetric":1349},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161281,"ongoing_decode_tokens":161281,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":1349,"score_linear":161281.0,"score_lmetric":4047}],"chosen_score_linear":5110.0,"chosen_score_lmetric":1349,"t_first_token":339200.840688003,"t_first_token_unix":1779770065.854632,"t_done":339203.584238367,"t_done_unix":1779770068.5981815},{"request_id":"1313181:6:1379623:639","session_id":"1313181","input_length":61543,"t_proxy_recv":339202.585704544,"t_decision_unix":1779770067.599646,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9983263734299596,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (103 < 8000)","cache_hit":61440,"estimated_new_tokens":103,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":61543,"score_linear":70990.0,"score_lmetric":132533},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128201,"ongoing_decode_tokens":128201,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1504,"cache_hit":0,"new_prefill":61543,"score_linear":128201.0,"score_lmetric":61543},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":61543,"score_linear":98853.0,"score_lmetric":133442},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":12800,"new_prefill":48743,"score_linear":244863.0,"score_lmetric":147654},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":153371,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":61543,"score_linear":153371.0,"score_lmetric":123086},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":173308,"ongoing_decode_tokens":89904,"pending_prefill_tokens":11724,"num_requests":4,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":0,"new_prefill":61543,"score_linear":173308.0,"score_lmetric":293068},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24990,"ongoing_decode_tokens":24990,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":12800,"new_prefill":48743,"score_linear":12190.0,"score_lmetric":97486},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":61440,"new_prefill":103,"score_linear":-61440.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":339202.672582416,"t_first_token_unix":1779770067.686526,"t_done":339206.339635562,"t_done_unix":1779770071.3535793},{"request_id":"1391584:1:1391584:713","session_id":"1391584","input_length":15302,"t_proxy_recv":339204.810633561,"t_decision_unix":1779770069.8245747,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":15302,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":15302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":15302,"score_linear":91673.0,"score_lmetric":174014},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":28307,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":15302,"score_linear":28307.0,"score_lmetric":43609},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":15302,"score_linear":98853.0,"score_lmetric":87201},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":15302,"score_linear":257663.0,"score_lmetric":47331},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":153371,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":15302,"score_linear":153371.0,"score_lmetric":30604},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":173308,"ongoing_decode_tokens":89904,"pending_prefill_tokens":11724,"num_requests":4,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":0,"new_prefill":15302,"score_linear":173308.0,"score_lmetric":108104},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":23641,"ongoing_decode_tokens":23641,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":15302,"score_linear":23641.0,"score_lmetric":15302},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61543,"ongoing_decode_tokens":61543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":0,"new_prefill":15302,"score_linear":61543.0,"score_lmetric":15302}],"chosen_score_linear":23641.0,"chosen_score_lmetric":15302,"t_first_token":339206.192834278,"t_first_token_unix":1779770071.2067776,"t_done":339206.892040757,"t_done_unix":1779770071.9059837},{"request_id":"1363093:4:1390050:704","session_id":"1363093","input_length":68187,"t_proxy_recv":339199.681441655,"t_decision_unix":1779770064.695383,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9536128587560678,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3163 < 8000)","cache_hit":65024,"estimated_new_tokens":3163,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":68187,"score_linear":70990.0,"score_lmetric":139177},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":127267,"ongoing_decode_tokens":127267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":68187,"score_linear":127267.0,"score_lmetric":68187},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":68187,"score_linear":98853.0,"score_lmetric":140086},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":68187,"score_linear":315319.0,"score_lmetric":274648},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":85184,"ongoing_decode_tokens":85184,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":65024,"new_prefill":3163,"score_linear":20160.0,"score_lmetric":3163},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160408,"ongoing_decode_tokens":134262,"pending_prefill_tokens":5154,"num_requests":4,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":68187,"score_linear":160408.0,"score_lmetric":293364},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":68187,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":161281,"ongoing_decode_tokens":0,"pending_prefill_tokens":52737,"num_requests":3,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":68187,"score_linear":161281.0,"score_lmetric":362772}],"chosen_score_linear":20160.0,"chosen_score_lmetric":3163,"t_first_token":339201.079327938,"t_first_token_unix":1779770066.093271,"t_done":339207.184783429,"t_done_unix":1779770072.1987274},{"request_id":"1370129:4:1384856:678","session_id":"1370129","input_length":85184,"t_proxy_recv":339187.06609231,"t_decision_unix":1779770052.0800335,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":61952,"affinity_cache_ratio":0.7272727272727273,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=7680 chosen=61952)","cache_hit":61952,"estimated_new_tokens":23232,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":7680,"new_prefill":77504,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":85184,"score_linear":126340.0,"score_lmetric":211524},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":7680,"new_prefill":77504,"score_linear":-18978.0,"score_lmetric":-77504},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":195979,"pending_prefill_tokens":1811,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":85184,"score_linear":351902.0,"score_lmetric":434975},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":63536,"ongoing_decode_tokens":63536,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":61952,"new_prefill":23232,"score_linear":1584.0,"score_lmetric":23232},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":62684,"pending_prefill_tokens":15508,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":7680,"new_prefill":77504,"score_linear":216944.0,"score_lmetric":465060},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":23457,"ongoing_decode_tokens":23457,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1547,"cache_hit":7680,"new_prefill":77504,"score_linear":15777.0,"score_lmetric":77504},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":85184,"score_linear":123039.0,"score_lmetric":275134}],"chosen_score_linear":1584.0,"chosen_score_lmetric":23232,"t_first_token":339197.852037689,"t_first_token_unix":1779770062.865981,"t_done":339207.264214238,"t_done_unix":1779770072.2781582},{"request_id":"1340290:5:1386504:688","session_id":"1340290","input_length":27220,"t_proxy_recv":339201.325097666,"t_decision_unix":1779770066.3390393,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9592946362968405,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1108 < 8000)","cache_hit":26112,"estimated_new_tokens":1108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":27220,"score_linear":70990.0,"score_lmetric":98210},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128201,"ongoing_decode_tokens":0,"pending_prefill_tokens":1225,"num_requests":1,"active_p_offloads":0,"cached_blocks":1504,"cache_hit":0,"new_prefill":27220,"score_linear":128201.0,"score_lmetric":28445},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":27220,"score_linear":98853.0,"score_lmetric":99119},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":27220,"score_linear":315319.0,"score_lmetric":110780},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":153371,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":27220,"score_linear":153371.0,"score_lmetric":54440},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":134392,"ongoing_decode_tokens":134392,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1842,"cache_hit":26112,"new_prefill":1108,"score_linear":108280.0,"score_lmetric":3324},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24990,"ongoing_decode_tokens":1349,"pending_prefill_tokens":2137,"num_requests":2,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":1024,"new_prefill":26196,"score_linear":23966.0,"score_lmetric":56666},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":106305,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1593,"cache_hit":0,"new_prefill":27220,"score_linear":106305.0,"score_lmetric":27220}],"chosen_score_linear":108280.0,"chosen_score_lmetric":3324,"t_first_token":339201.618263822,"t_first_token_unix":1779770066.6322076,"t_done":339207.780838758,"t_done_unix":1779770072.7947822},{"request_id":"1391584:2:1392466:715","session_id":"1391584","input_length":15483,"t_proxy_recv":339208.130314848,"t_decision_unix":1779770073.1442564,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14848,"affinity_cache_ratio":0.958987276367629,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (635 < 8000)","cache_hit":14848,"estimated_new_tokens":635,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":15483,"score_linear":91673.0,"score_lmetric":174376},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28307,"ongoing_decode_tokens":28307,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":15483,"score_linear":28307.0,"score_lmetric":15483},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":15483,"score_linear":98853.0,"score_lmetric":87382},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":15483,"score_linear":257663.0,"score_lmetric":47874},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":87687,"ongoing_decode_tokens":0,"pending_prefill_tokens":2183,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":15483,"score_linear":87687.0,"score_lmetric":17666},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":173373,"ongoing_decode_tokens":173373,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1845,"cache_hit":0,"new_prefill":15483,"score_linear":173373.0,"score_lmetric":61932},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":52023,"ongoing_decode_tokens":52023,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1603,"cache_hit":14848,"new_prefill":635,"score_linear":37175.0,"score_lmetric":1270},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":15483,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":37175.0,"chosen_score_lmetric":1270,"t_first_token":339208.215448194,"t_first_token_unix":1779770073.229391,"t_done":339208.938829568,"t_done_unix":1779770073.9527776},{"request_id":"1355484:12:1391879:714","session_id":"1355484","input_length":28382,"t_proxy_recv":339205.948327751,"t_decision_unix":1779770070.962269,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9380593333803114,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1758 < 8000)","cache_hit":26624,"estimated_new_tokens":1758,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":28382,"score_linear":91673.0,"score_lmetric":200174},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":28307,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":28382,"score_linear":28307.0,"score_lmetric":56689},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":28382,"score_linear":98853.0,"score_lmetric":100281},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":19456,"new_prefill":8926,"score_linear":238207.0,"score_lmetric":28203},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":153371,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":28382,"score_linear":153371.0,"score_lmetric":56764},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":173308,"ongoing_decode_tokens":89904,"pending_prefill_tokens":11724,"num_requests":4,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":0,"new_prefill":28382,"score_linear":173308.0,"score_lmetric":160424},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":38943,"ongoing_decode_tokens":23641,"pending_prefill_tokens":15302,"num_requests":2,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":26624,"new_prefill":1758,"score_linear":12319.0,"score_lmetric":34120},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61543,"ongoing_decode_tokens":61543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":12800,"new_prefill":15582,"score_linear":48743.0,"score_lmetric":15582}],"chosen_score_linear":12319.0,"chosen_score_lmetric":34120,"t_first_token":339206.523482937,"t_first_token_unix":1779770071.537426,"t_done":339208.940375532,"t_done_unix":1779770073.954319},{"request_id":"1363943:9:1390468:707","session_id":"1363943","input_length":23641,"t_proxy_recv":339201.213550505,"t_decision_unix":1779770066.2274919,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9096061926314454,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2137 < 8000)","cache_hit":21504,"estimated_new_tokens":2137,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":8704,"new_prefill":14937,"score_linear":62286.0,"score_lmetric":85927},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128201,"ongoing_decode_tokens":0,"pending_prefill_tokens":1225,"num_requests":1,"active_p_offloads":0,"cached_blocks":1504,"cache_hit":0,"new_prefill":23641,"score_linear":128201.0,"score_lmetric":24866},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":4096,"new_prefill":19545,"score_linear":94757.0,"score_lmetric":91444},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":23641,"score_linear":315319.0,"score_lmetric":96464},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":153371,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":4096,"new_prefill":19545,"score_linear":149275.0,"score_lmetric":39090},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":160538,"ongoing_decode_tokens":160538,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":4096,"new_prefill":19545,"score_linear":156442.0,"score_lmetric":78180},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":1349,"ongoing_decode_tokens":1349,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":21504,"new_prefill":2137,"score_linear":-20155.0,"score_lmetric":2137},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":106305,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1593,"cache_hit":0,"new_prefill":23641,"score_linear":106305.0,"score_lmetric":23641}],"chosen_score_linear":-20155.0,"chosen_score_lmetric":2137,"t_first_token":339201.575687633,"t_first_token_unix":1779770066.5896318,"t_done":339209.100779712,"t_done_unix":1779770074.1147246},{"request_id":"1391404:1:1391404:712","session_id":"1391404","input_length":28307,"t_proxy_recv":339204.296915761,"t_decision_unix":1779770069.310857,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":28307,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":28307,"score_linear":91673.0,"score_lmetric":200024},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":28307,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":28307,"score_linear":98853.0,"score_lmetric":100206},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":28307,"score_linear":257663.0,"score_lmetric":86346},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":153371,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":2560,"new_prefill":25747,"score_linear":150811.0,"score_lmetric":51494},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":173308,"ongoing_decode_tokens":89904,"pending_prefill_tokens":11724,"num_requests":4,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":0,"new_prefill":28307,"score_linear":173308.0,"score_lmetric":160124},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":23641,"ongoing_decode_tokens":23641,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":28307,"score_linear":23641.0,"score_lmetric":28307},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61543,"ongoing_decode_tokens":61543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":0,"new_prefill":28307,"score_linear":61543.0,"score_lmetric":28307}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339207.849589433,"t_first_token_unix":1779770072.8635333,"t_done":339209.172303252,"t_done_unix":1779770074.1862469},{"request_id":"1366128:14:1389156:701","session_id":"1366128","input_length":83404,"t_proxy_recv":339201.997345954,"t_decision_unix":1779770067.0112875,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":71680,"affinity_cache_ratio":0.8594312023404154,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":71680,"estimated_new_tokens":11724,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":83404,"score_linear":70990.0,"score_lmetric":154394},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":128201,"ongoing_decode_tokens":128201,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1504,"cache_hit":0,"new_prefill":83404,"score_linear":128201.0,"score_lmetric":83404},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":83404,"score_linear":98853.0,"score_lmetric":155303},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":83404,"score_linear":315319.0,"score_lmetric":335516},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":153371,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":83404,"score_linear":153371.0,"score_lmetric":166808},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":89904,"ongoing_decode_tokens":89904,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":71680,"new_prefill":11724,"score_linear":18224.0,"score_lmetric":35172},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24990,"ongoing_decode_tokens":24990,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":0,"new_prefill":83404,"score_linear":24990.0,"score_lmetric":166808},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106305,"ongoing_decode_tokens":106305,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1593,"cache_hit":0,"new_prefill":83404,"score_linear":106305.0,"score_lmetric":83404}],"chosen_score_linear":18224.0,"chosen_score_lmetric":35172,"t_first_token":339207.74675347,"t_first_token_unix":1779770072.760697,"t_done":339209.344018189,"t_done_unix":1779770074.357962},{"request_id":"1390240:2:1392954:717","session_id":"1390240","input_length":7440,"t_proxy_recv":339209.8448434,"t_decision_unix":1779770074.8587847,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":5120,"affinity_cache_ratio":0.6881720430107527,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2320 < 8000)","cache_hit":5120,"estimated_new_tokens":2320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":7440,"score_linear":91673.0,"score_lmetric":158290},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":83622,"ongoing_decode_tokens":0,"pending_prefill_tokens":83622,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":7440,"score_linear":83622.0,"score_lmetric":91062},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":7440,"score_linear":98853.0,"score_lmetric":79339},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":7440,"score_linear":257663.0,"score_lmetric":23745},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":87687,"ongoing_decode_tokens":87687,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":7440,"score_linear":87687.0,"score_lmetric":7440},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":89969,"ongoing_decode_tokens":89969,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":7440,"score_linear":89969.0,"score_lmetric":22320},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1612,"cache_hit":5120,"new_prefill":2320,"score_linear":-5120.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-5120.0,"chosen_score_lmetric":0,"t_first_token":339210.072530392,"t_first_token_unix":1779770075.086474,"t_done":339210.293618692,"t_done_unix":1779770075.3075619},{"request_id":"1340290:6:1387365:695","session_id":"1340290","input_length":27285,"t_proxy_recv":339207.78659174,"t_decision_unix":1779770072.8005333,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9945391240608393,"affinity_num_requests":3,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (149 < 8000)","cache_hit":27136,"estimated_new_tokens":149,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":27285,"score_linear":91673.0,"score_lmetric":197980},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":28307,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":27285,"score_linear":28307.0,"score_lmetric":55592},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":27285,"score_linear":98853.0,"score_lmetric":99184},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":27285,"score_linear":257663.0,"score_lmetric":83280},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":87687,"ongoing_decode_tokens":0,"pending_prefill_tokens":2183,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":27285,"score_linear":87687.0,"score_lmetric":29468},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":146088,"ongoing_decode_tokens":146088,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1845,"cache_hit":27136,"new_prefill":149,"score_linear":118952.0,"score_lmetric":447},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":52023,"ongoing_decode_tokens":52023,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1603,"cache_hit":1024,"new_prefill":26261,"score_linear":50999.0,"score_lmetric":52522},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":27285,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":118952.0,"chosen_score_lmetric":447,"t_first_token":339207.883311376,"t_first_token_unix":1779770072.8972545,"t_done":339210.401740206,"t_done_unix":1779770075.4156835},{"request_id":"1364090:3:1381959:653","session_id":"1364090","input_length":47401,"t_proxy_recv":339171.113891897,"t_decision_unix":1779770036.1278334,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":27648,"affinity_cache_ratio":0.5832788337798781,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=11264 chosen=27648)","cache_hit":27648,"estimated_new_tokens":19753,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49877,"ongoing_decode_tokens":49877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":7680,"new_prefill":39721,"score_linear":42197.0,"score_lmetric":79442},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":47401,"score_linear":126340.0,"score_lmetric":173741},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":91220,"ongoing_decode_tokens":102518,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":7680,"new_prefill":39721,"score_linear":83540.0,"score_lmetric":39721},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":351902,"ongoing_decode_tokens":159396,"pending_prefill_tokens":2042,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":47401,"score_linear":351902.0,"score_lmetric":247215},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":141954,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":7680,"new_prefill":39721,"score_linear":183757.0,"score_lmetric":158884},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74657,"ongoing_decode_tokens":74657,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1802,"cache_hit":27648,"new_prefill":19753,"score_linear":47009.0,"score_lmetric":39506},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1397,"cache_hit":11264,"new_prefill":36137,"score_linear":8542.0,"score_lmetric":36137},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":60858,"ongoing_decode_tokens":60858,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":0,"new_prefill":47401,"score_linear":60858.0,"score_lmetric":47401}],"chosen_score_linear":47009.0,"chosen_score_lmetric":39506,"t_first_token":339176.253314963,"t_first_token_unix":1779770041.2672584,"t_done":339211.831167408,"t_done_unix":1779770076.8451111},{"request_id":"1370129:5:1390477:708","session_id":"1370129","input_length":87687,"t_proxy_recv":339207.280463453,"t_decision_unix":1779770072.2944052,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9751046335260642,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2183 < 8000)","cache_hit":85504,"estimated_new_tokens":2183,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":7680,"new_prefill":80007,"score_linear":83993.0,"score_lmetric":303424},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":28307,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":87687,"score_linear":28307.0,"score_lmetric":115994},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":7680,"new_prefill":80007,"score_linear":91173.0,"score_lmetric":151906},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":87687,"score_linear":257663.0,"score_lmetric":264486},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":85504,"new_prefill":2183,"score_linear":-85504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":173308,"ongoing_decode_tokens":89904,"pending_prefill_tokens":11724,"num_requests":4,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":7680,"new_prefill":80007,"score_linear":165628.0,"score_lmetric":366924},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":52023,"ongoing_decode_tokens":52023,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1603,"cache_hit":7680,"new_prefill":80007,"score_linear":44343.0,"score_lmetric":160014},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":87687,"score_linear":121811.0,"score_lmetric":147546}],"chosen_score_linear":-85504.0,"chosen_score_lmetric":0,"t_first_token":339208.537243131,"t_first_token_unix":1779770073.5511868,"t_done":339213.039809548,"t_done_unix":1779770078.0537539},{"request_id":"1388507:1:1388507:697","session_id":"1388507","input_length":71899,"t_proxy_recv":339193.927715217,"t_decision_unix":1779770058.941656,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":71899,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":71899,"score_linear":50793.0,"score_lmetric":94532},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":126340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":71899,"score_linear":126340.0,"score_lmetric":71899},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26954,"ongoing_decode_tokens":0,"pending_prefill_tokens":20844,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":71899,"score_linear":26954.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":71899,"score_linear":315319.0,"score_lmetric":289496},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":71899,"score_linear":148720.0,"score_lmetric":190262},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":203403,"pending_prefill_tokens":229,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":71899,"score_linear":224624.0,"score_lmetric":360640},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26837,"ongoing_decode_tokens":26837,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":71899,"score_linear":26837.0,"score_lmetric":71899},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":71899,"score_linear":123039.0,"score_lmetric":248564}],"chosen_score_linear":26954.0,"chosen_score_lmetric":0,"t_first_token":339214.143279311,"t_first_token_unix":1779770079.1572227,"t_done":339214.607250044,"t_done_unix":1779770079.621194},{"request_id":"1394167:1:1394167:721","session_id":"1394167","input_length":4675,"t_proxy_recv":339214.108373945,"t_decision_unix":1779770079.1223152,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4675 < 8000)","cache_hit":0,"estimated_new_tokens":4675,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":4675,"score_linear":91673.0,"score_lmetric":152760},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":4675,"score_linear":218746.0,"score_lmetric":366135},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":4675,"score_linear":98853.0,"score_lmetric":76574},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":4675,"score_linear":257663.0,"score_lmetric":15450},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":4675,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":4675,"score_linear":64227.0,"score_lmetric":9350},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":4675,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":4675,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339214.408575261,"t_first_token_unix":1779770079.4225185,"t_done":339214.780821977,"t_done_unix":1779770079.7947671},{"request_id":"1355484:13:1394384:723","session_id":"1355484","input_length":28665,"t_proxy_recv":339214.854720104,"t_decision_unix":1779770079.8686612,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":28160,"affinity_cache_ratio":0.9823826966684109,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (505 < 8000)","cache_hit":28160,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":28665,"score_linear":91673.0,"score_lmetric":200740},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":28665,"score_linear":218746.0,"score_lmetric":438105},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26954,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2180,"cache_hit":0,"new_prefill":28665,"score_linear":26954.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":19456,"new_prefill":9209,"score_linear":238207.0,"score_lmetric":29052},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":28665,"score_linear":106143.0,"score_lmetric":46744},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":28665,"score_linear":64227.0,"score_lmetric":57330},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":28160,"new_prefill":505,"score_linear":-28160.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":12800,"new_prefill":15865,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":339214.925214056,"t_first_token_unix":1779770079.9391575,"t_done":339215.631956874,"t_done_unix":1779770080.6459007},{"request_id":"1394482:1:1394482:726","session_id":"1394482","input_length":4428,"t_proxy_recv":339215.268869937,"t_decision_unix":1779770080.2828112,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":4428,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4428 < 8000)","cache_hit":0,"estimated_new_tokens":4428,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":4428,"score_linear":91673.0,"score_lmetric":152266},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":4428,"score_linear":218746.0,"score_lmetric":365394},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":31690,"ongoing_decode_tokens":38252,"pending_prefill_tokens":4736,"num_requests":1,"active_p_offloads":0,"cached_blocks":2180,"cache_hit":0,"new_prefill":4428,"score_linear":31690.0,"score_lmetric":9164},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":4428,"score_linear":257663.0,"score_lmetric":14709},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":4428,"score_linear":106143.0,"score_lmetric":22507},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":4428,"score_linear":64227.0,"score_lmetric":8856},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28665,"ongoing_decode_tokens":28665,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":4428,"score_linear":28665.0,"score_lmetric":4428},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":4428,"score_linear":982.0,"score_lmetric":5410}],"chosen_score_linear":28665.0,"chosen_score_lmetric":4428,"t_first_token":339215.556960522,"t_first_token_unix":1779770080.5709038,"t_done":339215.671159949,"t_done_unix":1779770080.6851034},{"request_id":"1394432:1:1394432:725","session_id":"1394432","input_length":4736,"t_proxy_recv":339215.014787709,"t_decision_unix":1779770080.0287294,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4736 < 8000)","cache_hit":0,"estimated_new_tokens":4736,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":4736,"score_linear":91673.0,"score_lmetric":152882},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":4736,"score_linear":218746.0,"score_lmetric":366318},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26954,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2180,"cache_hit":0,"new_prefill":4736,"score_linear":26954.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":4736,"score_linear":257663.0,"score_lmetric":15633},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":4736,"score_linear":106143.0,"score_lmetric":22815},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":4736,"score_linear":64227.0,"score_lmetric":9472},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28665,"ongoing_decode_tokens":28665,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":4736,"score_linear":28665.0,"score_lmetric":4736},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":4736,"score_linear":982.0,"score_lmetric":5718}],"chosen_score_linear":26954.0,"chosen_score_lmetric":0,"t_first_token":339215.324475875,"t_first_token_unix":1779770080.3384194,"t_done":339215.738110164,"t_done_unix":1779770080.7520533},{"request_id":"1384947:3:1390898:710","session_id":"1384947","input_length":20683,"t_proxy_recv":339202.949540179,"t_decision_unix":1779770067.9634814,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9654305468258957,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (715 < 8000)","cache_hit":19968,"estimated_new_tokens":715,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":0,"pending_prefill_tokens":70990,"num_requests":1,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":19968,"new_prefill":715,"score_linear":51022.0,"score_lmetric":71705},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":20683,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":20683,"score_linear":98853.0,"score_lmetric":92582},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":20683,"score_linear":257663.0,"score_lmetric":63474},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":153371,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":20683,"score_linear":153371.0,"score_lmetric":41366},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":173308,"ongoing_decode_tokens":89904,"pending_prefill_tokens":11724,"num_requests":4,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":0,"new_prefill":20683,"score_linear":173308.0,"score_lmetric":129628},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24990,"ongoing_decode_tokens":24990,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":0,"new_prefill":20683,"score_linear":24990.0,"score_lmetric":41366},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":61543,"ongoing_decode_tokens":61543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1595,"cache_hit":0,"new_prefill":20683,"score_linear":61543.0,"score_lmetric":20683}],"chosen_score_linear":51022.0,"chosen_score_lmetric":71705,"t_first_token":339215.60728526,"t_first_token_unix":1779770080.6212282,"t_done":339217.717750213,"t_done_unix":1779770082.7316933},{"request_id":"1364090:4:1392998:718","session_id":"1364090","input_length":48944,"t_proxy_recv":339211.840704163,"t_decision_unix":1779770076.8546457,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9833278849297156,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (816 < 8000)","cache_hit":48128,"estimated_new_tokens":816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":7680,"new_prefill":41264,"score_linear":83993.0,"score_lmetric":225938},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":83622,"ongoing_decode_tokens":0,"pending_prefill_tokens":83622,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":48944,"score_linear":83622.0,"score_lmetric":132566},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":7680,"new_prefill":41264,"score_linear":91173.0,"score_lmetric":113163},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":48944,"score_linear":257663.0,"score_lmetric":148257},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":87687,"ongoing_decode_tokens":87687,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":7680,"new_prefill":41264,"score_linear":80007.0,"score_lmetric":41264},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":48128,"new_prefill":816,"score_linear":-32845.0,"score_lmetric":816},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":11264,"new_prefill":37680,"score_linear":-11264.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":48944,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-32845.0,"chosen_score_lmetric":816,"t_first_token":339212.207673881,"t_first_token_unix":1779770077.2216172,"t_done":339217.886931583,"t_done_unix":1779770082.900875},{"request_id":"1394167:2:1395165:728","session_id":"1394167","input_length":4863,"t_proxy_recv":339217.540233934,"t_decision_unix":1779770082.5541751,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":4608,"affinity_cache_ratio":0.9475632325724861,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (255 < 8000)","cache_hit":4608,"estimated_new_tokens":255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":91673,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":4863,"score_linear":91673.0,"score_lmetric":9726},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":4863,"score_linear":218746.0,"score_lmetric":366699},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62404,"ongoing_decode_tokens":38252,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":0,"new_prefill":4863,"score_linear":62404.0,"score_lmetric":40313},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":4863,"score_linear":257663.0,"score_lmetric":16014},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":4863,"score_linear":106143.0,"score_lmetric":22942},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":4863,"score_linear":64227.0,"score_lmetric":9726},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":4608,"new_prefill":255,"score_linear":-4608.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":4863,"score_linear":982.0,"score_lmetric":5845}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":339217.583507733,"t_first_token_unix":1779770082.5974507,"t_done":339218.184778737,"t_done_unix":1779770083.1987221},{"request_id":"1342921:4:1389290:702","session_id":"1342921","input_length":70990,"t_proxy_recv":339198.115335251,"t_decision_unix":1779770063.129277,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":0,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9736582617270038,"affinity_num_requests":4,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":70990,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":70990,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":126340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":70990,"score_linear":126340.0,"score_lmetric":70990},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":70990,"score_linear":98853.0,"score_lmetric":142889},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":0,"new_prefill":70990,"score_linear":315319.0,"score_lmetric":285860},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":148720,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":70990,"score_linear":148720.0,"score_lmetric":141980},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":155159,"ongoing_decode_tokens":155159,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1809,"cache_hit":69120,"new_prefill":1870,"score_linear":86039.0,"score_lmetric":7480},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":70990,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":70990,"score_linear":123039.0,"score_lmetric":246746}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339215.606111884,"t_first_token_unix":1779770080.6200554,"t_done":339218.461568426,"t_done_unix":1779770083.4755123},{"request_id":"1390240:3:1395387:730","session_id":"1390240","input_length":10890,"t_proxy_recv":339218.398548126,"t_decision_unix":1779770083.4124858,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":7168,"affinity_cache_ratio":0.65821854912764,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3722 < 8000)","cache_hit":7168,"estimated_new_tokens":3722,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":70990,"ongoing_decode_tokens":70990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1502,"cache_hit":0,"new_prefill":10890,"score_linear":70990.0,"score_lmetric":10890},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":10890,"score_linear":218746.0,"score_lmetric":384780},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":69296,"ongoing_decode_tokens":38252,"pending_prefill_tokens":37734,"num_requests":2,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":0,"new_prefill":10890,"score_linear":69296.0,"score_lmetric":97248},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":10890,"score_linear":257663.0,"score_lmetric":34095},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":10890,"score_linear":106143.0,"score_lmetric":28969},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1910,"cache_hit":0,"new_prefill":10890,"score_linear":15283.0,"score_lmetric":10890},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":7168,"new_prefill":3722,"score_linear":-7168.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":10890,"score_linear":982.0,"score_lmetric":11872}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":339218.766965236,"t_first_token_unix":1779770083.7809086,"t_done":339219.343408933,"t_done_unix":1779770084.3573523},{"request_id":"1378979:1:1378979:633","session_id":"1378979","input_length":15283,"t_proxy_recv":339160.644935289,"t_decision_unix":1779770025.658877,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":15283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1330,"cache_hit":0,"new_prefill":15283,"score_linear":35888.0,"score_lmetric":15283},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":15283,"score_linear":126340.0,"score_lmetric":141623},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":15283,"score_linear":89568.0,"score_lmetric":94778},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":378027,"ongoing_decode_tokens":230647,"pending_prefill_tokens":4875,"num_requests":7,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":14848,"new_prefill":435,"score_linear":363179.0,"score_lmetric":37170},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":15283,"score_linear":191437.0,"score_lmetric":239636},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19806,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":15283,"score_linear":19806.0,"score_lmetric":15283},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":15283,"score_linear":143943.0,"score_lmetric":123892}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339162.00865374,"t_first_token_unix":1779770027.0225973,"t_done":339219.725214554,"t_done_unix":1779770084.7391586},{"request_id":"1394432:2:1395186:729","session_id":"1394432","input_length":6892,"t_proxy_recv":339217.602852316,"t_decision_unix":1779770082.6167939,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":4608,"affinity_cache_ratio":0.6686012768427162,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2284 < 8000)","cache_hit":4608,"estimated_new_tokens":2284,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":91673,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":6892,"score_linear":91673.0,"score_lmetric":13784},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":6892,"score_linear":218746.0,"score_lmetric":372786},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62404,"ongoing_decode_tokens":38252,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":4608,"new_prefill":2284,"score_linear":57796.0,"score_lmetric":37734},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":6892,"score_linear":257663.0,"score_lmetric":22101},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":6892,"score_linear":106143.0,"score_lmetric":24971},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":6892,"score_linear":64227.0,"score_lmetric":13784},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":4863,"ongoing_decode_tokens":4863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":6892,"score_linear":4863.0,"score_lmetric":6892},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":6892,"score_linear":982.0,"score_lmetric":7874}],"chosen_score_linear":57796.0,"chosen_score_lmetric":37734,"t_first_token":339221.385105503,"t_first_token_unix":1779770086.3990488,"t_done":339221.887253909,"t_done_unix":1779770086.9011972},{"request_id":"1394696:1:1394696:727","session_id":"1394696","input_length":35450,"t_proxy_recv":339216.045537284,"t_decision_unix":1779770081.0594785,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":0,"estimated_new_tokens":35450,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":91673,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":35450,"score_linear":91673.0,"score_lmetric":70900},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":35450,"score_linear":218746.0,"score_lmetric":458460},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26954,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":0,"new_prefill":35450,"score_linear":26954.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":35450,"score_linear":257663.0,"score_lmetric":107775},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":35450,"score_linear":106143.0,"score_lmetric":53529},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":35450,"score_linear":64227.0,"score_lmetric":70900},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":35450,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":35450,"score_linear":982.0,"score_lmetric":36432}],"chosen_score_linear":26954.0,"chosen_score_lmetric":0,"t_first_token":339221.380990393,"t_first_token_unix":1779770086.3949344,"t_done":339222.043874388,"t_done_unix":1779770087.057818},{"request_id":"1268861:7:1374579:614","session_id":"1268861","input_length":41508,"t_proxy_recv":339145.492697435,"t_decision_unix":1779770010.5066385,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":36864,"affinity_cache_ratio":0.8881179531656548,"affinity_num_requests":6,"avg_num_requests":2.5,"fallback_score":42918,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4644 < 8000)","cache_hit":36864,"estimated_new_tokens":4644,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32113,"ongoing_decode_tokens":0,"pending_prefill_tokens":18497,"num_requests":2,"active_p_offloads":0,"cached_blocks":1233,"cache_hit":0,"new_prefill":41508,"score_linear":32113.0,"score_lmetric":120010},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":41508,"score_linear":126340.0,"score_lmetric":167848},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":41508,"score_linear":89568.0,"score_lmetric":121003},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":336145,"ongoing_decode_tokens":250361,"pending_prefill_tokens":2509,"num_requests":6,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":36864,"new_prefill":4644,"score_linear":299281.0,"score_lmetric":42918},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":209467,"ongoing_decode_tokens":18301,"pending_prefill_tokens":6211,"num_requests":5,"active_p_offloads":0,"cached_blocks":1427,"cache_hit":0,"new_prefill":41508,"score_linear":209467.0,"score_lmetric":238595},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":73469,"ongoing_decode_tokens":26705,"pending_prefill_tokens":11948,"num_requests":2,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":41508,"score_linear":73469.0,"score_lmetric":106912},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":100108,"ongoing_decode_tokens":80246,"pending_prefill_tokens":19862,"num_requests":2,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":41508,"score_linear":100108.0,"score_lmetric":122740},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":0,"new_prefill":41508,"score_linear":105902.0,"score_lmetric":86994}],"chosen_score_linear":299281.0,"chosen_score_lmetric":42918,"t_first_token":339169.9130681,"t_first_token_unix":1779770034.9270113,"t_done":339222.892951235,"t_done_unix":1779770087.9068952},{"request_id":"1277428:7:1372183:602","session_id":"1277428","input_length":38252,"t_proxy_recv":339190.485846721,"t_decision_unix":1779770055.4997883,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":2,"affinity_cache_hit":36352,"affinity_cache_ratio":0.9503293945414619,"affinity_num_requests":4,"avg_num_requests":1.875,"fallback_score":-20844,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":17408,"estimated_new_tokens":20844,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1455,"cache_hit":0,"new_prefill":38252,"score_linear":50793.0,"score_lmetric":60885},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":4608,"new_prefill":33644,"score_linear":121732.0,"score_lmetric":159984},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":17408,"new_prefill":20844,"score_linear":-28706.0,"score_lmetric":-20844},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":315319,"ongoing_decode_tokens":217052,"pending_prefill_tokens":475,"num_requests":4,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":36352,"new_prefill":1900,"score_linear":278967.0,"score_lmetric":9500},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":148720,"ongoing_decode_tokens":63536,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":38252,"score_linear":148720.0,"score_lmetric":122968},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":224624,"ongoing_decode_tokens":132149,"pending_prefill_tokens":12091,"num_requests":5,"active_p_offloads":0,"cached_blocks":1803,"cache_hit":0,"new_prefill":38252,"score_linear":224624.0,"score_lmetric":251715},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":25226,"ongoing_decode_tokens":25226,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1548,"cache_hit":0,"new_prefill":38252,"score_linear":25226.0,"score_lmetric":38252},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":123039,"ongoing_decode_tokens":0,"pending_prefill_tokens":52383,"num_requests":2,"active_p_offloads":0,"cached_blocks":1579,"cache_hit":0,"new_prefill":38252,"score_linear":123039.0,"score_lmetric":181270}],"chosen_score_linear":-28706.0,"chosen_score_lmetric":-20844,"t_first_token":339196.355108966,"t_first_token_unix":1779770061.3690526,"t_done":339223.784601625,"t_done_unix":1779770088.7985451},{"request_id":"1305906:3:1396717:733","session_id":"1305906","input_length":17047,"t_proxy_recv":339222.92761651,"t_decision_unix":1779770087.9415576,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":16384,"affinity_cache_ratio":0.9611075262509532,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (663 < 8000)","cache_hit":16384,"estimated_new_tokens":663,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":17047,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":17047,"score_linear":218746.0,"score_lmetric":403251},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26954,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2262,"cache_hit":0,"new_prefill":17047,"score_linear":26954.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":216155,"ongoing_decode_tokens":216155,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1274,"cache_hit":0,"new_prefill":17047,"score_linear":216155.0,"score_lmetric":34094},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":17047,"score_linear":106143.0,"score_lmetric":35126},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21241,"ongoing_decode_tokens":21241,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1945,"cache_hit":16384,"new_prefill":663,"score_linear":4857.0,"score_lmetric":663},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":17047,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":17047,"score_linear":982.0,"score_lmetric":18029}],"chosen_score_linear":4857.0,"chosen_score_lmetric":663,"t_first_token":339224.565778601,"t_first_token_unix":1779770089.5797217,"t_done":339225.539184017,"t_done_unix":1779770090.5531278},{"request_id":"1395933:1:1395933:732","session_id":"1395933","input_length":21241,"t_proxy_recv":339220.339116078,"t_decision_unix":1779770085.353057,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":11776,"estimated_new_tokens":9465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":11776,"new_prefill":9465,"score_linear":-11776.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":21241,"score_linear":218746.0,"score_lmetric":415833},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":69296,"ongoing_decode_tokens":38252,"pending_prefill_tokens":37734,"num_requests":2,"active_p_offloads":0,"cached_blocks":2189,"cache_hit":0,"new_prefill":21241,"score_linear":69296.0,"score_lmetric":117950},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":257663,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":21241,"score_linear":257663.0,"score_lmetric":63723},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":21241,"score_linear":106143.0,"score_lmetric":39320},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1945,"cache_hit":11776,"new_prefill":9465,"score_linear":-11776.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":11776,"new_prefill":9465,"score_linear":-11776.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":21241,"score_linear":982.0,"score_lmetric":22223}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":339222.578369922,"t_first_token_unix":1779770087.5923133,"t_done":339225.637850933,"t_done_unix":1779770090.6517942},{"request_id":"1360649:1:1360649:527","session_id":"1360649","input_length":117888,"t_proxy_recv":339096.601465217,"t_decision_unix":1779769961.6153994,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":117888,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156948,"ongoing_decode_tokens":0,"pending_prefill_tokens":123156,"num_requests":2,"active_p_offloads":0,"cached_blocks":991,"cache_hit":0,"new_prefill":117888,"score_linear":156948.0,"score_lmetric":482088},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":130877,"ongoing_decode_tokens":130877,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1148,"cache_hit":0,"new_prefill":117888,"score_linear":130877.0,"score_lmetric":235776},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58266,"ongoing_decode_tokens":69564,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":117888,"score_linear":58266.0,"score_lmetric":117888},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1254,"cache_hit":0,"new_prefill":117888,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1222,"cache_hit":0,"new_prefill":117888,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1309,"cache_hit":0,"new_prefill":117888,"score_linear":83526.0,"score_lmetric":201414},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1147,"cache_hit":0,"new_prefill":117888,"score_linear":80246.0,"score_lmetric":117888},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":34694,"num_requests":1,"active_p_offloads":0,"cached_blocks":1306,"cache_hit":0,"new_prefill":117888,"score_linear":58758.0,"score_lmetric":152582}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339143.701528998,"t_first_token_unix":1779770008.7154732,"t_done":339226.501802398,"t_done_unix":1779770091.515746},{"request_id":"1398145:1:1398145:739","session_id":"1398145","input_length":3213,"t_proxy_recv":339228.088595646,"t_decision_unix":1779770093.102537,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3213 < 8000)","cache_hit":0,"estimated_new_tokens":3213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":3213,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":3213,"score_linear":218746.0,"score_lmetric":361749},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1836,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":3213,"score_linear":1836.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":217319,"ongoing_decode_tokens":217319,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":0,"new_prefill":3213,"score_linear":217319.0,"score_lmetric":6426},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":3213,"score_linear":106143.0,"score_lmetric":3213},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":3213,"score_linear":93040.0,"score_lmetric":95741},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":16550,"ongoing_decode_tokens":0,"pending_prefill_tokens":16550,"num_requests":1,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":3213,"score_linear":16550.0,"score_lmetric":19763},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":3213,"score_linear":982.0,"score_lmetric":4195}],"chosen_score_linear":1836.0,"chosen_score_lmetric":0,"t_first_token":339228.292602316,"t_first_token_unix":1779770093.3065548,"t_done":339228.439167962,"t_done_unix":1779770093.453111},{"request_id":"1398213:1:1398213:742","session_id":"1398213","input_length":3891,"t_proxy_recv":339228.229909879,"t_decision_unix":1779770093.2438514,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3891 < 8000)","cache_hit":0,"estimated_new_tokens":3891,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":3891,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":3891,"score_linear":218746.0,"score_lmetric":363783},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":5049,"ongoing_decode_tokens":13134,"pending_prefill_tokens":3213,"num_requests":1,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":3891,"score_linear":5049.0,"score_lmetric":7104},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":217319,"ongoing_decode_tokens":217319,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":0,"new_prefill":3891,"score_linear":217319.0,"score_lmetric":7782},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":3891,"score_linear":106143.0,"score_lmetric":3891},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":3891,"score_linear":103776.0,"score_lmetric":193830},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47774,"ongoing_decode_tokens":0,"pending_prefill_tokens":19102,"num_requests":2,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":3891,"score_linear":47774.0,"score_lmetric":45986},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":3891,"score_linear":982.0,"score_lmetric":4873}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339228.474510796,"t_first_token_unix":1779770093.488454,"t_done":339228.797147624,"t_done_unix":1779770093.811091},{"request_id":"1360649:2:1367022:561","session_id":"1360649","input_length":119052,"t_proxy_recv":339226.527266533,"t_decision_unix":1779770091.541208,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":118272,"affinity_cache_ratio":0.9934482411047273,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (780 < 8000)","cache_hit":118272,"estimated_new_tokens":780,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":119052,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":1024,"new_prefill":118028,"score_linear":217722.0,"score_lmetric":706194},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1836,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":119052,"score_linear":1836.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":118272,"new_prefill":780,"score_linear":-20005.0,"score_lmetric":780},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":119052,"score_linear":106143.0,"score_lmetric":119052},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":4096,"new_prefill":114956,"score_linear":88944.0,"score_lmetric":207484},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":119052,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":119052,"score_linear":982.0,"score_lmetric":120034}],"chosen_score_linear":-20005.0,"chosen_score_lmetric":780,"t_first_token":339227.117208546,"t_first_token_unix":1779770092.1311522,"t_done":339229.218896059,"t_done_unix":1779770094.2328393},{"request_id":"1397840:1:1397840:737","session_id":"1397840","input_length":16550,"t_proxy_recv":339227.019471405,"t_decision_unix":1779770092.0334127,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":16550,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":16550,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":16550,"score_linear":218746.0,"score_lmetric":401760},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1836,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":16550,"score_linear":1836.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":217319,"ongoing_decode_tokens":98267,"pending_prefill_tokens":780,"num_requests":2,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":0,"new_prefill":16550,"score_linear":217319.0,"score_lmetric":34660},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":16550,"score_linear":106143.0,"score_lmetric":16550},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":16550,"score_linear":93040.0,"score_lmetric":109078},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":16550,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":16550,"score_linear":982.0,"score_lmetric":17532}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339228.59603726,"t_first_token_unix":1779770093.6099803,"t_done":339229.523838662,"t_done_unix":1779770094.537782},{"request_id":"1398594:1:1398594:744","session_id":"1398594","input_length":7877,"t_proxy_recv":339229.457891169,"t_decision_unix":1779770094.4718323,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7877 < 8000)","cache_hit":0,"estimated_new_tokens":7877,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1648,"cache_hit":0,"new_prefill":7877,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":7877,"score_linear":218746.0,"score_lmetric":375741},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":7877,"score_linear":65094.0,"score_lmetric":19935},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":7877,"score_linear":98267.0,"score_lmetric":7877},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":7877,"score_linear":106143.0,"score_lmetric":7877},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":7877,"score_linear":103776.0,"score_lmetric":201802},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47774,"ongoing_decode_tokens":47774,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":7877,"score_linear":47774.0,"score_lmetric":15754},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":7877,"score_linear":982.0,"score_lmetric":8859}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339230.004902555,"t_first_token_unix":1779770095.0188463,"t_done":339230.122578556,"t_done_unix":1779770095.136522},{"request_id":"1355484:14:1398178:740","session_id":"1355484","input_length":31224,"t_proxy_recv":339228.167923426,"t_decision_unix":1779770093.181865,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9182679989751473,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2552 < 8000)","cache_hit":28672,"estimated_new_tokens":2552,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":31224,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":31224,"score_linear":218746.0,"score_lmetric":445782},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":5049,"ongoing_decode_tokens":13134,"pending_prefill_tokens":3213,"num_requests":1,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":31224,"score_linear":5049.0,"score_lmetric":34437},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":217319,"ongoing_decode_tokens":217319,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":19456,"new_prefill":11768,"score_linear":197863.0,"score_lmetric":23536},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":31224,"score_linear":106143.0,"score_lmetric":31224},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":31224,"score_linear":93040.0,"score_lmetric":123752},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":16550,"ongoing_decode_tokens":0,"pending_prefill_tokens":16550,"num_requests":1,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":28672,"new_prefill":2552,"score_linear":-12122.0,"score_lmetric":19102},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":12800,"new_prefill":18424,"score_linear":-11818.0,"score_lmetric":19406}],"chosen_score_linear":-12122.0,"chosen_score_lmetric":19102,"t_first_token":339229.200240989,"t_first_token_unix":1779770094.2141845,"t_done":339230.664558909,"t_done_unix":1779770095.6785023},{"request_id":"1399948:1:1399948:748","session_id":"1399948","input_length":4991,"t_proxy_recv":339234.268924206,"t_decision_unix":1779770099.2828655,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4991 < 8000)","cache_hit":0,"estimated_new_tokens":4991,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1663,"cache_hit":0,"new_prefill":4991,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":4991,"score_linear":218746.0,"score_lmetric":367083},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":4991,"score_linear":65094.0,"score_lmetric":17049},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":4991,"score_linear":98267.0,"score_lmetric":4991},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":4991,"score_linear":106143.0,"score_lmetric":4991},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":4991,"score_linear":103776.0,"score_lmetric":196030},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1678,"cache_hit":0,"new_prefill":4991,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":4991,"score_linear":982.0,"score_lmetric":5973}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339234.589229437,"t_first_token_unix":1779770099.603173,"t_done":339235.262439845,"t_done_unix":1779770100.2763832},{"request_id":"1400379:1:1400379:751","session_id":"1400379","input_length":3707,"t_proxy_recv":339235.584298767,"t_decision_unix":1779770100.59824,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3707 < 8000)","cache_hit":0,"estimated_new_tokens":3707,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1672,"cache_hit":0,"new_prefill":3707,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":3707,"score_linear":218746.0,"score_lmetric":112365},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":3707,"score_linear":65094.0,"score_lmetric":15765},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":3707,"score_linear":98267.0,"score_lmetric":3707},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":3707,"score_linear":106143.0,"score_lmetric":3707},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":3707,"score_linear":103776.0,"score_lmetric":193462},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57909,"ongoing_decode_tokens":0,"pending_prefill_tokens":10293,"num_requests":2,"active_p_offloads":0,"cached_blocks":1678,"cache_hit":0,"new_prefill":3707,"score_linear":57909.0,"score_lmetric":28000},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":3707,"score_linear":982.0,"score_lmetric":4689}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339235.819020609,"t_first_token_unix":1779770100.8329647,"t_done":339236.105643404,"t_done_unix":1779770101.1195872},{"request_id":"1397840:2:1400216:749","session_id":"1397840","input_length":26440,"t_proxy_recv":339235.068917361,"t_decision_unix":1779770100.0828583,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":16384,"affinity_cache_ratio":0.6196671709531013,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":16384,"estimated_new_tokens":10056,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":4991,"ongoing_decode_tokens":4991,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1663,"cache_hit":0,"new_prefill":26440,"score_linear":4991.0,"score_lmetric":26440},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":26440,"score_linear":218746.0,"score_lmetric":180564},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":26440,"score_linear":65094.0,"score_lmetric":38498},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":26440,"score_linear":98267.0,"score_lmetric":26440},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":26440,"score_linear":106143.0,"score_lmetric":26440},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":26440,"score_linear":103776.0,"score_lmetric":238928},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1678,"cache_hit":16384,"new_prefill":10056,"score_linear":-16384.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":26440,"score_linear":982.0,"score_lmetric":27422}],"chosen_score_linear":-16384.0,"chosen_score_lmetric":0,"t_first_token":339236.794757866,"t_first_token_unix":1779770101.8087018,"t_done":339237.066350462,"t_done_unix":1779770102.080294},{"request_id":"1355484:15:1400281:750","session_id":"1355484","input_length":31469,"t_proxy_recv":339235.273680294,"t_decision_unix":1779770100.287622,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9924687787981823,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (237 < 8000)","cache_hit":31232,"estimated_new_tokens":237,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1672,"cache_hit":0,"new_prefill":31469,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":31469,"score_linear":218746.0,"score_lmetric":195651},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":31469,"score_linear":65094.0,"score_lmetric":43527},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":19456,"new_prefill":12013,"score_linear":78811.0,"score_lmetric":12013},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":31469,"score_linear":106143.0,"score_lmetric":31469},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":31469,"score_linear":103776.0,"score_lmetric":248986},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26440,"ongoing_decode_tokens":0,"pending_prefill_tokens":10056,"num_requests":1,"active_p_offloads":0,"cached_blocks":1678,"cache_hit":31232,"new_prefill":237,"score_linear":-4792.0,"score_lmetric":10293},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":12800,"new_prefill":18669,"score_linear":-11818.0,"score_lmetric":19651}],"chosen_score_linear":-4792.0,"chosen_score_lmetric":10293,"t_first_token":339236.818097909,"t_first_token_unix":1779770101.8320417,"t_done":339237.38737459,"t_done_unix":1779770102.4013178},{"request_id":"1394428:1:1394428:724","session_id":"1394428","input_length":982,"t_proxy_recv":339214.996921417,"t_decision_unix":1779770080.0108628,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (982 < 8000)","cache_hit":0,"estimated_new_tokens":982,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":512,"new_prefill":470,"score_linear":91161.0,"score_lmetric":144350},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":982,"score_linear":218746.0,"score_lmetric":355056},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26954,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2180,"cache_hit":0,"new_prefill":982,"score_linear":26954.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":982,"score_linear":257663.0,"score_lmetric":4371},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":982,"score_linear":106143.0,"score_lmetric":19061},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":982,"score_linear":64227.0,"score_lmetric":1964},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28665,"ongoing_decode_tokens":28665,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":982,"score_linear":28665.0,"score_lmetric":982},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":982,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339240.123280617,"t_first_token_unix":1779770105.1372237,"t_done":339240.183982852,"t_done_unix":1779770105.1979263},{"request_id":"1313181:7:1327489:383","session_id":"1313181","input_length":121811,"t_proxy_recv":339206.365632403,"t_decision_unix":1779770071.379574,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61952,"affinity_cache_ratio":0.5085911781366215,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":61952,"estimated_new_tokens":59859,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":121811,"score_linear":91673.0,"score_lmetric":387032},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":28307,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":121811,"score_linear":28307.0,"score_lmetric":150118},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":121811,"score_linear":98853.0,"score_lmetric":193710},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":12800,"new_prefill":109011,"score_linear":244863.0,"score_lmetric":328458},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153371,"ongoing_decode_tokens":153371,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1673,"cache_hit":0,"new_prefill":121811,"score_linear":153371.0,"score_lmetric":243622},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":173308,"ongoing_decode_tokens":89904,"pending_prefill_tokens":11724,"num_requests":4,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":0,"new_prefill":121811,"score_linear":173308.0,"score_lmetric":534140},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":67325,"ongoing_decode_tokens":38943,"pending_prefill_tokens":1758,"num_requests":3,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":12800,"new_prefill":109011,"score_linear":54525.0,"score_lmetric":332307},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":61952,"new_prefill":59859,"score_linear":-61952.0,"score_lmetric":0}],"chosen_score_linear":-61952.0,"chosen_score_lmetric":0,"t_first_token":339240.122153252,"t_first_token_unix":1779770105.1360967,"t_done":339241.396761819,"t_done_unix":1779770106.4107056},{"request_id":"1397223:1:1397223:734","session_id":"1397223","input_length":13134,"t_proxy_recv":339224.572785641,"t_decision_unix":1779770089.5867271,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-13134,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":13134,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":13134,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":13134,"score_linear":218746.0,"score_lmetric":391512},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":13134,"score_linear":-11298.0,"score_lmetric":-13134},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":216155,"ongoing_decode_tokens":216155,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1274,"cache_hit":0,"new_prefill":13134,"score_linear":216155.0,"score_lmetric":26268},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":0,"pending_prefill_tokens":18079,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":13134,"score_linear":106143.0,"score_lmetric":31213},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":38288,"ongoing_decode_tokens":38288,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1945,"cache_hit":0,"new_prefill":13134,"score_linear":38288.0,"score_lmetric":26268},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":13134,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":13134,"score_linear":982.0,"score_lmetric":14116}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-13134,"t_first_token":339225.684153668,"t_first_token_unix":1779770090.6980968,"t_done":339241.495377915,"t_done_unix":1779770106.5093215},{"request_id":"1253743:8:1398344:743","session_id":"1253743","input_length":63258,"t_proxy_recv":339228.606233179,"t_decision_unix":1779770093.6201746,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":51200,"affinity_cache_ratio":0.8093837933542003,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=49664 chosen=51200)","cache_hit":51200,"estimated_new_tokens":12058,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":3891,"ongoing_decode_tokens":3891,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":63258,"score_linear":3891.0,"score_lmetric":63258},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":4608,"new_prefill":58650,"score_linear":214138.0,"score_lmetric":528060},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1836,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":51200,"new_prefill":12058,"score_linear":-49364.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":217319,"ongoing_decode_tokens":217319,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":49664,"new_prefill":13594,"score_linear":167655.0,"score_lmetric":27188},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":63258,"score_linear":106143.0,"score_lmetric":63258},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":63258,"score_linear":103776.0,"score_lmetric":312564},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47774,"ongoing_decode_tokens":16550,"pending_prefill_tokens":2552,"num_requests":2,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":63258,"score_linear":47774.0,"score_lmetric":131620},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":63258,"score_linear":982.0,"score_lmetric":64240}],"chosen_score_linear":-49364.0,"chosen_score_lmetric":0,"t_first_token":339241.130526229,"t_first_token_unix":1779770106.1444697,"t_done":339242.276432248,"t_done_unix":1779770107.290376},{"request_id":"1363943:10:1401002:755","session_id":"1363943","input_length":24442,"t_proxy_recv":339237.981455731,"t_decision_unix":1779770102.9953969,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9845348171180754,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (378 < 8000)","cache_hit":24064,"estimated_new_tokens":378,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":79523,"ongoing_decode_tokens":11414,"pending_prefill_tokens":17421,"num_requests":2,"active_p_offloads":0,"cached_blocks":1679,"cache_hit":8704,"new_prefill":15738,"score_linear":70819.0,"score_lmetric":66318},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":24442,"score_linear":218746.0,"score_lmetric":174570},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":4096,"new_prefill":20346,"score_linear":60998.0,"score_lmetric":32404},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":24442,"score_linear":98267.0,"score_lmetric":24442},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":4096,"new_prefill":20346,"score_linear":102047.0,"score_lmetric":20346},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":4096,"new_prefill":20346,"score_linear":99680.0,"score_lmetric":226740},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":24064,"new_prefill":378,"score_linear":-24064.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":24442,"score_linear":982.0,"score_lmetric":25424}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":339238.058626238,"t_first_token_unix":1779770103.0725696,"t_done":339243.923767413,"t_done_unix":1779770108.9377105},{"request_id":"1397840:3:1402568:761","session_id":"1397840","input_length":28603,"t_proxy_recv":339243.416795383,"t_decision_unix":1779770108.4307363,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9129112330874384,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2491 < 8000)","cache_hit":26112,"estimated_new_tokens":2491,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83386,"ongoing_decode_tokens":11414,"pending_prefill_tokens":17700,"num_requests":3,"active_p_offloads":0,"cached_blocks":1679,"cache_hit":0,"new_prefill":28603,"score_linear":83386.0,"score_lmetric":138909},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":28603,"score_linear":218746.0,"score_lmetric":187053},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":28603,"score_linear":-11298.0,"score_lmetric":-28603},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":28603,"score_linear":98267.0,"score_lmetric":28603},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":28603,"score_linear":106143.0,"score_lmetric":28603},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":28603,"score_linear":166406.0,"score_lmetric":405315},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":56774,"ongoing_decode_tokens":56774,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":26112,"new_prefill":2491,"score_linear":30662.0,"score_lmetric":4982},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-59793,"ongoing_decode_tokens":62018,"pending_prefill_tokens":-59859,"num_requests":0,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":28603,"score_linear":-59793.0,"score_lmetric":0}],"chosen_score_linear":30662.0,"chosen_score_lmetric":4982,"t_first_token":339243.919865308,"t_first_token_unix":1779770108.9338088,"t_done":339244.146433753,"t_done_unix":1779770109.1603773},{"request_id":"1400379:2:1401201:757","session_id":"1400379","input_length":3863,"t_proxy_recv":339238.69999868,"t_decision_unix":1779770103.7139397,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":3584,"affinity_cache_ratio":0.92777633963241,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (279 < 8000)","cache_hit":3584,"estimated_new_tokens":279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":79523,"ongoing_decode_tokens":11414,"pending_prefill_tokens":17421,"num_requests":2,"active_p_offloads":0,"cached_blocks":1679,"cache_hit":3584,"new_prefill":279,"score_linear":75939.0,"score_lmetric":35400},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":3863,"score_linear":218746.0,"score_lmetric":112833},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":3863,"score_linear":65094.0,"score_lmetric":15921},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":3863,"score_linear":98267.0,"score_lmetric":3863},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":3863,"score_linear":106143.0,"score_lmetric":3863},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":3863,"score_linear":166406.0,"score_lmetric":331095},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":0,"new_prefill":3863,"score_linear":24442.0,"score_lmetric":3863},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":3863,"score_linear":982.0,"score_lmetric":4845}],"chosen_score_linear":75939.0,"chosen_score_lmetric":35400,"t_first_token":339243.700632561,"t_first_token_unix":1779770108.7145798,"t_done":339244.407125353,"t_done_unix":1779770109.4210694},{"request_id":"1398213:2:1400590:753","session_id":"1398213","input_length":11414,"t_proxy_recv":339236.374535203,"t_decision_unix":1779770101.3884764,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":3584,"affinity_cache_ratio":0.3140003504468197,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7830 < 8000)","cache_hit":3584,"estimated_new_tokens":7830,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1679,"cache_hit":3584,"new_prefill":7830,"score_linear":-3584.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":11414,"score_linear":218746.0,"score_lmetric":135486},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":11414,"score_linear":65094.0,"score_lmetric":23472},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":11414,"score_linear":98267.0,"score_lmetric":11414},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":11414,"score_linear":106143.0,"score_lmetric":11414},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":11414,"score_linear":103776.0,"score_lmetric":208876},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57909,"ongoing_decode_tokens":0,"pending_prefill_tokens":10293,"num_requests":2,"active_p_offloads":0,"cached_blocks":1678,"cache_hit":0,"new_prefill":11414,"score_linear":57909.0,"score_lmetric":43414},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":11414,"score_linear":982.0,"score_lmetric":12396}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":339237.05759704,"t_first_token_unix":1779770102.07154,"t_done":339244.427661885,"t_done_unix":1779770109.4416053},{"request_id":"1355484:16:1401320:758","session_id":"1355484","input_length":32332,"t_proxy_recv":339239.155481776,"t_decision_unix":1779770104.169423,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9659779784733391,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1100 < 8000)","cache_hit":31232,"estimated_new_tokens":1100,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83386,"ongoing_decode_tokens":11414,"pending_prefill_tokens":17700,"num_requests":3,"active_p_offloads":0,"cached_blocks":1679,"cache_hit":0,"new_prefill":32332,"score_linear":83386.0,"score_lmetric":150096},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":32332,"score_linear":218746.0,"score_lmetric":198240},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":32332,"score_linear":65094.0,"score_lmetric":44390},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":19456,"new_prefill":12876,"score_linear":78811.0,"score_lmetric":12876},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":32332,"score_linear":106143.0,"score_lmetric":32332},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":32332,"score_linear":166406.0,"score_lmetric":416502},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":31232,"new_prefill":1100,"score_linear":-6790.0,"score_lmetric":1100},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":12800,"new_prefill":19532,"score_linear":-11818.0,"score_lmetric":20514}],"chosen_score_linear":-6790.0,"chosen_score_lmetric":1100,"t_first_token":339239.433240308,"t_first_token_unix":1779770104.4471843,"t_done":339244.714359661,"t_done_unix":1779770109.7283032},{"request_id":"1313181:7:1382851:662","session_id":"1313181","input_length":62018,"t_proxy_recv":339241.405921476,"t_decision_unix":1779770106.419863,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61952,"affinity_cache_ratio":0.9989357928343384,"affinity_num_requests":-1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (66 < 8000)","cache_hit":61952,"estimated_new_tokens":66,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":83386,"ongoing_decode_tokens":11414,"pending_prefill_tokens":17700,"num_requests":3,"active_p_offloads":0,"cached_blocks":1679,"cache_hit":0,"new_prefill":62018,"score_linear":83386.0,"score_lmetric":239154},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":62018,"score_linear":218746.0,"score_lmetric":287298},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":76392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":62018,"score_linear":65094.0,"score_lmetric":62018},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":12800,"new_prefill":49218,"score_linear":85467.0,"score_lmetric":49218},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":62018,"score_linear":106143.0,"score_lmetric":62018},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":62018,"score_linear":166406.0,"score_lmetric":505560},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":56774,"ongoing_decode_tokens":56774,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":12800,"new_prefill":49218,"score_linear":43974.0,"score_lmetric":98436},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":61952,"new_prefill":66,"score_linear":-183763.0,"score_lmetric":59793}],"chosen_score_linear":-183763.0,"chosen_score_lmetric":59793,"t_first_token":339241.4997307,"t_first_token_unix":1779770106.5136738,"t_done":339244.785406427,"t_done_unix":1779770109.7993505},{"request_id":"1279412:25:1366203:554","session_id":"1279412","input_length":98267,"t_proxy_recv":339162.733505876,"t_decision_unix":1779770027.7474475,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9951662307794071,"affinity_num_requests":5,"avg_num_requests":2.375,"fallback_score":33430,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (475 < 8000)","cache_hit":97792,"estimated_new_tokens":475,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1332,"cache_hit":0,"new_prefill":98267,"score_linear":35888.0,"score_lmetric":98267},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":1,"active_p_offloads":0,"cached_blocks":1256,"cache_hit":0,"new_prefill":98267,"score_linear":126340.0,"score_lmetric":224607},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":89568,"ongoing_decode_tokens":21371,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1741,"cache_hit":0,"new_prefill":98267,"score_linear":89568.0,"score_lmetric":177762},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":273441,"ongoing_decode_tokens":117888,"pending_prefill_tokens":6211,"num_requests":5,"active_p_offloads":0,"cached_blocks":1263,"cache_hit":97792,"new_prefill":475,"score_linear":175649.0,"score_lmetric":33430},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":191437,"ongoing_decode_tokens":63536,"pending_prefill_tokens":44626,"num_requests":4,"active_p_offloads":0,"cached_blocks":1478,"cache_hit":0,"new_prefill":98267,"score_linear":191437.0,"score_lmetric":571572},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":17915,"ongoing_decode_tokens":15283,"pending_prefill_tokens":2632,"num_requests":2,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":98267,"score_linear":17915.0,"score_lmetric":201798},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":55570,"ongoing_decode_tokens":0,"pending_prefill_tokens":15284,"num_requests":3,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":98267,"score_linear":55570.0,"score_lmetric":340653},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":143943,"ongoing_decode_tokens":0,"pending_prefill_tokens":46663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1465,"cache_hit":12800,"new_prefill":85467,"score_linear":131143.0,"score_lmetric":264260}],"chosen_score_linear":175649.0,"chosen_score_lmetric":33430,"t_first_token":339220.239197979,"t_first_token_unix":1779770085.2531424,"t_done":339245.512760284,"t_done_unix":1779770110.526704},{"request_id":"1400379:3:1402972:763","session_id":"1400379","input_length":11216,"t_proxy_recv":339244.912859937,"t_decision_unix":1779770109.926801,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":6,"affinity_cache_hit":3584,"affinity_cache_ratio":0.3195435092724679,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81019,"ongoing_decode_tokens":81019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1694,"cache_hit":3584,"new_prefill":7632,"score_linear":77435.0,"score_lmetric":15264},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":11216,"score_linear":218746.0,"score_lmetric":134892},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":112792,"ongoing_decode_tokens":0,"pending_prefill_tokens":124090,"num_requests":0,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":11216,"score_linear":112792.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":11216,"score_linear":98267.0,"score_lmetric":11216},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":11216,"score_linear":106143.0,"score_lmetric":11216},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":11216,"score_linear":166406.0,"score_lmetric":353154},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1705,"cache_hit":0,"new_prefill":11216,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":11216,"score_linear":-121811.0,"score_lmetric":48643}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339245.814778163,"t_first_token_unix":1779770110.828721,"t_done":339246.048710206,"t_done_unix":1779770111.0626554},{"request_id":"1381387:4:1400724:754","session_id":"1381387","input_length":68109,"t_proxy_recv":339236.950933247,"t_decision_unix":1779770101.964871,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":50688,"affinity_cache_ratio":0.7442188257058538,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":50688,"estimated_new_tokens":17421,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11414,"ongoing_decode_tokens":0,"pending_prefill_tokens":7830,"num_requests":1,"active_p_offloads":0,"cached_blocks":1679,"cache_hit":50688,"new_prefill":17421,"score_linear":-39274.0,"score_lmetric":25251},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":68109,"score_linear":218746.0,"score_lmetric":305571},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":0,"new_prefill":68109,"score_linear":65094.0,"score_lmetric":80167},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":68109,"score_linear":98267.0,"score_lmetric":68109},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":68109,"score_linear":106143.0,"score_lmetric":68109},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":68109,"score_linear":103776.0,"score_lmetric":322266},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":57909,"ongoing_decode_tokens":57909,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1678,"cache_hit":0,"new_prefill":68109,"score_linear":57909.0,"score_lmetric":136218},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":68109,"score_linear":982.0,"score_lmetric":69091}],"chosen_score_linear":-39274.0,"chosen_score_lmetric":25251,"t_first_token":339243.695960255,"t_first_token_unix":1779770108.7099042,"t_done":339246.7862717,"t_done_unix":1779770111.8002155},{"request_id":"1279412:26:1378269:628","session_id":"1279412","input_length":100340,"t_proxy_recv":339245.530391233,"t_decision_unix":1779770110.5443325,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":99840,"affinity_cache_ratio":0.9950169423958541,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (500 < 8000)","cache_hit":99840,"estimated_new_tokens":500,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81019,"ongoing_decode_tokens":81019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1694,"cache_hit":0,"new_prefill":100340,"score_linear":81019.0,"score_lmetric":200680},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":100340,"score_linear":218746.0,"score_lmetric":402264},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":112792,"ongoing_decode_tokens":0,"pending_prefill_tokens":124090,"num_requests":0,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":100340,"score_linear":112792.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1508,"cache_hit":99840,"new_prefill":500,"score_linear":-99840.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":100340,"score_linear":106143.0,"score_lmetric":100340},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":100340,"score_linear":166406.0,"score_lmetric":620526},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":11216,"ongoing_decode_tokens":0,"pending_prefill_tokens":11216,"num_requests":1,"active_p_offloads":0,"cached_blocks":1705,"cache_hit":12800,"new_prefill":87540,"score_linear":-1584.0,"score_lmetric":98756},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":12800,"new_prefill":87540,"score_linear":-134611.0,"score_lmetric":-27681}],"chosen_score_linear":-99840.0,"chosen_score_lmetric":0,"t_first_token":339245.746219741,"t_first_token_unix":1779770110.7601633,"t_done":339247.108901889,"t_done_unix":1779770112.1228456},{"request_id":"1398213:3:1401507:759","session_id":"1398213","input_length":12910,"t_proxy_recv":339244.432332056,"t_decision_unix":1779770109.4462736,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.8725019364833463,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1646 < 8000)","cache_hit":11264,"estimated_new_tokens":1646,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":68109,"ongoing_decode_tokens":68109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1694,"cache_hit":11264,"new_prefill":1646,"score_linear":56845.0,"score_lmetric":1646},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":12910,"score_linear":218746.0,"score_lmetric":139974},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":12910,"score_linear":-11298.0,"score_lmetric":-12910},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":12910,"score_linear":98267.0,"score_lmetric":12910},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":12910,"score_linear":106143.0,"score_lmetric":12910},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":12910,"score_linear":166406.0,"score_lmetric":358236},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32332,"ongoing_decode_tokens":32332,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1702,"cache_hit":0,"new_prefill":12910,"score_linear":32332.0,"score_lmetric":12910},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-59793,"ongoing_decode_tokens":62018,"pending_prefill_tokens":-59859,"num_requests":0,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":12910,"score_linear":-59793.0,"score_lmetric":0}],"chosen_score_linear":56845.0,"chosen_score_lmetric":1646,"t_first_token":339244.657675153,"t_first_token_unix":1779770109.6716187,"t_done":339248.196262054,"t_done_unix":1779770113.2102058},{"request_id":"1403989:1:1403989:767","session_id":"1403989","input_length":1954,"t_proxy_recv":339248.306781905,"t_decision_unix":1779770113.320723,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1954 < 8000)","cache_hit":0,"estimated_new_tokens":1954,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1732,"cache_hit":0,"new_prefill":1954,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":1954,"score_linear":218746.0,"score_lmetric":107106},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":121640,"ongoing_decode_tokens":0,"pending_prefill_tokens":132938,"num_requests":1,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":1954,"score_linear":121640.0,"score_lmetric":134892},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":144686,"ongoing_decode_tokens":100538,"pending_prefill_tokens":2676,"num_requests":2,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":1954,"score_linear":144686.0,"score_lmetric":9260},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":1954,"score_linear":106143.0,"score_lmetric":1954},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":1954,"score_linear":166406.0,"score_lmetric":325368},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":1954,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":1954,"score_linear":-121811.0,"score_lmetric":57905}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339248.434871626,"t_first_token_unix":1779770113.4488158,"t_done":339248.787125174,"t_done_unix":1779770113.8010688},{"request_id":"1279412:27:1380696:646","session_id":"1279412","input_length":100538,"t_proxy_recv":339247.125567205,"t_decision_unix":1779770112.1395087,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":100352,"affinity_cache_ratio":0.9981499532515069,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (186 < 8000)","cache_hit":100352,"estimated_new_tokens":186,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":12910,"ongoing_decode_tokens":12910,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":100538,"score_linear":12910.0,"score_lmetric":100538},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":100538,"score_linear":218746.0,"score_lmetric":402858},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":121640,"ongoing_decode_tokens":0,"pending_prefill_tokens":132938,"num_requests":1,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":100538,"score_linear":121640.0,"score_lmetric":233476},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":100352,"new_prefill":186,"score_linear":-100352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":100538,"score_linear":106143.0,"score_lmetric":100538},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":100538,"score_linear":166406.0,"score_lmetric":621120},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":12800,"new_prefill":87738,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":12800,"new_prefill":87738,"score_linear":-134611.0,"score_lmetric":-27879}],"chosen_score_linear":-100352.0,"chosen_score_lmetric":0,"t_first_token":339247.339841272,"t_first_token_unix":1779770112.3537853,"t_done":339250.532891047,"t_done_unix":1779770115.5468352},{"request_id":"1268861:8:1403753:765","session_id":"1268861","input_length":44148,"t_proxy_recv":339247.560371334,"t_decision_unix":1779770112.5743124,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":41472,"affinity_cache_ratio":0.939385702636586,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2676 < 8000)","cache_hit":41472,"estimated_new_tokens":2676,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":12910,"ongoing_decode_tokens":12910,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":44148,"score_linear":12910.0,"score_lmetric":44148},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":44148,"score_linear":218746.0,"score_lmetric":233688},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":121640,"ongoing_decode_tokens":0,"pending_prefill_tokens":132938,"num_requests":1,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":44148,"score_linear":121640.0,"score_lmetric":177086},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100538,"ongoing_decode_tokens":100538,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":41472,"new_prefill":2676,"score_linear":59066.0,"score_lmetric":2676},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":44148,"score_linear":106143.0,"score_lmetric":44148},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":44148,"score_linear":166406.0,"score_lmetric":451950},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":44148,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":44148,"score_linear":-121811.0,"score_lmetric":15711}],"chosen_score_linear":59066.0,"chosen_score_lmetric":2676,"t_first_token":339248.333303042,"t_first_token_unix":1779770113.3472466,"t_done":339251.895821493,"t_done_unix":1779770116.9097655},{"request_id":"1382089:2:1404847:773","session_id":"1382089","input_length":39203,"t_proxy_recv":339251.328807043,"t_decision_unix":1779770116.3427482,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9664566487258628,"affinity_num_requests":-1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1315 < 8000)","cache_hit":37888,"estimated_new_tokens":1315,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":90517,"ongoing_decode_tokens":69508,"pending_prefill_tokens":13329,"num_requests":2,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":39203,"score_linear":90517.0,"score_lmetric":105064},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":39203,"score_linear":218746.0,"score_lmetric":218853},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":185283,"ongoing_decode_tokens":0,"pending_prefill_tokens":133605,"num_requests":2,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":39203,"score_linear":185283.0,"score_lmetric":345616},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":144914,"ongoing_decode_tokens":144914,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":39203,"score_linear":144914.0,"score_lmetric":78406},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":39203,"score_linear":106143.0,"score_lmetric":39203},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":39203,"score_linear":166406.0,"score_lmetric":437115},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21684,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":39203,"score_linear":21684.0,"score_lmetric":60887},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":37888,"new_prefill":1315,"score_linear":-159699.0,"score_lmetric":58544}],"chosen_score_linear":-159699.0,"chosen_score_lmetric":58544,"t_first_token":339251.714516199,"t_first_token_unix":1779770116.7284598,"t_done":339252.023628781,"t_done_unix":1779770117.0375724},{"request_id":"1279412:28:1382526:657","session_id":"1279412","input_length":100766,"t_proxy_recv":339250.549878909,"t_decision_unix":1779770115.5638206,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":100352,"affinity_cache_ratio":0.9958914713296152,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (414 < 8000)","cache_hit":100352,"estimated_new_tokens":414,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69508,"ongoing_decode_tokens":69508,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":100766,"score_linear":69508.0,"score_lmetric":100766},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":100766,"score_linear":218746.0,"score_lmetric":403542},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":185283,"ongoing_decode_tokens":0,"pending_prefill_tokens":133605,"num_requests":2,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":100766,"score_linear":185283.0,"score_lmetric":468742},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":44148,"ongoing_decode_tokens":44148,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":100352,"new_prefill":414,"score_linear":-56204.0,"score_lmetric":414},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":100766,"score_linear":106143.0,"score_lmetric":100766},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":100766,"score_linear":166406.0,"score_lmetric":621804},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21684,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":12800,"new_prefill":87966,"score_linear":8884.0,"score_lmetric":109650},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":12800,"new_prefill":87966,"score_linear":-134611.0,"score_lmetric":-28107}],"chosen_score_linear":-56204.0,"chosen_score_lmetric":414,"t_first_token":339250.73397429,"t_first_token_unix":1779770115.7479181,"t_done":339253.075090323,"t_done_unix":1779770118.0890343},{"request_id":"1279412:29:1385688:685","session_id":"1279412","input_length":101200,"t_proxy_recv":339253.09206811,"t_decision_unix":1779770118.1060095,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":100864,"affinity_cache_ratio":0.9966798418972332,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (336 < 8000)","cache_hit":100864,"estimated_new_tokens":336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":101200,"score_linear":209924.0,"score_lmetric":423676},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":101200,"score_linear":218746.0,"score_lmetric":404844},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":101200,"score_linear":270721.0,"score_lmetric":716505},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1515,"cache_hit":100864,"new_prefill":336,"score_linear":-100864.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":101200,"score_linear":106143.0,"score_lmetric":101200},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":101200,"score_linear":166406.0,"score_lmetric":623106},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":12800,"new_prefill":88400,"score_linear":8884.0,"score_lmetric":88400},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-77364,"ongoing_decode_tokens":0,"pending_prefill_tokens":-15412,"num_requests":1,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":12800,"new_prefill":88400,"score_linear":-90164.0,"score_lmetric":72988}],"chosen_score_linear":-100864.0,"chosen_score_lmetric":0,"t_first_token":339253.402884284,"t_first_token_unix":1779770118.416828,"t_done":339254.104896188,"t_done_unix":1779770119.1188395},{"request_id":"1279412:30:1387344:694","session_id":"1279412","input_length":101396,"t_proxy_recv":339254.123658354,"t_decision_unix":1779770119.1375997,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":100864,"affinity_cache_ratio":0.9947532447039331,"affinity_num_requests":0,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (532 < 8000)","cache_hit":100864,"estimated_new_tokens":532,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":101396,"score_linear":209924.0,"score_lmetric":424460},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":101396,"score_linear":218746.0,"score_lmetric":405432},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":101396,"score_linear":270721.0,"score_lmetric":717093},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1515,"cache_hit":100864,"new_prefill":532,"score_linear":-100864.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":101396,"score_linear":106143.0,"score_lmetric":101396},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":101396,"score_linear":166406.0,"score_lmetric":623694},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":41650,"ongoing_decode_tokens":21684,"pending_prefill_tokens":9214,"num_requests":2,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":12800,"new_prefill":88596,"score_linear":28850.0,"score_lmetric":195620},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-77364,"ongoing_decode_tokens":0,"pending_prefill_tokens":-15412,"num_requests":1,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":12800,"new_prefill":88596,"score_linear":-90164.0,"score_lmetric":73184}],"chosen_score_linear":-100864.0,"chosen_score_lmetric":0,"t_first_token":339254.348436848,"t_first_token_unix":1779770119.3623805,"t_done":339255.141896581,"t_done_unix":1779770120.1558409},{"request_id":"1405199:1:1405199:776","session_id":"1405199","input_length":11503,"t_proxy_recv":339252.556956543,"t_decision_unix":1779770117.5708978,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11503,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":90517,"ongoing_decode_tokens":69508,"pending_prefill_tokens":13329,"num_requests":2,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":11503,"score_linear":90517.0,"score_lmetric":49664},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":11503,"score_linear":218746.0,"score_lmetric":135753},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":11503,"score_linear":270721.0,"score_lmetric":447414},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100766,"ongoing_decode_tokens":100766,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1514,"cache_hit":0,"new_prefill":11503,"score_linear":100766.0,"score_lmetric":11503},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":11503,"score_linear":106143.0,"score_lmetric":11503},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":11503,"score_linear":166406.0,"score_lmetric":354015},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":11503,"score_linear":21684.0,"score_lmetric":11503},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-88867,"ongoing_decode_tokens":0,"pending_prefill_tokens":-26915,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":11503,"score_linear":-88867.0,"score_lmetric":0}],"chosen_score_linear":-88867.0,"chosen_score_lmetric":0,"t_first_token":339257.882503852,"t_first_token_unix":1779770122.8964477,"t_done":339258.248905321,"t_done_unix":1779770123.2628489},{"request_id":"1279412:31:1391216:711","session_id":"1279412","input_length":101993,"t_proxy_recv":339255.163663064,"t_decision_unix":1779770120.1776044,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":101376,"affinity_cache_ratio":0.993950565234869,"affinity_num_requests":0,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (617 < 8000)","cache_hit":101376,"estimated_new_tokens":617,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":101993,"score_linear":209924.0,"score_lmetric":426848},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":101993,"score_linear":218746.0,"score_lmetric":407223},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":101993,"score_linear":270721.0,"score_lmetric":718884},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1516,"cache_hit":101376,"new_prefill":617,"score_linear":-101376.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":101993,"score_linear":106143.0,"score_lmetric":101993},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":93040,"pending_prefill_tokens":13974,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":101993,"score_linear":166406.0,"score_lmetric":347901},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":12800,"new_prefill":89193,"score_linear":148827.0,"score_lmetric":278838},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-77364,"ongoing_decode_tokens":0,"pending_prefill_tokens":-15412,"num_requests":1,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":12800,"new_prefill":89193,"score_linear":-90164.0,"score_lmetric":73781}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":339255.672255852,"t_first_token_unix":1779770120.6861997,"t_done":339258.431246922,"t_done_unix":1779770123.4451907},{"request_id":"1406966:1:1406966:785","session_id":"1406966","input_length":2115,"t_proxy_recv":339258.80773759,"t_decision_unix":1779770123.8216786,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2115 < 8000)","cache_hit":0,"estimated_new_tokens":2115,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":2115,"score_linear":209924.0,"score_lmetric":27336},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":2115,"score_linear":218746.0,"score_lmetric":107589},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":2115,"score_linear":270721.0,"score_lmetric":419250},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1517,"cache_hit":0,"new_prefill":2115,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":2115,"score_linear":106143.0,"score_lmetric":2115},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":103776,"pending_prefill_tokens":13478,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":2115,"score_linear":166406.0,"score_lmetric":46779},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":2115,"score_linear":161627.0,"score_lmetric":17604},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-88867,"ongoing_decode_tokens":32944,"pending_prefill_tokens":-59859,"num_requests":0,"active_p_offloads":0,"cached_blocks":1653,"cache_hit":0,"new_prefill":2115,"score_linear":-88867.0,"score_lmetric":0}],"chosen_score_linear":-88867.0,"chosen_score_lmetric":0,"t_first_token":339258.991281539,"t_first_token_unix":1779770124.0052252,"t_done":339259.353871325,"t_done_unix":1779770124.367815},{"request_id":"1362265:5:1398180:741","session_id":"1362265","input_length":10736,"t_proxy_recv":339228.172578707,"t_decision_unix":1779770093.1865203,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":10240,"affinity_cache_ratio":0.9538002980625931,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (496 < 8000)","cache_hit":10240,"estimated_new_tokens":496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":10736,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":10736,"score_linear":218746.0,"score_lmetric":384318},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":5049,"ongoing_decode_tokens":13134,"pending_prefill_tokens":3213,"num_requests":1,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":10736,"score_linear":5049.0,"score_lmetric":13949},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":217319,"ongoing_decode_tokens":217319,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":0,"new_prefill":10736,"score_linear":217319.0,"score_lmetric":21472},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":10736,"score_linear":106143.0,"score_lmetric":10736},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":10240,"new_prefill":496,"score_linear":82800.0,"score_lmetric":93024},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47774,"ongoing_decode_tokens":0,"pending_prefill_tokens":19102,"num_requests":2,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":10736,"score_linear":47774.0,"score_lmetric":59676},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":6656,"new_prefill":4080,"score_linear":-5674.0,"score_lmetric":5062}],"chosen_score_linear":82800.0,"chosen_score_lmetric":93024,"t_first_token":339255.648363431,"t_first_token_unix":1779770120.662307,"t_done":339260.015311714,"t_done_unix":1779770125.0292554},{"request_id":"1362265:6:1399937:747","session_id":"1362265","input_length":10816,"t_proxy_recv":339260.018405708,"t_decision_unix":1779770125.0323474,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9940828402366864,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (64 < 8000)","cache_hit":10752,"estimated_new_tokens":64,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":10816,"score_linear":209924.0,"score_lmetric":62140},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":10816,"score_linear":218746.0,"score_lmetric":133692},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":10816,"score_linear":270721.0,"score_lmetric":445353},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1517,"cache_hit":0,"new_prefill":10816,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":10816,"score_linear":106143.0,"score_lmetric":10816},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":155670,"ongoing_decode_tokens":155670,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1965,"cache_hit":10752,"new_prefill":64,"score_linear":144918.0,"score_lmetric":128},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":10816,"score_linear":161627.0,"score_lmetric":43707},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-88867,"ongoing_decode_tokens":32944,"pending_prefill_tokens":-59859,"num_requests":0,"active_p_offloads":0,"cached_blocks":1657,"cache_hit":6656,"new_prefill":4160,"score_linear":-95523.0,"score_lmetric":0}],"chosen_score_linear":144918.0,"chosen_score_lmetric":128,"t_first_token":339260.082314317,"t_first_token_unix":1779770125.0962577,"t_done":339260.798900824,"t_done_unix":1779770125.8128443},{"request_id":"1407494:1:1407494:787","session_id":"1407494","input_length":1531,"t_proxy_recv":339260.603915319,"t_decision_unix":1779770125.6178565,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1531 < 8000)","cache_hit":0,"estimated_new_tokens":1531,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":1531,"score_linear":209924.0,"score_lmetric":25000},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":1531,"score_linear":218746.0,"score_lmetric":105837},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":1531,"score_linear":270721.0,"score_lmetric":417498},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1517,"cache_hit":0,"new_prefill":1531,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":1531,"score_linear":106143.0,"score_lmetric":1531},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166486,"ongoing_decode_tokens":166486,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1965,"cache_hit":0,"new_prefill":1531,"score_linear":166486.0,"score_lmetric":4593},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":1531,"score_linear":161627.0,"score_lmetric":15852},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-88867,"ongoing_decode_tokens":32944,"pending_prefill_tokens":-59859,"num_requests":0,"active_p_offloads":0,"cached_blocks":1657,"cache_hit":0,"new_prefill":1531,"score_linear":-88867.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339260.712485839,"t_first_token_unix":1779770125.726429,"t_done":339260.982659116,"t_done_unix":1779770125.9966023},{"request_id":"1406966:2:1407710:789","session_id":"1406966","input_length":4789,"t_proxy_recv":339261.286694236,"t_decision_unix":1779770126.3006353,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":2048,"affinity_cache_ratio":0.42764669033201086,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":-57118,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2741 < 8000)","cache_hit":2048,"estimated_new_tokens":2741,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":4789,"score_linear":209924.0,"score_lmetric":38032},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":4789,"score_linear":218746.0,"score_lmetric":115611},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":4789,"score_linear":270721.0,"score_lmetric":427272},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":4789,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":4789,"score_linear":106143.0,"score_lmetric":4789},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":155670,"ongoing_decode_tokens":155670,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1965,"cache_hit":0,"new_prefill":4789,"score_linear":155670.0,"score_lmetric":9578},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":4789,"score_linear":161627.0,"score_lmetric":25626},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-87366,"ongoing_decode_tokens":34445,"pending_prefill_tokens":-59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1657,"cache_hit":2048,"new_prefill":2741,"score_linear":-89414.0,"score_lmetric":-57118}],"chosen_score_linear":-89414.0,"chosen_score_lmetric":-57118,"t_first_token":339261.496732802,"t_first_token_unix":1779770126.5106761,"t_done":339261.905337138,"t_done_unix":1779770126.9192805},{"request_id":"1397515:1:1397515:736","session_id":"1397515","input_length":93040,"t_proxy_recv":339225.677193422,"t_decision_unix":1779770090.691135,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":512,"estimated_new_tokens":92528,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":93040,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":93040,"score_linear":218746.0,"score_lmetric":631230},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1836,"ongoing_decode_tokens":0,"pending_prefill_tokens":13134,"num_requests":0,"active_p_offloads":0,"cached_blocks":2303,"cache_hit":0,"new_prefill":93040,"score_linear":1836.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":216155,"ongoing_decode_tokens":216155,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1274,"cache_hit":0,"new_prefill":93040,"score_linear":216155.0,"score_lmetric":186080},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":93040,"score_linear":106143.0,"score_lmetric":93040},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":512,"new_prefill":92528,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":93040,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":93040,"score_linear":982.0,"score_lmetric":94022}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":339254.306681138,"t_first_token_unix":1779770119.3206246,"t_done":339262.433071219,"t_done_unix":1779770127.4470146},{"request_id":"1407599:1:1407599:788","session_id":"1407599","input_length":1501,"t_proxy_recv":339260.952077462,"t_decision_unix":1779770125.9660187,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1501 < 8000)","cache_hit":0,"estimated_new_tokens":1501,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":1501,"score_linear":209924.0,"score_lmetric":24880},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":1501,"score_linear":218746.0,"score_lmetric":105747},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":1501,"score_linear":270721.0,"score_lmetric":417408},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1531,"ongoing_decode_tokens":1531,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1517,"cache_hit":0,"new_prefill":1501,"score_linear":1531.0,"score_lmetric":1501},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":1501,"score_linear":106143.0,"score_lmetric":1501},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":155670,"ongoing_decode_tokens":155670,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1965,"cache_hit":0,"new_prefill":1501,"score_linear":155670.0,"score_lmetric":3002},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":1501,"score_linear":161627.0,"score_lmetric":15762},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-88867,"ongoing_decode_tokens":32944,"pending_prefill_tokens":-59859,"num_requests":0,"active_p_offloads":0,"cached_blocks":1657,"cache_hit":0,"new_prefill":1501,"score_linear":-88867.0,"score_lmetric":0}],"chosen_score_linear":-88867.0,"chosen_score_lmetric":0,"t_first_token":339261.108020228,"t_first_token_unix":1779770126.1219635,"t_done":339263.119142084,"t_done_unix":1779770128.133085},{"request_id":"1407995:1:1407995:793","session_id":"1407995","input_length":2944,"t_proxy_recv":339262.42039272,"t_decision_unix":1779770127.4343345,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":-56915,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2944 < 8000)","cache_hit":0,"estimated_new_tokens":2944,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":2944,"score_linear":209924.0,"score_lmetric":30652},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":218746,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":2944,"score_linear":218746.0,"score_lmetric":8832},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":2944,"score_linear":270721.0,"score_lmetric":421737},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":2944,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":106143,"pending_prefill_tokens":10240,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":2944,"score_linear":207007.0,"score_lmetric":39552},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":155670,"ongoing_decode_tokens":155670,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1965,"cache_hit":0,"new_prefill":2944,"score_linear":155670.0,"score_lmetric":5888},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":2944,"score_linear":161627.0,"score_lmetric":20091},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-87366,"ongoing_decode_tokens":34445,"pending_prefill_tokens":-59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1662,"cache_hit":0,"new_prefill":2944,"score_linear":-87366.0,"score_lmetric":-56915}],"chosen_score_linear":-87366.0,"chosen_score_lmetric":-56915,"t_first_token":339262.643169441,"t_first_token_unix":1779770127.6571126,"t_done":339263.287272651,"t_done_unix":1779770128.3012156},{"request_id":"1366128:15:1390663:709","session_id":"1366128","input_length":83622,"t_proxy_recv":339209.358521597,"t_decision_unix":1779770074.372463,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":1,"affinity_cache_hit":83456,"affinity_cache_ratio":0.9980148764679151,"affinity_num_requests":3,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":83622,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":83622,"score_linear":91673.0,"score_lmetric":310654},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":83622,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":83622,"score_linear":98853.0,"score_lmetric":155521},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":83622,"score_linear":257663.0,"score_lmetric":252291},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":87687,"ongoing_decode_tokens":87687,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":83622,"score_linear":87687.0,"score_lmetric":83622},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":89969,"ongoing_decode_tokens":89969,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":83456,"new_prefill":166,"score_linear":6513.0,"score_lmetric":498},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1612,"cache_hit":0,"new_prefill":83622,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":83622,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339234.523889515,"t_first_token_unix":1779770099.5378337,"t_done":339264.209035504,"t_done_unix":1779770129.2229795},{"request_id":"1340290:7:1408176:795","session_id":"1340290","input_length":27786,"t_proxy_recv":339263.106475091,"t_decision_unix":1779770128.1204162,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9766069243503923,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (650 < 8000)","cache_hit":27136,"estimated_new_tokens":650,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":27786,"score_linear":209924.0,"score_lmetric":130020},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":218746,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":27786,"score_linear":218746.0,"score_lmetric":83358},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":27786,"score_linear":270721.0,"score_lmetric":496263},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":27786,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":106143,"pending_prefill_tokens":10240,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":27786,"score_linear":207007.0,"score_lmetric":114078},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62630,"ongoing_decode_tokens":62630,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":27136,"new_prefill":650,"score_linear":35494.0,"score_lmetric":650},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":1024,"new_prefill":26762,"score_linear":160603.0,"score_lmetric":91545},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-84422,"ongoing_decode_tokens":37389,"pending_prefill_tokens":-59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1662,"cache_hit":0,"new_prefill":27786,"score_linear":-84422.0,"score_lmetric":-64146}],"chosen_score_linear":35494.0,"chosen_score_lmetric":650,"t_first_token":339263.275428838,"t_first_token_unix":1779770128.2893724,"t_done":339264.394909375,"t_done_unix":1779770129.408853},{"request_id":"1405155:1:1405155:775","session_id":"1405155","input_length":32944,"t_proxy_recv":339252.427052692,"t_decision_unix":1779770117.440994,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":26915,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":32944,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":90517,"ongoing_decode_tokens":69508,"pending_prefill_tokens":13329,"num_requests":2,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":512,"new_prefill":32432,"score_linear":90005.0,"score_lmetric":91522},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":32944,"score_linear":218746.0,"score_lmetric":200076},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":32944,"score_linear":270721.0,"score_lmetric":511737},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100766,"ongoing_decode_tokens":100766,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1514,"cache_hit":0,"new_prefill":32944,"score_linear":100766.0,"score_lmetric":32944},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":32944,"score_linear":106143.0,"score_lmetric":32944},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":32944,"score_linear":166406.0,"score_lmetric":418338},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":32944,"score_linear":21684.0,"score_lmetric":32944},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":32944,"score_linear":-121811.0,"score_lmetric":26915}],"chosen_score_linear":-121811.0,"chosen_score_lmetric":26915,"t_first_token":339257.519501634,"t_first_token_unix":1779770122.5334454,"t_done":339265.051663372,"t_done_unix":1779770130.065607},{"request_id":"1408430:1:1408430:797","session_id":"1408430","input_length":7553,"t_proxy_recv":339263.949621251,"t_decision_unix":1779770128.9635623,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7553 < 8000)","cache_hit":0,"estimated_new_tokens":7553,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":7553,"score_linear":209924.0,"score_lmetric":49088},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":218746,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":7553,"score_linear":218746.0,"score_lmetric":22659},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":7553,"score_linear":270721.0,"score_lmetric":435564},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":7553,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":179331,"pending_prefill_tokens":5660,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":7553,"score_linear":207007.0,"score_lmetric":39639},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":90416,"ongoing_decode_tokens":90416,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":7553,"score_linear":90416.0,"score_lmetric":15106},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":7553,"score_linear":161627.0,"score_lmetric":33918},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-88867,"ongoing_decode_tokens":32944,"pending_prefill_tokens":-59859,"num_requests":0,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":7553,"score_linear":-88867.0,"score_lmetric":0}],"chosen_score_linear":-88867.0,"chosen_score_lmetric":0,"t_first_token":339264.479895453,"t_first_token_unix":1779770129.493839,"t_done":339265.291525188,"t_done_unix":1779770130.305469},{"request_id":"1408779:1:1408779:802","session_id":"1408779","input_length":1801,"t_proxy_recv":339265.049161289,"t_decision_unix":1779770130.0631027,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":-116116,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1801 < 8000)","cache_hit":0,"estimated_new_tokens":1801,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":1801,"score_linear":209924.0,"score_lmetric":26080},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220021,"ongoing_decode_tokens":220021,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":1801,"score_linear":220021.0,"score_lmetric":5403},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":1801,"score_linear":270721.0,"score_lmetric":418308},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":1801,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":207007,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":1801,"score_linear":207007.0,"score_lmetric":5403},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":73700,"ongoing_decode_tokens":62630,"pending_prefill_tokens":318,"num_requests":2,"active_p_offloads":0,"cached_blocks":2147,"cache_hit":0,"new_prefill":1801,"score_linear":73700.0,"score_lmetric":4238},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":1801,"score_linear":161627.0,"score_lmetric":16662},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-74937,"ongoing_decode_tokens":46874,"pending_prefill_tokens":-59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":1801,"score_linear":-74937.0,"score_lmetric":-116116}],"chosen_score_linear":-74937.0,"chosen_score_lmetric":-116116,"t_first_token":339265.180172549,"t_first_token_unix":1779770130.1941156,"t_done":339265.659589606,"t_done_unix":1779770130.6735327},{"request_id":"1408889:1:1408889:804","session_id":"1408889","input_length":3230,"t_proxy_recv":339265.419982765,"t_decision_unix":1779770130.4339244,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":-56629,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3230 < 8000)","cache_hit":0,"estimated_new_tokens":3230,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":3230,"score_linear":209924.0,"score_lmetric":31796},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220021,"ongoing_decode_tokens":220021,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":3230,"score_linear":220021.0,"score_lmetric":9690},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":3230,"score_linear":270721.0,"score_lmetric":422595},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":3230,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":207007,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":3230,"score_linear":207007.0,"score_lmetric":9690},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":73700,"ongoing_decode_tokens":73700,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2147,"cache_hit":0,"new_prefill":3230,"score_linear":73700.0,"score_lmetric":6460},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":3230,"score_linear":161627.0,"score_lmetric":20949},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-113633,"ongoing_decode_tokens":8178,"pending_prefill_tokens":-59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":3230,"score_linear":-113633.0,"score_lmetric":-56629}],"chosen_score_linear":-113633.0,"chosen_score_lmetric":-56629,"t_first_token":339265.628216853,"t_first_token_unix":1779770130.6421604,"t_done":339265.780363105,"t_done_unix":1779770130.7943063},{"request_id":"1362265:7:1408772:800","session_id":"1362265","input_length":11070,"t_proxy_recv":339265.032957991,"t_decision_unix":1779770130.0468996,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9712737127371274,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (318 < 8000)","cache_hit":10752,"estimated_new_tokens":318,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":11070,"score_linear":209924.0,"score_lmetric":63156},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220021,"ongoing_decode_tokens":220021,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":11070,"score_linear":220021.0,"score_lmetric":33210},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":11070,"score_linear":270721.0,"score_lmetric":446115},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":11070,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":207007,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":11070,"score_linear":207007.0,"score_lmetric":33210},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":62630,"ongoing_decode_tokens":62630,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2147,"cache_hit":10752,"new_prefill":318,"score_linear":51878.0,"score_lmetric":318},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":11070,"score_linear":161627.0,"score_lmetric":44469},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-74937,"ongoing_decode_tokens":46874,"pending_prefill_tokens":-59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":6656,"new_prefill":4414,"score_linear":-81593.0,"score_lmetric":-110890}],"chosen_score_linear":51878.0,"chosen_score_lmetric":318,"t_first_token":339265.124540914,"t_first_token_unix":1779770130.1384842,"t_done":339265.986844258,"t_done_unix":1779770131.0007877},{"request_id":"1364090:5:1401054:756","session_id":"1364090","input_length":62630,"t_proxy_recv":339238.214502338,"t_decision_unix":1779770103.2284431,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49152,"affinity_cache_ratio":0.7847996167970621,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=2 decode_tok=0)","cache_hit":49152,"estimated_new_tokens":13478,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":79523,"ongoing_decode_tokens":11414,"pending_prefill_tokens":17421,"num_requests":2,"active_p_offloads":0,"cached_blocks":1679,"cache_hit":7680,"new_prefill":54950,"score_linear":71843.0,"score_lmetric":144742},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":62630,"score_linear":218746.0,"score_lmetric":289134},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":65094,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":1,"active_p_offloads":0,"cached_blocks":2309,"cache_hit":7680,"new_prefill":54950,"score_linear":57414.0,"score_lmetric":67008},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":0,"new_prefill":62630,"score_linear":98267.0,"score_lmetric":62630},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":7680,"new_prefill":54950,"score_linear":98463.0,"score_lmetric":54950},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103776,"ongoing_decode_tokens":0,"pending_prefill_tokens":93024,"num_requests":2,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":49152,"new_prefill":13478,"score_linear":54624.0,"score_lmetric":213004},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":11264,"new_prefill":51366,"score_linear":13178.0,"score_lmetric":51366},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":62630,"score_linear":982.0,"score_lmetric":63612}],"chosen_score_linear":54624.0,"chosen_score_lmetric":213004,"t_first_token":339259.57718663,"t_first_token_unix":1779770124.59113,"t_done":339265.99732956,"t_done_unix":1779770131.011273},{"request_id":"1408578:1:1408578:798","session_id":"1408578","input_length":6377,"t_proxy_recv":339264.370673041,"t_decision_unix":1779770129.3846142,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":-45929,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6377 < 8000)","cache_hit":0,"estimated_new_tokens":6377,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":6377,"score_linear":209924.0,"score_lmetric":44384},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220021,"ongoing_decode_tokens":135124,"pending_prefill_tokens":1441,"num_requests":3,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":6377,"score_linear":220021.0,"score_lmetric":23454},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":6377,"score_linear":270721.0,"score_lmetric":432036},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":6377,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":179331,"pending_prefill_tokens":5660,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":6377,"score_linear":207007.0,"score_lmetric":36111},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":90416,"ongoing_decode_tokens":90416,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":0,"new_prefill":6377,"score_linear":90416.0,"score_lmetric":12754},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":6377,"score_linear":161627.0,"score_lmetric":30390},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-81314,"ongoing_decode_tokens":32944,"pending_prefill_tokens":-52306,"num_requests":1,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":6377,"score_linear":-81314.0,"score_lmetric":-45929}],"chosen_score_linear":-81314.0,"chosen_score_lmetric":-45929,"t_first_token":339264.894349066,"t_first_token_unix":1779770129.908293,"t_done":339266.413292725,"t_done_unix":1779770131.427236},{"request_id":"1363093:5:1407810:790","session_id":"1363093","input_length":73188,"t_proxy_recv":339261.746165185,"t_decision_unix":1779770126.7601063,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":68608,"affinity_cache_ratio":0.9374214352079576,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4580 < 8000)","cache_hit":68608,"estimated_new_tokens":4580,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":73188,"score_linear":209924.0,"score_lmetric":311628},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":73188,"score_linear":218746.0,"score_lmetric":320808},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":73188,"score_linear":270721.0,"score_lmetric":632469},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":73188,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":68608,"new_prefill":4580,"score_linear":37535.0,"score_lmetric":4580},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":155670,"ongoing_decode_tokens":155670,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1965,"cache_hit":0,"new_prefill":73188,"score_linear":155670.0,"score_lmetric":146376},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":73188,"score_linear":161627.0,"score_lmetric":230823},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-82577,"ongoing_decode_tokens":39234,"pending_prefill_tokens":-59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1657,"cache_hit":0,"new_prefill":73188,"score_linear":-82577.0,"score_lmetric":26658}],"chosen_score_linear":37535.0,"chosen_score_lmetric":4580,"t_first_token":339263.920084264,"t_first_token_unix":1779770128.9340281,"t_done":339266.462862101,"t_done_unix":1779770131.4768057},{"request_id":"1398594:2:1404702:771","session_id":"1398594","input_length":21009,"t_proxy_recv":339250.852950516,"t_decision_unix":1779770115.8668911,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":7680,"affinity_cache_ratio":0.3655576181636441,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":13329,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":7680,"estimated_new_tokens":13329,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69508,"ongoing_decode_tokens":69508,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":7680,"new_prefill":13329,"score_linear":61828.0,"score_lmetric":13329},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":21009,"score_linear":218746.0,"score_lmetric":164271},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":185283,"ongoing_decode_tokens":0,"pending_prefill_tokens":133605,"num_requests":2,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":21009,"score_linear":185283.0,"score_lmetric":309228},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":144914,"ongoing_decode_tokens":144914,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":21009,"score_linear":144914.0,"score_lmetric":42018},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":21009,"score_linear":106143.0,"score_lmetric":21009},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":21009,"score_linear":166406.0,"score_lmetric":382533},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21684,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":21009,"score_linear":21684.0,"score_lmetric":42693},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":21009,"score_linear":-121811.0,"score_lmetric":38850}],"chosen_score_linear":61828.0,"chosen_score_lmetric":13329,"t_first_token":339252.55929822,"t_first_token_unix":1779770117.5732417,"t_done":339267.872323361,"t_done_unix":1779770132.8862667},{"request_id":"1370129:6:1394177:722","session_id":"1370129","input_length":106143,"t_proxy_recv":339214.167320459,"t_decision_unix":1779770079.1812618,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":88064,"affinity_cache_ratio":0.8296731767521174,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":88064,"estimated_new_tokens":18079,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":7680,"new_prefill":98463,"score_linear":83993.0,"score_lmetric":340336},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":0,"pending_prefill_tokens":117370,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":106143,"score_linear":218746.0,"score_lmetric":670539},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":110151,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":7680,"new_prefill":98463,"score_linear":91173.0,"score_lmetric":98463},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":106143,"score_linear":257663.0,"score_lmetric":319854},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":88064,"new_prefill":18079,"score_linear":-88064.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":7680,"new_prefill":98463,"score_linear":56547.0,"score_lmetric":196926},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":4675,"ongoing_decode_tokens":0,"pending_prefill_tokens":4675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":7680,"new_prefill":98463,"score_linear":-3005.0,"score_lmetric":103138},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":106143,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-88064.0,"chosen_score_lmetric":0,"t_first_token":339225.170069475,"t_first_token_unix":1779770090.1840134,"t_done":339268.375323256,"t_done_unix":1779770133.3892667},{"request_id":"1286804:5:1407989:792","session_id":"1286804","input_length":27676,"t_proxy_recv":339262.393783141,"t_decision_unix":1779770127.4077249,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":22016,"affinity_cache_ratio":0.795490677843619,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5660 < 8000)","cache_hit":22016,"estimated_new_tokens":5660,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":27676,"score_linear":209924.0,"score_lmetric":129580},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":218746,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":2560,"new_prefill":25116,"score_linear":216186.0,"score_lmetric":75348},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":27676,"score_linear":270721.0,"score_lmetric":495933},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":27676,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":179331,"ongoing_decode_tokens":106143,"pending_prefill_tokens":4580,"num_requests":2,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":22016,"new_prefill":5660,"score_linear":157315.0,"score_lmetric":20480},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":155670,"ongoing_decode_tokens":155670,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1965,"cache_hit":0,"new_prefill":27676,"score_linear":155670.0,"score_lmetric":55352},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":27676,"score_linear":161627.0,"score_lmetric":94287},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-87366,"ongoing_decode_tokens":34445,"pending_prefill_tokens":-59859,"num_requests":1,"active_p_offloads":0,"cached_blocks":1662,"cache_hit":0,"new_prefill":27676,"score_linear":-87366.0,"score_lmetric":-32183}],"chosen_score_linear":157315.0,"chosen_score_lmetric":20480,"t_first_token":339264.914250266,"t_first_token_unix":1779770129.9281936,"t_done":339268.388690984,"t_done_unix":1779770133.4026341},{"request_id":"1366128:16:1392704:716","session_id":"1366128","input_length":84897,"t_proxy_recv":339264.224307323,"t_decision_unix":1779770129.238249,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":83456,"affinity_cache_ratio":0.9830264909242965,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1441 < 8000)","cache_hit":83456,"estimated_new_tokens":1441,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":84897,"score_linear":209924.0,"score_lmetric":358464},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":83456,"new_prefill":1441,"score_linear":51668.0,"score_lmetric":2882},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":84897,"score_linear":270721.0,"score_lmetric":667596},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":84897,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":179331,"pending_prefill_tokens":5660,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":84897,"score_linear":207007.0,"score_lmetric":271671},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":90416,"ongoing_decode_tokens":90416,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2146,"cache_hit":83456,"new_prefill":1441,"score_linear":6960.0,"score_lmetric":2882},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":84897,"score_linear":161627.0,"score_lmetric":265950},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-81314,"ongoing_decode_tokens":32944,"pending_prefill_tokens":-52306,"num_requests":1,"active_p_offloads":0,"cached_blocks":1670,"cache_hit":0,"new_prefill":84897,"score_linear":-81314.0,"score_lmetric":32591}],"chosen_score_linear":51668.0,"chosen_score_lmetric":2882,"t_first_token":339264.974346869,"t_first_token_unix":1779770129.9882903,"t_done":339268.736474325,"t_done_unix":1779770133.750418},{"request_id":"1408779:2:1410032:807","session_id":"1408779","input_length":5001,"t_proxy_recv":339269.570443851,"t_decision_unix":1779770134.584385,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":1536,"affinity_cache_ratio":0.30713857228554287,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3465 < 8000)","cache_hit":1536,"estimated_new_tokens":3465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":188915,"ongoing_decode_tokens":69508,"pending_prefill_tokens":4719,"num_requests":3,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":5001,"score_linear":188915.0,"score_lmetric":29160},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220400,"ongoing_decode_tokens":220400,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1727,"cache_hit":0,"new_prefill":5001,"score_linear":220400.0,"score_lmetric":15003},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":5001,"score_linear":270721.0,"score_lmetric":427908},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":5001,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":5001,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103122,"ongoing_decode_tokens":0,"pending_prefill_tokens":12498,"num_requests":2,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":0,"new_prefill":5001,"score_linear":103122.0,"score_lmetric":34998},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":5001,"score_linear":161627.0,"score_lmetric":26262},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":1536,"new_prefill":3465,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":339269.820044534,"t_first_token_unix":1779770134.833988,"t_done":339270.230180575,"t_done_unix":1779770135.2441247},{"request_id":"1406966:3:1410097:808","session_id":"1406966","input_length":7893,"t_proxy_recv":339269.803810092,"t_decision_unix":1779770134.8177516,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":4608,"affinity_cache_ratio":0.5838084378563284,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3285 < 8000)","cache_hit":4608,"estimated_new_tokens":3285,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":188915,"ongoing_decode_tokens":69508,"pending_prefill_tokens":4719,"num_requests":3,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":7893,"score_linear":188915.0,"score_lmetric":37836},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220400,"ongoing_decode_tokens":220400,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1727,"cache_hit":0,"new_prefill":7893,"score_linear":220400.0,"score_lmetric":23679},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":7893,"score_linear":270721.0,"score_lmetric":436584},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":7893,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":7893,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103122,"ongoing_decode_tokens":0,"pending_prefill_tokens":12498,"num_requests":2,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":0,"new_prefill":7893,"score_linear":103122.0,"score_lmetric":40782},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":7893,"score_linear":161627.0,"score_lmetric":34938},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":5001,"ongoing_decode_tokens":0,"pending_prefill_tokens":3465,"num_requests":1,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":4608,"new_prefill":3285,"score_linear":393.0,"score_lmetric":6750}],"chosen_score_linear":393.0,"chosen_score_lmetric":6750,"t_first_token":339270.101290199,"t_first_token_unix":1779770135.1152341,"t_done":339270.315929251,"t_done_unix":1779770135.3298728},{"request_id":"1366128:17:1395596:731","session_id":"1366128","input_length":85276,"t_proxy_recv":339268.751930287,"t_decision_unix":1779770133.7658718,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":84992,"affinity_cache_ratio":0.9966696374126366,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (284 < 8000)","cache_hit":84992,"estimated_new_tokens":284,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":188915,"ongoing_decode_tokens":69508,"pending_prefill_tokens":4719,"num_requests":3,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":85276,"score_linear":188915.0,"score_lmetric":269985},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1727,"cache_hit":84992,"new_prefill":284,"score_linear":50132.0,"score_lmetric":568},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":85276,"score_linear":270721.0,"score_lmetric":668733},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":85276,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":85276,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103122,"ongoing_decode_tokens":0,"pending_prefill_tokens":12498,"num_requests":2,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":83456,"new_prefill":1820,"score_linear":19666.0,"score_lmetric":28636},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":85276,"score_linear":161627.0,"score_lmetric":267087},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":85276,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":50132.0,"chosen_score_lmetric":568,"t_first_token":339269.028808872,"t_first_token_unix":1779770134.0427525,"t_done":339270.838896355,"t_done_unix":1779770135.8528397},{"request_id":"1407494:2:1410400:809","session_id":"1407494","input_length":4728,"t_proxy_recv":339270.668335429,"t_decision_unix":1779770135.682277,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":1536,"affinity_cache_ratio":0.3248730964467005,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3192 < 8000)","cache_hit":1536,"estimated_new_tokens":3192,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":188915,"ongoing_decode_tokens":69508,"pending_prefill_tokens":4719,"num_requests":3,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":4728,"score_linear":188915.0,"score_lmetric":28341},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220400,"ongoing_decode_tokens":220400,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1727,"cache_hit":0,"new_prefill":4728,"score_linear":220400.0,"score_lmetric":14184},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":4728,"score_linear":270721.0,"score_lmetric":427089},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":1536,"new_prefill":3192,"score_linear":-1536.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":4728,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103122,"ongoing_decode_tokens":73392,"pending_prefill_tokens":2082,"num_requests":2,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":0,"new_prefill":4728,"score_linear":103122.0,"score_lmetric":13620},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":4728,"score_linear":161627.0,"score_lmetric":25443},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1782,"cache_hit":0,"new_prefill":4728,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":339270.898222695,"t_first_token_unix":1779770135.9121666,"t_done":339271.035781679,"t_done_unix":1779770136.0497248},{"request_id":"1408889:2:1410453:810","session_id":"1408889","input_length":4825,"t_proxy_recv":339270.806751462,"t_decision_unix":1779770135.8206928,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":3072,"affinity_cache_ratio":0.6366839378238341,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1753 < 8000)","cache_hit":3072,"estimated_new_tokens":1753,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":188915,"ongoing_decode_tokens":69508,"pending_prefill_tokens":4719,"num_requests":3,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":4825,"score_linear":188915.0,"score_lmetric":28632},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220400,"ongoing_decode_tokens":220400,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1727,"cache_hit":0,"new_prefill":4825,"score_linear":220400.0,"score_lmetric":14475},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":4825,"score_linear":270721.0,"score_lmetric":427380},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":4728,"ongoing_decode_tokens":0,"pending_prefill_tokens":3192,"num_requests":1,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":4825,"score_linear":4728.0,"score_lmetric":8017},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":4825,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103122,"ongoing_decode_tokens":73392,"pending_prefill_tokens":2082,"num_requests":2,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":0,"new_prefill":4825,"score_linear":103122.0,"score_lmetric":13814},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":4825,"score_linear":161627.0,"score_lmetric":25734},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1782,"cache_hit":3072,"new_prefill":1753,"score_linear":-3072.0,"score_lmetric":0}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":339270.953786527,"t_first_token_unix":1779770135.9677298,"t_done":339271.238552793,"t_done_unix":1779770136.2524967},{"request_id":"1381387:5:1404170:768","session_id":"1381387","input_length":69508,"t_proxy_recv":339248.921018709,"t_decision_unix":1779770113.93496,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9796857915635611,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1412 < 8000)","cache_hit":68096,"estimated_new_tokens":1412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":68096,"new_prefill":1412,"score_linear":-68096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":69508,"score_linear":218746.0,"score_lmetric":309768},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":121640,"ongoing_decode_tokens":0,"pending_prefill_tokens":132938,"num_requests":1,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":69508,"score_linear":121640.0,"score_lmetric":202446},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":144686,"ongoing_decode_tokens":144686,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":69508,"score_linear":144686.0,"score_lmetric":139016},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":69508,"score_linear":106143.0,"score_lmetric":69508},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":69508,"score_linear":166406.0,"score_lmetric":528030},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":69508,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":69508,"score_linear":-121811.0,"score_lmetric":-9649}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":339249.54549058,"t_first_token_unix":1779770114.5594404,"t_done":339271.450081615,"t_done_unix":1779770136.4640248},{"request_id":"1366128:18:1397382:735","session_id":"1366128","input_length":85504,"t_proxy_recv":339270.852731266,"t_decision_unix":1779770135.866673,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":84992,"affinity_cache_ratio":0.9940119760479041,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (512 < 8000)","cache_hit":84992,"estimated_new_tokens":512,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":188915,"ongoing_decode_tokens":69508,"pending_prefill_tokens":4719,"num_requests":3,"active_p_offloads":0,"cached_blocks":1761,"cache_hit":0,"new_prefill":85504,"score_linear":188915.0,"score_lmetric":270669},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1727,"cache_hit":84992,"new_prefill":512,"score_linear":50132.0,"score_lmetric":1024},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":85504,"score_linear":270721.0,"score_lmetric":669417},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":4728,"ongoing_decode_tokens":0,"pending_prefill_tokens":3192,"num_requests":1,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":85504,"score_linear":4728.0,"score_lmetric":88696},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":85504,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103122,"ongoing_decode_tokens":73392,"pending_prefill_tokens":2082,"num_requests":2,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":83456,"new_prefill":2048,"score_linear":19666.0,"score_lmetric":8260},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":85504,"score_linear":161627.0,"score_lmetric":267771},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":4825,"ongoing_decode_tokens":0,"pending_prefill_tokens":1753,"num_requests":1,"active_p_offloads":0,"cached_blocks":1782,"cache_hit":0,"new_prefill":85504,"score_linear":4825.0,"score_lmetric":87257}],"chosen_score_linear":50132.0,"chosen_score_lmetric":1024,"t_first_token":339271.136707703,"t_first_token_unix":1779770136.1506512,"t_done":339272.717028974,"t_done_unix":1779770137.7309725},{"request_id":"1340290:8:1409129:805","session_id":"1340290","input_length":29730,"t_proxy_recv":339266.431392438,"t_decision_unix":1779770131.4453335,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":27648,"affinity_cache_ratio":0.9299697275479314,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2082 < 8000)","cache_hit":27648,"estimated_new_tokens":2082,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":29730,"score_linear":209924.0,"score_lmetric":137796},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220021,"ongoing_decode_tokens":220021,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":29730,"score_linear":220021.0,"score_lmetric":89190},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":29730,"score_linear":270721.0,"score_lmetric":502095},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":29730,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":207007,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":29730,"score_linear":207007.0,"score_lmetric":89190},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":73392,"ongoing_decode_tokens":0,"pending_prefill_tokens":10416,"num_requests":1,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":27648,"new_prefill":2082,"score_linear":45744.0,"score_lmetric":12498},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":1024,"new_prefill":28706,"score_linear":160603.0,"score_lmetric":97377},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":29730,"score_linear":-121811.0,"score_lmetric":30129}],"chosen_score_linear":45744.0,"chosen_score_lmetric":12498,"t_first_token":339270.950272729,"t_first_token_unix":1779770135.964217,"t_done":339273.087681763,"t_done_unix":1779770138.1016254},{"request_id":"1366128:19:1398824:745","session_id":"1366128","input_length":85663,"t_proxy_recv":339272.731277165,"t_decision_unix":1779770137.7452188,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9981438894271739,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (159 < 8000)","cache_hit":85504,"estimated_new_tokens":159,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":85663,"score_linear":190977.0,"score_lmetric":276960},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":85504,"new_prefill":159,"score_linear":49620.0,"score_lmetric":318},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":85663,"score_linear":270721.0,"score_lmetric":669894},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":85663,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":85663,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103122,"ongoing_decode_tokens":103122,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":83456,"new_prefill":2207,"score_linear":19666.0,"score_lmetric":4414},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":85663,"score_linear":161627.0,"score_lmetric":268248},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1785,"cache_hit":0,"new_prefill":85663,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":49620.0,"chosen_score_lmetric":318,"t_first_token":339272.943673958,"t_first_token_unix":1779770137.9576182,"t_done":339273.340811964,"t_done_unix":1779770138.3547554},{"request_id":"1340290:9:1410595:811","session_id":"1340290","input_length":29958,"t_proxy_recv":339273.101264239,"t_decision_unix":1779770138.115206,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":29696,"affinity_cache_ratio":0.9912544228586688,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (262 < 8000)","cache_hit":29696,"estimated_new_tokens":262,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":29958,"score_linear":190977.0,"score_lmetric":109845},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220787,"ongoing_decode_tokens":220787,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":29958,"score_linear":220787.0,"score_lmetric":89874},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":29958,"score_linear":270721.0,"score_lmetric":502779},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":29958,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":29958,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":73392,"ongoing_decode_tokens":73392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":29696,"new_prefill":262,"score_linear":43696.0,"score_lmetric":262},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":1024,"new_prefill":28934,"score_linear":160603.0,"score_lmetric":98061},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1785,"cache_hit":0,"new_prefill":29958,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":43696.0,"chosen_score_lmetric":262,"t_first_token":339273.187008534,"t_first_token_unix":1779770138.200953,"t_done":339273.712089368,"t_done_unix":1779770138.726033},{"request_id":"1408779:3:1411249:815","session_id":"1408779","input_length":7110,"t_proxy_recv":339273.583920983,"t_decision_unix":1779770138.5978622,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":4608,"affinity_cache_ratio":0.6481012658227848,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2502 < 8000)","cache_hit":4608,"estimated_new_tokens":2502,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":7110,"score_linear":190977.0,"score_lmetric":41301},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":232781,"ongoing_decode_tokens":135124,"pending_prefill_tokens":12153,"num_requests":3,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":7110,"score_linear":232781.0,"score_lmetric":57789},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":7110,"score_linear":270721.0,"score_lmetric":434235},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":7110,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":7110,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103350,"ongoing_decode_tokens":103350,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":7110,"score_linear":103350.0,"score_lmetric":14220},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":7110,"score_linear":161627.0,"score_lmetric":32589},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1785,"cache_hit":4608,"new_prefill":2502,"score_linear":-4608.0,"score_lmetric":0}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":339273.785984663,"t_first_token_unix":1779770138.7999284,"t_done":339273.946047637,"t_done_unix":1779770138.9599917},{"request_id":"1364090:6:1408777:801","session_id":"1364090","input_length":73392,"t_proxy_recv":339266.01517048,"t_decision_unix":1779770131.029112,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":62976,"affinity_cache_ratio":0.8580771746239372,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":62976,"estimated_new_tokens":10416,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":7680,"new_prefill":65712,"score_linear":202244.0,"score_lmetric":281724},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220021,"ongoing_decode_tokens":220021,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1724,"cache_hit":0,"new_prefill":73392,"score_linear":220021.0,"score_lmetric":220176},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":7680,"new_prefill":65712,"score_linear":263041.0,"score_lmetric":610041},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1520,"cache_hit":0,"new_prefill":73392,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":207007,"ongoing_decode_tokens":207007,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":7680,"new_prefill":65712,"score_linear":199327.0,"score_lmetric":197136},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":62976,"new_prefill":10416,"score_linear":-62976.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":11264,"new_prefill":62128,"score_linear":150363.0,"score_lmetric":197643},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-115434,"ongoing_decode_tokens":6377,"pending_prefill_tokens":-59859,"num_requests":0,"active_p_offloads":0,"cached_blocks":1758,"cache_hit":0,"new_prefill":73392,"score_linear":-115434.0,"score_lmetric":0}],"chosen_score_linear":-62976.0,"chosen_score_lmetric":0,"t_first_token":339270.504970796,"t_first_token_unix":1779770135.5189142,"t_done":339275.489528041,"t_done_unix":1779770140.5034719},{"request_id":"1340290:10:1411397:816","session_id":"1340290","input_length":30154,"t_proxy_recv":339274.212928817,"t_decision_unix":1779770139.2268698,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":29696,"affinity_cache_ratio":0.9848113019831531,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (458 < 8000)","cache_hit":29696,"estimated_new_tokens":458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":30154,"score_linear":190977.0,"score_lmetric":110433},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":232781,"ongoing_decode_tokens":135124,"pending_prefill_tokens":12153,"num_requests":3,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":30154,"score_linear":232781.0,"score_lmetric":126921},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":30154,"score_linear":270721.0,"score_lmetric":503367},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":30154,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":30154,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":73392,"ongoing_decode_tokens":73392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":29696,"new_prefill":458,"score_linear":43696.0,"score_lmetric":458},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":1024,"new_prefill":29130,"score_linear":160603.0,"score_lmetric":98649},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":30154,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":43696.0,"chosen_score_lmetric":458,"t_first_token":339274.334580689,"t_first_token_unix":1779770139.3485239,"t_done":339276.647011206,"t_done_unix":1779770141.6609547},{"request_id":"1366128:20:1400446:752","session_id":"1366128","input_length":97657,"t_proxy_recv":339273.361335635,"t_decision_unix":1779770138.3752773,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":85504,"affinity_cache_ratio":0.8755542357434695,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=83456 chosen=85504)","cache_hit":85504,"estimated_new_tokens":12153,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":97657,"score_linear":190977.0,"score_lmetric":312942},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":85504,"new_prefill":12153,"score_linear":49620.0,"score_lmetric":24306},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":97657,"score_linear":270721.0,"score_lmetric":705876},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":97657,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":97657,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103350,"ongoing_decode_tokens":103350,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":83456,"new_prefill":14201,"score_linear":19894.0,"score_lmetric":28402},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":97657,"score_linear":161627.0,"score_lmetric":304230},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1785,"cache_hit":0,"new_prefill":97657,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":49620.0,"chosen_score_lmetric":24306,"t_first_token":339280.310695474,"t_first_token_unix":1779770145.3246396,"t_done":339281.957326656,"t_done_unix":1779770146.9712698},{"request_id":"1407494:3:1413526:825","session_id":"1407494","input_length":9801,"t_proxy_recv":339281.618684037,"t_decision_unix":1779770146.632625,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":4608,"affinity_cache_ratio":0.4701561065197429,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5193 < 8000)","cache_hit":4608,"estimated_new_tokens":5193,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":9801,"score_linear":190977.0,"score_lmetric":49374},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":232781,"ongoing_decode_tokens":232781,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":9801,"score_linear":232781.0,"score_lmetric":29403},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":9801,"score_linear":310994.0,"score_lmetric":597236},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":4608,"new_prefill":5193,"score_linear":-4608.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":9801,"score_linear":107292.0,"score_lmetric":23909},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":121215,"ongoing_decode_tokens":0,"pending_prefill_tokens":17279,"num_requests":2,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":0,"new_prefill":9801,"score_linear":121215.0,"score_lmetric":54160},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":9801,"score_linear":161627.0,"score_lmetric":40662},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":9801,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":339282.088184433,"t_first_token_unix":1779770147.102128,"t_done":339282.424731616,"t_done_unix":1779770147.4386752},{"request_id":"1366128:21:1402408:760","session_id":"1366128","input_length":97814,"t_proxy_recv":339281.973203958,"t_decision_unix":1779770146.9871454,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97280,"affinity_cache_ratio":0.9945406588013985,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (534 < 8000)","cache_hit":97280,"estimated_new_tokens":534,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":97814,"score_linear":190977.0,"score_lmetric":313413},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1751,"cache_hit":97280,"new_prefill":534,"score_linear":37844.0,"score_lmetric":1068},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":97814,"score_linear":310994.0,"score_lmetric":949288},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9801,"ongoing_decode_tokens":0,"pending_prefill_tokens":5193,"num_requests":1,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":97814,"score_linear":9801.0,"score_lmetric":103007},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":97814,"score_linear":107292.0,"score_lmetric":111922},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":121215,"ongoing_decode_tokens":0,"pending_prefill_tokens":17279,"num_requests":2,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":83456,"new_prefill":14358,"score_linear":37759.0,"score_lmetric":63274},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":97814,"score_linear":161627.0,"score_lmetric":304701},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":97814,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":37844.0,"chosen_score_lmetric":1068,"t_first_token":339282.159545662,"t_first_token_unix":1779770147.1734886,"t_done":339282.931234265,"t_done_unix":1779770147.9451776},{"request_id":"1366128:22:1403837:766","session_id":"1366128","input_length":97917,"t_proxy_recv":339282.947865387,"t_decision_unix":1779770147.961807,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9987234086011622,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (125 < 8000)","cache_hit":97792,"estimated_new_tokens":125,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":97917,"score_linear":190977.0,"score_lmetric":313722},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":97792,"new_prefill":125,"score_linear":37332.0,"score_lmetric":250},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":97917,"score_linear":310994.0,"score_lmetric":949700},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":97917,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":97917,"score_linear":107292.0,"score_lmetric":112025},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":121215,"ongoing_decode_tokens":0,"pending_prefill_tokens":17279,"num_requests":2,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":83456,"new_prefill":14461,"score_linear":37759.0,"score_lmetric":63480},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":97917,"score_linear":161627.0,"score_lmetric":305010},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22392,"ongoing_decode_tokens":0,"pending_prefill_tokens":19832,"num_requests":1,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":97917,"score_linear":22392.0,"score_lmetric":117749}],"chosen_score_linear":37332.0,"chosen_score_lmetric":250,"t_first_token":339283.159148552,"t_first_token_unix":1779770148.173092,"t_done":339283.341565695,"t_done_unix":1779770148.3555093},{"request_id":"1408889:3:1414177:827","session_id":"1408889","input_length":10633,"t_proxy_recv":339283.749087699,"t_decision_unix":1779770148.7630293,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":3,"affinity_cache_hit":4608,"affinity_cache_ratio":0.43336781717295214,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":10633,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":10633,"score_linear":190977.0,"score_lmetric":51870},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":244698,"ongoing_decode_tokens":135124,"pending_prefill_tokens":11782,"num_requests":3,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":0,"new_prefill":10633,"score_linear":244698.0,"score_lmetric":67245},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":10633,"score_linear":310994.0,"score_lmetric":600564},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":10633,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":10633,"score_linear":107292.0,"score_lmetric":24741},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":121215,"ongoing_decode_tokens":0,"pending_prefill_tokens":17279,"num_requests":2,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":0,"new_prefill":10633,"score_linear":121215.0,"score_lmetric":55824},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":10633,"score_linear":161627.0,"score_lmetric":43158},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22392,"ongoing_decode_tokens":0,"pending_prefill_tokens":19832,"num_requests":1,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":4608,"new_prefill":6025,"score_linear":17784.0,"score_lmetric":25857}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339284.581927398,"t_first_token_unix":1779770149.595871,"t_done":339285.598012566,"t_done_unix":1779770150.6119561},{"request_id":"1407995:2:1413725:826","session_id":"1407995","input_length":22392,"t_proxy_recv":339282.24733884,"t_decision_unix":1779770147.2612803,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":2560,"affinity_cache_ratio":0.1143265451947124,"affinity_num_requests":0,"avg_num_requests":2.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":2560,"estimated_new_tokens":19832,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":22392,"score_linear":190977.0,"score_lmetric":87147},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":232938,"ongoing_decode_tokens":232938,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1751,"cache_hit":0,"new_prefill":22392,"score_linear":232938.0,"score_lmetric":67176},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":22392,"score_linear":310994.0,"score_lmetric":647600},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9801,"ongoing_decode_tokens":9801,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":22392,"score_linear":9801.0,"score_lmetric":22392},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":22392,"score_linear":107292.0,"score_lmetric":36500},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":121215,"ongoing_decode_tokens":0,"pending_prefill_tokens":17279,"num_requests":2,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":0,"new_prefill":22392,"score_linear":121215.0,"score_lmetric":79342},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":22392,"score_linear":161627.0,"score_lmetric":78435},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":2560,"new_prefill":19832,"score_linear":-2560.0,"score_lmetric":0}],"chosen_score_linear":-2560.0,"chosen_score_lmetric":0,"t_first_token":339284.547256849,"t_first_token_unix":1779770149.5612004,"t_done":339286.88422583,"t_done_unix":1779770151.8981695},{"request_id":"1340290:11:1413050:822","session_id":"1340290","input_length":35022,"t_proxy_recv":339279.834849117,"t_decision_unix":1779770144.8487906,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":30208,"affinity_cache_ratio":0.8625435440580207,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4814 < 8000)","cache_hit":30208,"estimated_new_tokens":4814,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":35022,"score_linear":190977.0,"score_lmetric":125037},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":232781,"ongoing_decode_tokens":135124,"pending_prefill_tokens":12153,"num_requests":3,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":35022,"score_linear":232781.0,"score_lmetric":141525},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":35022,"score_linear":310994.0,"score_lmetric":698120},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":35022,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":35022,"score_linear":107292.0,"score_lmetric":49130},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":86193,"ongoing_decode_tokens":0,"pending_prefill_tokens":12465,"num_requests":1,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":30208,"new_prefill":4814,"score_linear":55985.0,"score_lmetric":17279},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":1024,"new_prefill":33998,"score_linear":160603.0,"score_lmetric":113253},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":35022,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":55985.0,"chosen_score_lmetric":17279,"t_first_token":339285.724079021,"t_first_token_unix":1779770150.7380273,"t_done":339288.667712215,"t_done_unix":1779770153.6816568},{"request_id":"1405155:2:1414214:828","session_id":"1405155","input_length":40330,"t_proxy_recv":339283.901924257,"t_decision_unix":1779770148.915866,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33280,"affinity_cache_ratio":0.825192164641706,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7050 < 8000)","cache_hit":33280,"estimated_new_tokens":7050,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":512,"new_prefill":39818,"score_linear":190465.0,"score_lmetric":139425},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":244698,"ongoing_decode_tokens":135124,"pending_prefill_tokens":11782,"num_requests":3,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":0,"new_prefill":40330,"score_linear":244698.0,"score_lmetric":156336},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":40330,"score_linear":310994.0,"score_lmetric":719352},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":10633,"ongoing_decode_tokens":0,"pending_prefill_tokens":10633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":40330,"score_linear":10633.0,"score_lmetric":50963},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":40330,"score_linear":107292.0,"score_lmetric":54438},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":121215,"ongoing_decode_tokens":0,"pending_prefill_tokens":17279,"num_requests":2,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":0,"new_prefill":40330,"score_linear":121215.0,"score_lmetric":115218},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":40330,"score_linear":161627.0,"score_lmetric":132249},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22392,"ongoing_decode_tokens":0,"pending_prefill_tokens":19832,"num_requests":1,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":33280,"new_prefill":7050,"score_linear":-10888.0,"score_lmetric":26882}],"chosen_score_linear":-10888.0,"chosen_score_lmetric":26882,"t_first_token":339286.237898011,"t_first_token_unix":1779770151.2518413,"t_done":339290.184027665,"t_done_unix":1779770155.1979718},{"request_id":"1364090:7:1412644:820","session_id":"1364090","input_length":86193,"t_proxy_recv":339278.445006353,"t_decision_unix":1779770143.458939,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":73728,"affinity_cache_ratio":0.8553826876892555,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":73728,"estimated_new_tokens":12465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":7680,"new_prefill":78513,"score_linear":183297.0,"score_lmetric":255510},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":232781,"ongoing_decode_tokens":135124,"pending_prefill_tokens":12153,"num_requests":3,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":86193,"score_linear":232781.0,"score_lmetric":295038},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":7680,"new_prefill":78513,"score_linear":263041.0,"score_lmetric":648444},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":86193,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":7680,"new_prefill":78513,"score_linear":99612.0,"score_lmetric":92621},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":73728,"new_prefill":12465,"score_linear":-73728.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":11264,"new_prefill":74929,"score_linear":150363.0,"score_lmetric":236046},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":86193,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":339284.684707585,"t_first_token_unix":1779770149.698651,"t_done":339291.208574047,"t_done_unix":1779770156.2225182},{"request_id":"1366128:23:1405900:781","session_id":"1366128","input_length":109574,"t_proxy_recv":339283.364347912,"t_decision_unix":1779770148.3782897,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.8924744921240441,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=83456 chosen=97792)","cache_hit":97792,"estimated_new_tokens":11782,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":109574,"score_linear":190977.0,"score_lmetric":348693},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":97792,"new_prefill":11782,"score_linear":37332.0,"score_lmetric":23564},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":109574,"score_linear":310994.0,"score_lmetric":996328},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":109574,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":109574,"score_linear":107292.0,"score_lmetric":123682},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":121215,"ongoing_decode_tokens":0,"pending_prefill_tokens":17279,"num_requests":2,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":83456,"new_prefill":26118,"score_linear":37759.0,"score_lmetric":86794},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":109574,"score_linear":161627.0,"score_lmetric":339981},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22392,"ongoing_decode_tokens":0,"pending_prefill_tokens":19832,"num_requests":1,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":109574,"score_linear":22392.0,"score_lmetric":129406}],"chosen_score_linear":37332.0,"chosen_score_lmetric":23564,"t_first_token":339290.933304232,"t_first_token_unix":1779770155.9472487,"t_done":339293.066884103,"t_done_unix":1779770158.0808277},{"request_id":"1366128:24:1407996:794","session_id":"1366128","input_length":109775,"t_proxy_recv":339293.086695881,"t_decision_unix":1779770158.1006374,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":109568,"affinity_cache_ratio":0.998114324755181,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (207 < 8000)","cache_hit":109568,"estimated_new_tokens":207,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":109775,"score_linear":190977.0,"score_lmetric":349296},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":109568,"new_prefill":207,"score_linear":25556.0,"score_lmetric":414},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":109775,"score_linear":310994.0,"score_lmetric":997132},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":109775,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":109775,"score_linear":107292.0,"score_lmetric":123883},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":83456,"new_prefill":26319,"score_linear":-83456.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":109775,"score_linear":161627.0,"score_lmetric":340584},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":109775,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":25556.0,"chosen_score_lmetric":414,"t_first_token":339293.352015868,"t_first_token_unix":1779770158.3659592,"t_done":339294.076909028,"t_done_unix":1779770159.0908537},{"request_id":"1399948:2:1405276:778","session_id":"1399948","input_length":6149,"t_proxy_recv":339252.778731357,"t_decision_unix":1779770117.7926729,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":4608,"affinity_cache_ratio":0.7493901447389819,"affinity_num_requests":3,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1541 < 8000)","cache_hit":4608,"estimated_new_tokens":1541,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":203775,"ongoing_decode_tokens":90517,"pending_prefill_tokens":3178,"num_requests":3,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":4608,"new_prefill":1541,"score_linear":199167.0,"score_lmetric":14157},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":6149,"score_linear":218746.0,"score_lmetric":119691},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":6149,"score_linear":270721.0,"score_lmetric":431352},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100766,"ongoing_decode_tokens":100766,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1514,"cache_hit":0,"new_prefill":6149,"score_linear":100766.0,"score_lmetric":6149},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":6149,"score_linear":106143.0,"score_lmetric":6149},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":6149,"score_linear":166406.0,"score_lmetric":337953},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":6149,"score_linear":21684.0,"score_lmetric":6149},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-77364,"ongoing_decode_tokens":0,"pending_prefill_tokens":-15412,"num_requests":1,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":6149,"score_linear":-77364.0,"score_lmetric":-9263}],"chosen_score_linear":199167.0,"chosen_score_lmetric":14157,"t_first_token":339294.408680131,"t_first_token_unix":1779770159.4226232,"t_done":339295.183577564,"t_done_unix":1779770160.1975207},{"request_id":"1399948:3:1405988:782","session_id":"1399948","input_length":7995,"t_proxy_recv":339295.186898148,"t_decision_unix":1779770160.2008398,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":6144,"affinity_cache_ratio":0.7684803001876173,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1851 < 8000)","cache_hit":6144,"estimated_new_tokens":1851,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":184828,"ongoing_decode_tokens":184828,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":6144,"new_prefill":1851,"score_linear":178684.0,"score_lmetric":3702},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":252870,"ongoing_decode_tokens":135124,"pending_prefill_tokens":8178,"num_requests":3,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":7995,"score_linear":252870.0,"score_lmetric":48519},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":7995,"score_linear":310994.0,"score_lmetric":55592},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":7995,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":7995,"score_linear":107292.0,"score_lmetric":22103},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":96849,"ongoing_decode_tokens":0,"pending_prefill_tokens":10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":7995,"score_linear":96849.0,"score_lmetric":18316},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":7995,"score_linear":161627.0,"score_lmetric":35244},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":0,"pending_prefill_tokens":2803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":7995,"score_linear":43763.0,"score_lmetric":10798}],"chosen_score_linear":178684.0,"chosen_score_lmetric":3702,"t_first_token":339295.517335979,"t_first_token_unix":1779770160.5312793,"t_done":339296.42833461,"t_done_unix":1779770161.4422781},{"request_id":"1399948:4:1407140:786","session_id":"1399948","input_length":8540,"t_proxy_recv":339296.431736989,"t_decision_unix":1779770161.4456787,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":7680,"affinity_cache_ratio":0.8992974238875878,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (860 < 8000)","cache_hit":7680,"estimated_new_tokens":860,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":184828,"ongoing_decode_tokens":184828,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":7680,"new_prefill":860,"score_linear":177148.0,"score_lmetric":1720},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":252870,"ongoing_decode_tokens":135124,"pending_prefill_tokens":8178,"num_requests":3,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":8540,"score_linear":252870.0,"score_lmetric":50154},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":8540,"score_linear":310994.0,"score_lmetric":57772},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":8540,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":8540,"score_linear":107292.0,"score_lmetric":22648},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":96849,"ongoing_decode_tokens":0,"pending_prefill_tokens":10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":8540,"score_linear":96849.0,"score_lmetric":18861},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":8540,"score_linear":161627.0,"score_lmetric":36879},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":0,"pending_prefill_tokens":2803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":8540,"score_linear":43763.0,"score_lmetric":11343}],"chosen_score_linear":177148.0,"chosen_score_lmetric":1720,"t_first_token":339296.550933395,"t_first_token_unix":1779770161.5648766,"t_done":339296.910922597,"t_done_unix":1779770161.9248655},{"request_id":"1366128:25:1411126:813","session_id":"1366128","input_length":117746,"t_proxy_recv":339294.101506708,"t_decision_unix":1779770159.115448,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":109568,"affinity_cache_ratio":0.9305454113090891,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=83456 chosen=109568)","cache_hit":109568,"estimated_new_tokens":8178,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":117746,"score_linear":190977.0,"score_lmetric":373209},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":109568,"new_prefill":8178,"score_linear":25556.0,"score_lmetric":16356},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":0,"pending_prefill_tokens":139508,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":117746,"score_linear":310994.0,"score_lmetric":1029016},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":117746,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":117746,"score_linear":107292.0,"score_lmetric":131854},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":83456,"new_prefill":34290,"score_linear":-83456.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":117746,"score_linear":161627.0,"score_lmetric":364497},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":117746,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":25556.0,"chosen_score_lmetric":16356,"t_first_token":339299.648641181,"t_first_token_unix":1779770164.6625848,"t_done":339300.987009407,"t_done_unix":1779770166.0009537},{"request_id":"1400379:4:1405483:779","session_id":"1400379","input_length":19966,"t_proxy_recv":339253.440051226,"t_decision_unix":1779770118.4539924,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":10752,"affinity_cache_ratio":0.5385154763097265,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (3584 < 4000)","cache_hit":10752,"estimated_new_tokens":9214,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":3584,"new_prefill":16382,"score_linear":206340.0,"score_lmetric":84404},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":19966,"score_linear":218746.0,"score_lmetric":161142},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":19966,"score_linear":270721.0,"score_lmetric":472803},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":101200,"ongoing_decode_tokens":101200,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1515,"cache_hit":0,"new_prefill":19966,"score_linear":101200.0,"score_lmetric":19966},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":19966,"score_linear":106143.0,"score_lmetric":19966},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":19966,"score_linear":166406.0,"score_lmetric":379404},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":10752,"new_prefill":9214,"score_linear":10932.0,"score_lmetric":9214},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-77364,"ongoing_decode_tokens":0,"pending_prefill_tokens":-15412,"num_requests":1,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":19966,"score_linear":-77364.0,"score_lmetric":4554}],"chosen_score_linear":10932.0,"chosen_score_lmetric":9214,"t_first_token":339254.678365792,"t_first_token_unix":1779770119.6923091,"t_done":339301.453236108,"t_done_unix":1779770166.4671795},{"request_id":"1381387:6:1409195:806","session_id":"1381387","input_length":71570,"t_proxy_recv":339271.463288298,"t_decision_unix":1779770136.47723,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":69632,"affinity_cache_ratio":0.9729216152019002,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1938 < 8000)","cache_hit":69632,"estimated_new_tokens":1938,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":119407,"ongoing_decode_tokens":0,"pending_prefill_tokens":4719,"num_requests":2,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":69632,"new_prefill":1938,"score_linear":49775.0,"score_lmetric":13314},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":220628,"ongoing_decode_tokens":220628,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1727,"cache_hit":0,"new_prefill":71570,"score_linear":220628.0,"score_lmetric":214710},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":71570,"score_linear":270721.0,"score_lmetric":627615},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":71570,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":71570,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103122,"ongoing_decode_tokens":103122,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":0,"new_prefill":71570,"score_linear":103122.0,"score_lmetric":143140},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":71570,"score_linear":161627.0,"score_lmetric":225969},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1785,"cache_hit":512,"new_prefill":71058,"score_linear":-512.0,"score_lmetric":0}],"chosen_score_linear":49775.0,"chosen_score_lmetric":13314,"t_first_token":339295.058622377,"t_first_token_unix":1779770160.072566,"t_done":339302.847723041,"t_done_unix":1779770167.861667},{"request_id":"1400379:5:1406380:784","session_id":"1400379","input_length":22009,"t_proxy_recv":339301.458750839,"t_decision_unix":1779770166.4726925,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9072652096869462,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2041 < 8000)","cache_hit":19968,"estimated_new_tokens":2041,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":230838,"ongoing_decode_tokens":193440,"pending_prefill_tokens":22,"num_requests":4,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":3584,"new_prefill":18425,"score_linear":227254.0,"score_lmetric":73788},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263113,"ongoing_decode_tokens":135124,"pending_prefill_tokens":10229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":22009,"score_linear":263113.0,"score_lmetric":96714},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":22009,"score_linear":310994.0,"score_lmetric":111648},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":22009,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":22009,"score_linear":107292.0,"score_lmetric":36117},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":96849,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":22009,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":141661,"ongoing_decode_tokens":141661,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":19968,"new_prefill":2041,"score_linear":121693.0,"score_lmetric":4082},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":0,"pending_prefill_tokens":2803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":22009,"score_linear":43763.0,"score_lmetric":24812}],"chosen_score_linear":121693.0,"chosen_score_lmetric":4082,"t_first_token":339301.867271906,"t_first_token_unix":1779770166.8812153,"t_done":339302.8673946,"t_done_unix":1779770167.881338},{"request_id":"1275274:2:1405679:780","session_id":"1275274","input_length":119977,"t_proxy_recv":339254.231228343,"t_decision_unix":1779770119.2451699,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":116224,"affinity_cache_ratio":0.9687190044758579,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3753 < 8000)","cache_hit":116224,"estimated_new_tokens":3753,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":209924,"ongoing_decode_tokens":90517,"pending_prefill_tokens":4719,"num_requests":4,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":119977,"score_linear":209924.0,"score_lmetric":498784},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":119977,"score_linear":218746.0,"score_lmetric":461175},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":119977,"score_linear":270721.0,"score_lmetric":772836},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":101396,"ongoing_decode_tokens":0,"pending_prefill_tokens":532,"num_requests":1,"active_p_offloads":0,"cached_blocks":1515,"cache_hit":0,"new_prefill":119977,"score_linear":101396.0,"score_lmetric":120509},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":119977,"score_linear":106143.0,"score_lmetric":119977},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":119977,"score_linear":166406.0,"score_lmetric":679437},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":41650,"ongoing_decode_tokens":21684,"pending_prefill_tokens":9214,"num_requests":2,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":116224,"new_prefill":3753,"score_linear":-74574.0,"score_lmetric":25934},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-77364,"ongoing_decode_tokens":0,"pending_prefill_tokens":-15412,"num_requests":1,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":119977,"score_linear":-77364.0,"score_lmetric":104565}],"chosen_score_linear":-74574.0,"chosen_score_lmetric":25934,"t_first_token":339300.821193565,"t_first_token_unix":1779770165.8351374,"t_done":339303.001085273,"t_done_unix":1779770168.015029},{"request_id":"1399948:5:1407922:791","session_id":"1399948","input_length":8612,"t_proxy_recv":339296.91443859,"t_decision_unix":1779770161.9283803,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":8192,"affinity_cache_ratio":0.9512308406874129,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (420 < 8000)","cache_hit":8192,"estimated_new_tokens":420,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":184828,"ongoing_decode_tokens":184828,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":8192,"new_prefill":420,"score_linear":176636.0,"score_lmetric":840},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":252870,"ongoing_decode_tokens":135124,"pending_prefill_tokens":8178,"num_requests":3,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":8612,"score_linear":252870.0,"score_lmetric":50370},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":8612,"score_linear":310994.0,"score_lmetric":58060},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":8612,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":8612,"score_linear":107292.0,"score_lmetric":22720},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":96849,"ongoing_decode_tokens":0,"pending_prefill_tokens":10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":8612,"score_linear":96849.0,"score_lmetric":18933},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":8612,"score_linear":161627.0,"score_lmetric":37095},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":0,"pending_prefill_tokens":2803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":8612,"score_linear":43763.0,"score_lmetric":11415}],"chosen_score_linear":176636.0,"chosen_score_lmetric":840,"t_first_token":339296.99581642,"t_first_token_unix":1779770162.0097597,"t_done":339303.58190152,"t_done_unix":1779770168.5958447},{"request_id":"1399948:6:1410696:812","session_id":"1399948","input_length":11381,"t_proxy_recv":339303.586654128,"t_decision_unix":1779770168.6005957,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":8192,"affinity_cache_ratio":0.7197961514805378,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3189 < 8000)","cache_hit":8192,"estimated_new_tokens":3189,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":8192,"new_prefill":3189,"score_linear":215072.0,"score_lmetric":9567},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263113,"ongoing_decode_tokens":135124,"pending_prefill_tokens":10229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":11381,"score_linear":263113.0,"score_lmetric":64830},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":11381,"score_linear":310994.0,"score_lmetric":69136},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":11381,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":11381,"score_linear":107292.0,"score_lmetric":25489},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43311,"ongoing_decode_tokens":96849,"pending_prefill_tokens":13534,"num_requests":2,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":11381,"score_linear":43311.0,"score_lmetric":49830},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1755,"cache_hit":0,"new_prefill":11381,"score_linear":21684.0,"score_lmetric":11381},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":11381,"score_linear":43763.0,"score_lmetric":11381}],"chosen_score_linear":215072.0,"chosen_score_lmetric":9567,"t_first_token":339303.957295875,"t_first_token_unix":1779770168.9712396,"t_done":339304.282302407,"t_done_unix":1779770169.2962465},{"request_id":"1399948:7:1413205:824","session_id":"1399948","input_length":11537,"t_proxy_recv":339304.286247639,"t_decision_unix":1779770169.3001895,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9763370026870071,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (273 < 8000)","cache_hit":11264,"estimated_new_tokens":273,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":11264,"new_prefill":273,"score_linear":212000.0,"score_lmetric":819},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263113,"ongoing_decode_tokens":135124,"pending_prefill_tokens":10229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":11537,"score_linear":263113.0,"score_lmetric":65298},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":11537,"score_linear":310994.0,"score_lmetric":69760},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":11537,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":11537,"score_linear":107292.0,"score_lmetric":25645},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43311,"ongoing_decode_tokens":96849,"pending_prefill_tokens":13534,"num_requests":2,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":11537,"score_linear":43311.0,"score_lmetric":50142},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1755,"cache_hit":0,"new_prefill":11537,"score_linear":21684.0,"score_lmetric":11537},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":11537,"score_linear":43763.0,"score_lmetric":11537}],"chosen_score_linear":212000.0,"chosen_score_lmetric":819,"t_first_token":339304.387632835,"t_first_token_unix":1779770169.4015768,"t_done":339305.198791437,"t_done_unix":1779770170.2127354},{"request_id":"1362265:8:1419710:834","session_id":"1362265","input_length":11132,"t_proxy_recv":339303.333436413,"t_decision_unix":1779770168.347378,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9658641753503413,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (380 < 8000)","cache_hit":10752,"estimated_new_tokens":380,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":231876,"ongoing_decode_tokens":159268,"pending_prefill_tokens":928,"num_requests":4,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":11132,"score_linear":231876.0,"score_lmetric":48240},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263113,"ongoing_decode_tokens":135124,"pending_prefill_tokens":10229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":11132,"score_linear":263113.0,"score_lmetric":64083},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":11132,"score_linear":310994.0,"score_lmetric":68140},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":11132,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":11132,"score_linear":107292.0,"score_lmetric":25240},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32179,"ongoing_decode_tokens":96849,"pending_prefill_tokens":13154,"num_requests":1,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":10752,"new_prefill":380,"score_linear":21427.0,"score_lmetric":13534},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1755,"cache_hit":0,"new_prefill":11132,"score_linear":21684.0,"score_lmetric":11132},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":6656,"new_prefill":4476,"score_linear":37107.0,"score_lmetric":4476}],"chosen_score_linear":21427.0,"chosen_score_lmetric":13534,"t_first_token":339307.132997062,"t_first_token_unix":1779770172.1469405,"t_done":339307.911703551,"t_done_unix":1779770172.9256468},{"request_id":"1404395:1:1404395:770","session_id":"1404395","input_length":21684,"t_proxy_recv":339249.724454198,"t_decision_unix":1779770114.7383955,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":21684,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69508,"ongoing_decode_tokens":69508,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":21684,"score_linear":69508.0,"score_lmetric":21684},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":512,"new_prefill":21172,"score_linear":218234.0,"score_lmetric":164760},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":185283,"ongoing_decode_tokens":0,"pending_prefill_tokens":133605,"num_requests":2,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":21684,"score_linear":185283.0,"score_lmetric":310578},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":144686,"ongoing_decode_tokens":144686,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":21684,"score_linear":144686.0,"score_lmetric":43368},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":512,"new_prefill":21172,"score_linear":105631.0,"score_lmetric":21172},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":21684,"score_linear":166406.0,"score_lmetric":384558},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":21684,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":21684,"score_linear":-121811.0,"score_lmetric":38175}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339252.083837078,"t_first_token_unix":1779770117.0977807,"t_done":339308.293196217,"t_done_unix":1779770173.3071396},{"request_id":"1364090:8:1417347:830","session_id":"1364090","input_length":96849,"t_proxy_recv":339295.10181627,"t_decision_unix":1779770160.1157575,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":86528,"affinity_cache_ratio":0.893432043696889,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":86528,"estimated_new_tokens":10321,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":190977,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":7680,"new_prefill":89169,"score_linear":183297.0,"score_lmetric":267507},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":252870,"ongoing_decode_tokens":135124,"pending_prefill_tokens":8178,"num_requests":3,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":96849,"score_linear":252870.0,"score_lmetric":315081},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":7680,"new_prefill":89169,"score_linear":303314.0,"score_lmetric":380288},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":96849,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":7680,"new_prefill":89169,"score_linear":99612.0,"score_lmetric":103277},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":86528,"new_prefill":10321,"score_linear":-86528.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":11264,"new_prefill":85585,"score_linear":150363.0,"score_lmetric":268014},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":0,"pending_prefill_tokens":2803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":96849,"score_linear":43763.0,"score_lmetric":99652}],"chosen_score_linear":-86528.0,"chosen_score_lmetric":0,"t_first_token":339301.001446593,"t_first_token_unix":1779770166.01539,"t_done":339310.539803425,"t_done_unix":1779770175.553747},{"request_id":"1340290:12:1421294:835","session_id":"1340290","input_length":35303,"t_proxy_recv":339308.984024069,"t_decision_unix":1779770173.9979653,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9862051383735093,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (487 < 8000)","cache_hit":34816,"estimated_new_tokens":487,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":35303,"score_linear":223264.0,"score_lmetric":105909},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263113,"ongoing_decode_tokens":263113,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":35303,"score_linear":263113.0,"score_lmetric":105909},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":35303,"score_linear":310994.0,"score_lmetric":164824},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":35303,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":35303,"score_linear":107292.0,"score_lmetric":49411},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32179,"ongoing_decode_tokens":129028,"pending_prefill_tokens":-10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":34816,"new_prefill":487,"score_linear":-2637.0,"score_lmetric":-9834},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":1024,"new_prefill":34279,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":35303,"score_linear":43763.0,"score_lmetric":35303}],"chosen_score_linear":-2637.0,"chosen_score_lmetric":-9834,"t_first_token":339309.085508281,"t_first_token_unix":1779770174.0994523,"t_done":339311.628724489,"t_done_unix":1779770176.642668},{"request_id":"1362265:9:1422050:838","session_id":"1362265","input_length":11879,"t_proxy_recv":339311.800722421,"t_decision_unix":1779770176.814664,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9051266941661756,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1127 < 8000)","cache_hit":10752,"estimated_new_tokens":1127,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":11879,"score_linear":223264.0,"score_lmetric":35637},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263113,"ongoing_decode_tokens":263113,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":11879,"score_linear":263113.0,"score_lmetric":35637},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":11879,"score_linear":310994.0,"score_lmetric":71128},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49308,"ongoing_decode_tokens":49308,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":11879,"score_linear":49308.0,"score_lmetric":11879},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":11879,"score_linear":107292.0,"score_lmetric":25987},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-64670,"ongoing_decode_tokens":32179,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2256,"cache_hit":10752,"new_prefill":1127,"score_linear":-75422.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":11879,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":6656,"new_prefill":5223,"score_linear":37107.0,"score_lmetric":5223}],"chosen_score_linear":-75422.0,"chosen_score_lmetric":0,"t_first_token":339311.938474576,"t_first_token_unix":1779770176.9524179,"t_done":339312.207200023,"t_done_unix":1779770177.2211435},{"request_id":"1268861:9:1421319:836","session_id":"1268861","input_length":49308,"t_proxy_recv":339309.066631095,"t_decision_unix":1779770174.0805724,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":44032,"affinity_cache_ratio":0.8929991076498742,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5276 < 8000)","cache_hit":44032,"estimated_new_tokens":5276,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":49308,"score_linear":223264.0,"score_lmetric":147924},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263113,"ongoing_decode_tokens":263113,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":49308,"score_linear":263113.0,"score_lmetric":147924},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":49308,"score_linear":310994.0,"score_lmetric":220844},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":44032,"new_prefill":5276,"score_linear":-44032.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":49308,"score_linear":107292.0,"score_lmetric":63416},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":67482,"ongoing_decode_tokens":129028,"pending_prefill_tokens":-9834,"num_requests":2,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":49308,"score_linear":67482.0,"score_lmetric":78948},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":49308,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":49308,"score_linear":43763.0,"score_lmetric":49308}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":339310.638715948,"t_first_token_unix":1779770175.6526597,"t_done":339312.316086254,"t_done_unix":1779770177.3300295},{"request_id":"1270340:5:1417075:829","session_id":"1270340","input_length":43763,"t_proxy_recv":339294.299807224,"t_decision_unix":1779770159.3137488,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":40960,"affinity_cache_ratio":0.9359504604346137,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2803 < 8000)","cache_hit":40960,"estimated_new_tokens":2803,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":43763,"score_linear":190977.0,"score_lmetric":151260},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":252870,"ongoing_decode_tokens":135124,"pending_prefill_tokens":8178,"num_requests":3,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":43763,"score_linear":252870.0,"score_lmetric":155823},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":124090,"pending_prefill_tokens":15418,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":43763,"score_linear":310994.0,"score_lmetric":236724},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":43763,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":1536,"new_prefill":42227,"score_linear":105756.0,"score_lmetric":56335},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":43763,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":43763,"score_linear":161627.0,"score_lmetric":142548},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":40960,"new_prefill":2803,"score_linear":-40960.0,"score_lmetric":0}],"chosen_score_linear":-40960.0,"chosen_score_lmetric":0,"t_first_token":339301.741763517,"t_first_token_unix":1779770166.7557075,"t_done":339312.719315396,"t_done_unix":1779770177.7332587},{"request_id":"1366128:26:1413108:823","session_id":"1366128","input_length":127989,"t_proxy_recv":339301.018301168,"t_decision_unix":1779770166.0322425,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":117760,"affinity_cache_ratio":0.9200790692950176,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=83456 chosen=117760)","cache_hit":117760,"estimated_new_tokens":10229,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":230838,"ongoing_decode_tokens":193440,"pending_prefill_tokens":22,"num_requests":4,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":127989,"score_linear":230838.0,"score_lmetric":512044},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":117760,"new_prefill":10229,"score_linear":17364.0,"score_lmetric":20458},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":127989,"score_linear":310994.0,"score_lmetric":535568},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":127989,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":127989,"score_linear":107292.0,"score_lmetric":142097},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":96849,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":83456,"new_prefill":44533,"score_linear":-83456.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":161627,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":127989,"score_linear":161627.0,"score_lmetric":383967},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":0,"pending_prefill_tokens":2803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":127989,"score_linear":43763.0,"score_lmetric":130792}],"chosen_score_linear":17364.0,"chosen_score_lmetric":20458,"t_first_token":339308.772925269,"t_first_token_unix":1779770173.786869,"t_done":339313.841215136,"t_done_unix":1779770178.855159},{"request_id":"1270606:2:1411654:817","session_id":"1270606","input_length":107292,"t_proxy_recv":339275.236746182,"t_decision_unix":1779770140.2506876,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":93184,"affinity_cache_ratio":0.8685083696827349,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":93184,"estimated_new_tokens":14108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":107292,"score_linear":190977.0,"score_lmetric":341847},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":232781,"ongoing_decode_tokens":135124,"pending_prefill_tokens":12153,"num_requests":3,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":0,"new_prefill":107292,"score_linear":232781.0,"score_lmetric":358335},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":107292,"score_linear":270721.0,"score_lmetric":734781},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":0,"new_prefill":107292,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":93184,"new_prefill":14108,"score_linear":-93184.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103546,"ongoing_decode_tokens":103546,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":107292,"score_linear":103546.0,"score_lmetric":214584},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":107292,"score_linear":161627.0,"score_lmetric":333135},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":1536,"new_prefill":105756,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":339312.879939122,"t_first_token_unix":1779770177.8938825,"t_done":339314.23263551,"t_done_unix":1779770179.2465792},{"request_id":"1366128:27:1417788:831","session_id":"1366128","input_length":128433,"t_proxy_recv":339313.861085403,"t_decision_unix":1779770178.875027,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":128000,"affinity_cache_ratio":0.9966285923399749,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (433 < 8000)","cache_hit":128000,"estimated_new_tokens":433,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":128433,"score_linear":223264.0,"score_lmetric":385299},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1811,"cache_hit":128000,"new_prefill":433,"score_linear":7124.0,"score_lmetric":866},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":128433,"score_linear":310994.0,"score_lmetric":537344},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":128433,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":107292,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":128433,"score_linear":107292.0,"score_lmetric":128433},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-64670,"ongoing_decode_tokens":32179,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":83456,"new_prefill":44977,"score_linear":-148126.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":128433,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":0,"new_prefill":128433,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":7124.0,"chosen_score_lmetric":866,"t_first_token":339314.260159245,"t_first_token_unix":1779770179.2741024,"t_done":339316.076875807,"t_done_unix":1779770181.09082},{"request_id":"1313181:8:1329951:394","session_id":"1313181","input_length":124090,"t_proxy_recv":339244.807077365,"t_decision_unix":1779770109.821019,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":2,"affinity_cache_hit":61952,"affinity_cache_ratio":0.499250543960029,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":-124090,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":124090,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81019,"ongoing_decode_tokens":81019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1694,"cache_hit":0,"new_prefill":124090,"score_linear":81019.0,"score_lmetric":248180},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":124090,"score_linear":218746.0,"score_lmetric":473514},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":124090,"score_linear":-11298.0,"score_lmetric":-124090},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1506,"cache_hit":12800,"new_prefill":111290,"score_linear":85467.0,"score_lmetric":111290},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":124090,"score_linear":106143.0,"score_lmetric":124090},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":124090,"score_linear":166406.0,"score_lmetric":691776},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1705,"cache_hit":12800,"new_prefill":111290,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":61952,"new_prefill":62138,"score_linear":-183763.0,"score_lmetric":-2279}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-124090,"t_first_token":339294.266265257,"t_first_token_unix":1779770159.2802086,"t_done":339317.033697912,"t_done_unix":1779770182.0476418},{"request_id":"1366128:28:1421998:837","session_id":"1366128","input_length":128642,"t_proxy_recv":339316.097860833,"t_decision_unix":1779770181.111802,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":128512,"affinity_cache_ratio":0.9989894435720837,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (130 < 8000)","cache_hit":128512,"estimated_new_tokens":130,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":128642,"score_linear":223264.0,"score_lmetric":385926},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":128512,"new_prefill":130,"score_linear":6612.0,"score_lmetric":260},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":128642,"score_linear":310994.0,"score_lmetric":538180},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":128642,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":128642,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-64670,"ongoing_decode_tokens":32179,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":83456,"new_prefill":45186,"score_linear":-148126.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":128642,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":0,"new_prefill":128642,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":6612.0,"chosen_score_lmetric":260,"t_first_token":339316.462255989,"t_first_token_unix":1779770181.4762,"t_done":339317.427930891,"t_done_unix":1779770182.4418745},{"request_id":"1313181:8:1385690:686","session_id":"1313181","input_length":62444,"t_proxy_recv":339317.043476154,"t_decision_unix":1779770182.0574176,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":7,"affinity_cache_hit":61952,"affinity_cache_ratio":0.9921209403625648,"affinity_num_requests":3,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (492 < 8000)","cache_hit":61952,"estimated_new_tokens":492,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":62444,"score_linear":223264.0,"score_lmetric":187332},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263766,"ongoing_decode_tokens":263766,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":62444,"score_linear":263766.0,"score_lmetric":187332},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186904,"ongoing_decode_tokens":72491,"pending_prefill_tokens":5903,"num_requests":3,"active_p_offloads":0,"cached_blocks":2599,"cache_hit":61952,"new_prefill":492,"score_linear":124952.0,"score_lmetric":19185},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":12800,"new_prefill":49644,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":62444,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-64670,"ongoing_decode_tokens":32179,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":0,"new_prefill":62444,"score_linear":-64670.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":12800,"new_prefill":49644,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":61952,"new_prefill":492,"score_linear":-61952.0,"score_lmetric":0}],"chosen_score_linear":-61952.0,"chosen_score_lmetric":0,"t_first_token":339317.155617048,"t_first_token_unix":1779770182.1695611,"t_done":339317.657280882,"t_done_unix":1779770182.671225},{"request_id":"1363943:11:1424840:842","session_id":"1363943","input_length":25225,"t_proxy_recv":339321.644519809,"t_decision_unix":1779770186.6584609,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24576,"affinity_cache_ratio":0.9742715559960357,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (649 < 8000)","cache_hit":24576,"estimated_new_tokens":649,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":8704,"new_prefill":16521,"score_linear":214560.0,"score_lmetric":49563},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":25225,"score_linear":261322.0,"score_lmetric":75675},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186904,"ongoing_decode_tokens":157929,"pending_prefill_tokens":1873,"num_requests":3,"active_p_offloads":0,"cached_blocks":2599,"cache_hit":4096,"new_prefill":21129,"score_linear":182808.0,"score_lmetric":69006},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":25225,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":4096,"new_prefill":21129,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":36762,"ongoing_decode_tokens":133611,"pending_prefill_tokens":-10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":4096,"new_prefill":21129,"score_linear":32666.0,"score_lmetric":10808},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":24576,"new_prefill":649,"score_linear":-24576.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":67209,"num_requests":1,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":25225,"score_linear":129673.0,"score_lmetric":92434}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":339321.748829596,"t_first_token_unix":1779770186.7627728,"t_done":339322.728868689,"t_done_unix":1779770187.7428122},{"request_id":"1264679:2:1405259:777","session_id":"1264679","input_length":113258,"t_proxy_recv":339252.754251984,"t_decision_unix":1779770117.7681932,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":110080,"affinity_cache_ratio":0.9719401719966801,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3178 < 8000)","cache_hit":110080,"estimated_new_tokens":3178,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":90517,"ongoing_decode_tokens":90517,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":110080,"new_prefill":3178,"score_linear":-19563.0,"score_lmetric":6356},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":113258,"score_linear":218746.0,"score_lmetric":441018},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":113258,"score_linear":270721.0,"score_lmetric":752679},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100766,"ongoing_decode_tokens":100766,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1514,"cache_hit":0,"new_prefill":113258,"score_linear":100766.0,"score_lmetric":113258},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":113258,"score_linear":106143.0,"score_lmetric":113258},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":113258,"score_linear":166406.0,"score_lmetric":659280},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":113258,"score_linear":21684.0,"score_lmetric":113258},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-77364,"ongoing_decode_tokens":0,"pending_prefill_tokens":-15412,"num_requests":1,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":113258,"score_linear":-77364.0,"score_lmetric":97846}],"chosen_score_linear":-19563.0,"chosen_score_lmetric":6356,"t_first_token":339294.407314647,"t_first_token_unix":1779770159.4212582,"t_done":339322.976077425,"t_done_unix":1779770187.9900212},{"request_id":"1399948:8:1425345:844","session_id":"1399948","input_length":11736,"t_proxy_recv":339323.527627063,"t_decision_unix":1779770188.5415683,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9597818677573279,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (472 < 8000)","cache_hit":11264,"estimated_new_tokens":472,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110006,"ongoing_decode_tokens":110006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":11264,"new_prefill":472,"score_linear":98742.0,"score_lmetric":944},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":11736,"score_linear":261322.0,"score_lmetric":35208},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186904,"ongoing_decode_tokens":157929,"pending_prefill_tokens":1873,"num_requests":3,"active_p_offloads":0,"cached_blocks":2599,"cache_hit":0,"new_prefill":11736,"score_linear":186904.0,"score_lmetric":40827},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":11736,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":11736,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":36762,"ongoing_decode_tokens":133611,"pending_prefill_tokens":-10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":0,"new_prefill":11736,"score_linear":36762.0,"score_lmetric":1415},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":11736,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":169462,"ongoing_decode_tokens":0,"pending_prefill_tokens":69110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":11736,"score_linear":169462.0,"score_lmetric":161692}],"chosen_score_linear":98742.0,"chosen_score_lmetric":944,"t_first_token":339323.619957085,"t_first_token_unix":1779770188.6339004,"t_done":339323.93594871,"t_done_unix":1779770188.9498923},{"request_id":"1364128:3:1419644:833","session_id":"1364128","input_length":32179,"t_proxy_recv":339303.111566052,"t_decision_unix":1779770168.1255076,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":5,"affinity_cache_hit":8704,"affinity_cache_ratio":0.2704869635476553,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_not_meaningfully_more_cache (src=8704 chosen=8704)","cache_hit":8704,"estimated_new_tokens":23475,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":231876,"ongoing_decode_tokens":159268,"pending_prefill_tokens":928,"num_requests":4,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":32179,"score_linear":231876.0,"score_lmetric":132428},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263113,"ongoing_decode_tokens":135124,"pending_prefill_tokens":10229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":32179,"score_linear":263113.0,"score_lmetric":127224},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":32179,"score_linear":310994.0,"score_lmetric":152328},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":32179,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":32179,"score_linear":107292.0,"score_lmetric":46287},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":96849,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":8704,"new_prefill":23475,"score_linear":-8704.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1755,"cache_hit":0,"new_prefill":32179,"score_linear":21684.0,"score_lmetric":32179},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":8704,"new_prefill":23475,"score_linear":35059.0,"score_lmetric":23475}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":339307.129774246,"t_first_token_unix":1779770172.1437185,"t_done":339324.595956533,"t_done_unix":1779770189.6098998},{"request_id":"1364090:9:1423736:840","session_id":"1364090","input_length":101432,"t_proxy_recv":339317.918262723,"t_decision_unix":1779770182.932204,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":97280,"affinity_cache_ratio":0.9590661724110734,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4152 < 8000)","cache_hit":97280,"estimated_new_tokens":4152,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":7680,"new_prefill":93752,"score_linear":215584.0,"score_lmetric":281256},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":101432,"score_linear":135124.0,"score_lmetric":202864},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186904,"ongoing_decode_tokens":72491,"pending_prefill_tokens":5903,"num_requests":3,"active_p_offloads":0,"cached_blocks":2599,"cache_hit":7680,"new_prefill":93752,"score_linear":179224.0,"score_lmetric":298965},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":101432,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":7680,"new_prefill":93752,"score_linear":-7680.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-64670,"ongoing_decode_tokens":32179,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":97280,"new_prefill":4152,"score_linear":-161950.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":11264,"new_prefill":90168,"score_linear":-11264.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":67209,"num_requests":1,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":101432,"score_linear":129673.0,"score_lmetric":168641}],"chosen_score_linear":-161950.0,"chosen_score_lmetric":0,"t_first_token":339320.587387945,"t_first_token_unix":1779770185.6013317,"t_done":339324.863429251,"t_done_unix":1779770189.8773766},{"request_id":"1253743:8:1404284:769","session_id":"1253743","input_length":63643,"t_proxy_recv":339249.340760382,"t_decision_unix":1779770114.3547015,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9895196643778578,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (667 < 8000)","cache_hit":62976,"estimated_new_tokens":667,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69508,"ongoing_decode_tokens":0,"pending_prefill_tokens":1412,"num_requests":1,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":63643,"score_linear":69508.0,"score_lmetric":65055},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":4608,"new_prefill":59035,"score_linear":214138.0,"score_lmetric":278349},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":121640,"ongoing_decode_tokens":0,"pending_prefill_tokens":132938,"num_requests":1,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":62976,"new_prefill":667,"score_linear":58664.0,"score_lmetric":133605},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":144686,"ongoing_decode_tokens":144686,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":49664,"new_prefill":13979,"score_linear":95022.0,"score_lmetric":27958},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":63643,"score_linear":106143.0,"score_lmetric":63643},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":63643,"score_linear":166406.0,"score_lmetric":510435},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":63643,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":63643,"score_linear":-121811.0,"score_lmetric":-3784}],"chosen_score_linear":58664.0,"chosen_score_lmetric":133605,"t_first_token":339294.920852221,"t_first_token_unix":1779770159.9347963,"t_done":339325.128928122,"t_done_unix":1779770190.1428723},{"request_id":"1403188:1:1403188:764","session_id":"1403188","input_length":8848,"t_proxy_recv":339245.813279489,"t_decision_unix":1779770110.8272204,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":8848,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81019,"ongoing_decode_tokens":81019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1694,"cache_hit":0,"new_prefill":8848,"score_linear":81019.0,"score_lmetric":17696},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":8848,"score_linear":218746.0,"score_lmetric":127788},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":112792,"ongoing_decode_tokens":0,"pending_prefill_tokens":124090,"num_requests":0,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":8848,"score_linear":112792.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":100340,"ongoing_decode_tokens":100340,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1508,"cache_hit":0,"new_prefill":8848,"score_linear":100340.0,"score_lmetric":8848},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":8848,"score_linear":106143.0,"score_lmetric":8848},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":8848,"score_linear":166406.0,"score_lmetric":346050},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":11216,"ongoing_decode_tokens":0,"pending_prefill_tokens":11216,"num_requests":1,"active_p_offloads":0,"cached_blocks":1705,"cache_hit":0,"new_prefill":8848,"score_linear":11216.0,"score_lmetric":20064},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-121811,"ongoing_decode_tokens":0,"pending_prefill_tokens":-59859,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":8848,"score_linear":-121811.0,"score_lmetric":51011}],"chosen_score_linear":112792.0,"chosen_score_lmetric":0,"t_first_token":339294.922075796,"t_first_token_unix":1779770159.9360194,"t_done":339325.610018824,"t_done_unix":1779770190.6239624},{"request_id":"1403188:2:1408249:796","session_id":"1403188","input_length":14728,"t_proxy_recv":339325.620582578,"t_decision_unix":1779770190.634524,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":8704,"affinity_cache_ratio":0.5909831613253667,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6024 < 8000)","cache_hit":8704,"estimated_new_tokens":6024,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110006,"ongoing_decode_tokens":110006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":14728,"score_linear":110006.0,"score_lmetric":29456},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":14728,"score_linear":261322.0,"score_lmetric":44184},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":114413,"ongoing_decode_tokens":125711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2617,"cache_hit":8704,"new_prefill":6024,"score_linear":105709.0,"score_lmetric":6024},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":14728,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":14728,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-96849,"ongoing_decode_tokens":0,"pending_prefill_tokens":-10321,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":0,"new_prefill":14728,"score_linear":-96849.0,"score_lmetric":-4407},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":14728,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":169462,"ongoing_decode_tokens":0,"pending_prefill_tokens":69110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":14728,"score_linear":169462.0,"score_lmetric":167676}],"chosen_score_linear":105709.0,"chosen_score_lmetric":6024,"t_first_token":339326.32158069,"t_first_token_unix":1779770191.3355246,"t_done":339327.065910309,"t_done_unix":1779770192.0798545},{"request_id":"1277428:8:1412675:821","session_id":"1277428","input_length":40273,"t_proxy_recv":339278.54192133,"t_decision_unix":1779770143.5558627,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9534924142725896,"affinity_num_requests":3,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1873 < 8000)","cache_hit":38400,"estimated_new_tokens":1873,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":190977,"ongoing_decode_tokens":0,"pending_prefill_tokens":6657,"num_requests":3,"active_p_offloads":0,"cached_blocks":1764,"cache_hit":0,"new_prefill":40273,"score_linear":190977.0,"score_lmetric":140790},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":232781,"ongoing_decode_tokens":135124,"pending_prefill_tokens":12153,"num_requests":3,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":4608,"new_prefill":35665,"score_linear":228173.0,"score_lmetric":143454},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":270721,"ongoing_decode_tokens":0,"pending_prefill_tokens":137635,"num_requests":3,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":38400,"new_prefill":1873,"score_linear":232321.0,"score_lmetric":418524},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1526,"cache_hit":36352,"new_prefill":3921,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":40273,"score_linear":107292.0,"score_lmetric":54381},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":86193,"ongoing_decode_tokens":0,"pending_prefill_tokens":12465,"num_requests":1,"active_p_offloads":0,"cached_blocks":2200,"cache_hit":0,"new_prefill":40273,"score_linear":86193.0,"score_lmetric":52738},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":40273,"score_linear":161627.0,"score_lmetric":132078},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":40273,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":232321.0,"chosen_score_lmetric":418524,"t_first_token":339325.046983535,"t_first_token_unix":1779770190.0609272,"t_done":339328.404219688,"t_done_unix":1779770193.4181633},{"request_id":"1373577:3:1404917:774","session_id":"1373577","input_length":85438,"t_proxy_recv":339251.62502822,"t_decision_unix":1779770116.6389697,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":81408,"affinity_cache_ratio":0.9528312928673424,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4030 < 8000)","cache_hit":81408,"estimated_new_tokens":4030,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":90517,"ongoing_decode_tokens":69508,"pending_prefill_tokens":13329,"num_requests":2,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":85438,"score_linear":90517.0,"score_lmetric":197534},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":218746,"ongoing_decode_tokens":83622,"pending_prefill_tokens":33748,"num_requests":3,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":0,"new_prefill":85438,"score_linear":218746.0,"score_lmetric":357558},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":185283,"ongoing_decode_tokens":0,"pending_prefill_tokens":133605,"num_requests":2,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":81408,"new_prefill":4030,"score_linear":103875.0,"score_lmetric":275270},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":144914,"ongoing_decode_tokens":144914,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1509,"cache_hit":0,"new_prefill":85438,"score_linear":144914.0,"score_lmetric":170876},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":2560,"new_prefill":82878,"score_linear":103583.0,"score_lmetric":82878},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":166406,"ongoing_decode_tokens":0,"pending_prefill_tokens":106502,"num_requests":3,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":85438,"score_linear":166406.0,"score_lmetric":575820},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21684,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":85438,"score_linear":21684.0,"score_lmetric":107122},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":-82608,"ongoing_decode_tokens":0,"pending_prefill_tokens":-58544,"num_requests":0,"active_p_offloads":0,"cached_blocks":1629,"cache_hit":0,"new_prefill":85438,"score_linear":-82608.0,"score_lmetric":0}],"chosen_score_linear":103875.0,"chosen_score_lmetric":275270,"t_first_token":339320.404993163,"t_first_token_unix":1779770185.4189372,"t_done":339329.643488375,"t_done_unix":1779770194.6574326},{"request_id":"1270606:3:1427164:848","session_id":"1270606","input_length":108318,"t_proxy_recv":339329.96948758,"t_decision_unix":1779770194.983429,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":107008,"affinity_cache_ratio":0.987905980538784,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1310 < 8000)","cache_hit":107008,"estimated_new_tokens":1310,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110006,"ongoing_decode_tokens":110006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":108318,"score_linear":110006.0,"score_lmetric":216636},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":108318,"score_linear":261322.0,"score_lmetric":324954},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":121862,"ongoing_decode_tokens":42342,"pending_prefill_tokens":5314,"num_requests":1,"active_p_offloads":0,"cached_blocks":2640,"cache_hit":0,"new_prefill":108318,"score_linear":121862.0,"score_lmetric":113632},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":108318,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":107008,"new_prefill":1310,"score_linear":-107008.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":0,"pending_prefill_tokens":6644,"num_requests":1,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":0,"new_prefill":108318,"score_linear":108020.0,"score_lmetric":114962},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":26923,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":108318,"score_linear":26923.0,"score_lmetric":108318},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":169462,"ongoing_decode_tokens":0,"pending_prefill_tokens":69110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":1536,"new_prefill":106782,"score_linear":167926.0,"score_lmetric":351784}],"chosen_score_linear":-107008.0,"chosen_score_lmetric":0,"t_first_token":339330.713241437,"t_first_token_unix":1779770195.7271852,"t_done":339331.008503193,"t_done_unix":1779770196.022447},{"request_id":"1363943:12:1426646:845","session_id":"1363943","input_length":26923,"t_proxy_recv":339328.117901561,"t_decision_unix":1779770193.1318426,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9318426624076068,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1835 < 8000)","cache_hit":25088,"estimated_new_tokens":1835,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110006,"ongoing_decode_tokens":110006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":8704,"new_prefill":18219,"score_linear":101302.0,"score_lmetric":36438},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":26923,"score_linear":261322.0,"score_lmetric":80769},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":114413,"ongoing_decode_tokens":125711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2628,"cache_hit":4096,"new_prefill":22827,"score_linear":110317.0,"score_lmetric":22827},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":26923,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":4096,"new_prefill":22827,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":4096,"new_prefill":22827,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":25088,"new_prefill":1835,"score_linear":-25088.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":169462,"ongoing_decode_tokens":0,"pending_prefill_tokens":69110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":26923,"score_linear":169462.0,"score_lmetric":192066}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":0,"t_first_token":339328.477634106,"t_first_token_unix":1779770193.4915774,"t_done":339332.361572125,"t_done_unix":1779770197.3755162},{"request_id":"1301929:3:1417873:832","session_id":"1301929","input_length":37398,"t_proxy_recv":339296.93614422,"t_decision_unix":1779770161.9500856,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":37376,"affinity_cache_ratio":0.9994117332477672,"affinity_num_requests":3,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (22 < 8000)","cache_hit":37376,"estimated_new_tokens":22,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":193440,"ongoing_decode_tokens":184828,"pending_prefill_tokens":420,"num_requests":3,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":37376,"new_prefill":22,"score_linear":156064.0,"score_lmetric":1326},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":252870,"ongoing_decode_tokens":135124,"pending_prefill_tokens":8178,"num_requests":3,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":0,"new_prefill":37398,"score_linear":252870.0,"score_lmetric":136728},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":37398,"score_linear":310994.0,"score_lmetric":173204},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":37398,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":37398,"score_linear":107292.0,"score_lmetric":51506},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":96849,"ongoing_decode_tokens":0,"pending_prefill_tokens":10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":37398,"score_linear":96849.0,"score_lmetric":47719},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":161627,"ongoing_decode_tokens":41650,"pending_prefill_tokens":3753,"num_requests":3,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":37398,"score_linear":161627.0,"score_lmetric":123453},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":0,"pending_prefill_tokens":2803,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":37398,"score_linear":43763.0,"score_lmetric":40201}],"chosen_score_linear":156064.0,"chosen_score_lmetric":1326,"t_first_token":339302.753788432,"t_first_token_unix":1779770167.767732,"t_done":339332.900602395,"t_done_unix":1779770197.9145584},{"request_id":"1366128:29:1424316:841","session_id":"1366128","input_length":126198,"t_proxy_recv":339319.833191144,"t_decision_unix":1779770184.8471305,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":125952,"affinity_cache_ratio":0.9980506822612085,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (246 < 8000)","cache_hit":125952,"estimated_new_tokens":246,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":126198,"score_linear":223264.0,"score_lmetric":378594},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":125952,"new_prefill":246,"score_linear":9172.0,"score_lmetric":492},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186904,"ongoing_decode_tokens":72491,"pending_prefill_tokens":5903,"num_requests":3,"active_p_offloads":0,"cached_blocks":2599,"cache_hit":0,"new_prefill":126198,"score_linear":186904.0,"score_lmetric":396303},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":126198,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":126198,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":36762,"ongoing_decode_tokens":32179,"pending_prefill_tokens":-6169,"num_requests":1,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":83456,"new_prefill":42742,"score_linear":-46694.0,"score_lmetric":36573},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":126198,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":67209,"num_requests":1,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":126198,"score_linear":129673.0,"score_lmetric":193407}],"chosen_score_linear":9172.0,"chosen_score_lmetric":492,"t_first_token":339320.005405026,"t_first_token_unix":1779770185.0193484,"t_done":339333.292201059,"t_done_unix":1779770198.306145},{"request_id":"1373577:4:1408642:799","session_id":"1373577","input_length":90818,"t_proxy_recv":339329.660971804,"t_decision_unix":1779770194.6749132,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9414873703450858,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5314 < 8000)","cache_hit":85504,"estimated_new_tokens":5314,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110006,"ongoing_decode_tokens":110006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":90818,"score_linear":110006.0,"score_lmetric":181636},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":90818,"score_linear":261322.0,"score_lmetric":272454},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":31044,"ongoing_decode_tokens":42342,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2640,"cache_hit":85504,"new_prefill":5314,"score_linear":-54460.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":90818,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":2560,"new_prefill":88258,"score_linear":-2560.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":0,"pending_prefill_tokens":6644,"num_requests":1,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":0,"new_prefill":90818,"score_linear":108020.0,"score_lmetric":97462},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":26923,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":90818,"score_linear":26923.0,"score_lmetric":90818},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":169462,"ongoing_decode_tokens":0,"pending_prefill_tokens":69110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":90818,"score_linear":169462.0,"score_lmetric":319856}],"chosen_score_linear":-54460.0,"chosen_score_lmetric":0,"t_first_token":339332.578263466,"t_first_token_unix":1779770197.5922077,"t_done":339334.955556085,"t_done_unix":1779770199.9694996},{"request_id":"1277428:8:1426669:846","session_id":"1277428","input_length":42342,"t_proxy_recv":339328.412931201,"t_decision_unix":1779770193.4268727,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9552690000472345,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1894 < 8000)","cache_hit":40448,"estimated_new_tokens":1894,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110006,"ongoing_decode_tokens":110006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":42342,"score_linear":110006.0,"score_lmetric":84684},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":4608,"new_prefill":37734,"score_linear":256714.0,"score_lmetric":113202},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":74140,"ongoing_decode_tokens":85438,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2632,"cache_hit":40448,"new_prefill":1894,"score_linear":33692.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":36352,"new_prefill":5990,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":42342,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":0,"new_prefill":42342,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":0,"pending_prefill_tokens":1835,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":42342,"score_linear":26923.0,"score_lmetric":44177},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":169462,"ongoing_decode_tokens":0,"pending_prefill_tokens":69110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":42342,"score_linear":169462.0,"score_lmetric":222904}],"chosen_score_linear":33692.0,"chosen_score_lmetric":0,"t_first_token":339329.064825941,"t_first_token_unix":1779770194.0787692,"t_done":339337.468803629,"t_done_unix":1779770202.4827476},{"request_id":"1362265:10:1427606:850","session_id":"1362265","input_length":12596,"t_proxy_recv":339331.221721199,"t_decision_unix":1779770196.2356627,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":11776,"affinity_cache_ratio":0.934899968243887,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (820 < 8000)","cache_hit":11776,"estimated_new_tokens":820,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110006,"ongoing_decode_tokens":110006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":12596,"score_linear":110006.0,"score_lmetric":25192},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":12596,"score_linear":261322.0,"score_lmetric":37788},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":121862,"ongoing_decode_tokens":42342,"pending_prefill_tokens":5314,"num_requests":1,"active_p_offloads":0,"cached_blocks":2640,"cache_hit":0,"new_prefill":12596,"score_linear":121862.0,"score_lmetric":17910},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":12596,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":12596,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":0,"pending_prefill_tokens":6644,"num_requests":1,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":11776,"new_prefill":820,"score_linear":96244.0,"score_lmetric":7464},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":26923,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":12596,"score_linear":26923.0,"score_lmetric":12596},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":169462,"ongoing_decode_tokens":0,"pending_prefill_tokens":69110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":6656,"new_prefill":5940,"score_linear":162806.0,"score_lmetric":150100}],"chosen_score_linear":96244.0,"chosen_score_lmetric":7464,"t_first_token":339333.276561086,"t_first_token_unix":1779770198.2905052,"t_done":339337.519012536,"t_done_unix":1779770202.5329561},{"request_id":"1391404:2:1393919:720","session_id":"1391404","input_length":33321,"t_proxy_recv":339213.154067308,"t_decision_unix":1779770078.168009,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":28160,"affinity_cache_ratio":0.8451126916959275,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5161 < 8000)","cache_hit":28160,"estimated_new_tokens":5161,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":33321,"score_linear":91673.0,"score_lmetric":210052},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":185425,"ongoing_decode_tokens":0,"pending_prefill_tokens":112209,"num_requests":2,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":28160,"new_prefill":5161,"score_linear":157265.0,"score_lmetric":234740},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":33321,"score_linear":98853.0,"score_lmetric":105220},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":33321,"score_linear":257663.0,"score_lmetric":101388},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":2560,"new_prefill":30761,"score_linear":-2560.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":33321,"score_linear":64227.0,"score_lmetric":66642},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":33321,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":33321,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":157265.0,"chosen_score_lmetric":234740,"t_first_token":339262.268728054,"t_first_token_unix":1779770127.2826717,"t_done":339340.175188396,"t_done_unix":1779770205.1891322},{"request_id":"1399948:9:1433544:857","session_id":"1399948","input_length":11993,"t_proxy_recv":339352.420154479,"t_decision_unix":1779770217.4340959,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":4,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9392145418160593,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":11993,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":72608,"pending_prefill_tokens":14481,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":11264,"new_prefill":729,"score_linear":261169.0,"score_lmetric":45630},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":11993,"score_linear":101803.0,"score_lmetric":11993},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":29269,"ongoing_decode_tokens":0,"pending_prefill_tokens":40567,"num_requests":0,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":11993,"score_linear":29269.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":11993,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":11993,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":11993,"score_linear":108020.0,"score_lmetric":11993},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":11993,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":0,"pending_prefill_tokens":69414,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":11993,"score_linear":171302.0,"score_lmetric":244221}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339353.409681479,"t_first_token_unix":1779770218.423625,"t_done":339353.474308913,"t_done_unix":1779770218.4882522},{"request_id":"1314357:5:1393659:719","session_id":"1314357","input_length":101803,"t_proxy_recv":339212.217335134,"t_decision_unix":1779770077.2312765,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":73216,"affinity_cache_ratio":0.7191929510918146,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=1 decode_tok=0)","cache_hit":73216,"estimated_new_tokens":28587,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":91673,"ongoing_decode_tokens":0,"pending_prefill_tokens":71705,"num_requests":2,"active_p_offloads":0,"cached_blocks":1501,"cache_hit":0,"new_prefill":101803,"score_linear":91673.0,"score_lmetric":347016},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":83622,"ongoing_decode_tokens":0,"pending_prefill_tokens":83622,"num_requests":1,"active_p_offloads":0,"cached_blocks":1561,"cache_hit":73216,"new_prefill":28587,"score_linear":10406.0,"score_lmetric":112209},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98853,"ongoing_decode_tokens":38252,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":101803,"score_linear":98853.0,"score_lmetric":173702},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":257663,"ongoing_decode_tokens":159396,"pending_prefill_tokens":475,"num_requests":3,"active_p_offloads":0,"cached_blocks":1265,"cache_hit":0,"new_prefill":101803,"score_linear":257663.0,"score_lmetric":306834},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":87687,"ongoing_decode_tokens":87687,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1726,"cache_hit":0,"new_prefill":101803,"score_linear":87687.0,"score_lmetric":101803},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64227,"ongoing_decode_tokens":64227,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1908,"cache_hit":0,"new_prefill":101803,"score_linear":64227.0,"score_lmetric":203606},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":101803,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1596,"cache_hit":0,"new_prefill":101803,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":10406.0,"chosen_score_lmetric":112209,"t_first_token":339262.266684498,"t_first_token_unix":1779770127.280628,"t_done":339353.542593197,"t_done_unix":1779770218.556537},{"request_id":"1399948:10:1434418:858","session_id":"1399948","input_length":12032,"t_proxy_recv":339355.426832381,"t_decision_unix":1779770220.4407737,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9787234042553191,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (256 < 8000)","cache_hit":11776,"estimated_new_tokens":256,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":11264,"new_prefill":768,"score_linear":261169.0,"score_lmetric":10434},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":105709,"ongoing_decode_tokens":105709,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":12032,"score_linear":105709.0,"score_lmetric":12032},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":29269,"ongoing_decode_tokens":0,"pending_prefill_tokens":40567,"num_requests":0,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":12032,"score_linear":29269.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":12032,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":11776,"new_prefill":256,"score_linear":-11776.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":12032,"score_linear":108020.0,"score_lmetric":12032},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":12032,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":0,"pending_prefill_tokens":69414,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":12032,"score_linear":171302.0,"score_lmetric":244338}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":339355.464183958,"t_first_token_unix":1779770220.4781272,"t_done":339355.678133567,"t_done_unix":1779770220.6920772},{"request_id":"1314357:6:1422517:839","session_id":"1314357","input_length":105709,"t_proxy_recv":339353.560064007,"t_decision_unix":1779770218.5740056,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9977579960079085,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (237 < 8000)","cache_hit":105472,"estimated_new_tokens":237,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":105709,"score_linear":272433.0,"score_lmetric":325257},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":105472,"new_prefill":237,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":29269,"ongoing_decode_tokens":0,"pending_prefill_tokens":40567,"num_requests":0,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":105709,"score_linear":29269.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":105709,"score_linear":108020.0,"score_lmetric":105709},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":0,"pending_prefill_tokens":69414,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":105709,"score_linear":171302.0,"score_lmetric":525369}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":339353.726982411,"t_first_token_unix":1779770218.740926,"t_done":339359.376133235,"t_done_unix":1779770224.390077},{"request_id":"1382089:3:1433211:856","session_id":"1382089","input_length":40567,"t_proxy_recv":339351.196484037,"t_decision_unix":1779770216.2104256,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":2,"affinity_cache_hit":38912,"affinity_cache_ratio":0.9592032933172283,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":-40567,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":40567,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":72608,"pending_prefill_tokens":14481,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":40567,"score_linear":272433.0,"score_lmetric":165144},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":40567,"score_linear":101803.0,"score_lmetric":40567},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":40567,"score_linear":-11298.0,"score_lmetric":-40567},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":40567,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":40567,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":40567,"score_linear":108020.0,"score_lmetric":40567},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":40567,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":0,"pending_prefill_tokens":69414,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":38912,"new_prefill":1655,"score_linear":132390.0,"score_lmetric":213207}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-40567,"t_first_token":339357.696357901,"t_first_token_unix":1779770222.7103014,"t_done":339359.828351165,"t_done_unix":1779770224.8422947},{"request_id":"1355951:5:1425005:843","session_id":"1355951","input_length":39789,"t_proxy_recv":339322.294468319,"t_decision_unix":1779770187.308409,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9522229761994521,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1901 < 8000)","cache_hit":37888,"estimated_new_tokens":1901,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":39789,"score_linear":223264.0,"score_lmetric":119367},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":39789,"score_linear":261322.0,"score_lmetric":119367},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186904,"ongoing_decode_tokens":157929,"pending_prefill_tokens":1873,"num_requests":3,"active_p_offloads":0,"cached_blocks":2599,"cache_hit":0,"new_prefill":39789,"score_linear":186904.0,"score_lmetric":124986},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":39789,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":39789,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":36762,"ongoing_decode_tokens":133611,"pending_prefill_tokens":-10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":0,"new_prefill":39789,"score_linear":36762.0,"score_lmetric":29468},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":25225,"ongoing_decode_tokens":25225,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":0,"new_prefill":39789,"score_linear":25225.0,"score_lmetric":39789},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":67209,"num_requests":1,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":37888,"new_prefill":1901,"score_linear":91785.0,"score_lmetric":69110}],"chosen_score_linear":91785.0,"chosen_score_lmetric":69110,"t_first_token":339363.541760401,"t_first_token_unix":1779770228.5557044,"t_done":339363.89608494,"t_done_unix":1779770228.9100287},{"request_id":"1314357:7:1427957:851","session_id":"1314357","input_length":106279,"t_proxy_recv":339359.394642753,"t_decision_unix":1779770224.4085839,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105984,"affinity_cache_ratio":0.9972242870181315,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (295 < 8000)","cache_hit":105984,"estimated_new_tokens":295,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":106279,"score_linear":272433.0,"score_lmetric":326967},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":105984,"new_prefill":295,"score_linear":-105984.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":29269,"ongoing_decode_tokens":40567,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":106279,"score_linear":29269.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":106279,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57375,"ongoing_decode_tokens":0,"pending_prefill_tokens":44063,"num_requests":2,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":106279,"score_linear":57375.0,"score_lmetric":300684},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":106279,"score_linear":108020.0,"score_lmetric":106279},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":106279,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":129673,"pending_prefill_tokens":2205,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":106279,"score_linear":171302.0,"score_lmetric":325452}],"chosen_score_linear":-105984.0,"chosen_score_lmetric":0,"t_first_token":339359.599858864,"t_first_token_unix":1779770224.6138024,"t_done":339364.849172606,"t_done_unix":1779770229.863117},{"request_id":"1399948:11:1435163:860","session_id":"1399948","input_length":12106,"t_proxy_recv":339358.131341164,"t_decision_unix":1779770223.1452825,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9727407896910623,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (330 < 8000)","cache_hit":11776,"estimated_new_tokens":330,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":11264,"new_prefill":842,"score_linear":261169.0,"score_lmetric":10656},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":105709,"ongoing_decode_tokens":105709,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":12106,"score_linear":105709.0,"score_lmetric":12106},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":29269,"ongoing_decode_tokens":40567,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":12106,"score_linear":29269.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":12106,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45269,"ongoing_decode_tokens":0,"pending_prefill_tokens":43733,"num_requests":1,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":11776,"new_prefill":330,"score_linear":33493.0,"score_lmetric":44063},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":12106,"score_linear":108020.0,"score_lmetric":12106},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":12106,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":129673,"pending_prefill_tokens":2205,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":12106,"score_linear":171302.0,"score_lmetric":42933}],"chosen_score_linear":33493.0,"chosen_score_lmetric":44063,"t_first_token":339364.891941996,"t_first_token_unix":1779770229.905885,"t_done":339364.944131047,"t_done_unix":1779770229.9580743},{"request_id":"1399948:12:1436179:863","session_id":"1399948","input_length":12164,"t_proxy_recv":339364.947415233,"t_decision_unix":1779770229.9613569,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9681025978296613,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (388 < 8000)","cache_hit":11776,"estimated_new_tokens":388,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":11264,"new_prefill":900,"score_linear":261169.0,"score_lmetric":10830},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":106805,"ongoing_decode_tokens":0,"pending_prefill_tokens":309,"num_requests":1,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":12164,"score_linear":106805.0,"score_lmetric":12473},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":12164,"score_linear":-11298.0,"score_lmetric":-12164},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":12164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45269,"ongoing_decode_tokens":45269,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":11776,"new_prefill":388,"score_linear":33493.0,"score_lmetric":388},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":12164,"score_linear":108020.0,"score_lmetric":12164},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":29412,"ongoing_decode_tokens":29412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":12164,"score_linear":29412.0,"score_lmetric":12164},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131513,"ongoing_decode_tokens":131513,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1851,"cache_hit":0,"new_prefill":12164,"score_linear":131513.0,"score_lmetric":24328}],"chosen_score_linear":33493.0,"chosen_score_lmetric":388,"t_first_token":339365.001911302,"t_first_token_unix":1779770230.0158548,"t_done":339365.219435025,"t_done_unix":1779770230.233379},{"request_id":"1313181:9:1333349:415","session_id":"1313181","input_length":129673,"t_proxy_recv":339317.679440984,"t_decision_unix":1779770182.6933825,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":62464,"affinity_cache_ratio":0.481703978468918,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":62464,"estimated_new_tokens":67209,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":223264,"ongoing_decode_tokens":223264,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1781,"cache_hit":0,"new_prefill":129673,"score_linear":223264.0,"score_lmetric":389019},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":129673,"score_linear":135124.0,"score_lmetric":259346},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186904,"ongoing_decode_tokens":72491,"pending_prefill_tokens":5903,"num_requests":3,"active_p_offloads":0,"cached_blocks":2599,"cache_hit":61952,"new_prefill":67721,"score_linear":124952.0,"score_lmetric":220872},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":12800,"new_prefill":116873,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":129673,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-64670,"ongoing_decode_tokens":32179,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2258,"cache_hit":0,"new_prefill":129673,"score_linear":-64670.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1799,"cache_hit":12800,"new_prefill":116873,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":62464,"new_prefill":67209,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-62464.0,"chosen_score_lmetric":0,"t_first_token":339357.550590129,"t_first_token_unix":1779770222.5645332,"t_done":339365.511661448,"t_done_unix":1779770230.5256076},{"request_id":"1314357:8:1431749:854","session_id":"1314357","input_length":106805,"t_proxy_recv":339364.868170103,"t_decision_unix":1779770229.8821118,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":106496,"affinity_cache_ratio":0.9971068770188661,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (309 < 8000)","cache_hit":106496,"estimated_new_tokens":309,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":106805,"score_linear":272433.0,"score_lmetric":328545},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":106496,"new_prefill":309,"score_linear":-106496.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":106805,"score_linear":-11298.0,"score_lmetric":-106805},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":106805,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57375,"ongoing_decode_tokens":0,"pending_prefill_tokens":44063,"num_requests":2,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":106805,"score_linear":57375.0,"score_lmetric":301736},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":106805,"score_linear":108020.0,"score_lmetric":106805},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":29412,"ongoing_decode_tokens":29412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":106805,"score_linear":29412.0,"score_lmetric":106805},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131513,"ongoing_decode_tokens":131513,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1851,"cache_hit":0,"new_prefill":106805,"score_linear":131513.0,"score_lmetric":213610}],"chosen_score_linear":-106496.0,"chosen_score_lmetric":0,"t_first_token":339365.076570151,"t_first_token_unix":1779770230.0905137,"t_done":339366.347595082,"t_done_unix":1779770231.3615391},{"request_id":"1270340:6:1434904:859","session_id":"1270340","input_length":45269,"t_proxy_recv":339357.069686788,"t_decision_unix":1779770222.083628,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":4,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9952947933464402,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":1536,"estimated_new_tokens":43733,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":45269,"score_linear":272433.0,"score_lmetric":143937},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":105709,"ongoing_decode_tokens":105709,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":45269,"score_linear":105709.0,"score_lmetric":45269},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":29269,"ongoing_decode_tokens":0,"pending_prefill_tokens":40567,"num_requests":0,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":45269,"score_linear":29269.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":45269,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":1536,"new_prefill":43733,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":45269,"score_linear":108020.0,"score_lmetric":45269},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":45269,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":0,"pending_prefill_tokens":69414,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":45056,"new_prefill":213,"score_linear":126246.0,"score_lmetric":208881}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":339364.890814845,"t_first_token_unix":1779770229.9047616,"t_done":339366.442964698,"t_done_unix":1779770231.456909},{"request_id":"1314357:9:1435219:861","session_id":"1314357","input_length":106959,"t_proxy_recv":339366.367508008,"t_decision_unix":1779770231.3814497,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":106496,"affinity_cache_ratio":0.9956712385119532,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (463 < 8000)","cache_hit":106496,"estimated_new_tokens":463,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":106959,"score_linear":272433.0,"score_lmetric":329007},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":106496,"new_prefill":463,"score_linear":-106496.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":106959,"score_linear":-11298.0,"score_lmetric":-106959},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":106959,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45269,"ongoing_decode_tokens":45269,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":106959,"score_linear":45269.0,"score_lmetric":106959},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":106959,"score_linear":108020.0,"score_lmetric":106959},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":29412,"ongoing_decode_tokens":29412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":106959,"score_linear":29412.0,"score_lmetric":106959},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":106959,"score_linear":64353.0,"score_lmetric":213918}],"chosen_score_linear":-106496.0,"chosen_score_lmetric":0,"t_first_token":339366.531989461,"t_first_token_unix":1779770231.5459335,"t_done":339369.123464637,"t_done_unix":1779770234.1374085},{"request_id":"1314357:10:1439373:867","session_id":"1314357","input_length":107228,"t_proxy_recv":339372.240535926,"t_decision_unix":1779770237.254477,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":107008,"affinity_cache_ratio":0.9979482970865818,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (220 < 8000)","cache_hit":107008,"estimated_new_tokens":220,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":107228,"score_linear":272433.0,"score_lmetric":329814},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":107008,"new_prefill":220,"score_linear":-107008.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":116190,"ongoing_decode_tokens":0,"pending_prefill_tokens":127488,"num_requests":0,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":107228,"score_linear":116190.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":52920,"ongoing_decode_tokens":0,"pending_prefill_tokens":3768,"num_requests":1,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":107228,"score_linear":52920.0,"score_lmetric":110996},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":107228,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":107228,"score_linear":108020.0,"score_lmetric":107228},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58123,"ongoing_decode_tokens":58123,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":107228,"score_linear":58123.0,"score_lmetric":214456},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":107228,"score_linear":64353.0,"score_lmetric":214456}],"chosen_score_linear":-107008.0,"chosen_score_lmetric":0,"t_first_token":339372.395804602,"t_first_token_unix":1779770237.4097476,"t_done":339372.686748083,"t_done_unix":1779770237.7006917},{"request_id":"1363943:13:1438562:865","session_id":"1363943","input_length":28711,"t_proxy_recv":339369.527906963,"t_decision_unix":1779770234.5418484,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9451429765595069,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1575 < 8000)","cache_hit":27136,"estimated_new_tokens":1575,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":8704,"new_prefill":20007,"score_linear":263729.0,"score_lmetric":68151},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":28711,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":116190,"ongoing_decode_tokens":0,"pending_prefill_tokens":127488,"num_requests":0,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":4096,"new_prefill":24615,"score_linear":112094.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":28711,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":4096,"new_prefill":24615,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":4096,"new_prefill":24615,"score_linear":103924.0,"score_lmetric":24615},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":29412,"ongoing_decode_tokens":29412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":27136,"new_prefill":1575,"score_linear":2276.0,"score_lmetric":1575},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":28711,"score_linear":64353.0,"score_lmetric":57422}],"chosen_score_linear":2276.0,"chosen_score_lmetric":1575,"t_first_token":339369.848877978,"t_first_token_unix":1779770234.8628216,"t_done":339374.377884357,"t_done_unix":1779770239.391828},{"request_id":"1268861:10:1439325:866","session_id":"1268861","input_length":52920,"t_proxy_recv":339372.084442458,"t_decision_unix":1779770237.0983837,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9287981859410431,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3768 < 8000)","cache_hit":49152,"estimated_new_tokens":3768,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":52920,"score_linear":272433.0,"score_lmetric":166890},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":52920,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":116190,"ongoing_decode_tokens":0,"pending_prefill_tokens":127488,"num_requests":0,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":52920,"score_linear":116190.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":49152,"new_prefill":3768,"score_linear":-49152.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":52920,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":52920,"score_linear":108020.0,"score_lmetric":52920},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58123,"ongoing_decode_tokens":58123,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":52920,"score_linear":58123.0,"score_lmetric":105840},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":52920,"score_linear":64353.0,"score_lmetric":105840}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":339373.295737946,"t_first_token_unix":1779770238.3096814,"t_done":339374.408082274,"t_done_unix":1779770239.4220257},{"request_id":"1342921:5:1433192:855","session_id":"1342921","input_length":73878,"t_proxy_recv":339351.125256447,"t_decision_unix":1779770216.1391964,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9633179024878854,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2710 < 8000)","cache_hit":71168,"estimated_new_tokens":2710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198555,"ongoing_decode_tokens":72608,"pending_prefill_tokens":11771,"num_requests":2,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":71168,"new_prefill":2710,"score_linear":127387.0,"score_lmetric":28962},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":73878,"score_linear":101803.0,"score_lmetric":73878},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":73878,"score_linear":-11298.0,"score_lmetric":-73878},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":69120,"new_prefill":4758,"score_linear":38900.0,"score_lmetric":4758},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":0,"pending_prefill_tokens":69414,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":73878,"score_linear":171302.0,"score_lmetric":429876}],"chosen_score_linear":127387.0,"chosen_score_lmetric":28962,"t_first_token":339372.377725895,"t_first_token_unix":1779770237.3916693,"t_done":339375.16526099,"t_done_unix":1779770240.1792045},{"request_id":"1404395:2:1435941:862","session_id":"1404395","input_length":29412,"t_proxy_recv":339360.714770018,"t_decision_unix":1779770225.7287111,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":22528,"affinity_cache_ratio":0.7659458724330205,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6884 < 8000)","cache_hit":22528,"estimated_new_tokens":6884,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":29412,"score_linear":272433.0,"score_lmetric":96366},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":106279,"ongoing_decode_tokens":106279,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":512,"new_prefill":28900,"score_linear":105767.0,"score_lmetric":28900},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":29412,"score_linear":-11298.0,"score_lmetric":-29412},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":29412,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57375,"ongoing_decode_tokens":0,"pending_prefill_tokens":44063,"num_requests":2,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":512,"new_prefill":28900,"score_linear":56863.0,"score_lmetric":145926},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":29412,"score_linear":108020.0,"score_lmetric":29412},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":22528,"new_prefill":6884,"score_linear":-22528.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":129673,"pending_prefill_tokens":2205,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":29412,"score_linear":171302.0,"score_lmetric":94851}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":0,"t_first_token":339362.025543666,"t_first_token_unix":1779770227.0394883,"t_done":339381.538878213,"t_done_unix":1779770246.5528226},{"request_id":"1364090:10:1426859:847","session_id":"1364090","input_length":108020,"t_proxy_recv":339328.932381964,"t_decision_unix":1779770193.9463232,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9384928716904277,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6644 < 8000)","cache_hit":101376,"estimated_new_tokens":6644,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110006,"ongoing_decode_tokens":110006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":7680,"new_prefill":100340,"score_linear":102326.0,"score_lmetric":200680},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":261322,"ongoing_decode_tokens":261322,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":0,"new_prefill":108020,"score_linear":261322.0,"score_lmetric":324060},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":116482,"ongoing_decode_tokens":85438,"pending_prefill_tokens":1894,"num_requests":1,"active_p_offloads":0,"cached_blocks":2632,"cache_hit":7680,"new_prefill":100340,"score_linear":108802.0,"score_lmetric":102234},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":108020,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":7680,"new_prefill":100340,"score_linear":-7680.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":101376,"new_prefill":6644,"score_linear":-101376.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":26923,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":11264,"new_prefill":96756,"score_linear":15659.0,"score_lmetric":96756},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":169462,"ongoing_decode_tokens":0,"pending_prefill_tokens":69110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":108020,"score_linear":169462.0,"score_lmetric":354260}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":339333.090435848,"t_first_token_unix":1779770198.10438,"t_done":339384.181136612,"t_done_unix":1779770249.1950805},{"request_id":"1407599:2:1430593:852","session_id":"1407599","input_length":1840,"t_proxy_recv":339341.951302901,"t_decision_unix":1779770206.9652438,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":1536,"affinity_cache_ratio":0.8347826086956521,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (304 < 8000)","cache_hit":1536,"estimated_new_tokens":304,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":1840,"score_linear":72608.0,"score_lmetric":1840},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":1840,"score_linear":101803.0,"score_lmetric":1840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":1840,"score_linear":-11298.0,"score_lmetric":-1840},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":1840,"score_linear":108020.0,"score_lmetric":1840},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":169462,"ongoing_decode_tokens":0,"pending_prefill_tokens":69110,"num_requests":2,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":1536,"new_prefill":304,"score_linear":167926.0,"score_lmetric":138828}],"chosen_score_linear":167926.0,"chosen_score_lmetric":138828,"t_first_token":339363.542595926,"t_first_token_unix":1779770228.5565393,"t_done":339385.082277119,"t_done_unix":1779770250.0962205},{"request_id":"1313181:9:1386945:689","session_id":"1313181","input_length":62513,"t_proxy_recv":339365.520282663,"t_decision_unix":1779770230.5342243,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":62464,"affinity_cache_ratio":0.999216163038088,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (49 < 8000)","cache_hit":62464,"estimated_new_tokens":49,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":62513,"score_linear":272433.0,"score_lmetric":195669},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":106805,"ongoing_decode_tokens":106805,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":62513,"score_linear":106805.0,"score_lmetric":62513},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":61952,"new_prefill":561,"score_linear":-73250.0,"score_lmetric":-561},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":12800,"new_prefill":49713,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45269,"ongoing_decode_tokens":45269,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":62513,"score_linear":45269.0,"score_lmetric":62513},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":62513,"score_linear":108020.0,"score_lmetric":62513},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":29412,"ongoing_decode_tokens":29412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":12800,"new_prefill":49713,"score_linear":16612.0,"score_lmetric":49713},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":1840,"ongoing_decode_tokens":1840,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":62464,"new_prefill":49,"score_linear":-60624.0,"score_lmetric":49}],"chosen_score_linear":-60624.0,"chosen_score_lmetric":49,"t_first_token":339365.607508997,"t_first_token_unix":1779770230.6214526,"t_done":339385.502627578,"t_done_unix":1779770250.516571},{"request_id":"1314357:11:1442554:870","session_id":"1314357","input_length":108166,"t_proxy_recv":339382.729145991,"t_decision_unix":1779770247.7430873,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":107008,"affinity_cache_ratio":0.9892942329382616,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1158 < 8000)","cache_hit":107008,"estimated_new_tokens":1158,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":217611,"ongoing_decode_tokens":198555,"pending_prefill_tokens":5744,"num_requests":3,"active_p_offloads":0,"cached_blocks":1793,"cache_hit":0,"new_prefill":108166,"score_linear":217611.0,"score_lmetric":341730},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":107008,"new_prefill":1158,"score_linear":-107008.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":108166,"score_linear":179343.0,"score_lmetric":249655},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":108166,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":108166,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":108166,"score_linear":108020.0,"score_lmetric":108166},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":108166,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":108166,"score_linear":64353.0,"score_lmetric":216332}],"chosen_score_linear":-107008.0,"chosen_score_lmetric":0,"t_first_token":339383.484234288,"t_first_token_unix":1779770248.4981778,"t_done":339385.540711402,"t_done_unix":1779770250.5546558},{"request_id":"1371622:2:1442319:869","session_id":"1371622","input_length":19056,"t_proxy_recv":339381.933270242,"t_decision_unix":1779770246.947211,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":13312,"affinity_cache_ratio":0.6985726280436608,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5744 < 8000)","cache_hit":13312,"estimated_new_tokens":5744,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198555,"ongoing_decode_tokens":198555,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1793,"cache_hit":13312,"new_prefill":5744,"score_linear":185243.0,"score_lmetric":11488},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":19056,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":19056,"score_linear":179343.0,"score_lmetric":160545},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":19056,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":13312,"new_prefill":5744,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":19056,"score_linear":108020.0,"score_lmetric":19056},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":19056,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":19056,"score_linear":64353.0,"score_lmetric":38112}],"chosen_score_linear":185243.0,"chosen_score_lmetric":11488,"t_first_token":339383.957572133,"t_first_token_unix":1779770248.9715157,"t_done":339386.259660965,"t_done_unix":1779770251.2736046},{"request_id":"1381387:7:1411928:819","session_id":"1381387","input_length":72608,"t_proxy_recv":339302.859027933,"t_decision_unix":1779770167.8729694,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9872190392243279,"affinity_num_requests":3,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (928 < 8000)","cache_hit":71680,"estimated_new_tokens":928,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":159268,"ongoing_decode_tokens":159268,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1775,"cache_hit":71680,"new_prefill":928,"score_linear":87588.0,"score_lmetric":2784},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":263113,"ongoing_decode_tokens":135124,"pending_prefill_tokens":10229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":72608,"score_linear":263113.0,"score_lmetric":248511},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":310994,"ongoing_decode_tokens":196581,"pending_prefill_tokens":5903,"num_requests":4,"active_p_offloads":0,"cached_blocks":2358,"cache_hit":0,"new_prefill":72608,"score_linear":310994.0,"score_lmetric":314044},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1557,"cache_hit":0,"new_prefill":72608,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":0,"new_prefill":72608,"score_linear":107292.0,"score_lmetric":86716},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":96849,"pending_prefill_tokens":-10321,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":72608,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":163670,"ongoing_decode_tokens":163670,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1744,"cache_hit":0,"new_prefill":72608,"score_linear":163670.0,"score_lmetric":217824},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":43763,"ongoing_decode_tokens":43763,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":512,"new_prefill":72096,"score_linear":43251.0,"score_lmetric":72096}],"chosen_score_linear":87588.0,"chosen_score_lmetric":2784,"t_first_token":339303.510735949,"t_first_token_unix":1779770168.524679,"t_done":339386.83612144,"t_done_unix":1779770251.8500655},{"request_id":"1314357:12:1446681:873","session_id":"1314357","input_length":108409,"t_proxy_recv":339396.45721557,"t_decision_unix":1779770261.4711568,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":108032,"affinity_cache_ratio":0.996522428949626,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (377 < 8000)","cache_hit":108032,"estimated_new_tokens":377,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":145239,"ongoing_decode_tokens":145239,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1807,"cache_hit":0,"new_prefill":108409,"score_linear":145239.0,"score_lmetric":216818},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":108032,"new_prefill":377,"score_linear":-108032.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":108409,"score_linear":179343.0,"score_lmetric":249898},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":108409,"score_linear":129839.0,"score_lmetric":173736}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":339396.673673669,"t_first_token_unix":1779770261.6876173,"t_done":339397.281028105,"t_done_unix":1779770262.2949717},{"request_id":"1371622:3:1443876:871","session_id":"1371622","input_length":19292,"t_proxy_recv":339387.04259525,"t_decision_unix":1779770252.0565364,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":18944,"affinity_cache_ratio":0.9819614347916235,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (348 < 8000)","cache_hit":18944,"estimated_new_tokens":348,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1807,"cache_hit":18944,"new_prefill":348,"score_linear":107003.0,"score_lmetric":348},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":19292,"score_linear":179343.0,"score_lmetric":160781},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":13312,"new_prefill":5980,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":19292,"score_linear":129839.0,"score_lmetric":84619}],"chosen_score_linear":107003.0,"chosen_score_lmetric":348,"t_first_token":339387.139564839,"t_first_token_unix":1779770252.153509,"t_done":339402.002955829,"t_done_unix":1779770267.0168993},{"request_id":"1314357:13:1448346:875","session_id":"1314357","input_length":108791,"t_proxy_recv":339402.116995339,"t_decision_unix":1779770267.1309366,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":108032,"affinity_cache_ratio":0.9930233199437454,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (759 < 8000)","cache_hit":108032,"estimated_new_tokens":759,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1809,"cache_hit":0,"new_prefill":108791,"score_linear":125947.0,"score_lmetric":108791},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":108032,"new_prefill":759,"score_linear":-108032.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":108791,"score_linear":179343.0,"score_lmetric":250280},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":108791,"score_linear":129839.0,"score_lmetric":174118}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":339402.640464798,"t_first_token_unix":1779770267.6544087,"t_done":339404.450670125,"t_done_unix":1779770269.4646142},{"request_id":"1314357:14:1450740:878","session_id":"1314357","input_length":109010,"t_proxy_recv":339409.94765919,"t_decision_unix":1779770274.9616005,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":108544,"affinity_cache_ratio":0.9957251628290983,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (466 < 8000)","cache_hit":108544,"estimated_new_tokens":466,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":201708,"ongoing_decode_tokens":125947,"pending_prefill_tokens":2033,"num_requests":2,"active_p_offloads":0,"cached_blocks":1809,"cache_hit":0,"new_prefill":109010,"score_linear":201708.0,"score_lmetric":222086},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":108544,"new_prefill":466,"score_linear":-108544.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":109010,"score_linear":179343.0,"score_lmetric":250499},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":109010,"score_linear":129839.0,"score_lmetric":174337}],"chosen_score_linear":-108544.0,"chosen_score_lmetric":0,"t_first_token":339410.201533176,"t_first_token_unix":1779770275.2154765,"t_done":339411.475402453,"t_done_unix":1779770276.4893458},{"request_id":"1342921:6:1448548:876","session_id":"1342921","input_length":75761,"t_proxy_recv":339402.673147954,"t_decision_unix":1779770267.6870892,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9731656129142963,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2033 < 8000)","cache_hit":73728,"estimated_new_tokens":2033,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1809,"cache_hit":73728,"new_prefill":2033,"score_linear":52219.0,"score_lmetric":2033},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":108791,"ongoing_decode_tokens":108791,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1886,"cache_hit":0,"new_prefill":75761,"score_linear":108791.0,"score_lmetric":75761},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":75761,"score_linear":179343.0,"score_lmetric":217250},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":75761,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":75761,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":69120,"new_prefill":6641,"score_linear":-69120.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":75761,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":75761,"score_linear":129839.0,"score_lmetric":141088}],"chosen_score_linear":52219.0,"chosen_score_lmetric":2033,"t_first_token":339410.82669475,"t_first_token_unix":1779770275.8406384,"t_done":339411.537289267,"t_done_unix":1779770276.5512328},{"request_id":"1268861:11:1452208:880","session_id":"1268861","input_length":55050,"t_proxy_recv":339414.912159255,"t_decision_unix":1779770279.9261005,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9579654859218892,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2314 < 8000)","cache_hit":52736,"estimated_new_tokens":2314,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":55050,"score_linear":125947.0,"score_lmetric":55050},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":55050,"score_linear":179343.0,"score_lmetric":196539},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":52736,"new_prefill":2314,"score_linear":-52736.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":55050,"score_linear":129839.0,"score_lmetric":120377}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":339415.741537653,"t_first_token_unix":1779770280.755482,"t_done":339416.22072248,"t_done_unix":1779770281.2346663},{"request_id":"1371622:4:1452210:881","session_id":"1371622","input_length":20439,"t_proxy_recv":339414.919639713,"t_decision_unix":1779770279.9335814,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9769558197563482,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (471 < 8000)","cache_hit":19968,"estimated_new_tokens":471,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":19968,"new_prefill":471,"score_linear":105979.0,"score_lmetric":471},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":20439,"score_linear":179343.0,"score_lmetric":161928},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":55050,"ongoing_decode_tokens":0,"pending_prefill_tokens":2314,"num_requests":1,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":20439,"score_linear":55050.0,"score_lmetric":22753},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":13312,"new_prefill":7127,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":20439,"score_linear":129839.0,"score_lmetric":85766}],"chosen_score_linear":105979.0,"chosen_score_lmetric":471,"t_first_token":339415.011165924,"t_first_token_unix":1779770280.0251095,"t_done":339416.224906153,"t_done_unix":1779770281.2388496},{"request_id":"1314357:15:1452292:882","session_id":"1314357","input_length":109164,"t_proxy_recv":339415.183986529,"t_decision_unix":1779770280.1979282,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":109056,"affinity_cache_ratio":0.9990106628558866,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (108 < 8000)","cache_hit":109056,"estimated_new_tokens":108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":146386,"ongoing_decode_tokens":146386,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":109164,"score_linear":146386.0,"score_lmetric":218328},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":109056,"new_prefill":108,"score_linear":-109056.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":109164,"score_linear":179343.0,"score_lmetric":250653},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":55050,"ongoing_decode_tokens":0,"pending_prefill_tokens":2314,"num_requests":1,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":109164,"score_linear":55050.0,"score_lmetric":111478},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":109164,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":109164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":109164,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":0,"new_prefill":109164,"score_linear":129839.0,"score_lmetric":174491}],"chosen_score_linear":-109056.0,"chosen_score_lmetric":0,"t_first_token":339415.355691075,"t_first_token_unix":1779770280.369635,"t_done":339419.344671336,"t_done_unix":1779770284.3586204},{"request_id":"1264679:3:1431512:853","session_id":"1264679","input_length":125947,"t_proxy_recv":339344.946216835,"t_decision_unix":1779770209.960158,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":114176,"affinity_cache_ratio":0.9065400525617918,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"src_cache_below_threshold (0 < 4000)","cache_hit":114176,"estimated_new_tokens":11771,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":114176,"new_prefill":11771,"score_linear":-41568.0,"score_lmetric":11771},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":101803,"ongoing_decode_tokens":101803,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":125947,"score_linear":101803.0,"score_lmetric":125947},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2654,"cache_hit":0,"new_prefill":125947,"score_linear":-11298.0,"score_lmetric":-125947},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1817,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":125947,"score_linear":108020.0,"score_lmetric":125947},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":171302,"ongoing_decode_tokens":0,"pending_prefill_tokens":69414,"num_requests":3,"active_p_offloads":0,"cached_blocks":1849,"cache_hit":0,"new_prefill":125947,"score_linear":171302.0,"score_lmetric":586083}],"chosen_score_linear":-41568.0,"chosen_score_lmetric":11771,"t_first_token":339353.482117788,"t_first_token_unix":1779770218.4960608,"t_done":339420.676962392,"t_done_unix":1779770285.690906},{"request_id":"1313181:10:1338837:436","session_id":"1313181","input_length":129839,"t_proxy_recv":339385.523375541,"t_decision_unix":1779770250.537317,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":64512,"affinity_cache_ratio":0.4968614977009989,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":64512,"estimated_new_tokens":65327,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":217611,"ongoing_decode_tokens":217611,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1793,"cache_hit":0,"new_prefill":129839,"score_linear":217611.0,"score_lmetric":389517},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":108166,"ongoing_decode_tokens":108166,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":129839,"score_linear":108166.0,"score_lmetric":129839},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":0,"pending_prefill_tokens":141489,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":61952,"new_prefill":67887,"score_linear":117391.0,"score_lmetric":209376},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":12800,"new_prefill":117039,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":129839,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":129839,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":12800,"new_prefill":117039,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1876,"cache_hit":64512,"new_prefill":65327,"score_linear":-64512.0,"score_lmetric":0}],"chosen_score_linear":-64512.0,"chosen_score_lmetric":0,"t_first_token":339424.238398536,"t_first_token_unix":1779770289.2523422,"t_done":339425.119453741,"t_done_unix":1779770290.133398},{"request_id":"1371622:5:1455608:884","session_id":"1371622","input_length":21518,"t_proxy_recv":339426.698891337,"t_decision_unix":1779770291.7128325,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":20480,"affinity_cache_ratio":0.951761316107445,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1038 < 8000)","cache_hit":20480,"estimated_new_tokens":1038,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1838,"cache_hit":20480,"new_prefill":1038,"score_linear":-20480.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":127488,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":21518,"score_linear":179343.0,"score_lmetric":35519},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":13312,"new_prefill":8206,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64724,"ongoing_decode_tokens":64724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1877,"cache_hit":0,"new_prefill":21518,"score_linear":64724.0,"score_lmetric":21518}],"chosen_score_linear":-20480.0,"chosen_score_lmetric":0,"t_first_token":339426.941538634,"t_first_token_unix":1779770291.9554827,"t_done":339427.394683617,"t_done_unix":1779770292.4086275},{"request_id":"1313181:10:1399867:746","session_id":"1313181","input_length":64724,"t_proxy_recv":339425.130713134,"t_decision_unix":1779770290.1446545,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":64512,"affinity_cache_ratio":0.9967245534886595,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (212 < 8000)","cache_hit":64512,"estimated_new_tokens":212,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1838,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":127488,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":61952,"new_prefill":2772,"score_linear":117391.0,"score_lmetric":16773},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":12800,"new_prefill":51924,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":12800,"new_prefill":51924,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1877,"cache_hit":64512,"new_prefill":212,"score_linear":-64512.0,"score_lmetric":0}],"chosen_score_linear":-64512.0,"chosen_score_lmetric":0,"t_first_token":339425.223396796,"t_first_token_unix":1779770290.2373405,"t_done":339428.347508012,"t_done_unix":1779770293.3614523},{"request_id":"1356885:2:1438037:864","session_id":"1356885","input_length":127488,"t_proxy_recv":339367.820164939,"t_decision_unix":1779770232.834106,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":2,"affinity_cache_hit":122368,"affinity_cache_ratio":0.9598393574297188,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":-127488,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":0,"estimated_new_tokens":127488,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":272433,"ongoing_decode_tokens":198555,"pending_prefill_tokens":2710,"num_requests":3,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":122368,"new_prefill":5120,"score_linear":150065.0,"score_lmetric":23490},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":106959,"ongoing_decode_tokens":106959,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1883,"cache_hit":0,"new_prefill":127488,"score_linear":106959.0,"score_lmetric":127488},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":0,"new_prefill":127488,"score_linear":-11298.0,"score_lmetric":-127488},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1567,"cache_hit":0,"new_prefill":127488,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":127488,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":127488,"score_linear":108020.0,"score_lmetric":127488},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":29412,"ongoing_decode_tokens":29412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1804,"cache_hit":0,"new_prefill":127488,"score_linear":29412.0,"score_lmetric":127488},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":6656,"new_prefill":120832,"score_linear":57697.0,"score_lmetric":241664}],"chosen_score_linear":-11298.0,"chosen_score_lmetric":-127488,"t_first_token":339419.517972228,"t_first_token_unix":1779770284.5319161,"t_done":339435.571717053,"t_done_unix":1779770300.585661},{"request_id":"1363943:14:1458616:885","session_id":"1363943","input_length":29268,"t_proxy_recv":339436.646054396,"t_decision_unix":1779770301.6599956,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29184,"affinity_cache_ratio":0.997129971299713,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (84 < 8000)","cache_hit":29184,"estimated_new_tokens":84,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":8704,"new_prefill":20564,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":29268,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":180418,"ongoing_decode_tokens":63153,"pending_prefill_tokens":1075,"num_requests":1,"active_p_offloads":0,"cached_blocks":2982,"cache_hit":4096,"new_prefill":25172,"score_linear":176322.0,"score_lmetric":26247},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":29268,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":4096,"new_prefill":25172,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":4096,"new_prefill":25172,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":29184,"new_prefill":84,"score_linear":-29184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":29268,"score_linear":129931.0,"score_lmetric":94175}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":339436.737166332,"t_first_token_unix":1779770301.7511098,"t_done":339437.301408712,"t_done_unix":1779770302.3153517},{"request_id":"1294611:12:1441557:868","session_id":"1294611","input_length":63153,"t_proxy_recv":339379.200969152,"t_decision_unix":1779770244.2149103,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49152,"affinity_cache_ratio":0.7783003182746663,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":49152,"estimated_new_tokens":14001,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":198555,"ongoing_decode_tokens":198555,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1793,"cache_hit":0,"new_prefill":63153,"score_linear":198555.0,"score_lmetric":126306},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1884,"cache_hit":0,"new_prefill":63153,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":116190,"ongoing_decode_tokens":0,"pending_prefill_tokens":127488,"num_requests":0,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":49152,"new_prefill":14001,"score_linear":67038.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1574,"cache_hit":0,"new_prefill":63153,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":63153,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":12800,"new_prefill":50353,"score_linear":95220.0,"score_lmetric":50353},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":29412,"ongoing_decode_tokens":29412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1808,"cache_hit":12800,"new_prefill":50353,"score_linear":16612.0,"score_lmetric":50353},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64353,"ongoing_decode_tokens":64353,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1868,"cache_hit":0,"new_prefill":63153,"score_linear":64353.0,"score_lmetric":126306}],"chosen_score_linear":67038.0,"chosen_score_lmetric":0,"t_first_token":339433.431260791,"t_first_token_unix":1779770298.4452045,"t_done":339439.210613916,"t_done_unix":1779770304.2245576},{"request_id":"1294611:13:1446349:872","session_id":"1294611","input_length":63488,"t_proxy_recv":339439.221878118,"t_decision_unix":1779770304.2358198,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9919354838709677,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (512 < 8000)","cache_hit":62976,"estimated_new_tokens":512,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":117265,"ongoing_decode_tokens":128563,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3009,"cache_hit":62976,"new_prefill":512,"score_linear":54289.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":12800,"new_prefill":50688,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":12800,"new_prefill":50688,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":63488,"score_linear":129931.0,"score_lmetric":128395}],"chosen_score_linear":54289.0,"chosen_score_lmetric":0,"t_first_token":339439.36637433,"t_first_token_unix":1779770304.3803182,"t_done":339441.321894182,"t_done_unix":1779770306.3358378},{"request_id":"1404395:3:1460072:887","session_id":"1404395","input_length":34392,"t_proxy_recv":339441.288291234,"t_decision_unix":1779770306.3022323,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":31744,"affinity_cache_ratio":0.9230053500814143,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2648 < 8000)","cache_hit":31744,"estimated_new_tokens":2648,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26406,"ongoing_decode_tokens":0,"pending_prefill_tokens":294,"num_requests":1,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":34392,"score_linear":26406.0,"score_lmetric":34686},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":512,"new_prefill":33880,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":180753,"ongoing_decode_tokens":192051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3009,"cache_hit":0,"new_prefill":34392,"score_linear":180753.0,"score_lmetric":34392},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":512,"new_prefill":33880,"score_linear":-512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":31744,"new_prefill":2648,"score_linear":-31744.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":34392,"score_linear":129931.0,"score_lmetric":99299}],"chosen_score_linear":-31744.0,"chosen_score_lmetric":0,"t_first_token":339441.916892703,"t_first_token_unix":1779770306.9308395,"t_done":339444.335304647,"t_done_unix":1779770309.3492482},{"request_id":"1242838:5:1460026:886","session_id":"1242838","input_length":26406,"t_proxy_recv":339441.133758507,"t_decision_unix":1779770306.1476996,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9888661667802772,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (294 < 8000)","cache_hit":26112,"estimated_new_tokens":294,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":26112,"new_prefill":294,"score_linear":-26112.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":180753,"ongoing_decode_tokens":192051,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3009,"cache_hit":0,"new_prefill":26406,"score_linear":180753.0,"score_lmetric":26406},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":26406,"score_linear":129931.0,"score_lmetric":91313}],"chosen_score_linear":-26112.0,"chosen_score_lmetric":0,"t_first_token":339444.348131223,"t_first_token_unix":1779770309.362075,"t_done":339446.292436151,"t_done_unix":1779770311.3063796},{"request_id":"1363943:15:1461716:889","session_id":"1363943","input_length":29651,"t_proxy_recv":339446.568770958,"t_decision_unix":1779770311.5827124,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29184,"affinity_cache_ratio":0.984250109608445,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (467 < 8000)","cache_hit":29184,"estimated_new_tokens":467,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":8704,"new_prefill":20947,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":29651,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":180881,"ongoing_decode_tokens":192179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3010,"cache_hit":4096,"new_prefill":25555,"score_linear":176785.0,"score_lmetric":25555},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":29651,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81299,"ongoing_decode_tokens":0,"pending_prefill_tokens":8083,"num_requests":1,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":4096,"new_prefill":25555,"score_linear":77203.0,"score_lmetric":33638},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":4096,"new_prefill":25555,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":29184,"new_prefill":467,"score_linear":-29184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":29651,"score_linear":129931.0,"score_lmetric":94558}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":339446.753627097,"t_first_token_unix":1779770311.7675707,"t_done":339448.133552969,"t_done_unix":1779770313.1474967},{"request_id":"1294611:14:1449074:877","session_id":"1294611","input_length":63616,"t_proxy_recv":339441.331584459,"t_decision_unix":1779770306.345526,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":63488,"affinity_cache_ratio":0.9979879275653923,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (128 < 8000)","cache_hit":63488,"estimated_new_tokens":128,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26406,"ongoing_decode_tokens":0,"pending_prefill_tokens":294,"num_requests":1,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":63616,"score_linear":26406.0,"score_lmetric":63910},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":117265,"ongoing_decode_tokens":128563,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3010,"cache_hit":63488,"new_prefill":128,"score_linear":53777.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":12800,"new_prefill":50816,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34392,"ongoing_decode_tokens":0,"pending_prefill_tokens":2648,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":12800,"new_prefill":50816,"score_linear":21592.0,"score_lmetric":53464},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":63616,"score_linear":129931.0,"score_lmetric":128523}],"chosen_score_linear":53777.0,"chosen_score_lmetric":0,"t_first_token":339441.442707692,"t_first_token_unix":1779770306.4566514,"t_done":339450.759692452,"t_done_unix":1779770315.7736363},{"request_id":"1363943:16:1463626:890","session_id":"1363943","input_length":34671,"t_proxy_recv":339453.038632388,"t_decision_unix":1779770318.0525734,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29696,"affinity_cache_ratio":0.8565083210752502,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4975 < 8000)","cache_hit":29696,"estimated_new_tokens":4975,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":8704,"new_prefill":25967,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":34671,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":117265,"ongoing_decode_tokens":128563,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3011,"cache_hit":4096,"new_prefill":30575,"score_linear":113169.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":34671,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81299,"ongoing_decode_tokens":0,"pending_prefill_tokens":8083,"num_requests":1,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":4096,"new_prefill":30575,"score_linear":77203.0,"score_lmetric":38658},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":4096,"new_prefill":30575,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":29696,"new_prefill":4975,"score_linear":-29696.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":34671,"score_linear":129931.0,"score_lmetric":99578}],"chosen_score_linear":-29696.0,"chosen_score_lmetric":0,"t_first_token":339454.20490568,"t_first_token_unix":1779770319.2188501,"t_done":339455.669128908,"t_done_unix":1779770320.6830728},{"request_id":"1268861:12:1463977:891","session_id":"1268861","input_length":56147,"t_proxy_recv":339454.326194693,"t_decision_unix":1779770319.340136,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9757244376369174,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1363 < 8000)","cache_hit":54784,"estimated_new_tokens":1363,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":117265,"ongoing_decode_tokens":128563,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3011,"cache_hit":0,"new_prefill":56147,"score_linear":117265.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":54784,"new_prefill":1363,"score_linear":-54784.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81299,"ongoing_decode_tokens":0,"pending_prefill_tokens":8083,"num_requests":1,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":56147,"score_linear":81299.0,"score_lmetric":64230},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34671,"ongoing_decode_tokens":34671,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":56147,"score_linear":34671.0,"score_lmetric":56147},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":56147,"score_linear":129931.0,"score_lmetric":121054}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":339454.86088701,"t_first_token_unix":1779770319.8748314,"t_done":339455.745853454,"t_done_unix":1779770320.7597973},{"request_id":"1356885:3:1451307:879","session_id":"1356885","input_length":128563,"t_proxy_recv":339435.592125569,"t_decision_unix":1779770300.6060672,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":127488,"affinity_cache_ratio":0.9916383407356705,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1075 < 8000)","cache_hit":127488,"estimated_new_tokens":1075,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":122368,"new_prefill":6195,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51855,"ongoing_decode_tokens":63153,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2982,"cache_hit":127488,"new_prefill":1075,"score_linear":-75633.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":6656,"new_prefill":121907,"score_linear":123275.0,"score_lmetric":186814}],"chosen_score_linear":-75633.0,"chosen_score_lmetric":0,"t_first_token":339436.668221359,"t_first_token_unix":1779770301.6821647,"t_done":339456.910379112,"t_done_unix":1779770321.9243233},{"request_id":"1371622:6:1465333:892","session_id":"1371622","input_length":23079,"t_proxy_recv":339458.583448369,"t_decision_unix":1779770323.5973895,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9317561419472248,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1575 < 8000)","cache_hit":21504,"estimated_new_tokens":1575,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":21504,"new_prefill":1575,"score_linear":-21504.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":23079,"score_linear":-11298.0,"score_lmetric":-23079},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81299,"ongoing_decode_tokens":0,"pending_prefill_tokens":8083,"num_requests":1,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":13312,"new_prefill":9767,"score_linear":67987.0,"score_lmetric":17850},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1838,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":23079,"score_linear":129931.0,"score_lmetric":87986}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":339458.897816176,"t_first_token_unix":1779770323.9117603,"t_done":339459.422689303,"t_done_unix":1779770324.4366336},{"request_id":"1363943:17:1466811:893","session_id":"1363943","input_length":35991,"t_proxy_recv":339463.72117531,"t_decision_unix":1779770328.7351165,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9673529493484482,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1175 < 8000)","cache_hit":34816,"estimated_new_tokens":1175,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":8704,"new_prefill":27287,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":35991,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":4096,"new_prefill":31895,"score_linear":-15394.0,"score_lmetric":-31895},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":35991,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81299,"ongoing_decode_tokens":0,"pending_prefill_tokens":8083,"num_requests":1,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":4096,"new_prefill":31895,"score_linear":77203.0,"score_lmetric":39978},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":4096,"new_prefill":31895,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1838,"cache_hit":34816,"new_prefill":1175,"score_linear":-34816.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":35991,"score_linear":129931.0,"score_lmetric":100898}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":339464.088913055,"t_first_token_unix":1779770329.102857,"t_done":339465.211976258,"t_done_unix":1779770330.2259202},{"request_id":"1363093:6:1460799:888","session_id":"1363093","input_length":81299,"t_proxy_recv":339443.711477558,"t_decision_unix":1779770308.7254186,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9005768828644879,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":73216,"estimated_new_tokens":8083,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26406,"ongoing_decode_tokens":0,"pending_prefill_tokens":294,"num_requests":1,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":81299,"score_linear":26406.0,"score_lmetric":81593},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":180881,"ongoing_decode_tokens":192179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3010,"cache_hit":0,"new_prefill":81299,"score_linear":180881.0,"score_lmetric":81299},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":73216,"new_prefill":8083,"score_linear":-73216.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34392,"ongoing_decode_tokens":34392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":81299,"score_linear":34392.0,"score_lmetric":81299},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":81299,"score_linear":129931.0,"score_lmetric":146206}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":339466.126922815,"t_first_token_unix":1779770331.1408663,"t_done":339466.740256274,"t_done_unix":1779770331.7542},{"request_id":"1404395:4:1467473:894","session_id":"1404395","input_length":34863,"t_proxy_recv":339466.034444725,"t_decision_unix":1779770331.0483856,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9839658090238935,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (559 < 8000)","cache_hit":34304,"estimated_new_tokens":559,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":512,"new_prefill":34351,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":34863,"score_linear":-11298.0,"score_lmetric":-34863},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81299,"ongoing_decode_tokens":0,"pending_prefill_tokens":8083,"num_requests":1,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":512,"new_prefill":34351,"score_linear":80787.0,"score_lmetric":42434},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":34304,"new_prefill":559,"score_linear":-34304.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":129931,"ongoing_decode_tokens":0,"pending_prefill_tokens":64907,"num_requests":1,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":0,"new_prefill":34863,"score_linear":129931.0,"score_lmetric":99770}],"chosen_score_linear":-34304.0,"chosen_score_lmetric":0,"t_first_token":339466.145398346,"t_first_token_unix":1779770331.1593416,"t_done":339467.388980378,"t_done_unix":1779770332.4029238},{"request_id":"1313181:11:1344301:464","session_id":"1313181","input_length":129931,"t_proxy_recv":339428.368809468,"t_decision_unix":1779770293.382751,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":65024,"affinity_cache_ratio":0.5004502389729933,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":65024,"estimated_new_tokens":64907,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1840,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":179343,"ongoing_decode_tokens":127488,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2733,"cache_hit":61952,"new_prefill":67979,"score_linear":117391.0,"score_lmetric":81980},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1578,"cache_hit":12800,"new_prefill":117131,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":12800,"new_prefill":117131,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1878,"cache_hit":65024,"new_prefill":64907,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":339467.036108004,"t_first_token_unix":1779770332.050052,"t_done":339467.886896459,"t_done_unix":1779770332.9008408},{"request_id":"1313181:11:1402841:762","session_id":"1313181","input_length":65123,"t_proxy_recv":339467.895064152,"t_decision_unix":1779770332.9090059,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9984797997635244,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (99 < 8000)","cache_hit":65024,"estimated_new_tokens":99,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":61952,"new_prefill":3171,"score_linear":-73250.0,"score_lmetric":-3171},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":12800,"new_prefill":52323,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":12800,"new_prefill":52323,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":65024,"new_prefill":99,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":339467.985151851,"t_first_token_unix":1779770332.9990952,"t_done":339468.382128649,"t_done_unix":1779770333.3960721},{"request_id":"1371622:7:1468305:895","session_id":"1371622","input_length":24036,"t_proxy_recv":339468.804440985,"t_decision_unix":1779770333.8183823,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9585621567648527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (996 < 8000)","cache_hit":23040,"estimated_new_tokens":996,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":23040,"new_prefill":996,"score_linear":-23040.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":24036,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":13312,"new_prefill":10724,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":339469.022812702,"t_first_token_unix":1779770334.0367565,"t_done":339469.875070185,"t_done_unix":1779770334.889014},{"request_id":"1363943:18:1469143:896","session_id":"1363943","input_length":47303,"t_proxy_recv":339471.624268426,"t_decision_unix":1779770336.6382098,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35840,"affinity_cache_ratio":0.7576686468088705,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":35840,"estimated_new_tokens":11463,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1845,"cache_hit":8704,"new_prefill":38599,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":47303,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":4096,"new_prefill":43207,"score_linear":115085.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":47303,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":43207,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":4096,"new_prefill":43207,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":35840,"new_prefill":11463,"score_linear":-35840.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":47303,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35840.0,"chosen_score_lmetric":0,"t_first_token":339474.865138848,"t_first_token_unix":1779770339.8790836,"t_done":339477.618950637,"t_done_unix":1779770342.6328943},{"request_id":"1399948:13:1472968:898","session_id":"1399948","input_length":12247,"t_proxy_recv":339484.161809191,"t_decision_unix":1779770349.1757505,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9615416020249857,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (471 < 8000)","cache_hit":11776,"estimated_new_tokens":471,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1845,"cache_hit":11264,"new_prefill":983,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":12247,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":11776,"new_prefill":471,"score_linear":-11776.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1864,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":66246,"ongoing_decode_tokens":0,"pending_prefill_tokens":25798,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":12247,"score_linear":66246.0,"score_lmetric":38045}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":339484.220845267,"t_first_token_unix":1779770349.234789,"t_done":339484.3561743,"t_done_unix":1779770349.3701177},{"request_id":"1405155:3:1469167:897","session_id":"1405155","input_length":66246,"t_proxy_recv":339471.702053819,"t_decision_unix":1779770336.7159953,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":40448,"affinity_cache_ratio":0.610572713824231,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":40448,"estimated_new_tokens":25798,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1845,"cache_hit":512,"new_prefill":65734,"score_linear":-512.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":66246,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47303,"ongoing_decode_tokens":0,"pending_prefill_tokens":11463,"num_requests":1,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":0,"new_prefill":66246,"score_linear":47303.0,"score_lmetric":77709},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":40448,"new_prefill":25798,"score_linear":-40448.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":339487.154375304,"t_first_token_unix":1779770352.1683185,"t_done":339488.982414024,"t_done_unix":1779770353.9963577},{"request_id":"1342921:7:1473812:900","session_id":"1342921","input_length":76600,"t_proxy_recv":339487.120465888,"t_decision_unix":1779770352.1344073,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9892428198433421,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (824 < 8000)","cache_hit":75776,"estimated_new_tokens":824,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":24569,"ongoing_decode_tokens":24569,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1845,"cache_hit":75776,"new_prefill":824,"score_linear":-51207.0,"score_lmetric":824},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":76600,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":69120,"new_prefill":7480,"score_linear":-69120.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1864,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":66246,"ongoing_decode_tokens":0,"pending_prefill_tokens":25798,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":76600,"score_linear":66246.0,"score_lmetric":102398}],"chosen_score_linear":-51207.0,"chosen_score_lmetric":824,"t_first_token":339487.660311206,"t_first_token_unix":1779770352.674255,"t_done":339489.307099319,"t_done_unix":1779770354.3210433},{"request_id":"1371622:8:1473416:899","session_id":"1371622","input_length":24569,"t_proxy_recv":339485.826052625,"t_decision_unix":1779770350.839994,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9794456428833083,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (505 < 8000)","cache_hit":24064,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1845,"cache_hit":24064,"new_prefill":505,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":24569,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":13312,"new_prefill":11257,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1864,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":66246,"ongoing_decode_tokens":0,"pending_prefill_tokens":25798,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":24569,"score_linear":66246.0,"score_lmetric":50367}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":339485.988225823,"t_first_token_unix":1779770351.0021698,"t_done":339490.355843071,"t_done_unix":1779770355.3697867},{"request_id":"1363943:19:1473899:901","session_id":"1363943","input_length":56056,"t_proxy_recv":339487.399139382,"t_decision_unix":1779770352.4130807,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":47616,"affinity_cache_ratio":0.8494362780077066,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":47616,"estimated_new_tokens":8440,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":101169,"ongoing_decode_tokens":24569,"pending_prefill_tokens":824,"num_requests":2,"active_p_offloads":0,"cached_blocks":1845,"cache_hit":8704,"new_prefill":47352,"score_linear":92465.0,"score_lmetric":96352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":56056,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":4096,"new_prefill":51960,"score_linear":115085.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":56056,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":51960,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":4096,"new_prefill":51960,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1864,"cache_hit":47616,"new_prefill":8440,"score_linear":-47616.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":66246,"ongoing_decode_tokens":66246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":0,"new_prefill":56056,"score_linear":66246.0,"score_lmetric":56056}],"chosen_score_linear":-47616.0,"chosen_score_lmetric":0,"t_first_token":339490.37123655,"t_first_token_unix":1779770355.3851798,"t_done":339492.237393979,"t_done_unix":1779770357.2513375},{"request_id":"1363943:20:1476805:903","session_id":"1363943","input_length":60761,"t_proxy_recv":339497.366940241,"t_decision_unix":1779770362.380882,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":55808,"affinity_cache_ratio":0.9184838959200803,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4953 < 8000)","cache_hit":55808,"estimated_new_tokens":4953,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":8704,"new_prefill":52057,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":60761,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":4096,"new_prefill":56665,"score_linear":115085.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":60761,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":56665,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44949,"ongoing_decode_tokens":0,"pending_prefill_tokens":28053,"num_requests":1,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":4096,"new_prefill":56665,"score_linear":40853.0,"score_lmetric":84718},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1880,"cache_hit":55808,"new_prefill":4953,"score_linear":-55808.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":60761,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":339499.171346531,"t_first_token_unix":1779770364.1852908,"t_done":339502.637387896,"t_done_unix":1779770367.6513317},{"request_id":"1305906:4:1476757:902","session_id":"1305906","input_length":44949,"t_proxy_recv":339497.190269045,"t_decision_unix":1779770362.2042105,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":16896,"affinity_cache_ratio":0.375892678368818,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":16896,"estimated_new_tokens":28053,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":44949,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":16896,"new_prefill":28053,"score_linear":-16896.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1880,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":339504.939623408,"t_first_token_unix":1779770369.9535673,"t_done":339506.270992543,"t_done_unix":1779770371.2849364},{"request_id":"1371622:9:1479535:905","session_id":"1371622","input_length":25286,"t_proxy_recv":339506.504617584,"t_decision_unix":1779770371.518559,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":24576,"affinity_cache_ratio":0.9719212212291386,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (710 < 8000)","cache_hit":24576,"estimated_new_tokens":710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":0,"pending_prefill_tokens":427,"num_requests":1,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":24576,"new_prefill":710,"score_linear":14251.0,"score_lmetric":1137},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":25286,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":13312,"new_prefill":11974,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1890,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":14251.0,"chosen_score_lmetric":1137,"t_first_token":339510.660514206,"t_first_token_unix":1779770375.6744578,"t_done":339511.441124048,"t_done_unix":1779770376.4550676},{"request_id":"1404395:5:1480170:906","session_id":"1404395","input_length":36988,"t_proxy_recv":339508.743633668,"t_decision_unix":1779770373.7575746,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9412782524061858,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2172 < 8000)","cache_hit":34816,"estimated_new_tokens":2172,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64113,"ongoing_decode_tokens":0,"pending_prefill_tokens":1137,"num_requests":2,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":0,"new_prefill":36988,"score_linear":64113.0,"score_lmetric":76250},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":512,"new_prefill":36476,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":36988,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":512,"new_prefill":36476,"score_linear":-512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1890,"cache_hit":34816,"new_prefill":2172,"score_linear":-34816.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":339509.33638823,"t_first_token_unix":1779770374.3503325,"t_done":339511.986263922,"t_done_unix":1779770377.0002074},{"request_id":"1268861:13:1481831:907","session_id":"1268861","input_length":57130,"t_proxy_recv":339514.335273656,"t_decision_unix":1779770379.349215,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":55808,"affinity_cache_ratio":0.976859793453527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1322 < 8000)","cache_hit":55808,"estimated_new_tokens":1322,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":0,"new_prefill":57130,"score_linear":38827.0,"score_lmetric":57130},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":57130,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":55808,"new_prefill":1322,"score_linear":-55808.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":339514.760466252,"t_first_token_unix":1779770379.77441,"t_done":339516.052559681,"t_done_unix":1779770381.066504},{"request_id":"1363943:21:1482184:908","session_id":"1363943","input_length":65658,"t_proxy_recv":339515.66455839,"t_decision_unix":1779770380.6784997,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9279600353346127,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4730 < 8000)","cache_hit":60928,"estimated_new_tokens":4730,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":8704,"new_prefill":56954,"score_linear":30123.0,"score_lmetric":56954},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":65658,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":4096,"new_prefill":61562,"score_linear":115085.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":57130,"ongoing_decode_tokens":57130,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":65658,"score_linear":57130.0,"score_lmetric":65658},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":61562,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":61562,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":60928,"new_prefill":4730,"score_linear":-60928.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":65658,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":339517.602671856,"t_first_token_unix":1779770382.616616,"t_done":339520.748658982,"t_done_unix":1779770385.762606},{"request_id":"1301929:4:1479037:904","session_id":"1301929","input_length":38827,"t_proxy_recv":339504.760641602,"t_decision_unix":1779770369.7745829,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9890024982615191,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (427 < 8000)","cache_hit":38400,"estimated_new_tokens":427,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1847,"cache_hit":38400,"new_prefill":427,"score_linear":-38400.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":38827,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":44949,"ongoing_decode_tokens":0,"pending_prefill_tokens":28053,"num_requests":1,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":38827,"score_linear":44949.0,"score_lmetric":66880},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1890,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":339510.65687221,"t_first_token_unix":1779770375.6708162,"t_done":339522.456190746,"t_done_unix":1779770387.4701343},{"request_id":"1313181:12:1346194:470","session_id":"1313181","input_length":130479,"t_proxy_recv":339468.403182946,"t_decision_unix":1779770333.4171245,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":2,"affinity_cache_hit":65024,"affinity_cache_ratio":0.49834839322802904,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":-68527,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":61952,"estimated_new_tokens":68527,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1843,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":61952,"new_prefill":68527,"score_linear":-73250.0,"score_lmetric":-68527},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":12800,"new_prefill":117679,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1841,"cache_hit":12800,"new_prefill":117679,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1879,"cache_hit":65024,"new_prefill":65455,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-73250.0,"chosen_score_lmetric":-68527,"t_first_token":339522.279461471,"t_first_token_unix":1779770387.2934084,"t_done":339523.189578148,"t_done_unix":1779770388.203522},{"request_id":"1313181:12:1404813:772","session_id":"1313181","input_length":65179,"t_proxy_recv":339523.198480143,"t_decision_unix":1779770388.2124217,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9976219334448212,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (155 < 8000)","cache_hit":65024,"estimated_new_tokens":155,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":65024,"new_prefill":155,"score_linear":-76322.0,"score_lmetric":-155},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":12800,"new_prefill":52379,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37689,"ongoing_decode_tokens":37689,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1903,"cache_hit":12800,"new_prefill":52379,"score_linear":24889.0,"score_lmetric":52379},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":65024,"new_prefill":155,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-76322.0,"chosen_score_lmetric":-155,"t_first_token":339523.300394899,"t_first_token_unix":1779770388.3143382,"t_done":339523.84693693,"t_done_unix":1779770388.8608809},{"request_id":"1404395:6:1483967:909","session_id":"1404395","input_length":37689,"t_proxy_recv":339521.585571326,"t_decision_unix":1779770386.5995123,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9781103239672053,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (825 < 8000)","cache_hit":36864,"estimated_new_tokens":825,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":0,"new_prefill":37689,"score_linear":38827.0,"score_lmetric":37689},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":512,"new_prefill":37177,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":119181,"ongoing_decode_tokens":0,"pending_prefill_tokens":68527,"num_requests":0,"active_p_offloads":0,"cached_blocks":3015,"cache_hit":0,"new_prefill":37689,"score_linear":119181.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":512,"new_prefill":37177,"score_linear":-512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1903,"cache_hit":36864,"new_prefill":825,"score_linear":-36864.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-36864.0,"chosen_score_lmetric":0,"t_first_token":339521.80156928,"t_first_token_unix":1779770386.8155127,"t_done":339524.316402439,"t_done_unix":1779770389.3303478},{"request_id":"1363943:22:1485341:910","session_id":"1363943","input_length":70238,"t_proxy_recv":339526.321884139,"t_decision_unix":1779770391.3358252,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":65536,"affinity_cache_ratio":0.9330561804151599,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4702 < 8000)","cache_hit":65536,"estimated_new_tokens":4702,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":8704,"new_prefill":61534,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":70238,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":120401,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":0,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":4096,"new_prefill":66142,"score_linear":116305.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":70238,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":66142,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":66142,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":65536,"new_prefill":4702,"score_linear":-65536.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":70238,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-65536.0,"chosen_score_lmetric":0,"t_first_token":339528.260054392,"t_first_token_unix":1779770393.2739985,"t_done":339531.745479968,"t_done_unix":1779770396.7594235},{"request_id":"1242838:6:1486863:911","session_id":"1242838","input_length":29903,"t_proxy_recv":339531.338229345,"t_decision_unix":1779770396.3521705,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26624,"affinity_cache_ratio":0.8903454502892686,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3279 < 8000)","cache_hit":26624,"estimated_new_tokens":3279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":26624,"new_prefill":3279,"score_linear":-26624.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":120401,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":0,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":0,"new_prefill":29903,"score_linear":120401.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":70238,"ongoing_decode_tokens":70238,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1905,"cache_hit":0,"new_prefill":29903,"score_linear":70238.0,"score_lmetric":29903},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":339532.070499361,"t_first_token_unix":1779770397.0844429,"t_done":339533.424357372,"t_done_unix":1779770398.4383008},{"request_id":"1363943:23:1488713:912","session_id":"1363943","input_length":72516,"t_proxy_recv":339537.476221241,"t_decision_unix":1779770402.4901621,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":70144,"affinity_cache_ratio":0.9672899773843014,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2372 < 8000)","cache_hit":70144,"estimated_new_tokens":2372,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":8704,"new_prefill":63812,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":72516,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":120401,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":0,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":4096,"new_prefill":68420,"score_linear":116305.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":72516,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":68420,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":68420,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1914,"cache_hit":70144,"new_prefill":2372,"score_linear":-70144.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":72516,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70144.0,"chosen_score_lmetric":0,"t_first_token":339538.54199084,"t_first_token_unix":1779770403.555935,"t_done":339540.524828284,"t_done_unix":1779770405.5387719},{"request_id":"1404395:7:1491186:914","session_id":"1404395","input_length":39569,"t_proxy_recv":339546.129030649,"t_decision_unix":1779770411.1429722,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9575172483509818,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1681 < 8000)","cache_hit":37888,"estimated_new_tokens":1681,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":512,"new_prefill":39057,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":120401,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":0,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":0,"new_prefill":39569,"score_linear":120401.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":512,"new_prefill":39057,"score_linear":-512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":74939,"ongoing_decode_tokens":0,"pending_prefill_tokens":2235,"num_requests":1,"active_p_offloads":0,"cached_blocks":1919,"cache_hit":37888,"new_prefill":1681,"score_linear":37051.0,"score_lmetric":3916},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":37051.0,"chosen_score_lmetric":3916,"t_first_token":339547.432624526,"t_first_token_unix":1779770412.4465685,"t_done":339547.92810913,"t_done_unix":1779770412.942053},{"request_id":"1363943:24:1491020:913","session_id":"1363943","input_length":74939,"t_proxy_recv":339545.735297602,"t_decision_unix":1779770410.7492387,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":72704,"affinity_cache_ratio":0.9701757429375892,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2235 < 8000)","cache_hit":72704,"estimated_new_tokens":2235,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":8704,"new_prefill":66235,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":74939,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":120401,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":0,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":4096,"new_prefill":70843,"score_linear":116305.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":74939,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":70843,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":70843,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1919,"cache_hit":72704,"new_prefill":2235,"score_linear":-72704.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":74939,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72704.0,"chosen_score_lmetric":0,"t_first_token":339546.963160233,"t_first_token_unix":1779770411.977104,"t_done":339549.155135158,"t_done_unix":1779770414.169079},{"request_id":"1363943:25:1493325:915","session_id":"1363943","input_length":75711,"t_proxy_recv":339553.214474428,"t_decision_unix":1779770418.2284157,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":74752,"affinity_cache_ratio":0.9873334125820554,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (959 < 8000)","cache_hit":74752,"estimated_new_tokens":959,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":8704,"new_prefill":67007,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":75711,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":120401,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":0,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":4096,"new_prefill":71615,"score_linear":116305.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":75711,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":71615,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":71615,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":74752,"new_prefill":959,"score_linear":-74752.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":75711,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74752.0,"chosen_score_lmetric":0,"t_first_token":339553.622220925,"t_first_token_unix":1779770418.6361644,"t_done":339555.816868905,"t_done_unix":1779770420.8308127},{"request_id":"1404395:8:1494219:916","session_id":"1404395","input_length":39820,"t_proxy_recv":339556.240157355,"t_decision_unix":1779770421.254099,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":39424,"affinity_cache_ratio":0.9900552486187846,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (396 < 8000)","cache_hit":39424,"estimated_new_tokens":396,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":512,"new_prefill":39308,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":120401,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":0,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":0,"new_prefill":39820,"score_linear":120401.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":512,"new_prefill":39308,"score_linear":-512.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1928,"cache_hit":39424,"new_prefill":396,"score_linear":-39424.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39424.0,"chosen_score_lmetric":0,"t_first_token":339556.39124472,"t_first_token_unix":1779770421.4051886,"t_done":339559.998565928,"t_done_unix":1779770425.0125096},{"request_id":"1363943:26:1495648:917","session_id":"1363943","input_length":83861,"t_proxy_recv":339560.979379792,"t_decision_unix":1779770425.993321,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":75776,"affinity_cache_ratio":0.903590465174515,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":75776,"estimated_new_tokens":8085,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":8704,"new_prefill":75157,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":83861,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":120401,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":0,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":4096,"new_prefill":79765,"score_linear":116305.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":83861,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":79765,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":79765,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":75776,"new_prefill":8085,"score_linear":-75776.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":83861,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":339565.024852317,"t_first_token_unix":1779770430.038796,"t_done":339566.865762283,"t_done_unix":1779770431.8797066},{"request_id":"1363943:27:1497882:919","session_id":"1363943","input_length":84090,"t_proxy_recv":339568.252783489,"t_decision_unix":1779770433.2667248,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":83968,"affinity_cache_ratio":0.9985491735045784,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (122 < 8000)","cache_hit":83968,"estimated_new_tokens":122,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":8704,"new_prefill":75386,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":84090,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":196161,"ongoing_decode_tokens":131699,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":4096,"new_prefill":79994,"score_linear":192065.0,"score_lmetric":84074},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":84090,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":79994,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":79994,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1945,"cache_hit":83968,"new_prefill":122,"score_linear":-83968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":84090,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-83968.0,"chosen_score_lmetric":0,"t_first_token":339568.394253742,"t_first_token_unix":1779770433.4081976,"t_done":339571.970627157,"t_done_unix":1779770436.984571},{"request_id":"1268861:14:1498281:920","session_id":"1268861","input_length":61910,"t_proxy_recv":339569.724648495,"t_decision_unix":1779770434.73859,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":56832,"affinity_cache_ratio":0.9179777095784203,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5078 < 8000)","cache_hit":56832,"estimated_new_tokens":5078,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":196161,"ongoing_decode_tokens":131699,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":0,"new_prefill":61910,"score_linear":196161.0,"score_lmetric":65990},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":56832,"new_prefill":5078,"score_linear":-56832.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":84090,"ongoing_decode_tokens":84090,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1945,"cache_hit":0,"new_prefill":61910,"score_linear":84090.0,"score_lmetric":61910},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-56832.0,"chosen_score_lmetric":0,"t_first_token":339571.548586362,"t_first_token_unix":1779770436.56253,"t_done":339572.123910883,"t_done_unix":1779770437.1378546},{"request_id":"1242838:7:1499476:921","session_id":"1242838","input_length":33352,"t_proxy_recv":339573.558028065,"t_decision_unix":1779770438.5719693,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29696,"affinity_cache_ratio":0.8903813864236028,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3656 < 8000)","cache_hit":29696,"estimated_new_tokens":3656,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":29696,"new_prefill":3656,"score_linear":-29696.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":196161,"ongoing_decode_tokens":131699,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":0,"new_prefill":33352,"score_linear":196161.0,"score_lmetric":37432},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1945,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-29696.0,"chosen_score_lmetric":0,"t_first_token":339574.367286747,"t_first_token_unix":1779770439.3812306,"t_done":339576.397401819,"t_done_unix":1779770441.4113457},{"request_id":"1363943:28:1500984:922","session_id":"1363943","input_length":86984,"t_proxy_recv":339578.565200954,"t_decision_unix":1779770443.5791423,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":83968,"affinity_cache_ratio":0.9653269566816886,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3016 < 8000)","cache_hit":83968,"estimated_new_tokens":3016,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1861,"cache_hit":8704,"new_prefill":78280,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":86984,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":196161,"ongoing_decode_tokens":131699,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":4096,"new_prefill":82888,"score_linear":192065.0,"score_lmetric":86968},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":86984,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":82888,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":82888,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1945,"cache_hit":83968,"new_prefill":3016,"score_linear":-83968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":86984,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-83968.0,"chosen_score_lmetric":0,"t_first_token":339580.129122056,"t_first_token_unix":1779770445.1430666,"t_done":339582.310369203,"t_done_unix":1779770447.3243127},{"request_id":"1313181:13:1349104:484","session_id":"1313181","input_length":131699,"t_proxy_recv":339523.86813122,"t_decision_unix":1779770388.882073,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":65024,"affinity_cache_ratio":0.49373191899710706,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":-66675,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=-1 decode_tok=0)","cache_hit":65024,"estimated_new_tokens":66675,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1848,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":65024,"new_prefill":66675,"score_linear":-76322.0,"score_lmetric":-66675},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":12800,"new_prefill":118899,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37689,"ongoing_decode_tokens":37689,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1903,"cache_hit":12800,"new_prefill":118899,"score_linear":24889.0,"score_lmetric":118899},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":65024,"new_prefill":66675,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-76322.0,"chosen_score_lmetric":-66675,"t_first_token":339564.063953836,"t_first_token_unix":1779770429.077898,"t_done":339585.722625458,"t_done_unix":1779770450.7365694},{"request_id":"1388507:2:1496686:918","session_id":"1388507","input_length":75760,"t_proxy_recv":339564.34217983,"t_decision_unix":1779770429.356121,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9461457233368532,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4080 < 8000)","cache_hit":71680,"estimated_new_tokens":4080,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1854,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":120401,"ongoing_decode_tokens":131699,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":71680,"new_prefill":4080,"score_linear":48721.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1582,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":83861,"ongoing_decode_tokens":0,"pending_prefill_tokens":8085,"num_requests":1,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":75760,"score_linear":83861.0,"score_lmetric":83845},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":48721.0,"chosen_score_lmetric":0,"t_first_token":339584.102584708,"t_first_token_unix":1779770449.116529,"t_done":339586.839873144,"t_done_unix":1779770451.8538167},{"request_id":"1313181:13:1406095:783","session_id":"1313181","input_length":65279,"t_proxy_recv":339585.731765928,"t_decision_unix":1779770450.7457075,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9960936901606948,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (255 < 8000)","cache_hit":65024,"estimated_new_tokens":255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1861,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":64462,"ongoing_decode_tokens":75760,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3039,"cache_hit":65024,"new_prefill":255,"score_linear":-562.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":12800,"new_prefill":52479,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":65279,"score_linear":103893.0,"score_lmetric":75988},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":12800,"new_prefill":52479,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":65024,"new_prefill":255,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-562.0,"chosen_score_lmetric":0,"t_first_token":339585.832932282,"t_first_token_unix":1779770450.8468754,"t_done":339588.391122145,"t_done_unix":1779770453.405066},{"request_id":"1342921:8:1503584:925","session_id":"1342921","input_length":78722,"t_proxy_recv":339587.416859842,"t_decision_unix":1779770452.4308012,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":76288,"affinity_cache_ratio":0.9690810700947639,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2434 < 8000)","cache_hit":76288,"estimated_new_tokens":2434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1861,"cache_hit":76288,"new_prefill":2434,"score_linear":-76288.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":53981,"ongoing_decode_tokens":65279,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3047,"cache_hit":0,"new_prefill":78722,"score_linear":53981.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":69120,"new_prefill":9602,"score_linear":34773.0,"score_lmetric":20311},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89469,"ongoing_decode_tokens":0,"pending_prefill_tokens":2429,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":78722,"score_linear":89469.0,"score_lmetric":81151},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76288.0,"chosen_score_lmetric":0,"t_first_token":339588.584211796,"t_first_token_unix":1779770453.598155,"t_done":339588.737659676,"t_done_unix":1779770453.7516034},{"request_id":"1363943:29:1503515:924","session_id":"1363943","input_length":89469,"t_proxy_recv":339587.164080551,"t_decision_unix":1779770452.178022,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":87040,"affinity_cache_ratio":0.972850931607596,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2429 < 8000)","cache_hit":87040,"estimated_new_tokens":2429,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1861,"cache_hit":8704,"new_prefill":80765,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":89469,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":53981,"ongoing_decode_tokens":65279,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3047,"cache_hit":4096,"new_prefill":85373,"score_linear":49885.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":89469,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":4096,"new_prefill":85373,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":85373,"score_linear":99797.0,"score_lmetric":96082},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":87040,"new_prefill":2429,"score_linear":-87040.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":89469,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-87040.0,"chosen_score_lmetric":0,"t_first_token":339588.606129367,"t_first_token_unix":1779770453.6200743,"t_done":339590.646307556,"t_done_unix":1779770455.660251},{"request_id":"1399948:14:1504493:926","session_id":"1399948","input_length":14189,"t_proxy_recv":339590.197310571,"t_decision_unix":1779770455.2112517,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":11776,"affinity_cache_ratio":0.829938684896751,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2413 < 8000)","cache_hit":11776,"estimated_new_tokens":2413,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":11264,"new_prefill":2925,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":128059,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":0,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":14189,"score_linear":128059.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":11776,"new_prefill":2413,"score_linear":-11776.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":14189,"score_linear":103893.0,"score_lmetric":24898},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89469,"ongoing_decode_tokens":89469,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":14189,"score_linear":89469.0,"score_lmetric":14189},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":339590.489648188,"t_first_token_unix":1779770455.5035918,"t_done":339590.778665741,"t_done_unix":1779770455.792609},{"request_id":"1363943:30:1506171:927","session_id":"1363943","input_length":89713,"t_proxy_recv":339596.003798734,"t_decision_unix":1779770461.0177398,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":89600,"affinity_cache_ratio":0.9987404278086788,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (113 < 8000)","cache_hit":89600,"estimated_new_tokens":113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":8704,"new_prefill":81009,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":89713,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":128059,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":0,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":4096,"new_prefill":85617,"score_linear":123963.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":89713,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1943,"cache_hit":4096,"new_prefill":85617,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":85617,"score_linear":99797.0,"score_lmetric":96326},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":89600,"new_prefill":113,"score_linear":-89600.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":89713,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-89600.0,"chosen_score_lmetric":0,"t_first_token":339596.14967864,"t_first_token_unix":1779770461.163622,"t_done":339596.811115418,"t_done_unix":1779770461.8250592},{"request_id":"1268861:15:1508874:928","session_id":"1268861","input_length":63035,"t_proxy_recv":339605.060105768,"t_decision_unix":1779770470.074047,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":61952,"affinity_cache_ratio":0.9828190687713175,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1083 < 8000)","cache_hit":61952,"estimated_new_tokens":1083,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":128059,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":0,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":63035,"score_linear":128059.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":61952,"new_prefill":1083,"score_linear":-61952.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1943,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":63035,"score_linear":103893.0,"score_lmetric":73744},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-61952.0,"chosen_score_lmetric":0,"t_first_token":339605.636231114,"t_first_token_unix":1779770470.6501741,"t_done":339607.075194286,"t_done_unix":1779770472.0891383},{"request_id":"1397515:2:1502033:923","session_id":"1397515","input_length":103893,"t_proxy_recv":339582.123937501,"t_decision_unix":1779770447.137879,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":93184,"affinity_cache_ratio":0.8969227955685176,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":93184,"estimated_new_tokens":10709,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1861,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":196161,"ongoing_decode_tokens":131699,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":3035,"cache_hit":0,"new_prefill":103893,"score_linear":196161.0,"score_lmetric":107973},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":93184,"new_prefill":10709,"score_linear":-93184.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86984,"ongoing_decode_tokens":86984,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1945,"cache_hit":0,"new_prefill":103893,"score_linear":86984.0,"score_lmetric":103893},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":339608.602925072,"t_first_token_unix":1779770473.616869,"t_done":339612.354540486,"t_done_unix":1779770477.3684843},{"request_id":"1399948:15:1512300:930","session_id":"1399948","input_length":15669,"t_proxy_recv":339616.513531681,"t_decision_unix":1779770481.5274732,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":13824,"affinity_cache_ratio":0.8822515795519816,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1845 < 8000)","cache_hit":13824,"estimated_new_tokens":1845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":11264,"new_prefill":4405,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":128059,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":0,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":15669,"score_linear":128059.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1943,"cache_hit":13824,"new_prefill":1845,"score_linear":-13824.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":15669,"score_linear":89790.0,"score_lmetric":15669},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":339616.77767862,"t_first_token_unix":1779770481.791622,"t_done":339617.009122666,"t_done_unix":1779770482.0230656},{"request_id":"1399948:16:1513839:933","session_id":"1399948","input_length":15913,"t_proxy_recv":339621.694810448,"t_decision_unix":1779770486.708752,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":15360,"affinity_cache_ratio":0.9652485389304343,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (553 < 8000)","cache_hit":15360,"estimated_new_tokens":553,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":11264,"new_prefill":4649,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":0,"pending_prefill_tokens":10637,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":15913,"score_linear":302955.0,"score_lmetric":53100},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1946,"cache_hit":15360,"new_prefill":553,"score_linear":-15360.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":15913,"score_linear":89790.0,"score_lmetric":15913},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":339621.785034966,"t_first_token_unix":1779770486.798978,"t_done":339622.218993342,"t_done_unix":1779770487.2329369},{"request_id":"1399948:17:1518102:936","session_id":"1399948","input_length":16604,"t_proxy_recv":339635.806593848,"t_decision_unix":1779770500.820535,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":15872,"affinity_cache_ratio":0.9559142375331245,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (732 < 8000)","cache_hit":15872,"estimated_new_tokens":732,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":11264,"new_prefill":5340,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":16604,"score_linear":302955.0,"score_lmetric":38936},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1947,"cache_hit":15872,"new_prefill":732,"score_linear":-15872.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":16604,"score_linear":89790.0,"score_lmetric":16604},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-15872.0,"chosen_score_lmetric":0,"t_first_token":339635.946287691,"t_first_token_unix":1779770500.9602308,"t_done":339636.303119606,"t_done_unix":1779770501.317063},{"request_id":"1363943:31:1509815:929","session_id":"1363943","input_length":89790,"t_proxy_recv":339608.07701952,"t_decision_unix":1779770473.0909607,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":89600,"affinity_cache_ratio":0.9978839514422542,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (190 < 8000)","cache_hit":89600,"estimated_new_tokens":190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":8704,"new_prefill":81086,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":89790,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":128059,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":0,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":4096,"new_prefill":85694,"score_linear":123963.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":89790,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1943,"cache_hit":4096,"new_prefill":85694,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":4096,"new_prefill":85694,"score_linear":99797.0,"score_lmetric":96403},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":89600,"new_prefill":190,"score_linear":-89600.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":89790,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-89600.0,"chosen_score_lmetric":0,"t_first_token":339608.210152613,"t_first_token_unix":1779770473.224096,"t_done":339643.696941797,"t_done_unix":1779770508.7108853},{"request_id":"1399948:18:1524432:938","session_id":"1399948","input_length":17276,"t_proxy_recv":339656.861724955,"t_decision_unix":1779770521.8756664,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":16384,"affinity_cache_ratio":0.948367677703172,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (892 < 8000)","cache_hit":16384,"estimated_new_tokens":892,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":11264,"new_prefill":6012,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":17276,"score_linear":302955.0,"score_lmetric":40280},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":16384,"new_prefill":892,"score_linear":-16384.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16384.0,"chosen_score_lmetric":0,"t_first_token":339657.011521253,"t_first_token_unix":1779770522.0254648,"t_done":339657.358714007,"t_done_unix":1779770522.3726573},{"request_id":"1342921:9:1525898:939","session_id":"1342921","input_length":79086,"t_proxy_recv":339661.898612694,"t_decision_unix":1779770526.9125447,"policy":"unified_v2","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9905166527577574,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (750 < 8000)","cache_hit":78336,"estimated_new_tokens":750,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":78336,"new_prefill":750,"score_linear":-78336.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":79086,"score_linear":302955.0,"score_lmetric":163900},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1949,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":69120,"new_prefill":9966,"score_linear":-69120.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":339662.302112801,"t_first_token_unix":1779770527.3160563,"t_done":339662.389310224,"t_done_unix":1779770527.4032536},{"request_id":"1399948:19:1526739:941","session_id":"1399948","input_length":18091,"t_proxy_recv":339664.780384894,"t_decision_unix":1779770529.794326,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":16896,"affinity_cache_ratio":0.9339450555524846,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1195 < 8000)","cache_hit":16896,"estimated_new_tokens":1195,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":6827,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":18091,"score_linear":87576.0,"score_lmetric":28867},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":18091,"score_linear":302955.0,"score_lmetric":41910},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1949,"cache_hit":16896,"new_prefill":1195,"score_linear":-16896.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":339664.960668067,"t_first_token_unix":1779770529.9746118,"t_done":339665.245664257,"t_done_unix":1779770530.2596076},{"request_id":"1399948:20:1527857:942","session_id":"1399948","input_length":18164,"t_proxy_recv":339668.279767961,"t_decision_unix":1779770533.293709,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":17920,"affinity_cache_ratio":0.9865668354987888,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (244 < 8000)","cache_hit":17920,"estimated_new_tokens":244,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":6900,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":18164,"score_linear":87576.0,"score_lmetric":28940},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":18164,"score_linear":302955.0,"score_lmetric":42056},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":17920,"new_prefill":244,"score_linear":-17920.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-17920.0,"chosen_score_lmetric":0,"t_first_token":339668.33435144,"t_first_token_unix":1779770533.348295,"t_done":339668.872063319,"t_done_unix":1779770533.8860068},{"request_id":"1399948:21:1528978:944","session_id":"1399948","input_length":18388,"t_proxy_recv":339672.224755553,"t_decision_unix":1779770537.238697,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":17920,"affinity_cache_ratio":0.9745486186643463,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (468 < 8000)","cache_hit":17920,"estimated_new_tokens":468,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":7124,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":18388,"score_linear":87576.0,"score_lmetric":29164},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":18388,"score_linear":302955.0,"score_lmetric":42504},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":17920,"new_prefill":468,"score_linear":-17920.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":36568,"ongoing_decode_tokens":0,"pending_prefill_tokens":36568,"num_requests":1,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":18388,"score_linear":36568.0,"score_lmetric":54956}],"chosen_score_linear":-17920.0,"chosen_score_lmetric":0,"t_first_token":339672.311573732,"t_first_token_unix":1779770537.3255167,"t_done":339672.897331978,"t_done_unix":1779770537.9112751},{"request_id":"1399948:22:1530122:947","session_id":"1399948","input_length":18563,"t_proxy_recv":339676.054250359,"t_decision_unix":1779770541.0681915,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":18432,"affinity_cache_ratio":0.992942951031622,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (131 < 8000)","cache_hit":18432,"estimated_new_tokens":131,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":7299,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":18563,"score_linear":87576.0,"score_lmetric":29339},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":18563,"score_linear":302955.0,"score_lmetric":42854},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71192,"ongoing_decode_tokens":0,"pending_prefill_tokens":8216,"num_requests":1,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":18563,"score_linear":71192.0,"score_lmetric":26779},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":18432,"new_prefill":131,"score_linear":-18432.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":36568,"ongoing_decode_tokens":36568,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":18563,"score_linear":36568.0,"score_lmetric":18563}],"chosen_score_linear":-18432.0,"chosen_score_lmetric":0,"t_first_token":339676.117951093,"t_first_token_unix":1779770541.1318944,"t_done":339676.369678655,"t_done_unix":1779770541.383622},{"request_id":"1243831:2:1528168:943","session_id":"1243831","input_length":36568,"t_proxy_recv":339669.485203263,"t_decision_unix":1779770534.4991448,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":7,"affinity_cache_hit":11264,"affinity_cache_ratio":0.30802887770728504,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":0,"estimated_new_tokens":36568,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":25304,"score_linear":76312.0,"score_lmetric":36080},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":36568,"score_linear":302955.0,"score_lmetric":78864},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":339674.961283461,"t_first_token_unix":1779770539.975227,"t_done":339679.256433503,"t_done_unix":1779770544.270377},{"request_id":"1268861:16:1529894:946","session_id":"1268861","input_length":71192,"t_proxy_recv":339675.238877317,"t_decision_unix":1779770540.2528183,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":62976,"affinity_cache_ratio":0.8845937745814136,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":62976,"estimated_new_tokens":8216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":71192,"score_linear":87576.0,"score_lmetric":81968},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":71192,"score_linear":302955.0,"score_lmetric":148112},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":62976,"new_prefill":8216,"score_linear":-62976.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":36568,"ongoing_decode_tokens":36568,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":71192,"score_linear":36568.0,"score_lmetric":71192}],"chosen_score_linear":-62976.0,"chosen_score_lmetric":0,"t_first_token":339678.673473988,"t_first_token_unix":1779770543.6874175,"t_done":339679.807715427,"t_done_unix":1779770544.8216593},{"request_id":"1243831:3:1270336:110","session_id":"1243831","input_length":11584,"t_proxy_recv":339679.259722067,"t_decision_unix":1779770544.2736638,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9723756906077348,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (320 < 8000)","cache_hit":11264,"estimated_new_tokens":320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":320,"score_linear":76312.0,"score_lmetric":11096},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":11584,"score_linear":302955.0,"score_lmetric":28896},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71192,"ongoing_decode_tokens":71192,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":11584,"score_linear":71192.0,"score_lmetric":11584},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":11264,"new_prefill":320,"score_linear":-11264.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":339679.293900184,"t_first_token_unix":1779770544.3078442,"t_done":339680.139390778,"t_done_unix":1779770545.1533346},{"request_id":"1243831:4:1284306:183","session_id":"1243831","input_length":14515,"t_proxy_recv":339680.145485239,"t_decision_unix":1779770545.159427,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":11264,"affinity_cache_ratio":0.7760248019290389,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3251 < 8000)","cache_hit":11264,"estimated_new_tokens":3251,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":3251,"score_linear":76312.0,"score_lmetric":14027},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":14515,"score_linear":302955.0,"score_lmetric":34758},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":11264,"new_prefill":3251,"score_linear":-11264.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":339680.517577719,"t_first_token_unix":1779770545.5315223,"t_done":339680.951923307,"t_done_unix":1779770545.965867},{"request_id":"1243831:5:1303288:273","session_id":"1243831","input_length":14627,"t_proxy_recv":339680.956485397,"t_decision_unix":1779770545.970427,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":14336,"affinity_cache_ratio":0.980105284747385,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (291 < 8000)","cache_hit":14336,"estimated_new_tokens":291,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":3363,"score_linear":76312.0,"score_lmetric":14139},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":14627,"score_linear":302955.0,"score_lmetric":34982},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":14336,"new_prefill":291,"score_linear":-14336.0,"score_lmetric":0}],"chosen_score_linear":-14336.0,"chosen_score_lmetric":0,"t_first_token":339681.006947376,"t_first_token_unix":1779770546.0208907,"t_done":339681.462744182,"t_done_unix":1779770546.4766877},{"request_id":"1243831:6:1313694:320","session_id":"1243831","input_length":17410,"t_proxy_recv":339681.46831862,"t_decision_unix":1779770546.4822602,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":14336,"affinity_cache_ratio":0.8234348075818495,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3074 < 8000)","cache_hit":14336,"estimated_new_tokens":3074,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":6146,"score_linear":76312.0,"score_lmetric":16922},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":17410,"score_linear":302955.0,"score_lmetric":40548},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":14336,"new_prefill":3074,"score_linear":-14336.0,"score_lmetric":0}],"chosen_score_linear":-14336.0,"chosen_score_lmetric":0,"t_first_token":339681.884163511,"t_first_token_unix":1779770546.8981068,"t_done":339682.343398714,"t_done_unix":1779770547.357342},{"request_id":"1243831:7:1323865:366","session_id":"1243831","input_length":19460,"t_proxy_recv":339682.348838374,"t_decision_unix":1779770547.36278,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":17408,"affinity_cache_ratio":0.8945529290853032,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2052 < 8000)","cache_hit":17408,"estimated_new_tokens":2052,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":8196,"score_linear":76312.0,"score_lmetric":18972},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":19460,"score_linear":302955.0,"score_lmetric":44648},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":17408,"new_prefill":2052,"score_linear":-17408.0,"score_lmetric":0}],"chosen_score_linear":-17408.0,"chosen_score_lmetric":0,"t_first_token":339682.70370576,"t_first_token_unix":1779770547.7176495,"t_done":339683.311899706,"t_done_unix":1779770548.325843},{"request_id":"1243831:8:1333049:414","session_id":"1243831","input_length":24512,"t_proxy_recv":339683.318816217,"t_decision_unix":1779770548.332758,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19456,"affinity_cache_ratio":0.793733681462141,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5056 < 8000)","cache_hit":19456,"estimated_new_tokens":5056,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":13248,"score_linear":76312.0,"score_lmetric":24024},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":24512,"score_linear":302955.0,"score_lmetric":54752},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":19456,"new_prefill":5056,"score_linear":-19456.0,"score_lmetric":0}],"chosen_score_linear":-19456.0,"chosen_score_lmetric":0,"t_first_token":339684.189749541,"t_first_token_unix":1779770549.2036934,"t_done":339684.871877981,"t_done_unix":1779770549.8858213},{"request_id":"1243831:9:1348240:478","session_id":"1243831","input_length":25075,"t_proxy_recv":339684.877279377,"t_decision_unix":1779770549.891221,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":24576,"affinity_cache_ratio":0.980099700897308,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (499 < 8000)","cache_hit":24576,"estimated_new_tokens":499,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":13811,"score_linear":76312.0,"score_lmetric":24587},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":25075,"score_linear":302955.0,"score_lmetric":55878},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":24576,"new_prefill":499,"score_linear":-24576.0,"score_lmetric":0}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":339685.043877738,"t_first_token_unix":1779770550.0578218,"t_done":339685.453820193,"t_done_unix":1779770550.4677634},{"request_id":"1243831:10:1358921:521","session_id":"1243831","input_length":26343,"t_proxy_recv":339685.460113911,"t_decision_unix":1779770550.4740558,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9523592605246175,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1255 < 8000)","cache_hit":25088,"estimated_new_tokens":1255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":15079,"score_linear":76312.0,"score_lmetric":25855},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":26343,"score_linear":302955.0,"score_lmetric":58414},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2028,"cache_hit":25088,"new_prefill":1255,"score_linear":-25088.0,"score_lmetric":0}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":0,"t_first_token":339685.747101183,"t_first_token_unix":1779770550.7610447,"t_done":339686.45065605,"t_done_unix":1779770551.4645994},{"request_id":"1243831:11:1371191:594","session_id":"1243831","input_length":26818,"t_proxy_recv":339686.456650113,"t_decision_unix":1779770551.4705918,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9736743977925274,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (706 < 8000)","cache_hit":26112,"estimated_new_tokens":706,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":15554,"score_linear":76312.0,"score_lmetric":26330},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":26818,"score_linear":302955.0,"score_lmetric":59364},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2030,"cache_hit":26112,"new_prefill":706,"score_linear":-26112.0,"score_lmetric":0}],"chosen_score_linear":-26112.0,"chosen_score_lmetric":0,"t_first_token":339686.620784947,"t_first_token_unix":1779770551.6347282,"t_done":339687.083624656,"t_done_unix":1779770552.097568},{"request_id":"1243831:12:1385174:681","session_id":"1243831","input_length":27164,"t_proxy_recv":339687.089680765,"t_decision_unix":1779770552.1036224,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9801207480488883,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (540 < 8000)","cache_hit":26624,"estimated_new_tokens":540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":15900,"score_linear":76312.0,"score_lmetric":26676},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":27164,"score_linear":302955.0,"score_lmetric":60056},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2031,"cache_hit":26624,"new_prefill":540,"score_linear":-26624.0,"score_lmetric":0}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":339687.203062189,"t_first_token_unix":1779770552.2170055,"t_done":339687.617253838,"t_done_unix":1779770552.6311972},{"request_id":"1243831:13:1398072:738","session_id":"1243831","input_length":27286,"t_proxy_recv":339687.624465082,"t_decision_unix":1779770552.6384068,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9945026753646559,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (150 < 8000)","cache_hit":27136,"estimated_new_tokens":150,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":16022,"score_linear":76312.0,"score_lmetric":26798},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":27286,"score_linear":302955.0,"score_lmetric":60300},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2032,"cache_hit":27136,"new_prefill":150,"score_linear":-27136.0,"score_lmetric":0}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":339687.703787753,"t_first_token_unix":1779770552.7177308,"t_done":339688.237124053,"t_done_unix":1779770553.2510672},{"request_id":"1243831:14:1411189:814","session_id":"1243831","input_length":28085,"t_proxy_recv":339688.242897514,"t_decision_unix":1779770553.2568393,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9662097204913654,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (949 < 8000)","cache_hit":27136,"estimated_new_tokens":949,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":16821,"score_linear":76312.0,"score_lmetric":27597},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":269889,"pending_prefill_tokens":1868,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":28085,"score_linear":302955.0,"score_lmetric":59906},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2032,"cache_hit":27136,"new_prefill":949,"score_linear":-27136.0,"score_lmetric":0}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":339688.478778042,"t_first_token_unix":1779770553.4927216,"t_done":339689.014018354,"t_done_unix":1779770554.0279615},{"request_id":"1243831:15:1427593:849","session_id":"1243831","input_length":29382,"t_proxy_recv":339689.020231219,"t_decision_unix":1779770554.034173,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28160,"affinity_cache_ratio":0.9584099108297597,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1222 < 8000)","cache_hit":28160,"estimated_new_tokens":1222,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":18118,"score_linear":76312.0,"score_lmetric":28894},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":374942,"ongoing_decode_tokens":269889,"pending_prefill_tokens":9855,"num_requests":3,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":29382,"score_linear":374942.0,"score_lmetric":117711},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":28160,"new_prefill":1222,"score_linear":-28160.0,"score_lmetric":0}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":339689.33685743,"t_first_token_unix":1779770554.3508008,"t_done":339690.194098641,"t_done_unix":1779770555.2080424},{"request_id":"1243831:16:1446717:874","session_id":"1243831","input_length":29571,"t_proxy_recv":339690.200798943,"t_decision_unix":1779770555.2147403,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9869128538094755,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (387 < 8000)","cache_hit":29184,"estimated_new_tokens":387,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":18307,"score_linear":76312.0,"score_lmetric":29083},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":374942,"ongoing_decode_tokens":269889,"pending_prefill_tokens":9855,"num_requests":3,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":29571,"score_linear":374942.0,"score_lmetric":118278},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":29184,"new_prefill":387,"score_linear":-29184.0,"score_lmetric":0}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":339690.282887382,"t_first_token_unix":1779770555.2968311,"t_done":339690.71457778,"t_done_unix":1779770555.7285218},{"request_id":"1356885:4:1512902:931","session_id":"1356885","input_length":130532,"t_proxy_recv":339618.405183449,"t_decision_unix":1779770483.419125,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":129536,"affinity_cache_ratio":0.9923696871265284,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (996 < 8000)","cache_hit":129536,"estimated_new_tokens":996,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":122368,"new_prefill":8164,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":128059,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":0,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":129536,"new_prefill":996,"score_linear":-1477.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1946,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":130532,"score_linear":89790.0,"score_lmetric":130532},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":6656,"new_prefill":123876,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-1477.0,"chosen_score_lmetric":0,"t_first_token":339688.193290408,"t_first_token_unix":1779770553.2072334,"t_done":339694.543232965,"t_done_unix":1779770559.5571766},{"request_id":"1363943:32:1537767:950","session_id":"1363943","input_length":93368,"t_proxy_recv":339701.686240536,"t_decision_unix":1779770566.7001817,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9980293034015937,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (184 < 8000)","cache_hit":93184,"estimated_new_tokens":184,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":8704,"new_prefill":84664,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":87576,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":93368,"score_linear":87576.0,"score_lmetric":93368},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":375316,"ongoing_decode_tokens":139357,"pending_prefill_tokens":10201,"num_requests":3,"active_p_offloads":0,"cached_blocks":3050,"cache_hit":4096,"new_prefill":89272,"score_linear":371220.0,"score_lmetric":298419},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":93368,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":4096,"new_prefill":89272,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":4096,"new_prefill":89272,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":93184,"new_prefill":184,"score_linear":-93184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":93368,"score_linear":31161.0,"score_lmetric":93368}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":339701.895162364,"t_first_token_unix":1779770566.9091058,"t_done":339703.825586838,"t_done_unix":1779770568.8395305},{"request_id":"1253804:7:1526643:940","session_id":"1253804","input_length":87576,"t_proxy_recv":339664.346340081,"t_decision_unix":1779770529.360281,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":76800,"affinity_cache_ratio":0.8769525897506166,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":76800,"estimated_new_tokens":10776,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":76800,"new_prefill":10776,"score_linear":-76800.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2864,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":3584,"new_prefill":83992,"score_linear":299371.0,"score_lmetric":173712},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":3584,"new_prefill":83992,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1949,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":339690.21188679,"t_first_token_unix":1779770555.22583,"t_done":339705.114772019,"t_done_unix":1779770570.1287155},{"request_id":"1253804:8:1541192:953","session_id":"1253804","input_length":89116,"t_proxy_recv":339712.829012637,"t_decision_unix":1779770577.842954,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":88576,"affinity_cache_ratio":0.9939404820683154,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (540 < 8000)","cache_hit":88576,"estimated_new_tokens":540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1911,"cache_hit":88576,"new_prefill":540,"score_linear":-88576.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":375316,"ongoing_decode_tokens":183721,"pending_prefill_tokens":8333,"num_requests":3,"active_p_offloads":0,"cached_blocks":3050,"cache_hit":3584,"new_prefill":85532,"score_linear":371732.0,"score_lmetric":281595},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75693,"ongoing_decode_tokens":75693,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":3584,"new_prefill":85532,"score_linear":72109.0,"score_lmetric":85532},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93596,"ongoing_decode_tokens":93596,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":89116,"score_linear":93596.0,"score_lmetric":89116},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":89116,"score_linear":31161.0,"score_lmetric":89116}],"chosen_score_linear":-88576.0,"chosen_score_lmetric":0,"t_first_token":339712.973751952,"t_first_token_unix":1779770577.9876957,"t_done":339713.396343553,"t_done_unix":1779770578.4102871},{"request_id":"1268861:17:1540278:951","session_id":"1268861","input_length":75693,"t_proxy_recv":339709.766560205,"t_decision_unix":1779770574.7805016,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9402190427120077,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4525 < 8000)","cache_hit":71168,"estimated_new_tokens":4525,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1911,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":375316,"ongoing_decode_tokens":183721,"pending_prefill_tokens":8333,"num_requests":3,"active_p_offloads":0,"cached_blocks":3050,"cache_hit":0,"new_prefill":75693,"score_linear":375316.0,"score_lmetric":252078},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":71168,"new_prefill":4525,"score_linear":-71168.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":75693,"score_linear":31161.0,"score_lmetric":75693}],"chosen_score_linear":-71168.0,"chosen_score_lmetric":0,"t_first_token":339711.953909803,"t_first_token_unix":1779770576.9678533,"t_done":339713.402581554,"t_done_unix":1779770578.416525},{"request_id":"1363943:33:1541065:952","session_id":"1363943","input_length":93596,"t_proxy_recv":339712.413087927,"t_decision_unix":1779770577.4270296,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9955981024830121,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (412 < 8000)","cache_hit":93184,"estimated_new_tokens":412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":8704,"new_prefill":84892,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1911,"cache_hit":0,"new_prefill":93596,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":375316,"ongoing_decode_tokens":183721,"pending_prefill_tokens":8333,"num_requests":3,"active_p_offloads":0,"cached_blocks":3050,"cache_hit":4096,"new_prefill":89500,"score_linear":371220.0,"score_lmetric":293499},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75693,"ongoing_decode_tokens":75693,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":93596,"score_linear":75693.0,"score_lmetric":93596},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":4096,"new_prefill":89500,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":4096,"new_prefill":89500,"score_linear":-4096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":93184,"new_prefill":412,"score_linear":-93184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":93596,"score_linear":31161.0,"score_lmetric":93596}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":339712.568202368,"t_first_token_unix":1779770577.5821457,"t_done":339719.056951608,"t_done_unix":1779770584.0708952},{"request_id":"1277428:9:1513231:932","session_id":"1277428","input_length":44364,"t_proxy_recv":339619.529286063,"t_decision_unix":1779770484.5432277,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":42496,"affinity_cache_ratio":0.9578937877558381,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1868 < 8000)","cache_hit":42496,"estimated_new_tokens":1868,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1865,"cache_hit":0,"new_prefill":44364,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":4608,"new_prefill":39756,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":258591,"ongoing_decode_tokens":0,"pending_prefill_tokens":8769,"num_requests":1,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":42496,"new_prefill":1868,"score_linear":216095.0,"score_lmetric":10637},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":36352,"new_prefill":8012,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1946,"cache_hit":0,"new_prefill":44364,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":44364,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":44364,"score_linear":89790.0,"score_lmetric":44364},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":44364,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":216095.0,"chosen_score_lmetric":10637,"t_first_token":339701.822884163,"t_first_token_unix":1779770566.8368278,"t_done":339720.183864826,"t_done_unix":1779770585.1978083},{"request_id":"1399948:23:1544108:954","session_id":"1399948","input_length":20031,"t_proxy_recv":339722.275630877,"t_decision_unix":1779770587.289572,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9201737307173881,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1599 < 8000)","cache_hit":18432,"estimated_new_tokens":1599,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":8767,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":376803,"ongoing_decode_tokens":211344,"pending_prefill_tokens":2165,"num_requests":3,"active_p_offloads":0,"cached_blocks":3053,"cache_hit":0,"new_prefill":20031,"score_linear":376803.0,"score_lmetric":66588},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":18432,"new_prefill":1599,"score_linear":-18432.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":20031,"score_linear":31161.0,"score_lmetric":20031}],"chosen_score_linear":-18432.0,"chosen_score_lmetric":0,"t_first_token":339722.561133363,"t_first_token_unix":1779770587.5750768,"t_done":339723.000996819,"t_done_unix":1779770588.0149398},{"request_id":"1399948:24:1546940:955","session_id":"1399948","input_length":21809,"t_proxy_recv":339731.723456602,"t_decision_unix":1779770596.737398,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9155853088174607,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1841 < 8000)","cache_hit":19968,"estimated_new_tokens":1841,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":10545,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":376803,"ongoing_decode_tokens":211344,"pending_prefill_tokens":2165,"num_requests":3,"active_p_offloads":0,"cached_blocks":3053,"cache_hit":0,"new_prefill":21809,"score_linear":376803.0,"score_lmetric":71922},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1955,"cache_hit":19968,"new_prefill":1841,"score_linear":-19968.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":21809,"score_linear":31161.0,"score_lmetric":21809}],"chosen_score_linear":-19968.0,"chosen_score_lmetric":0,"t_first_token":339732.072436571,"t_first_token_unix":1779770597.0863798,"t_done":339732.327867807,"t_done_unix":1779770597.341811},{"request_id":"1294611:15:1533841:949","session_id":"1294611","input_length":71987,"t_proxy_recv":339688.359144279,"t_decision_unix":1779770553.3730855,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":64000,"affinity_cache_ratio":0.8890494116993346,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7987 < 8000)","cache_hit":64000,"estimated_new_tokens":7987,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":71987,"score_linear":87576.0,"score_lmetric":82763},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":302955,"ongoing_decode_tokens":269889,"pending_prefill_tokens":1868,"num_requests":2,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":64000,"new_prefill":7987,"score_linear":238955.0,"score_lmetric":19710},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":59187,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":12800,"new_prefill":59187,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28085,"ongoing_decode_tokens":0,"pending_prefill_tokens":949,"num_requests":1,"active_p_offloads":0,"cached_blocks":2032,"cache_hit":0,"new_prefill":71987,"score_linear":28085.0,"score_lmetric":72936}],"chosen_score_linear":238955.0,"chosen_score_lmetric":19710,"t_first_token":339719.482029518,"t_first_token_unix":1779770584.495973,"t_done":339737.348453415,"t_done_unix":1779770602.3623974},{"request_id":"1243831:17:1453633:883","session_id":"1243831","input_length":31161,"t_proxy_recv":339690.721749023,"t_decision_unix":1779770555.7356908,"policy":"unified_v2","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9365553095215173,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1977 < 8000)","cache_hit":29184,"estimated_new_tokens":1977,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":87576,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":11264,"new_prefill":19897,"score_linear":76312.0,"score_lmetric":19897},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":374942,"ongoing_decode_tokens":269889,"pending_prefill_tokens":9855,"num_requests":3,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":0,"new_prefill":31161,"score_linear":374942.0,"score_lmetric":123048},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":29184,"new_prefill":1977,"score_linear":-29184.0,"score_lmetric":0}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":339691.111879487,"t_first_token_unix":1779770556.1258233,"t_done":339743.686338412,"t_done_unix":1779770608.7002819},{"request_id":"1399948:25:1553234:957","session_id":"1399948","input_length":22250,"t_proxy_recv":339752.260357192,"t_decision_unix":1779770617.2742987,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9664719101123596,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (746 < 8000)","cache_hit":21504,"estimated_new_tokens":746,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":10986,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":304816,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2165,"num_requests":2,"active_p_offloads":0,"cached_blocks":3069,"cache_hit":0,"new_prefill":22250,"score_linear":304816.0,"score_lmetric":48830},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76017,"ongoing_decode_tokens":76017,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":22250,"score_linear":76017.0,"score_lmetric":22250},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":21504,"new_prefill":746,"score_linear":-21504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":339752.414867907,"t_first_token_unix":1779770617.428811,"t_done":339752.972952089,"t_done_unix":1779770617.9868968},{"request_id":"1268861:18:1553083:956","session_id":"1268861","input_length":76017,"t_proxy_recv":339751.818127475,"t_decision_unix":1779770616.832069,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9968296565241985,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (241 < 8000)","cache_hit":75776,"estimated_new_tokens":241,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":304816,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2165,"num_requests":2,"active_p_offloads":0,"cached_blocks":3069,"cache_hit":0,"new_prefill":76017,"score_linear":304816.0,"score_lmetric":156364},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":75776,"new_prefill":241,"score_linear":-75776.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":339752.074016892,"t_first_token_unix":1779770617.0879602,"t_done":339753.105431253,"t_done_unix":1779770618.1193755},{"request_id":"1399948:26:1557859:959","session_id":"1399948","input_length":23042,"t_proxy_recv":339768.21711882,"t_decision_unix":1779770633.23106,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9554726152243729,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1026 < 8000)","cache_hit":22016,"estimated_new_tokens":1026,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":11778,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":304816,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2165,"num_requests":2,"active_p_offloads":0,"cached_blocks":3069,"cache_hit":0,"new_prefill":23042,"score_linear":304816.0,"score_lmetric":50414},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1959,"cache_hit":22016,"new_prefill":1026,"score_linear":-22016.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-22016.0,"chosen_score_lmetric":0,"t_first_token":339768.429859518,"t_first_token_unix":1779770633.443803,"t_done":339768.714584445,"t_done_unix":1779770633.7285278},{"request_id":"1399948:27:1559017:960","session_id":"1399948","input_length":24139,"t_proxy_recv":339771.881336407,"t_decision_unix":1779770636.8952777,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9544720162392808,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1099 < 8000)","cache_hit":23040,"estimated_new_tokens":1099,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":12875,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":304816,"ongoing_decode_tokens":139357,"pending_prefill_tokens":2165,"num_requests":2,"active_p_offloads":0,"cached_blocks":3069,"cache_hit":0,"new_prefill":24139,"score_linear":304816.0,"score_lmetric":52608},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1961,"cache_hit":23040,"new_prefill":1099,"score_linear":-23040.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":339772.149363554,"t_first_token_unix":1779770637.163307,"t_done":339773.114846846,"t_done_unix":1779770638.1287906},{"request_id":"1313181:14:1351342:491","session_id":"1313181","input_length":139357,"t_proxy_recv":339588.41589468,"t_decision_unix":1779770453.4298363,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":131584,"affinity_cache_ratio":0.9442223928471479,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7773 < 8000)","cache_hit":131584,"estimated_new_tokens":7773,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":78722,"ongoing_decode_tokens":0,"pending_prefill_tokens":2434,"num_requests":1,"active_p_offloads":0,"cached_blocks":1861,"cache_hit":0,"new_prefill":139357,"score_linear":78722.0,"score_lmetric":141791},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":3048,"cache_hit":131584,"new_prefill":7773,"score_linear":-142882.0,"score_lmetric":-7773},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":12800,"new_prefill":126557,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103893,"ongoing_decode_tokens":0,"pending_prefill_tokens":10709,"num_requests":1,"active_p_offloads":0,"cached_blocks":2383,"cache_hit":0,"new_prefill":139357,"score_linear":103893.0,"score_lmetric":150066},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89469,"ongoing_decode_tokens":0,"pending_prefill_tokens":2429,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":12800,"new_prefill":126557,"score_linear":76669.0,"score_lmetric":128986},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":65024,"new_prefill":74333,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-142882.0,"chosen_score_lmetric":-7773,"t_first_token":339635.000910157,"t_first_token_unix":1779770500.0148537,"t_done":339777.2067123,"t_done_unix":1779770642.2206562},{"request_id":"1399948:28:1560859:961","session_id":"1399948","input_length":24386,"t_proxy_recv":339778.008253608,"t_decision_unix":1779770643.022195,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9867957024522267,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (322 < 8000)","cache_hit":24064,"estimated_new_tokens":322,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":13122,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":231335,"ongoing_decode_tokens":130906,"pending_prefill_tokens":2159,"num_requests":2,"active_p_offloads":0,"cached_blocks":3085,"cache_hit":0,"new_prefill":24386,"score_linear":231335.0,"score_lmetric":53090},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1963,"cache_hit":24064,"new_prefill":322,"score_linear":-24064.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":339778.096552796,"t_first_token_unix":1779770643.1104965,"t_done":339779.3738093,"t_done_unix":1779770644.387753},{"request_id":"1356885:5:1533696:948","session_id":"1356885","input_length":130906,"t_proxy_recv":339694.564503016,"t_decision_unix":1779770559.5784445,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":130560,"affinity_cache_ratio":0.9973568820374925,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (346 < 8000)","cache_hit":130560,"estimated_new_tokens":346,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":8538,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":87576,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":130906,"score_linear":87576.0,"score_lmetric":130906},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":244410,"ongoing_decode_tokens":139357,"pending_prefill_tokens":9855,"num_requests":2,"active_p_offloads":0,"cached_blocks":3050,"cache_hit":130560,"new_prefill":346,"score_linear":113850.0,"score_lmetric":20402},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1610,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1958,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":6656,"new_prefill":124250,"score_linear":24505.0,"score_lmetric":124250}],"chosen_score_linear":113850.0,"chosen_score_lmetric":20402,"t_first_token":339776.235542529,"t_first_token_unix":1779770641.249486,"t_done":339787.84944369,"t_done_unix":1779770652.8633876},{"request_id":"1277428:10:1517408:935","session_id":"1277428","input_length":45851,"t_proxy_recv":339720.193074367,"t_decision_unix":1779770585.207016,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":44032,"affinity_cache_ratio":0.9603280190181239,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1819 < 8000)","cache_hit":44032,"estimated_new_tokens":1819,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":4608,"new_prefill":41243,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":330952,"ongoing_decode_tokens":211344,"pending_prefill_tokens":346,"num_requests":2,"active_p_offloads":0,"cached_blocks":3053,"cache_hit":44032,"new_prefill":1819,"score_linear":286920.0,"score_lmetric":4330},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":36352,"new_prefill":9499,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1952,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":45851,"score_linear":31161.0,"score_lmetric":45851}],"chosen_score_linear":286920.0,"chosen_score_lmetric":4330,"t_first_token":339784.70779099,"t_first_token_unix":1779770649.7217348,"t_done":339789.322528372,"t_done_unix":1779770654.336472},{"request_id":"1277428:11:1520449:937","session_id":"1277428","input_length":46361,"t_proxy_recv":339789.331318021,"t_decision_unix":1779770654.3452597,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":46080,"affinity_cache_ratio":0.9939388710338432,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (281 < 8000)","cache_hit":46080,"estimated_new_tokens":281,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":4608,"new_prefill":41753,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":54578,"ongoing_decode_tokens":65876,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3090,"cache_hit":46080,"new_prefill":281,"score_linear":8498.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":36352,"new_prefill":10009,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":8498.0,"chosen_score_lmetric":0,"t_first_token":339789.498465649,"t_first_token_unix":1779770654.5124087,"t_done":339790.141956657,"t_done_unix":1779770655.1559},{"request_id":"1313181:14:1408792:803","session_id":"1313181","input_length":65876,"t_proxy_recv":339777.216647418,"t_decision_unix":1779770642.2305892,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":65536,"affinity_cache_ratio":0.9948387880259882,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (340 < 8000)","cache_hit":65536,"estimated_new_tokens":340,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":165459,"ongoing_decode_tokens":130906,"pending_prefill_tokens":1819,"num_requests":1,"active_p_offloads":0,"cached_blocks":3085,"cache_hit":65536,"new_prefill":340,"score_linear":99923.0,"score_lmetric":2159},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":12800,"new_prefill":53076,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1963,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":53076,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":65024,"new_prefill":852,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":99923.0,"chosen_score_lmetric":2159,"t_first_token":339784.712297597,"t_first_token_unix":1779770649.7262409,"t_done":339790.176903593,"t_done_unix":1779770655.190847},{"request_id":"1313181:15:1411723:818","session_id":"1313181","input_length":66259,"t_proxy_recv":339790.188230179,"t_decision_unix":1779770655.2021718,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":66048,"affinity_cache_ratio":0.9968155269472826,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (211 < 8000)","cache_hit":66048,"estimated_new_tokens":211,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":-11298,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":3091,"cache_hit":66048,"new_prefill":211,"score_linear":-77346.0,"score_lmetric":-211},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":12800,"new_prefill":53459,"score_linear":-12800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":53459,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":65024,"new_prefill":1235,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-77346.0,"chosen_score_lmetric":-211,"t_first_token":339790.300804906,"t_first_token_unix":1779770655.3147488,"t_done":339793.567363027,"t_done_unix":1779770658.5813072},{"request_id":"1399948:29:1571125:963","session_id":"1399948","input_length":28781,"t_proxy_recv":339811.878556756,"t_decision_unix":1779770676.8924983,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":24576,"affinity_cache_ratio":0.8538966679406553,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4205 < 8000)","cache_hit":24576,"estimated_new_tokens":4205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":17517,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":131413,"ongoing_decode_tokens":131413,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3092,"cache_hit":0,"new_prefill":28781,"score_linear":131413.0,"score_lmetric":28781},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":24576,"new_prefill":4205,"score_linear":-24576.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":339812.76427464,"t_first_token_unix":1779770677.778219,"t_done":339813.030793336,"t_done_unix":1779770678.0447378},{"request_id":"1356885:6:1570780:962","session_id":"1356885","input_length":131413,"t_proxy_recv":339810.861859945,"t_decision_unix":1779770675.8758013,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":131072,"affinity_cache_ratio":0.9974051273466096,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (341 < 8000)","cache_hit":131072,"estimated_new_tokens":341,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":9045,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3092,"cache_hit":131072,"new_prefill":341,"score_linear":-131072.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1964,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":124757,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-131072.0,"chosen_score_lmetric":0,"t_first_token":339811.490160833,"t_first_token_unix":1779770676.5041046,"t_done":339813.681313448,"t_done_unix":1779770678.6952577},{"request_id":"1270606:4:1578042:965","session_id":"1270606","input_length":109083,"t_proxy_recv":339835.47937138,"t_decision_unix":1779770700.4933126,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":108032,"affinity_cache_ratio":0.9903651348056067,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1051 < 8000)","cache_hit":108032,"estimated_new_tokens":1051,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1972,"cache_hit":108032,"new_prefill":1051,"score_linear":-108032.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":107547,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":339836.226847845,"t_first_token_unix":1779770701.2407918,"t_done":339836.653476085,"t_done_unix":1779770701.6674194},{"request_id":"1268861:19:1578134:966","session_id":"1268861","input_length":76963,"t_proxy_recv":339835.725980097,"t_decision_unix":1779770700.7399213,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9845770045346465,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1187 < 8000)","cache_hit":75776,"estimated_new_tokens":1187,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1619,"cache_hit":75776,"new_prefill":1187,"score_linear":-75776.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":1972,"cache_hit":0,"new_prefill":76963,"score_linear":109083.0,"score_lmetric":78014},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":339836.285638673,"t_first_token_unix":1779770701.2995822,"t_done":339837.044799771,"t_done_unix":1779770702.058743},{"request_id":"1399948:30:1578524:967","session_id":"1399948","input_length":30773,"t_proxy_recv":339837.065712205,"t_decision_unix":1779770702.079654,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9317258635817113,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2101 < 8000)","cache_hit":28672,"estimated_new_tokens":2101,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":19509,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1974,"cache_hit":28672,"new_prefill":2101,"score_linear":-28672.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-28672.0,"chosen_score_lmetric":0,"t_first_token":339837.573442088,"t_first_token_unix":1779770702.5873857,"t_done":339837.875390105,"t_done_unix":1779770702.8893332},{"request_id":"1399948:31:1579418:968","session_id":"1399948","input_length":30925,"t_proxy_recv":339840.291642137,"t_decision_unix":1779770705.3055837,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9933710590137429,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (205 < 8000)","cache_hit":30720,"estimated_new_tokens":205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":19661,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1978,"cache_hit":30720,"new_prefill":205,"score_linear":-30720.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":339840.390647503,"t_first_token_unix":1779770705.404591,"t_done":339841.733215722,"t_done_unix":1779770706.747159},{"request_id":"1356885:7:1580925:969","session_id":"1356885","input_length":131775,"t_proxy_recv":339845.603760774,"t_decision_unix":1779770710.6177022,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":131584,"affinity_cache_ratio":0.9985505596660975,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (191 < 8000)","cache_hit":131584,"estimated_new_tokens":191,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":9407,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":131584,"new_prefill":191,"score_linear":-131584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1978,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":125119,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-131584.0,"chosen_score_lmetric":0,"t_first_token":339846.003926995,"t_first_token_unix":1779770711.0178704,"t_done":339848.020413484,"t_done_unix":1779770713.0343602},{"request_id":"1399948:32:1581424:970","session_id":"1399948","input_length":31231,"t_proxy_recv":339847.054184727,"t_decision_unix":1779770712.0681262,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9836380519355769,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (511 < 8000)","cache_hit":30720,"estimated_new_tokens":511,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":19967,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":131775,"ongoing_decode_tokens":131775,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":0,"new_prefill":31231,"score_linear":131775.0,"score_lmetric":31231},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1978,"cache_hit":30720,"new_prefill":511,"score_linear":-30720.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":339847.161287498,"t_first_token_unix":1779770712.1752307,"t_done":339850.746910109,"t_done_unix":1779770715.7608535},{"request_id":"1270606:5:1581455:971","session_id":"1270606","input_length":111387,"t_proxy_recv":339847.206516335,"t_decision_unix":1779770712.2204578,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":109056,"affinity_cache_ratio":0.9790729618357618,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2331 < 8000)","cache_hit":109056,"estimated_new_tokens":2331,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":131775,"ongoing_decode_tokens":131775,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":0,"new_prefill":111387,"score_linear":131775.0,"score_lmetric":111387},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":31231,"ongoing_decode_tokens":31231,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1978,"cache_hit":109056,"new_prefill":2331,"score_linear":-77825.0,"score_lmetric":2331},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":109851,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-77825.0,"chosen_score_lmetric":2331,"t_first_token":339848.976712192,"t_first_token_unix":1779770713.9906557,"t_done":339852.12854272,"t_done_unix":1779770717.142487},{"request_id":"1399948:33:1583165:972","session_id":"1399948","input_length":31445,"t_proxy_recv":339852.95002483,"t_decision_unix":1779770717.9639657,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9932262680871363,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (213 < 8000)","cache_hit":31232,"estimated_new_tokens":213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":20181,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1984,"cache_hit":31232,"new_prefill":213,"score_linear":-31232.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":339853.045963743,"t_first_token_unix":1779770718.0599074,"t_done":339853.705903146,"t_done_unix":1779770718.7198465},{"request_id":"1277428:12:1586139:973","session_id":"1277428","input_length":47660,"t_proxy_recv":339863.092254069,"t_decision_unix":1779770728.1061952,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":46080,"affinity_cache_ratio":0.9668485102811583,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1580 < 8000)","cache_hit":46080,"estimated_new_tokens":1580,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":4608,"new_prefill":43052,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":46080,"new_prefill":1580,"score_linear":-46080.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":36352,"new_prefill":11308,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1984,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-46080.0,"chosen_score_lmetric":0,"t_first_token":339863.554796475,"t_first_token_unix":1779770728.56874,"t_done":339864.167627145,"t_done_unix":1779770729.1815712},{"request_id":"1399948:34:1586232:974","session_id":"1399948","input_length":32143,"t_proxy_recv":339863.44136646,"t_decision_unix":1779770728.4553077,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":31232,"affinity_cache_ratio":0.97165790374265,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (911 < 8000)","cache_hit":31232,"estimated_new_tokens":911,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":20879,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":47660,"ongoing_decode_tokens":0,"pending_prefill_tokens":1580,"num_requests":1,"active_p_offloads":0,"cached_blocks":3093,"cache_hit":0,"new_prefill":32143,"score_linear":47660.0,"score_lmetric":33723},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1984,"cache_hit":31232,"new_prefill":911,"score_linear":-31232.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":339863.648668518,"t_first_token_unix":1779770728.6626117,"t_done":339864.552928792,"t_done_unix":1779770729.5668724},{"request_id":"1268861:20:1595141:978","session_id":"1268861","input_length":77218,"t_proxy_recv":339893.976224582,"t_decision_unix":1779770758.990166,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":76800,"affinity_cache_ratio":0.9945867543836929,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (418 < 8000)","cache_hit":76800,"estimated_new_tokens":418,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124565,"ongoing_decode_tokens":0,"pending_prefill_tokens":4757,"num_requests":2,"active_p_offloads":0,"cached_blocks":3096,"cache_hit":0,"new_prefill":77218,"score_linear":124565.0,"score_lmetric":163950},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":76800,"new_prefill":418,"score_linear":-76800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":339894.245497225,"t_first_token_unix":1779770759.2594411,"t_done":339894.7668543,"t_done_unix":1779770759.7807977},{"request_id":"1277428:13:1593499:976","session_id":"1277428","input_length":49316,"t_proxy_recv":339888.270669852,"t_decision_unix":1779770753.2846112,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":47616,"affinity_cache_ratio":0.9655284289074539,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1700 < 8000)","cache_hit":47616,"estimated_new_tokens":1700,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":4608,"new_prefill":44708,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":75249,"ongoing_decode_tokens":0,"pending_prefill_tokens":3057,"num_requests":1,"active_p_offloads":0,"cached_blocks":3096,"cache_hit":47616,"new_prefill":1700,"score_linear":27633.0,"score_lmetric":4757},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":36352,"new_prefill":12964,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":27633.0,"chosen_score_lmetric":4757,"t_first_token":339896.575513309,"t_first_token_unix":1779770761.5894566,"t_done":339899.081539143,"t_done_unix":1779770764.0954826},{"request_id":"1294611:15:1590202:975","session_id":"1294611","input_length":75249,"t_proxy_recv":339876.640273274,"t_decision_unix":1779770741.6542144,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":72192,"affinity_cache_ratio":0.9593748754136268,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3057 < 8000)","cache_hit":72192,"estimated_new_tokens":3057,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3096,"cache_hit":72192,"new_prefill":3057,"score_linear":-72192.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":62449,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":62449,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72192.0,"chosen_score_lmetric":0,"t_first_token":339896.57340231,"t_first_token_unix":1779770761.5873463,"t_done":339900.000219213,"t_done_unix":1779770765.014163},{"request_id":"1294611:15:1594038:977","session_id":"1294611","input_length":75584,"t_proxy_recv":339900.015963961,"t_decision_unix":1779770765.0299056,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":75264,"affinity_cache_ratio":0.995766299745978,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (320 < 8000)","cache_hit":75264,"estimated_new_tokens":320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3105,"cache_hit":75264,"new_prefill":320,"score_linear":-75264.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":62784,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":62784,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":339900.136776849,"t_first_token_unix":1779770765.15072,"t_done":339901.25011777,"t_done_unix":1779770766.2640617},{"request_id":"1399948:35:1602048:980","session_id":"1399948","input_length":33533,"t_proxy_recv":339917.344691809,"t_decision_unix":1779770782.3586328,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":32256,"affinity_cache_ratio":0.9619181105179972,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1277 < 8000)","cache_hit":32256,"estimated_new_tokens":1277,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":22269,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":75716,"ongoing_decode_tokens":75716,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3105,"cache_hit":0,"new_prefill":33533,"score_linear":75716.0,"score_lmetric":33533},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":32256,"new_prefill":1277,"score_linear":-32256.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-32256.0,"chosen_score_lmetric":0,"t_first_token":339917.75201458,"t_first_token_unix":1779770782.765958,"t_done":339918.774320496,"t_done_unix":1779770783.788264},{"request_id":"1253804:9:1605110:981","session_id":"1253804","input_length":90656,"t_proxy_recv":339927.334138533,"t_decision_unix":1779770792.34808,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":89088,"affinity_cache_ratio":0.9827038475114719,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1568 < 8000)","cache_hit":89088,"estimated_new_tokens":1568,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":89088,"new_prefill":1568,"score_linear":-89088.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":75716,"ongoing_decode_tokens":75716,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3105,"cache_hit":3584,"new_prefill":87072,"score_linear":72132.0,"score_lmetric":87072},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":3584,"new_prefill":87072,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-89088.0,"chosen_score_lmetric":0,"t_first_token":339928.303531308,"t_first_token_unix":1779770793.317475,"t_done":339931.942025605,"t_done_unix":1779770796.955969},{"request_id":"1294611:15:1596163:979","session_id":"1294611","input_length":75716,"t_proxy_recv":339901.267854838,"t_decision_unix":1779770766.2817965,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9940303238417243,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (452 < 8000)","cache_hit":75264,"estimated_new_tokens":452,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1912,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3105,"cache_hit":75264,"new_prefill":452,"score_linear":-75264.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":62916,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":62916,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":339901.381098635,"t_first_token_unix":1779770766.3950424,"t_done":339939.805574708,"t_done_unix":1779770804.8195186},{"request_id":"1268861:21:1609831:982","session_id":"1268861","input_length":77293,"t_proxy_recv":339943.077936165,"t_decision_unix":1779770808.0918777,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":76800,"affinity_cache_ratio":0.9936216733727504,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (493 < 8000)","cache_hit":76800,"estimated_new_tokens":493,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1915,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":76800,"new_prefill":493,"score_linear":-76800.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":339943.194727599,"t_first_token_unix":1779770808.2086706,"t_done":339944.79339176,"t_done_unix":1779770809.8073354},{"request_id":"1253804:10:1611505:983","session_id":"1253804","input_length":91103,"t_proxy_recv":339948.558092085,"t_decision_unix":1779770813.5720332,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":90624,"affinity_cache_ratio":0.9947422148557128,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (479 < 8000)","cache_hit":90624,"estimated_new_tokens":479,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1915,"cache_hit":90624,"new_prefill":479,"score_linear":-90624.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":87519,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":3584,"new_prefill":87519,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-90624.0,"chosen_score_lmetric":0,"t_first_token":339948.73210708,"t_first_token_unix":1779770813.7460504,"t_done":339948.865272796,"t_done_unix":1779770813.8792164},{"request_id":"1399948:36:1612467:984","session_id":"1399948","input_length":34773,"t_proxy_recv":339951.664015048,"t_decision_unix":1779770816.6779563,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":33280,"affinity_cache_ratio":0.9570643890374716,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1493 < 8000)","cache_hit":33280,"estimated_new_tokens":1493,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":23509,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1915,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":33280,"new_prefill":1493,"score_linear":-33280.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-33280.0,"chosen_score_lmetric":0,"t_first_token":339952.021438401,"t_first_token_unix":1779770817.035382,"t_done":339952.570192828,"t_done_unix":1779770817.5841367},{"request_id":"1253804:11:1613030:985","session_id":"1253804","input_length":91131,"t_proxy_recv":339953.893802704,"t_decision_unix":1779770818.907744,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":90624,"affinity_cache_ratio":0.9944365803074695,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (507 < 8000)","cache_hit":90624,"estimated_new_tokens":507,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1915,"cache_hit":90624,"new_prefill":507,"score_linear":-90624.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":87547,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":3584,"new_prefill":87547,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1991,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-90624.0,"chosen_score_lmetric":0,"t_first_token":339954.013148975,"t_first_token_unix":1779770819.0270925,"t_done":339956.105340127,"t_done_unix":1779770821.1192832},{"request_id":"1399948:37:1613628:986","session_id":"1399948","input_length":35308,"t_proxy_recv":339955.745912578,"t_decision_unix":1779770820.7598536,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9860654809108418,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (492 < 8000)","cache_hit":34816,"estimated_new_tokens":492,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":24044,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":91131,"ongoing_decode_tokens":91131,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1915,"cache_hit":0,"new_prefill":35308,"score_linear":91131.0,"score_lmetric":35308},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1991,"cache_hit":34816,"new_prefill":492,"score_linear":-34816.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":339955.958064186,"t_first_token_unix":1779770820.9720073,"t_done":339957.10529753,"t_done_unix":1779770822.1192417},{"request_id":"1399948:38:1616890:987","session_id":"1399948","input_length":35581,"t_proxy_recv":339966.924474104,"t_decision_unix":1779770831.9384155,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9928894634776988,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (253 < 8000)","cache_hit":35328,"estimated_new_tokens":253,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":24317,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1992,"cache_hit":35328,"new_prefill":253,"score_linear":-35328.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":339967.038769026,"t_first_token_unix":1779770832.0527124,"t_done":339967.557221816,"t_done_unix":1779770832.571165},{"request_id":"1399948:39:1618250:989","session_id":"1399948","input_length":35682,"t_proxy_recv":339971.617016735,"t_decision_unix":1779770836.630958,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9900790314444258,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (354 < 8000)","cache_hit":35328,"estimated_new_tokens":354,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":24418,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1992,"cache_hit":35328,"new_prefill":354,"score_linear":-35328.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":339971.691817795,"t_first_token_unix":1779770836.7057612,"t_done":339972.136893036,"t_done_unix":1779770837.1508365},{"request_id":"1399948:40:1620970:991","session_id":"1399948","input_length":35958,"t_proxy_recv":339980.604540547,"t_decision_unix":1779770845.6184819,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9824795594860671,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (630 < 8000)","cache_hit":35328,"estimated_new_tokens":630,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":24694,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":78439,"ongoing_decode_tokens":0,"pending_prefill_tokens":1127,"num_requests":1,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":0,"new_prefill":35958,"score_linear":78439.0,"score_lmetric":37085},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1992,"cache_hit":35328,"new_prefill":630,"score_linear":-35328.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":339980.743309439,"t_first_token_unix":1779770845.757253,"t_done":339981.110551859,"t_done_unix":1779770846.124495},{"request_id":"1268861:22:1620830:990","session_id":"1268861","input_length":78439,"t_proxy_recv":339980.139181743,"t_decision_unix":1779770845.153123,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":77312,"affinity_cache_ratio":0.9856321472736777,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1127 < 8000)","cache_hit":77312,"estimated_new_tokens":1127,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":77312,"new_prefill":1127,"score_linear":-77312.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1992,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-77312.0,"chosen_score_lmetric":0,"t_first_token":339980.8341905,"t_first_token_unix":1779770845.8481338,"t_done":339982.10645125,"t_done_unix":1779770847.120395},{"request_id":"1253804:12:1621957:992","session_id":"1253804","input_length":91389,"t_proxy_recv":339984.054242586,"t_decision_unix":1779770849.068184,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":91136,"affinity_cache_ratio":0.9972316143080677,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (253 < 8000)","cache_hit":91136,"estimated_new_tokens":253,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":91136,"new_prefill":253,"score_linear":-91136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":87805,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":3584,"new_prefill":87805,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-91136.0,"chosen_score_lmetric":0,"t_first_token":339984.238599247,"t_first_token_unix":1779770849.2525425,"t_done":339984.530708578,"t_done_unix":1779770849.5446522},{"request_id":"1399948:41:1624143:993","session_id":"1399948","input_length":36168,"t_proxy_recv":339991.415914571,"t_decision_unix":1779770856.4298558,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":35840,"affinity_cache_ratio":0.990931209909312,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (328 < 8000)","cache_hit":35840,"estimated_new_tokens":328,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":24904,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":35840,"new_prefill":328,"score_linear":-35840.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35840.0,"chosen_score_lmetric":0,"t_first_token":339991.550502953,"t_first_token_unix":1779770856.5644467,"t_done":339992.224374776,"t_done_unix":1779770857.238318},{"request_id":"1253804:13:1624613:994","session_id":"1253804","input_length":91440,"t_proxy_recv":339992.989975053,"t_decision_unix":1779770858.003916,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":91136,"affinity_cache_ratio":0.9966754155730534,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (304 < 8000)","cache_hit":91136,"estimated_new_tokens":304,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":91136,"new_prefill":304,"score_linear":-91136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":87856,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":3584,"new_prefill":87856,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-91136.0,"chosen_score_lmetric":0,"t_first_token":339993.118114511,"t_first_token_unix":1779770858.1320581,"t_done":339994.125144347,"t_done_unix":1779770859.1390886},{"request_id":"1253804:14:1626575:995","session_id":"1253804","input_length":92367,"t_proxy_recv":339999.507131979,"t_decision_unix":1779770864.5210733,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":91136,"affinity_cache_ratio":0.9866727294380028,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1231 < 8000)","cache_hit":91136,"estimated_new_tokens":1231,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":91136,"new_prefill":1231,"score_linear":-91136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":88783,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":3584,"new_prefill":88783,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-91136.0,"chosen_score_lmetric":0,"t_first_token":340000.144010754,"t_first_token_unix":1779770865.157955,"t_done":340000.956828989,"t_done_unix":1779770865.9707723},{"request_id":"1253804:15:1628485:997","session_id":"1253804","input_length":92701,"t_proxy_recv":340005.832628045,"t_decision_unix":1779770870.8465695,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":92160,"affinity_cache_ratio":0.9941640327504557,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (541 < 8000)","cache_hit":92160,"estimated_new_tokens":541,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":92160,"new_prefill":541,"score_linear":-92160.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":89117,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":3584,"new_prefill":89117,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":36984,"ongoing_decode_tokens":0,"pending_prefill_tokens":1144,"num_requests":1,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":0,"new_prefill":92701,"score_linear":36984.0,"score_lmetric":93845},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-92160.0,"chosen_score_lmetric":0,"t_first_token":340006.124311273,"t_first_token_unix":1779770871.1382546,"t_done":340006.50802085,"t_done_unix":1779770871.521965},{"request_id":"1399948:42:1628429:996","session_id":"1399948","input_length":36984,"t_proxy_recv":340005.609453183,"t_decision_unix":1779770870.6233945,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":35840,"affinity_cache_ratio":0.9690677049534934,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1144 < 8000)","cache_hit":35840,"estimated_new_tokens":1144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":25720,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1918,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":35840,"new_prefill":1144,"score_linear":-35840.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-35840.0,"chosen_score_lmetric":0,"t_first_token":340005.90526838,"t_first_token_unix":1779770870.9192119,"t_done":340007.122976362,"t_done_unix":1779770872.13692},{"request_id":"1253804:16:1629713:999","session_id":"1253804","input_length":92953,"t_proxy_recv":340009.880878306,"t_decision_unix":1779770874.8948193,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":92672,"affinity_cache_ratio":0.9969769668542167,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (281 < 8000)","cache_hit":92672,"estimated_new_tokens":281,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1919,"cache_hit":92672,"new_prefill":281,"score_linear":-92672.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79074,"ongoing_decode_tokens":0,"pending_prefill_tokens":3298,"num_requests":1,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":89369,"score_linear":75490.0,"score_lmetric":92667},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":3584,"new_prefill":89369,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-92672.0,"chosen_score_lmetric":0,"t_first_token":340010.170784299,"t_first_token_unix":1779770875.1847281,"t_done":340011.157134318,"t_done_unix":1779770876.1710782},{"request_id":"1253804:17:1631602:1002","session_id":"1253804","input_length":93219,"t_proxy_recv":340015.95221379,"t_decision_unix":1779770880.9661555,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":92672,"affinity_cache_ratio":0.9941320975337645,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (547 < 8000)","cache_hit":92672,"estimated_new_tokens":547,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1919,"cache_hit":92672,"new_prefill":547,"score_linear":-92672.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":158964,"ongoing_decode_tokens":0,"pending_prefill_tokens":3828,"num_requests":2,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":89635,"score_linear":155380.0,"score_lmetric":186926},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":79693,"ongoing_decode_tokens":0,"pending_prefill_tokens":1357,"num_requests":1,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":3584,"new_prefill":89635,"score_linear":76109.0,"score_lmetric":90992},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-92672.0,"chosen_score_lmetric":0,"t_first_token":340016.262560559,"t_first_token_unix":1779770881.276504,"t_done":340017.012519179,"t_done_unix":1779770882.0264626},{"request_id":"1268861:23:1631587:1001","session_id":"1268861","input_length":79693,"t_proxy_recv":340015.877826731,"t_decision_unix":1779770880.891768,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9829721556472965,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1357 < 8000)","cache_hit":78336,"estimated_new_tokens":1357,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1919,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":158964,"ongoing_decode_tokens":0,"pending_prefill_tokens":3828,"num_requests":2,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":0,"new_prefill":79693,"score_linear":158964.0,"score_lmetric":167042},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":78336,"new_prefill":1357,"score_linear":-78336.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":340016.588528728,"t_first_token_unix":1779770881.6024723,"t_done":340017.228075388,"t_done_unix":1779770882.2420187},{"request_id":"1253804:18:1633257:1003","session_id":"1253804","input_length":93509,"t_proxy_recv":340021.403738484,"t_decision_unix":1779770886.4176798,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9965243987209788,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (325 < 8000)","cache_hit":93184,"estimated_new_tokens":325,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1920,"cache_hit":93184,"new_prefill":325,"score_linear":-93184.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":158964,"ongoing_decode_tokens":0,"pending_prefill_tokens":3828,"num_requests":2,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":89925,"score_linear":155380.0,"score_lmetric":187506},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":3584,"new_prefill":89925,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":340021.6919877,"t_first_token_unix":1779770886.7059312,"t_done":340022.587471289,"t_done_unix":1779770887.601415},{"request_id":"1388507:3:1629072:998","session_id":"1388507","input_length":79074,"t_proxy_recv":340007.917639861,"t_decision_unix":1779770872.9315808,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9582922325922554,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3298 < 8000)","cache_hit":75776,"estimated_new_tokens":3298,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1919,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":75776,"new_prefill":3298,"score_linear":-75776.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":340029.23122389,"t_first_token_unix":1779770894.2451673,"t_done":340030.100446287,"t_done_unix":1779770895.1143901},{"request_id":"1253804:19:1634903:1005","session_id":"1253804","input_length":93664,"t_proxy_recv":340026.953429778,"t_decision_unix":1779770891.967371,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9948752989408951,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (480 < 8000)","cache_hit":93184,"estimated_new_tokens":480,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1920,"cache_hit":93184,"new_prefill":480,"score_linear":-93184.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":158964,"ongoing_decode_tokens":0,"pending_prefill_tokens":3828,"num_requests":2,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":3584,"new_prefill":90080,"score_linear":155380.0,"score_lmetric":187816},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":3584,"new_prefill":90080,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":340027.134714755,"t_first_token_unix":1779770892.148658,"t_done":340031.168064956,"t_done_unix":1779770896.1820095},{"request_id":"1294611:15:1631116:1000","session_id":"1294611","input_length":79890,"t_proxy_recv":340014.465303648,"t_decision_unix":1779770879.4792447,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":79360,"affinity_cache_ratio":0.9933658780823632,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (530 < 8000)","cache_hit":79360,"estimated_new_tokens":530,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1919,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79074,"ongoing_decode_tokens":0,"pending_prefill_tokens":3298,"num_requests":1,"active_p_offloads":0,"cached_blocks":3107,"cache_hit":79360,"new_prefill":530,"score_linear":-286.0,"score_lmetric":3828},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1624,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":67090,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":67090,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-286.0,"chosen_score_lmetric":3828,"t_first_token":340029.238622112,"t_first_token_unix":1779770894.2525659,"t_done":340032.296876089,"t_done_unix":1779770897.3108213},{"request_id":"1294611:15:1634779:1004","session_id":"1294611","input_length":80221,"t_proxy_recv":340032.312580222,"t_decision_unix":1779770897.3265219,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":79872,"affinity_cache_ratio":0.995649518205956,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (349 < 8000)","cache_hit":79872,"estimated_new_tokens":349,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1921,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3114,"cache_hit":79872,"new_prefill":349,"score_linear":-79872.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":67421,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":67421,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":340032.44045855,"t_first_token_unix":1779770897.454402,"t_done":340033.001296235,"t_done_unix":1779770898.0152397},{"request_id":"1253804:20:1638889:1007","session_id":"1253804","input_length":94111,"t_proxy_recv":340040.125047188,"t_decision_unix":1779770905.1389883,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93696,"affinity_cache_ratio":0.9955903135658956,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (415 < 8000)","cache_hit":93696,"estimated_new_tokens":415,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1921,"cache_hit":93696,"new_prefill":415,"score_linear":-93696.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3114,"cache_hit":3584,"new_prefill":90527,"score_linear":77133.0,"score_lmetric":90527},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":3584,"new_prefill":90527,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93696.0,"chosen_score_lmetric":0,"t_first_token":340040.318592673,"t_first_token_unix":1779770905.3325357,"t_done":340041.143547397,"t_done_unix":1779770906.1574914},{"request_id":"1258908:2:1639411:1008","session_id":"1258908","input_length":9804,"t_proxy_recv":340041.925900229,"t_decision_unix":1779770906.9398413,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.9400244798041616,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (588 < 8000)","cache_hit":9216,"estimated_new_tokens":588,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1921,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3114,"cache_hit":0,"new_prefill":9804,"score_linear":80717.0,"score_lmetric":9804},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":9216,"new_prefill":588,"score_linear":-9216.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":340042.683180136,"t_first_token_unix":1779770907.6971235,"t_done":340044.260573164,"t_done_unix":1779770909.2745166},{"request_id":"1268861:24:1643502:1011","session_id":"1268861","input_length":80019,"t_proxy_recv":340055.755315866,"t_decision_unix":1779770920.7692573,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":79360,"affinity_cache_ratio":0.9917644559417138,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (659 < 8000)","cache_hit":79360,"estimated_new_tokens":659,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":94269,"ongoing_decode_tokens":94269,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1921,"cache_hit":0,"new_prefill":80019,"score_linear":94269.0,"score_lmetric":80019},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3114,"cache_hit":0,"new_prefill":80019,"score_linear":80717.0,"score_lmetric":80019},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1627,"cache_hit":79360,"new_prefill":659,"score_linear":-79360.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":38102,"ongoing_decode_tokens":0,"pending_prefill_tokens":1238,"num_requests":1,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":80019,"score_linear":38102.0,"score_lmetric":81257},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-79360.0,"chosen_score_lmetric":0,"t_first_token":340056.035025585,"t_first_token_unix":1779770921.0489697,"t_done":340056.494124391,"t_done_unix":1779770921.5080683},{"request_id":"1399948:43:1643413:1010","session_id":"1399948","input_length":38102,"t_proxy_recv":340055.403483047,"t_decision_unix":1779770920.417424,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9675082672825573,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1238 < 8000)","cache_hit":36864,"estimated_new_tokens":1238,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":26838,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":94269,"ongoing_decode_tokens":94269,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1921,"cache_hit":0,"new_prefill":38102,"score_linear":94269.0,"score_lmetric":38102},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3114,"cache_hit":0,"new_prefill":38102,"score_linear":80717.0,"score_lmetric":38102},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1627,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":36864,"new_prefill":1238,"score_linear":-36864.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-36864.0,"chosen_score_lmetric":0,"t_first_token":340055.78189338,"t_first_token_unix":1779770920.7958364,"t_done":340056.576997281,"t_done_unix":1779770921.5909407},{"request_id":"1294611:15:1636645:1006","session_id":"1294611","input_length":80717,"t_proxy_recv":340033.020531947,"t_decision_unix":1779770898.0344737,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":79872,"affinity_cache_ratio":0.9895313254952488,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (845 < 8000)","cache_hit":79872,"estimated_new_tokens":845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1921,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3114,"cache_hit":79872,"new_prefill":845,"score_linear":-79872.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":67917,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":67917,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":340033.427781914,"t_first_token_unix":1779770898.4417257,"t_done":340056.634790349,"t_done_unix":1779770921.648734},{"request_id":"1253804:21:1640487:1009","session_id":"1253804","input_length":94269,"t_proxy_recv":340045.41282903,"t_decision_unix":1779770910.4267704,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93696,"affinity_cache_ratio":0.9939216497470006,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (573 < 8000)","cache_hit":93696,"estimated_new_tokens":573,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1921,"cache_hit":93696,"new_prefill":573,"score_linear":-93696.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3114,"cache_hit":3584,"new_prefill":90685,"score_linear":77133.0,"score_lmetric":90685},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1627,"cache_hit":3584,"new_prefill":90685,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93696.0,"chosen_score_lmetric":0,"t_first_token":340045.637066818,"t_first_token_unix":1779770910.6510105,"t_done":340060.098398989,"t_done_unix":1779770925.1123426},{"request_id":"1399948:44:1644970:1012","session_id":"1399948","input_length":38308,"t_proxy_recv":340060.527966427,"t_decision_unix":1779770925.5419078,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9890362326407017,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (420 < 8000)","cache_hit":37888,"estimated_new_tokens":420,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":27044,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1923,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1628,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1997,"cache_hit":37888,"new_prefill":420,"score_linear":-37888.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-37888.0,"chosen_score_lmetric":0,"t_first_token":340060.650871641,"t_first_token_unix":1779770925.6648152,"t_done":340062.174193866,"t_done_unix":1779770927.1881373},{"request_id":"1253804:22:1650001:1013","session_id":"1253804","input_length":95715,"t_proxy_recv":340077.121119584,"t_decision_unix":1779770942.1350608,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":95232,"affinity_cache_ratio":0.9949537690017238,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (483 < 8000)","cache_hit":95232,"estimated_new_tokens":483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1923,"cache_hit":95232,"new_prefill":483,"score_linear":-95232.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":3584,"new_prefill":92131,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1628,"cache_hit":3584,"new_prefill":92131,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-95232.0,"chosen_score_lmetric":0,"t_first_token":340077.313547471,"t_first_token_unix":1779770942.3274906,"t_done":340077.8119167,"t_done_unix":1779770942.82586},{"request_id":"1399948:45:1650679:1014","session_id":"1399948","input_length":38758,"t_proxy_recv":340079.569322186,"t_decision_unix":1779770944.5832636,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9907631972754012,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (358 < 8000)","cache_hit":38400,"estimated_new_tokens":358,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":27494,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1628,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":38400,"new_prefill":358,"score_linear":-38400.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":340079.722634245,"t_first_token_unix":1779770944.7365775,"t_done":340080.315137614,"t_done_unix":1779770945.3290808},{"request_id":"1253804:23:1651448:1015","session_id":"1253804","input_length":96944,"t_proxy_recv":340081.86772995,"t_decision_unix":1779770946.8816714,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9876217197557353,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1200 < 8000)","cache_hit":95744,"estimated_new_tokens":1200,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1924,"cache_hit":95744,"new_prefill":1200,"score_linear":-95744.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":3584,"new_prefill":93360,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1628,"cache_hit":3584,"new_prefill":93360,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-95744.0,"chosen_score_lmetric":0,"t_first_token":340082.71088763,"t_first_token_unix":1779770947.7248316,"t_done":340083.123636358,"t_done_unix":1779770948.1375804},{"request_id":"1399948:46:1658183:1018","session_id":"1399948","input_length":39837,"t_proxy_recv":340104.704452267,"t_decision_unix":1779770969.7183936,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9639280066270051,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1437 < 8000)","cache_hit":38400,"estimated_new_tokens":1437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":28573,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":0,"new_prefill":39837,"score_linear":83204.0,"score_lmetric":39837},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":80406,"ongoing_decode_tokens":80406,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1628,"cache_hit":0,"new_prefill":39837,"score_linear":80406.0,"score_lmetric":39837},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":38400,"new_prefill":1437,"score_linear":-38400.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":340105.104130719,"t_first_token_unix":1779770970.1180751,"t_done":340105.955808599,"t_done_unix":1779770970.9697518},{"request_id":"1253804:24:1659020:1019","session_id":"1253804","input_length":97769,"t_proxy_recv":340107.600853889,"t_decision_unix":1779770972.6147947,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":96768,"affinity_cache_ratio":0.9897615808691916,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1001 < 8000)","cache_hit":96768,"estimated_new_tokens":1001,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":96768,"new_prefill":1001,"score_linear":-96768.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":3584,"new_prefill":94185,"score_linear":79620.0,"score_lmetric":94185},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":80406,"ongoing_decode_tokens":80406,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1628,"cache_hit":3584,"new_prefill":94185,"score_linear":76822.0,"score_lmetric":94185},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-96768.0,"chosen_score_lmetric":0,"t_first_token":340108.273726179,"t_first_token_unix":1779770973.2876704,"t_done":340108.686299886,"t_done_unix":1779770973.7002437},{"request_id":"1268861:25:1653400:1016","session_id":"1268861","input_length":80406,"t_proxy_recv":340088.303620426,"t_decision_unix":1779770953.3175616,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":79872,"affinity_cache_ratio":0.9933587045742855,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (534 < 8000)","cache_hit":79872,"estimated_new_tokens":534,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1628,"cache_hit":79872,"new_prefill":534,"score_linear":-79872.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":340088.716205635,"t_first_token_unix":1779770953.7301493,"t_done":340113.096366708,"t_done_unix":1779770978.11031},{"request_id":"1253804:25:1662273:1020","session_id":"1253804","input_length":98187,"t_proxy_recv":340118.463028313,"t_decision_unix":1779770983.4769697,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9959770641734649,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (395 < 8000)","cache_hit":97792,"estimated_new_tokens":395,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1928,"cache_hit":97792,"new_prefill":395,"score_linear":-97792.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":3584,"new_prefill":94603,"score_linear":79620.0,"score_lmetric":94603},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":94603,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-97792.0,"chosen_score_lmetric":0,"t_first_token":340118.945917839,"t_first_token_unix":1779770983.9598615,"t_done":340119.357350281,"t_done_unix":1779770984.371294},{"request_id":"1294611:15:1657992:1017","session_id":"1294611","input_length":83204,"t_proxy_recv":340104.226857851,"t_decision_unix":1779770969.2407992,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":82944,"affinity_cache_ratio":0.9968751502331619,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (260 < 8000)","cache_hit":82944,"estimated_new_tokens":260,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":82944,"new_prefill":260,"score_linear":-82944.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":80406,"ongoing_decode_tokens":80406,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1628,"cache_hit":0,"new_prefill":83204,"score_linear":80406.0,"score_lmetric":83204},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":70404,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":70404,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-82944.0,"chosen_score_lmetric":0,"t_first_token":340104.416694425,"t_first_token_unix":1779770969.4306383,"t_done":340126.48037722,"t_done_unix":1779770991.4943209},{"request_id":"1399948:47:1664702:1022","session_id":"1399948","input_length":40261,"t_proxy_recv":340127.065528407,"t_decision_unix":1779770992.07947,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9919276719405876,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (325 < 8000)","cache_hit":39936,"estimated_new_tokens":325,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":28997,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1928,"cache_hit":0,"new_prefill":40261,"score_linear":98254.0,"score_lmetric":40261},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3119,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":39936,"new_prefill":325,"score_linear":-39936.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":340127.295930174,"t_first_token_unix":1779770992.309874,"t_done":340127.715718611,"t_done_unix":1779770992.7296624},{"request_id":"1399948:48:1666394:1024","session_id":"1399948","input_length":40343,"t_proxy_recv":340132.695891793,"t_decision_unix":1779770997.7098331,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9899115088119377,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (407 < 8000)","cache_hit":39936,"estimated_new_tokens":407,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":29079,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1928,"cache_hit":0,"new_prefill":40343,"score_linear":98254.0,"score_lmetric":40343},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":132537,"ongoing_decode_tokens":0,"pending_prefill_tokens":953,"num_requests":1,"active_p_offloads":0,"cached_blocks":3119,"cache_hit":0,"new_prefill":40343,"score_linear":132537.0,"score_lmetric":41296},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":39936,"new_prefill":407,"score_linear":-39936.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":340132.776644519,"t_first_token_unix":1779770997.790588,"t_done":340134.169917915,"t_done_unix":1779770999.1838627},{"request_id":"1399948:49:1668825:1025","session_id":"1399948","input_length":40642,"t_proxy_recv":340141.258581604,"t_decision_unix":1779771006.2725227,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9952266128635402,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (194 < 8000)","cache_hit":40448,"estimated_new_tokens":194,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":29378,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1928,"cache_hit":0,"new_prefill":40642,"score_linear":98254.0,"score_lmetric":40642},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":132537,"ongoing_decode_tokens":0,"pending_prefill_tokens":953,"num_requests":1,"active_p_offloads":0,"cached_blocks":3119,"cache_hit":0,"new_prefill":40642,"score_linear":132537.0,"score_lmetric":41595},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2002,"cache_hit":40448,"new_prefill":194,"score_linear":-40448.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":340141.387558002,"t_first_token_unix":1779771006.4015017,"t_done":340142.466341796,"t_done_unix":1779771007.4802852},{"request_id":"1253804:26:1663639:1021","session_id":"1253804","input_length":98254,"t_proxy_recv":340123.301182744,"t_decision_unix":1779770988.3151238,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9952979013577056,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (462 < 8000)","cache_hit":97792,"estimated_new_tokens":462,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1928,"cache_hit":97792,"new_prefill":462,"score_linear":-97792.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3117,"cache_hit":3584,"new_prefill":94670,"score_linear":79620.0,"score_lmetric":94670},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":94670,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-97792.0,"chosen_score_lmetric":0,"t_first_token":340123.442993485,"t_first_token_unix":1779770988.4569366,"t_done":340159.928196948,"t_done_unix":1779771024.9421403},{"request_id":"1356885:8:1665690:1023","session_id":"1356885","input_length":132537,"t_proxy_recv":340130.285841425,"t_decision_unix":1779770995.2997828,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":131584,"affinity_cache_ratio":0.9928095550676415,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (953 < 8000)","cache_hit":131584,"estimated_new_tokens":953,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":10169,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1928,"cache_hit":0,"new_prefill":132537,"score_linear":98254.0,"score_lmetric":132537},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3119,"cache_hit":131584,"new_prefill":953,"score_linear":-131584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":125881,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-131584.0,"chosen_score_lmetric":0,"t_first_token":340170.139504112,"t_first_token_unix":1779771035.153448,"t_done":340172.713545469,"t_done_unix":1779771037.7274892},{"request_id":"1294611:15:1673197:1026","session_id":"1294611","input_length":85542,"t_proxy_recv":340156.47443992,"t_decision_unix":1779771021.488381,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":84992,"affinity_cache_ratio":0.9935704098571462,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (550 < 8000)","cache_hit":84992,"estimated_new_tokens":550,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1928,"cache_hit":0,"new_prefill":85542,"score_linear":98254.0,"score_lmetric":85542},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":132537,"ongoing_decode_tokens":0,"pending_prefill_tokens":953,"num_requests":1,"active_p_offloads":0,"cached_blocks":3119,"cache_hit":84992,"new_prefill":550,"score_linear":47545.0,"score_lmetric":1503},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2002,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":72742,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":72742,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":47545.0,"chosen_score_lmetric":1503,"t_first_token":340170.142936585,"t_first_token_unix":1779771035.1568801,"t_done":340172.875217711,"t_done_unix":1779771037.8891616},{"request_id":"1399948:50:1677482:1028","session_id":"1399948","input_length":42648,"t_proxy_recv":340171.117716348,"t_decision_unix":1779771036.1316574,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9484149315325455,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2200 < 8000)","cache_hit":40448,"estimated_new_tokens":2200,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":31384,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":218079,"ongoing_decode_tokens":218079,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":3119,"cache_hit":0,"new_prefill":42648,"score_linear":218079.0,"score_lmetric":85296},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2002,"cache_hit":40448,"new_prefill":2200,"score_linear":-40448.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":340171.724297688,"t_first_token_unix":1779771036.7382412,"t_done":340175.393899145,"t_done_unix":1779771040.4078426},{"request_id":"1294611:15:1676489:1027","session_id":"1294611","input_length":85722,"t_proxy_recv":340172.893239156,"t_decision_unix":1779771037.9071805,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9974568955460675,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (218 < 8000)","cache_hit":85504,"estimated_new_tokens":218,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3122,"cache_hit":85504,"new_prefill":218,"score_linear":-85504.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":42648,"ongoing_decode_tokens":42648,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2002,"cache_hit":0,"new_prefill":85722,"score_linear":42648.0,"score_lmetric":85722},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":72922,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":72922,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-85504.0,"chosen_score_lmetric":0,"t_first_token":340173.025358987,"t_first_token_unix":1779771038.039302,"t_done":340175.627818764,"t_done_unix":1779771040.6417623},{"request_id":"1294611:15:1679070:1029","session_id":"1294611","input_length":86602,"t_proxy_recv":340176.540317141,"t_decision_unix":1779771041.5542583,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":85504,"affinity_cache_ratio":0.987321308976698,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1098 < 8000)","cache_hit":85504,"estimated_new_tokens":1098,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3122,"cache_hit":85504,"new_prefill":1098,"score_linear":-85504.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":73802,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":73802,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-85504.0,"chosen_score_lmetric":0,"t_first_token":340177.000373884,"t_first_token_unix":1779771042.0143175,"t_done":340179.779590654,"t_done_unix":1779771044.7935343},{"request_id":"1253804:27:1683133:1031","session_id":"1253804","input_length":101743,"t_proxy_recv":340190.170509388,"t_decision_unix":1779771055.1844509,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9963928722369106,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (367 < 8000)","cache_hit":101376,"estimated_new_tokens":367,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":101376,"new_prefill":367,"score_linear":-101376.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":86925,"ongoing_decode_tokens":86925,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3124,"cache_hit":3584,"new_prefill":98159,"score_linear":83341.0,"score_lmetric":98159},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":98159,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":340190.369837515,"t_first_token_unix":1779771055.383781,"t_done":340190.685555097,"t_done_unix":1779771055.6994994},{"request_id":"1294611:15:1682649:1030","session_id":"1294611","input_length":86925,"t_proxy_recv":340188.450702906,"t_decision_unix":1779771053.4646444,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":86528,"affinity_cache_ratio":0.9954328444060973,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (397 < 8000)","cache_hit":86528,"estimated_new_tokens":397,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3124,"cache_hit":86528,"new_prefill":397,"score_linear":-86528.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":12800,"new_prefill":74125,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":12800,"new_prefill":74125,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-86528.0,"chosen_score_lmetric":0,"t_first_token":340188.586665129,"t_first_token_unix":1779771053.6006088,"t_done":340192.579973596,"t_done_unix":1779771057.5939171},{"request_id":"1253804:28:1685257:1032","session_id":"1253804","input_length":101794,"t_proxy_recv":340197.171814992,"t_decision_unix":1779771062.1857564,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9958936676031986,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (418 < 8000)","cache_hit":101376,"estimated_new_tokens":418,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":101376,"new_prefill":418,"score_linear":-101376.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":98210,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":98210,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":340197.316204834,"t_first_token_unix":1779771062.3301485,"t_done":340198.037898388,"t_done_unix":1779771063.0518417},{"request_id":"1399948:51:1686626:1033","session_id":"1399948","input_length":43407,"t_proxy_recv":340202.102075647,"t_decision_unix":1779771067.1160169,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":43008,"affinity_cache_ratio":0.9908079342041606,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (399 < 8000)","cache_hit":43008,"estimated_new_tokens":399,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":32143,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":43008,"new_prefill":399,"score_linear":-43008.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-43008.0,"chosen_score_lmetric":0,"t_first_token":340202.274873669,"t_first_token_unix":1779771067.2888172,"t_done":340203.722919677,"t_done_unix":1779771068.736863},{"request_id":"1253804:29:1688398:1034","session_id":"1253804","input_length":102067,"t_proxy_recv":340208.157148036,"t_decision_unix":1779771073.1710894,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9932299371981149,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (691 < 8000)","cache_hit":101376,"estimated_new_tokens":691,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":101376,"new_prefill":691,"score_linear":-101376.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":98483,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":98483,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":340208.469390251,"t_first_token_unix":1779771073.4833343,"t_done":340209.49692533,"t_done_unix":1779771074.5108693},{"request_id":"1399948:52:1689144:1035","session_id":"1399948","input_length":43673,"t_proxy_recv":340210.739822083,"t_decision_unix":1779771075.7537634,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9964966913195796,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (153 < 8000)","cache_hit":43520,"estimated_new_tokens":153,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":32409,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":43520,"new_prefill":153,"score_linear":-43520.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":340210.853864501,"t_first_token_unix":1779771075.8678086,"t_done":340211.594404602,"t_done_unix":1779771076.6083484},{"request_id":"1356885:9:1689871:1036","session_id":"1356885","input_length":132823,"t_proxy_recv":340213.116337167,"t_decision_unix":1779771078.1302783,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":132608,"affinity_cache_ratio":0.9983813044427546,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (215 < 8000)","cache_hit":132608,"estimated_new_tokens":215,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":10455,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":132608,"new_prefill":215,"score_linear":-132608.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":126167,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-132608.0,"chosen_score_lmetric":0,"t_first_token":340213.39816213,"t_first_token_unix":1779771078.4121056,"t_done":340215.095754279,"t_done_unix":1779771080.1096978},{"request_id":"1253804:30:1690362:1037","session_id":"1253804","input_length":102214,"t_proxy_recv":340214.76926352,"t_decision_unix":1779771079.7832048,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101888,"affinity_cache_ratio":0.9968106130275696,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (326 < 8000)","cache_hit":101888,"estimated_new_tokens":326,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":101888,"new_prefill":326,"score_linear":-101888.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":132823,"ongoing_decode_tokens":132823,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":98630,"score_linear":129239.0,"score_lmetric":98630},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":98630,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101888.0,"chosen_score_lmetric":0,"t_first_token":340214.990559544,"t_first_token_unix":1779771080.0045033,"t_done":340215.74092527,"t_done_unix":1779771080.7548687},{"request_id":"1253804:31:1691981:1038","session_id":"1253804","input_length":102322,"t_proxy_recv":340220.346824746,"t_decision_unix":1779771085.360766,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101888,"affinity_cache_ratio":0.9957584879107132,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (434 < 8000)","cache_hit":101888,"estimated_new_tokens":434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":101888,"new_prefill":434,"score_linear":-101888.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":98738,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":98738,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101888.0,"chosen_score_lmetric":0,"t_first_token":340220.500375908,"t_first_token_unix":1779771085.5143192,"t_done":340220.890987389,"t_done_unix":1779771085.9049306},{"request_id":"1399948:53:1692115:1039","session_id":"1399948","input_length":43938,"t_proxy_recv":340220.860980544,"t_decision_unix":1779771085.874922,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9904865947471437,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (418 < 8000)","cache_hit":43520,"estimated_new_tokens":418,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":32674,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102322,"ongoing_decode_tokens":102322,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":43938,"score_linear":102322.0,"score_lmetric":43938},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":43520,"new_prefill":418,"score_linear":-43520.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":340221.024087706,"t_first_token_unix":1779771086.0380313,"t_done":340222.398462953,"t_done_unix":1779771087.4124062},{"request_id":"1253804:32:1693715:1040","session_id":"1253804","input_length":102509,"t_proxy_recv":340226.433147206,"t_decision_unix":1779771091.4470885,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101888,"affinity_cache_ratio":0.9939419953369948,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (621 < 8000)","cache_hit":101888,"estimated_new_tokens":621,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":101888,"new_prefill":621,"score_linear":-101888.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":98925,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":98925,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101888.0,"chosen_score_lmetric":0,"t_first_token":340226.740755335,"t_first_token_unix":1779771091.754699,"t_done":340227.529133591,"t_done_unix":1779771092.5430775},{"request_id":"1399948:54:1694570:1041","session_id":"1399948","input_length":44182,"t_proxy_recv":340229.520378713,"t_decision_unix":1779771094.53432,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":44032,"affinity_cache_ratio":0.9966049522429948,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (150 < 8000)","cache_hit":44032,"estimated_new_tokens":150,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":32918,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1931,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":44032,"new_prefill":150,"score_linear":-44032.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":340229.630650007,"t_first_token_unix":1779771094.6445935,"t_done":340229.912996425,"t_done_unix":1779771094.9269397},{"request_id":"1253804:33:1695393:1042","session_id":"1253804","input_length":102632,"t_proxy_recv":340232.182986167,"t_decision_unix":1779771097.1969278,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102400,"affinity_cache_ratio":0.9977394964533479,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (232 < 8000)","cache_hit":102400,"estimated_new_tokens":232,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1931,"cache_hit":102400,"new_prefill":232,"score_linear":-102400.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":99048,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":99048,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102400.0,"chosen_score_lmetric":0,"t_first_token":340232.374879095,"t_first_token_unix":1779771097.3888226,"t_done":340232.886037267,"t_done_unix":1779771097.8999808},{"request_id":"1253804:34:1697569:1043","session_id":"1253804","input_length":102792,"t_proxy_recv":340240.019702724,"t_decision_unix":1779771105.033644,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102400,"affinity_cache_ratio":0.9961864736555374,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (392 < 8000)","cache_hit":102400,"estimated_new_tokens":392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1931,"cache_hit":102400,"new_prefill":392,"score_linear":-102400.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":99208,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":99208,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102400.0,"chosen_score_lmetric":0,"t_first_token":340240.2678199,"t_first_token_unix":1779771105.2817636,"t_done":340243.684434903,"t_done_unix":1779771108.6983786},{"request_id":"1253804:35:1701250:1044","session_id":"1253804","input_length":103349,"t_proxy_recv":340252.563466036,"t_decision_unix":1779771117.5774071,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102912,"affinity_cache_ratio":0.9957716088205981,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (437 < 8000)","cache_hit":102912,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1932,"cache_hit":102912,"new_prefill":437,"score_linear":-102912.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":99765,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":99765,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102912.0,"chosen_score_lmetric":0,"t_first_token":340252.880988433,"t_first_token_unix":1779771117.894932,"t_done":340253.195012565,"t_done_unix":1779771118.208956},{"request_id":"1253804:36:1702697:1045","session_id":"1253804","input_length":103400,"t_proxy_recv":340257.713724066,"t_decision_unix":1779771122.7276654,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102912,"affinity_cache_ratio":0.9952804642166344,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (488 < 8000)","cache_hit":102912,"estimated_new_tokens":488,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1932,"cache_hit":102912,"new_prefill":488,"score_linear":-102912.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":99816,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":99816,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102912.0,"chosen_score_lmetric":0,"t_first_token":340257.855967851,"t_first_token_unix":1779771122.869911,"t_done":340258.310463443,"t_done_unix":1779771123.3244069},{"request_id":"1399948:55:1704272:1047","session_id":"1399948","input_length":45284,"t_proxy_recv":340262.986318965,"t_decision_unix":1779771128.0002604,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":44032,"affinity_cache_ratio":0.9723522657009098,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1252 < 8000)","cache_hit":44032,"estimated_new_tokens":1252,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":34020,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103471,"ongoing_decode_tokens":103471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1933,"cache_hit":0,"new_prefill":45284,"score_linear":103471.0,"score_lmetric":45284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":44032,"new_prefill":1252,"score_linear":-44032.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":340263.423977611,"t_first_token_unix":1779771128.437921,"t_done":340265.109253818,"t_done_unix":1779771130.123197},{"request_id":"1356885:10:1706624:1048","session_id":"1356885","input_length":133014,"t_proxy_recv":340271.312051333,"t_decision_unix":1779771136.3259923,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":132608,"affinity_cache_ratio":0.9969476897168719,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (406 < 8000)","cache_hit":132608,"estimated_new_tokens":406,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":10646,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103471,"ongoing_decode_tokens":103471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1933,"cache_hit":0,"new_prefill":133014,"score_linear":103471.0,"score_lmetric":133014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":132608,"new_prefill":406,"score_linear":-132608.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2011,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":126358,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-132608.0,"chosen_score_lmetric":0,"t_first_token":340271.570475051,"t_first_token_unix":1779771136.5844185,"t_done":340273.040710023,"t_done_unix":1779771138.0546536},{"request_id":"1253804:37:1704071:1046","session_id":"1253804","input_length":103471,"t_proxy_recv":340262.187487793,"t_decision_unix":1779771127.2014294,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":103424,"affinity_cache_ratio":0.9995457664466372,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (47 < 8000)","cache_hit":103424,"estimated_new_tokens":47,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1933,"cache_hit":103424,"new_prefill":47,"score_linear":-103424.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3125,"cache_hit":3584,"new_prefill":99887,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":99887,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-103424.0,"chosen_score_lmetric":0,"t_first_token":340262.331539993,"t_first_token_unix":1779771127.3454835,"t_done":340277.875084156,"t_done_unix":1779771142.889028},{"request_id":"1399948:56:1708284:1049","session_id":"1399948","input_length":45571,"t_proxy_recv":340277.02142779,"t_decision_unix":1779771142.035369,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9886989532816923,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (515 < 8000)","cache_hit":45056,"estimated_new_tokens":515,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":34307,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103471,"ongoing_decode_tokens":103471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1933,"cache_hit":0,"new_prefill":45571,"score_linear":103471.0,"score_lmetric":45571},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2011,"cache_hit":45056,"new_prefill":515,"score_linear":-45056.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-45056.0,"chosen_score_lmetric":0,"t_first_token":340277.138032133,"t_first_token_unix":1779771142.1519759,"t_done":340278.041087774,"t_done_unix":1779771143.0550315},{"request_id":"1399948:57:1711464:1050","session_id":"1399948","input_length":45756,"t_proxy_recv":340287.817082519,"t_decision_unix":1779771152.8310237,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":45568,"affinity_cache_ratio":0.995891249235073,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (188 < 8000)","cache_hit":45568,"estimated_new_tokens":188,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":34492,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1935,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2012,"cache_hit":45568,"new_prefill":188,"score_linear":-45568.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-45568.0,"chosen_score_lmetric":0,"t_first_token":340287.931830575,"t_first_token_unix":1779771152.945774,"t_done":340290.422865712,"t_done_unix":1779771155.4368095},{"request_id":"1253804:38:1712824:1051","session_id":"1253804","input_length":104953,"t_proxy_recv":340292.64170033,"t_decision_unix":1779771157.6556413,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":104448,"affinity_cache_ratio":0.9951883223919278,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (505 < 8000)","cache_hit":104448,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1935,"cache_hit":104448,"new_prefill":505,"score_linear":-104448.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":101369,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":101369,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-104448.0,"chosen_score_lmetric":0,"t_first_token":340292.849382321,"t_first_token_unix":1779771157.8633263,"t_done":340293.165342073,"t_done_unix":1779771158.179286},{"request_id":"1253804:39:1714767:1052","session_id":"1253804","input_length":105004,"t_proxy_recv":340299.225610961,"t_decision_unix":1779771164.2395446,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":104960,"affinity_cache_ratio":0.9995809683440631,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (44 < 8000)","cache_hit":104960,"estimated_new_tokens":44,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":104960,"new_prefill":44,"score_linear":-104960.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":101420,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":101420,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-104960.0,"chosen_score_lmetric":0,"t_first_token":340299.373064173,"t_first_token_unix":1779771164.387008,"t_done":340299.953031314,"t_done_unix":1779771164.9669752},{"request_id":"1253804:40:1716790:1053","session_id":"1253804","input_length":105156,"t_proxy_recv":340305.935552125,"t_decision_unix":1779771170.949493,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":104960,"affinity_cache_ratio":0.9981361025523984,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (196 < 8000)","cache_hit":104960,"estimated_new_tokens":196,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":104960,"new_prefill":196,"score_linear":-104960.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":101572,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":101572,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-104960.0,"chosen_score_lmetric":0,"t_first_token":340306.187321752,"t_first_token_unix":1779771171.2012663,"t_done":340310.874219339,"t_done_unix":1779771175.888163},{"request_id":"1305906:4:1718131:1054","session_id":"1305906","input_length":47861,"t_proxy_recv":340310.385845673,"t_decision_unix":1779771175.3997867,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9413927832682142,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2805 < 8000)","cache_hit":45056,"estimated_new_tokens":2805,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":105156,"ongoing_decode_tokens":105156,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":0,"new_prefill":47861,"score_linear":105156.0,"score_lmetric":47861},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":45056,"new_prefill":2805,"score_linear":-45056.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-45056.0,"chosen_score_lmetric":0,"t_first_token":340311.313900465,"t_first_token_unix":1779771176.327844,"t_done":340319.057942374,"t_done_unix":1779771184.0718863},{"request_id":"1305906:5:1514292:934","session_id":"1305906","input_length":49817,"t_proxy_recv":340319.070436348,"t_decision_unix":1779771184.0843778,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9660959110343859,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1689 < 8000)","cache_hit":48128,"estimated_new_tokens":1689,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37173,"ongoing_decode_tokens":37173,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2410,"cache_hit":48128,"new_prefill":1689,"score_linear":-10955.0,"score_lmetric":1689},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-10955.0,"chosen_score_lmetric":1689,"t_first_token":340319.729826755,"t_first_token_unix":1779771184.7437706,"t_done":340320.657332236,"t_done_unix":1779771185.6712763},{"request_id":"1340290:13:1718889:1055","session_id":"1340290","input_length":37173,"t_proxy_recv":340313.018626585,"t_decision_unix":1779771178.0325677,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9503672020014526,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1845 < 8000)","cache_hit":35328,"estimated_new_tokens":1845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":47861,"ongoing_decode_tokens":47861,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2404,"cache_hit":35328,"new_prefill":1845,"score_linear":12533.0,"score_lmetric":1845},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":36149,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":12533.0,"chosen_score_lmetric":1845,"t_first_token":340318.655095562,"t_first_token_unix":1779771183.6690404,"t_done":340321.320022105,"t_done_unix":1779771186.3339658},{"request_id":"1253804:41:1721979:1057","session_id":"1253804","input_length":105638,"t_proxy_recv":340323.614374416,"t_decision_unix":1779771188.6283154,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9984285957704614,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (166 < 8000)","cache_hit":105472,"estimated_new_tokens":166,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":105472,"new_prefill":166,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":102054,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":102054,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37438,"ongoing_decode_tokens":37438,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2417,"cache_hit":0,"new_prefill":105638,"score_linear":37438.0,"score_lmetric":105638},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":340323.813793016,"t_first_token_unix":1779771188.8277369,"t_done":340324.470930089,"t_done_unix":1779771189.4848742},{"request_id":"1340290:14:1721731:1056","session_id":"1340290","input_length":37438,"t_proxy_recv":340322.738254797,"t_decision_unix":1779771187.7521958,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":37376,"affinity_cache_ratio":0.998343928628666,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (62 < 8000)","cache_hit":37376,"estimated_new_tokens":62,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2417,"cache_hit":37376,"new_prefill":62,"score_linear":-37376.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":36414,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-37376.0,"chosen_score_lmetric":0,"t_first_token":340322.815343484,"t_first_token_unix":1779771187.8292875,"t_done":340324.597159243,"t_done_unix":1779771189.6111033},{"request_id":"1253804:42:1724453:1058","session_id":"1253804","input_length":105715,"t_proxy_recv":340332.117402115,"t_decision_unix":1779771197.1313431,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9977013668826562,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (243 < 8000)","cache_hit":105472,"estimated_new_tokens":243,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":105472,"new_prefill":243,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":102131,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":102131,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2417,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":340332.263779035,"t_first_token_unix":1779771197.2777228,"t_done":340333.053473128,"t_done_unix":1779771198.0674171},{"request_id":"1253804:43:1726322:1060","session_id":"1253804","input_length":105845,"t_proxy_recv":340338.189683279,"t_decision_unix":1779771203.2036245,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9964759790259341,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (373 < 8000)","cache_hit":105472,"estimated_new_tokens":373,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":105472,"new_prefill":373,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":102261,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":102261,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2417,"cache_hit":0,"new_prefill":105845,"score_linear":37724.0,"score_lmetric":105845},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":340338.388749928,"t_first_token_unix":1779771203.402694,"t_done":340339.089367177,"t_done_unix":1779771204.1033108},{"request_id":"1399948:58:1726693:1061","session_id":"1399948","input_length":46886,"t_proxy_recv":340339.40506013,"t_decision_unix":1779771204.4190016,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":46080,"affinity_cache_ratio":0.982809367401783,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (806 < 8000)","cache_hit":46080,"estimated_new_tokens":806,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":35622,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":46080,"new_prefill":806,"score_linear":-46080.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2417,"cache_hit":0,"new_prefill":46886,"score_linear":37724.0,"score_lmetric":46886},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-46080.0,"chosen_score_lmetric":0,"t_first_token":340339.771150767,"t_first_token_unix":1779771204.7850945,"t_done":340339.897223324,"t_done_unix":1779771204.9111664},{"request_id":"1399948:59:1728305:1062","session_id":"1399948","input_length":46927,"t_proxy_recv":340344.699888087,"t_decision_unix":1779771209.7138288,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":46592,"affinity_cache_ratio":0.9928612525838004,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (335 < 8000)","cache_hit":46592,"estimated_new_tokens":335,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":35663,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2014,"cache_hit":46592,"new_prefill":335,"score_linear":-46592.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2417,"cache_hit":0,"new_prefill":46927,"score_linear":37724.0,"score_lmetric":46927},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-46592.0,"chosen_score_lmetric":0,"t_first_token":340344.791420906,"t_first_token_unix":1779771209.8053644,"t_done":340344.933928726,"t_done_unix":1779771209.9478722},{"request_id":"1253804:44:1728625:1063","session_id":"1253804","input_length":106070,"t_proxy_recv":340345.892362366,"t_decision_unix":1779771210.9063034,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9943622136325069,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (598 < 8000)","cache_hit":105472,"estimated_new_tokens":598,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":105472,"new_prefill":598,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":102486,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":102486,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2014,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2417,"cache_hit":0,"new_prefill":106070,"score_linear":37724.0,"score_lmetric":106070},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":340346.213086034,"t_first_token_unix":1779771211.2270296,"t_done":340347.264510391,"t_done_unix":1779771212.278455},{"request_id":"1340290:15:1725612:1059","session_id":"1340290","input_length":37724,"t_proxy_recv":340335.923804206,"t_decision_unix":1779771200.9377422,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":37376,"affinity_cache_ratio":0.9907751033824621,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (348 < 8000)","cache_hit":37376,"estimated_new_tokens":348,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2013,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2417,"cache_hit":37376,"new_prefill":348,"score_linear":-37376.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":36700,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-37376.0,"chosen_score_lmetric":0,"t_first_token":340336.004591865,"t_first_token_unix":1779771201.0185354,"t_done":340353.615425834,"t_done_unix":1779771218.62937},{"request_id":"1253804:45:1732408:1064","session_id":"1253804","input_length":106394,"t_proxy_recv":340358.856976176,"t_decision_unix":1779771223.8709173,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105984,"affinity_cache_ratio":0.9961463992330395,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (410 < 8000)","cache_hit":105984,"estimated_new_tokens":410,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1938,"cache_hit":105984,"new_prefill":410,"score_linear":-105984.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":102810,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":102810,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2014,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105984.0,"chosen_score_lmetric":0,"t_first_token":340359.180349132,"t_first_token_unix":1779771224.1942928,"t_done":340359.558588774,"t_done_unix":1779771224.5725324},{"request_id":"1399948:60:1732843:1065","session_id":"1399948","input_length":47074,"t_proxy_recv":340360.450486995,"t_decision_unix":1779771225.4644282,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":46592,"affinity_cache_ratio":0.9897608021413095,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (482 < 8000)","cache_hit":46592,"estimated_new_tokens":482,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":35810,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1938,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2014,"cache_hit":46592,"new_prefill":482,"score_linear":-46592.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-46592.0,"chosen_score_lmetric":0,"t_first_token":340360.580578039,"t_first_token_unix":1779771225.5945215,"t_done":340360.951584245,"t_done_unix":1779771225.9655278},{"request_id":"1253804:46:1733935:1066","session_id":"1253804","input_length":106443,"t_proxy_recv":340363.949706579,"t_decision_unix":1779771228.963648,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105984,"affinity_cache_ratio":0.9956878329246639,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (459 < 8000)","cache_hit":105984,"estimated_new_tokens":459,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1938,"cache_hit":105984,"new_prefill":459,"score_linear":-105984.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":102859,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":102859,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105984.0,"chosen_score_lmetric":0,"t_first_token":340364.083702133,"t_first_token_unix":1779771229.0976455,"t_done":340364.804352411,"t_done_unix":1779771229.8182957},{"request_id":"1399948:61:1735933:1067","session_id":"1399948","input_length":47205,"t_proxy_recv":340370.646537708,"t_decision_unix":1779771235.6604793,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":47104,"affinity_cache_ratio":0.9978603961444762,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (101 < 8000)","cache_hit":47104,"estimated_new_tokens":101,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":35941,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":47104,"new_prefill":101,"score_linear":-47104.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-47104.0,"chosen_score_lmetric":0,"t_first_token":340370.757156926,"t_first_token_unix":1779771235.7711,"t_done":340372.299747392,"t_done_unix":1779771237.3136907},{"request_id":"1253804:47:1736337:1068","session_id":"1253804","input_length":107137,"t_proxy_recv":340371.925867771,"t_decision_unix":1779771236.939809,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":106496,"affinity_cache_ratio":0.994017006263009,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (641 < 8000)","cache_hit":106496,"estimated_new_tokens":641,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1939,"cache_hit":106496,"new_prefill":641,"score_linear":-106496.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":103553,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":103553,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":47205,"ongoing_decode_tokens":47205,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":0,"new_prefill":107137,"score_linear":47205.0,"score_lmetric":107137},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-106496.0,"chosen_score_lmetric":0,"t_first_token":340372.641907141,"t_first_token_unix":1779771237.6558506,"t_done":340373.063903608,"t_done_unix":1779771238.0778472},{"request_id":"1253804:48:1737694:1069","session_id":"1253804","input_length":108145,"t_proxy_recv":340376.362914454,"t_decision_unix":1779771241.3768542,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":107008,"affinity_cache_ratio":0.98948633778723,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1137 < 8000)","cache_hit":107008,"estimated_new_tokens":1137,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":107008,"new_prefill":1137,"score_linear":-107008.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":104561,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":104561,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-107008.0,"chosen_score_lmetric":0,"t_first_token":340377.281483915,"t_first_token_unix":1779771242.2954278,"t_done":340377.715048486,"t_done_unix":1779771242.728992},{"request_id":"1253804:49:1739030:1070","session_id":"1253804","input_length":109454,"t_proxy_recv":340380.747194657,"t_decision_unix":1779771245.761136,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":108032,"affinity_cache_ratio":0.9870082409048551,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1422 < 8000)","cache_hit":108032,"estimated_new_tokens":1422,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1942,"cache_hit":108032,"new_prefill":1422,"score_linear":-108032.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":105870,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":105870,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":340381.879852856,"t_first_token_unix":1779771246.893797,"t_done":340382.312628523,"t_done_unix":1779771247.3265727},{"request_id":"1253804:50:1741670:1071","session_id":"1253804","input_length":110854,"t_proxy_recv":340389.728279117,"t_decision_unix":1779771254.7422206,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":109056,"affinity_cache_ratio":0.9837804680029588,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1798 < 8000)","cache_hit":109056,"estimated_new_tokens":1798,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1944,"cache_hit":109056,"new_prefill":1798,"score_linear":-109056.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":107270,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":107270,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-109056.0,"chosen_score_lmetric":0,"t_first_token":340390.878126362,"t_first_token_unix":1779771255.8920698,"t_done":340391.318920986,"t_done_unix":1779771256.332865},{"request_id":"1253804:51:1744514:1072","session_id":"1253804","input_length":111661,"t_proxy_recv":340399.149828047,"t_decision_unix":1779771264.1637692,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":110592,"affinity_cache_ratio":0.9904263798461415,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1069 < 8000)","cache_hit":110592,"estimated_new_tokens":1069,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1947,"cache_hit":110592,"new_prefill":1069,"score_linear":-110592.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":108077,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":108077,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-110592.0,"chosen_score_lmetric":0,"t_first_token":340399.898634776,"t_first_token_unix":1779771264.9125788,"t_done":340400.338174721,"t_done_unix":1779771265.3521183},{"request_id":"1340290:16:1745847:1073","session_id":"1340290","input_length":40200,"t_proxy_recv":340403.757944202,"t_decision_unix":1779771268.7718854,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9934328358208955,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (264 < 8000)","cache_hit":39936,"estimated_new_tokens":264,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1949,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":39936,"new_prefill":264,"score_linear":-39936.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":39176,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":340403.837181844,"t_first_token_unix":1779771268.851125,"t_done":340404.570209264,"t_done_unix":1779771269.584153},{"request_id":"1253804:52:1746191:1074","session_id":"1253804","input_length":111724,"t_proxy_recv":340404.795459367,"t_decision_unix":1779771269.8093996,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9990333321399162,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (108 < 8000)","cache_hit":111616,"estimated_new_tokens":108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1949,"cache_hit":111616,"new_prefill":108,"score_linear":-111616.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":108140,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":108140,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":340404.955263197,"t_first_token_unix":1779771269.969207,"t_done":340405.740338963,"t_done_unix":1779771270.7542825},{"request_id":"1253804:53:1748377:1075","session_id":"1253804","input_length":111811,"t_proxy_recv":340412.130915797,"t_decision_unix":1779771277.1448572,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9982559855470392,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (195 < 8000)","cache_hit":111616,"estimated_new_tokens":195,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1949,"cache_hit":111616,"new_prefill":195,"score_linear":-111616.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":108227,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":108227,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":340412.283561753,"t_first_token_unix":1779771277.2975054,"t_done":340412.7805973,"t_done_unix":1779771277.7945411},{"request_id":"1399948:62:1749028:1076","session_id":"1399948","input_length":48212,"t_proxy_recv":340414.289873409,"t_decision_unix":1779771279.3038146,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":47104,"affinity_cache_ratio":0.9770181697502697,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1108 < 8000)","cache_hit":47104,"estimated_new_tokens":1108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":36948,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1949,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2015,"cache_hit":47104,"new_prefill":1108,"score_linear":-47104.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-47104.0,"chosen_score_lmetric":0,"t_first_token":340414.657017171,"t_first_token_unix":1779771279.6709611,"t_done":340415.21315312,"t_done_unix":1779771280.2270966},{"request_id":"1253804:54:1750056:1077","session_id":"1253804","input_length":112074,"t_proxy_recv":340417.721767731,"t_decision_unix":1779771282.735709,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9959134143512323,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (458 < 8000)","cache_hit":111616,"estimated_new_tokens":458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1949,"cache_hit":111616,"new_prefill":458,"score_linear":-111616.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":3584,"new_prefill":108490,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":108490,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":340418.059995895,"t_first_token_unix":1779771283.0739398,"t_done":340424.029035952,"t_done_unix":1779771289.0429802},{"request_id":"1399948:63:1753316:1079","session_id":"1399948","input_length":48636,"t_proxy_recv":340428.588466699,"t_decision_unix":1779771293.602408,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9895550620939222,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (508 < 8000)","cache_hit":48128,"estimated_new_tokens":508,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":37372,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":133552,"ongoing_decode_tokens":0,"pending_prefill_tokens":432,"num_requests":1,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":48636,"score_linear":133552.0,"score_lmetric":49068},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":48128,"new_prefill":508,"score_linear":-48128.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48128.0,"chosen_score_lmetric":0,"t_first_token":340428.849841438,"t_first_token_unix":1779771293.863785,"t_done":340428.961097512,"t_done_unix":1779771293.975041},{"request_id":"1399948:64:1754453:1080","session_id":"1399948","input_length":48693,"t_proxy_recv":340432.499142609,"t_decision_unix":1779771297.513084,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":48640,"affinity_cache_ratio":0.9989115478610888,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (53 < 8000)","cache_hit":48640,"estimated_new_tokens":53,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":37429,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":133552,"ongoing_decode_tokens":133552,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":0,"new_prefill":48693,"score_linear":133552.0,"score_lmetric":48693},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2018,"cache_hit":48640,"new_prefill":53,"score_linear":-48640.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48640.0,"chosen_score_lmetric":0,"t_first_token":340432.608490714,"t_first_token_unix":1779771297.6224413,"t_done":340433.900782609,"t_done_unix":1779771298.9147265},{"request_id":"1356885:11:1753167:1078","session_id":"1356885","input_length":133552,"t_proxy_recv":340428.162789395,"t_decision_unix":1779771293.1767309,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":133120,"affinity_cache_ratio":0.9967653048999641,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (432 < 8000)","cache_hit":133120,"estimated_new_tokens":432,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":11184,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3126,"cache_hit":133120,"new_prefill":432,"score_linear":-133120.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2017,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":126896,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-133120.0,"chosen_score_lmetric":0,"t_first_token":340428.808984053,"t_first_token_unix":1779771293.8229277,"t_done":340446.18151853,"t_done_unix":1779771311.1954625},{"request_id":"1399948:65:1759200:1081","session_id":"1399948","input_length":49220,"t_proxy_recv":340448.995929547,"t_decision_unix":1779771314.0098708,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":48640,"affinity_cache_ratio":0.9882161722876879,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (580 < 8000)","cache_hit":48640,"estimated_new_tokens":580,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":37956,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3128,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2018,"cache_hit":48640,"new_prefill":580,"score_linear":-48640.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48640.0,"chosen_score_lmetric":0,"t_first_token":340449.26363567,"t_first_token_unix":1779771314.2775793,"t_done":340454.867619664,"t_done_unix":1779771319.8815796},{"request_id":"1277428:14:1767578:1082","session_id":"1277428","input_length":49716,"t_proxy_recv":340477.845339324,"t_decision_unix":1779771342.8592806,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9886555636012552,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (564 < 8000)","cache_hit":49152,"estimated_new_tokens":564,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":4608,"new_prefill":45108,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3128,"cache_hit":49152,"new_prefill":564,"score_linear":-49152.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":13364,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":340487.112269032,"t_first_token_unix":1779771352.1262126,"t_done":340488.159991572,"t_done_unix":1779771353.1739352},{"request_id":"1277428:14:1770237:1083","session_id":"1277428","input_length":49925,"t_proxy_recv":340488.175752054,"t_decision_unix":1779771353.1896937,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49664,"affinity_cache_ratio":0.994772158237356,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (261 < 8000)","cache_hit":49664,"estimated_new_tokens":261,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":4608,"new_prefill":45317,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3129,"cache_hit":49664,"new_prefill":261,"score_linear":-49664.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":13573,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":340488.284197999,"t_first_token_unix":1779771353.2981415,"t_done":340488.60157772,"t_done_unix":1779771353.615521},{"request_id":"1277428:15:1771467:1084","session_id":"1277428","input_length":51398,"t_proxy_recv":340491.272976844,"t_decision_unix":1779771356.2869182,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9662632787267987,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1734 < 8000)","cache_hit":49664,"estimated_new_tokens":1734,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":4608,"new_prefill":46790,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3129,"cache_hit":49664,"new_prefill":1734,"score_linear":-49664.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":15046,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":340491.857892265,"t_first_token_unix":1779771356.8718355,"t_done":340495.372911425,"t_done_unix":1779771360.3868546},{"request_id":"1399948:66:1772336:1085","session_id":"1399948","input_length":51012,"t_proxy_recv":340494.223595641,"t_decision_unix":1779771359.2375367,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9735748451344781,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1348 < 8000)","cache_hit":49664,"estimated_new_tokens":1348,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":39748,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51398,"ongoing_decode_tokens":51398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3129,"cache_hit":0,"new_prefill":51012,"score_linear":51398.0,"score_lmetric":51012},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":49664,"new_prefill":1348,"score_linear":-49664.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":340494.705332374,"t_first_token_unix":1779771359.7192757,"t_done":340497.684029728,"t_done_unix":1779771362.6979735},{"request_id":"1399948:67:1777074:1086","session_id":"1399948","input_length":52331,"t_proxy_recv":340510.281651548,"t_decision_unix":1779771375.2955928,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9783875714203818,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1131 < 8000)","cache_hit":51200,"estimated_new_tokens":1131,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":41067,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3133,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2023,"cache_hit":51200,"new_prefill":1131,"score_linear":-51200.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-51200.0,"chosen_score_lmetric":0,"t_first_token":340510.762258502,"t_first_token_unix":1779771375.776202,"t_done":340515.10452427,"t_done_unix":1779771380.1184678},{"request_id":"1399948:68:1784799:1087","session_id":"1399948","input_length":53289,"t_proxy_recv":340536.769763251,"t_decision_unix":1779771401.7837045,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9896226238060387,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (553 < 8000)","cache_hit":52736,"estimated_new_tokens":553,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":42025,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3133,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2026,"cache_hit":52736,"new_prefill":553,"score_linear":-52736.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":340537.059588064,"t_first_token_unix":1779771402.0735316,"t_done":340541.386521602,"t_done_unix":1779771406.4004653},{"request_id":"1277428:16:1787662:1088","session_id":"1277428","input_length":52754,"t_proxy_recv":340547.227080911,"t_decision_unix":1779771412.241022,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":51712,"affinity_cache_ratio":0.9802479432839216,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1042 < 8000)","cache_hit":51712,"estimated_new_tokens":1042,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":4608,"new_prefill":48146,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3133,"cache_hit":51712,"new_prefill":1042,"score_linear":-51712.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":16402,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2028,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-51712.0,"chosen_score_lmetric":0,"t_first_token":340547.626069561,"t_first_token_unix":1779771412.640013,"t_done":340548.227979875,"t_done_unix":1779771413.2419236},{"request_id":"1305906:5:1788459:1089","session_id":"1305906","input_length":52461,"t_proxy_recv":340549.759941467,"t_decision_unix":1779771414.7738829,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49664,"affinity_cache_ratio":0.946684203503555,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2797 < 8000)","cache_hit":49664,"estimated_new_tokens":2797,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3135,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2028,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2418,"cache_hit":49664,"new_prefill":2797,"score_linear":-49664.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":340550.750882691,"t_first_token_unix":1779771415.764826,"t_done":340551.258994906,"t_done_unix":1779771416.2729385},{"request_id":"1399948:69:1790968:1090","session_id":"1399948","input_length":54141,"t_proxy_recv":340558.357069448,"t_decision_unix":1779771423.3710108,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":53760,"affinity_cache_ratio":0.9929628193051476,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (381 < 8000)","cache_hit":53760,"estimated_new_tokens":381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":42877,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3135,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2028,"cache_hit":53760,"new_prefill":381,"score_linear":-53760.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":51255,"ongoing_decode_tokens":51255,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2423,"cache_hit":0,"new_prefill":54141,"score_linear":51255.0,"score_lmetric":54141},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":340558.642963606,"t_first_token_unix":1779771423.6569076,"t_done":340562.548452897,"t_done_unix":1779771427.5623965},{"request_id":"1305906:6:1529600:945","session_id":"1305906","input_length":51255,"t_proxy_recv":340551.268795092,"t_decision_unix":1779771416.2827365,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9989269339576626,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (55 < 8000)","cache_hit":51200,"estimated_new_tokens":55,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3135,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2028,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2423,"cache_hit":51200,"new_prefill":55,"score_linear":-51200.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-51200.0,"chosen_score_lmetric":0,"t_first_token":340551.342904288,"t_first_token_unix":1779771416.3568475,"t_done":340566.395844369,"t_done_unix":1779771431.409788},{"request_id":"1277428:17:1793546:1091","session_id":"1277428","input_length":53574,"t_proxy_recv":340567.360293949,"t_decision_unix":1779771432.3742354,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9843580841452944,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (838 < 8000)","cache_hit":52736,"estimated_new_tokens":838,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":4608,"new_prefill":48966,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3135,"cache_hit":52736,"new_prefill":838,"score_linear":-52736.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":17222,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2029,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":340567.755816084,"t_first_token_unix":1779771432.7697604,"t_done":340569.086672731,"t_done_unix":1779771434.100617},{"request_id":"1399948:70:1795589:1092","session_id":"1399948","input_length":54654,"t_proxy_recv":340574.681625329,"t_decision_unix":1779771439.6955667,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":54272,"affinity_cache_ratio":0.9930105756211806,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (382 < 8000)","cache_hit":54272,"estimated_new_tokens":382,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":43390,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3136,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2029,"cache_hit":54272,"new_prefill":382,"score_linear":-54272.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54272.0,"chosen_score_lmetric":0,"t_first_token":340574.791584806,"t_first_token_unix":1779771439.8055282,"t_done":340575.712262071,"t_done_unix":1779771440.7262056},{"request_id":"1277428:18:1796824:1093","session_id":"1277428","input_length":53805,"t_proxy_recv":340579.278362329,"t_decision_unix":1779771444.2923036,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":53248,"affinity_cache_ratio":0.9896478022488616,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (557 < 8000)","cache_hit":53248,"estimated_new_tokens":557,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":4608,"new_prefill":49197,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3136,"cache_hit":53248,"new_prefill":557,"score_linear":-53248.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":17453,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2029,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53248.0,"chosen_score_lmetric":0,"t_first_token":340579.409160648,"t_first_token_unix":1779771444.423104,"t_done":340579.78951433,"t_done_unix":1779771444.8034575},{"request_id":"1399948:71:1800797:1094","session_id":"1399948","input_length":54920,"t_proxy_recv":340592.866977877,"t_decision_unix":1779771457.880919,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":54272,"affinity_cache_ratio":0.9882010196649672,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (648 < 8000)","cache_hit":54272,"estimated_new_tokens":648,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":43656,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2029,"cache_hit":54272,"new_prefill":648,"score_linear":-54272.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54272.0,"chosen_score_lmetric":0,"t_first_token":340593.04990727,"t_first_token_unix":1779771458.0638506,"t_done":340595.1408782,"t_done_unix":1779771460.1548214},{"request_id":"1399948:72:1804699:1097","session_id":"1399948","input_length":55246,"t_proxy_recv":340606.317190559,"t_decision_unix":1779771471.331132,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9916374036129313,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (462 < 8000)","cache_hit":54784,"estimated_new_tokens":462,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":43982,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":55246,"score_linear":112634.0,"score_lmetric":55246},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2030,"cache_hit":54784,"new_prefill":462,"score_linear":56952.0,"score_lmetric":462},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":56952.0,"chosen_score_lmetric":462,"t_first_token":340606.436588602,"t_first_token_unix":1779771471.4505317,"t_done":340607.198107219,"t_done_unix":1779771472.212051},{"request_id":"1305906:6:1806419:1099","session_id":"1305906","input_length":52932,"t_proxy_recv":340612.009392938,"t_decision_unix":1779771477.023334,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9962971359480087,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (196 < 8000)","cache_hit":52736,"estimated_new_tokens":196,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":52932,"score_linear":112634.0,"score_lmetric":52932},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":167295,"ongoing_decode_tokens":167295,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2031,"cache_hit":0,"new_prefill":52932,"score_linear":167295.0,"score_lmetric":105864},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":52736,"new_prefill":196,"score_linear":-52736.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":340612.101090715,"t_first_token_unix":1779771477.1150339,"t_done":340612.806196708,"t_done_unix":1779771477.8201404},{"request_id":"1399948:73:1805981:1098","session_id":"1399948","input_length":55559,"t_proxy_recv":340610.499529936,"t_decision_unix":1779771475.513471,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":55296,"affinity_cache_ratio":0.9952662934898037,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (263 < 8000)","cache_hit":55296,"estimated_new_tokens":263,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":44295,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":55559,"score_linear":112634.0,"score_lmetric":55559},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2031,"cache_hit":55296,"new_prefill":263,"score_linear":56440.0,"score_lmetric":263},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":56440.0,"chosen_score_lmetric":263,"t_first_token":340610.693299821,"t_first_token_unix":1779771475.7072437,"t_done":340613.115342793,"t_done_unix":1779771478.129286},{"request_id":"1305906:7:1572780:964","session_id":"1305906","input_length":54789,"t_proxy_recv":340612.81814566,"t_decision_unix":1779771477.8320873,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9625289747942105,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2053 < 8000)","cache_hit":52736,"estimated_new_tokens":2053,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":54789,"score_linear":112634.0,"score_lmetric":54789},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":167295,"ongoing_decode_tokens":167295,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2031,"cache_hit":0,"new_prefill":54789,"score_linear":167295.0,"score_lmetric":109578},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":52736,"new_prefill":2053,"score_linear":-52736.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":340613.543111038,"t_first_token_unix":1779771478.557055,"t_done":340613.846026427,"t_done_unix":1779771478.8599699},{"request_id":"1399948:74:1808473:1100","session_id":"1399948","input_length":55791,"t_proxy_recv":340619.205242793,"t_decision_unix":1779771484.2191842,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":55296,"affinity_cache_ratio":0.9911276012260042,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (495 < 8000)","cache_hit":55296,"estimated_new_tokens":495,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":44527,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":55791,"score_linear":112634.0,"score_lmetric":55791},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2031,"cache_hit":55296,"new_prefill":495,"score_linear":56440.0,"score_lmetric":495},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":56440.0,"chosen_score_lmetric":495,"t_first_token":340619.362476594,"t_first_token_unix":1779771484.37642,"t_done":340622.882166267,"t_done_unix":1779771487.8961103},{"request_id":"1399948:75:1817312:1101","session_id":"1399948","input_length":57308,"t_proxy_recv":340647.736746624,"t_decision_unix":1779771512.750688,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":55808,"affinity_cache_ratio":0.9738256438891604,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1500 < 8000)","cache_hit":55808,"estimated_new_tokens":1500,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":46044,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":57308,"score_linear":112634.0,"score_lmetric":57308},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2032,"cache_hit":55808,"new_prefill":1500,"score_linear":55928.0,"score_lmetric":1500},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":55928.0,"chosen_score_lmetric":1500,"t_first_token":340648.382740952,"t_first_token_unix":1779771513.396685,"t_done":340649.994909165,"t_done_unix":1779771515.0088522},{"request_id":"1270606:6:1804345:1096","session_id":"1270606","input_length":111736,"t_proxy_recv":340605.153225821,"t_decision_unix":1779771470.167167,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9989260399513138,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (120 < 8000)","cache_hit":111616,"estimated_new_tokens":120,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":111736,"score_linear":112634.0,"score_lmetric":111736},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2030,"cache_hit":111616,"new_prefill":120,"score_linear":-111616.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":110200,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":340605.447991834,"t_first_token_unix":1779771470.4619358,"t_done":340651.015169338,"t_done_unix":1779771516.0291128},{"request_id":"1399948:76:1821495:1102","session_id":"1399948","input_length":57659,"t_proxy_recv":340661.393312602,"t_decision_unix":1779771526.407254,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9945368459390554,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (315 < 8000)","cache_hit":57344,"estimated_new_tokens":315,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":11264,"new_prefill":46395,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":57659,"score_linear":112634.0,"score_lmetric":57659},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2037,"cache_hit":57344,"new_prefill":315,"score_linear":-57344.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":340661.590568716,"t_first_token_unix":1779771526.6045122,"t_done":340662.258032887,"t_done_unix":1779771527.2719767},{"request_id":"1270606:6:1822681:1103","session_id":"1270606","input_length":112180,"t_proxy_recv":340665.239602819,"t_decision_unix":1779771530.253544,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":112128,"affinity_cache_ratio":0.9995364592619005,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (52 < 8000)","cache_hit":112128,"estimated_new_tokens":52,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":112180,"score_linear":112634.0,"score_lmetric":112180},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2037,"cache_hit":112128,"new_prefill":52,"score_linear":-112128.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":110644,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-112128.0,"chosen_score_lmetric":0,"t_first_token":340665.379068466,"t_first_token_unix":1779771530.393012,"t_done":340666.274315565,"t_done_unix":1779771531.288259},{"request_id":"1270606:7:1825806:1104","session_id":"1270606","input_length":112645,"t_proxy_recv":340675.853838461,"t_decision_unix":1779771540.8677797,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":112128,"affinity_cache_ratio":0.9954103599804697,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (517 < 8000)","cache_hit":112128,"estimated_new_tokens":517,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":112645,"score_linear":112634.0,"score_lmetric":112645},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2037,"cache_hit":112128,"new_prefill":517,"score_linear":-112128.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":111109,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-112128.0,"chosen_score_lmetric":0,"t_first_token":340676.411265406,"t_first_token_unix":1779771541.4252088,"t_done":340680.485104228,"t_done_unix":1779771545.499048},{"request_id":"1277428:19:1834491:1105","session_id":"1277428","input_length":58474,"t_proxy_recv":340705.121926685,"t_decision_unix":1779771570.1358678,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":53760,"affinity_cache_ratio":0.9193829736293053,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (4714 < 8000)","cache_hit":53760,"estimated_new_tokens":4714,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":4608,"new_prefill":53866,"score_linear":108026.0,"score_lmetric":53866},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":53760,"new_prefill":4714,"score_linear":-53760.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":22122,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2038,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":340706.870055429,"t_first_token_unix":1779771571.8839989,"t_done":340707.207445879,"t_done_unix":1779771572.2213893},{"request_id":"1270606:8:1837578:1106","session_id":"1270606","input_length":113482,"t_proxy_recv":340715.951927897,"t_decision_unix":1779771580.9658694,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":112640,"affinity_cache_ratio":0.9925803211081934,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (842 < 8000)","cache_hit":112640,"estimated_new_tokens":842,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":0,"new_prefill":113482,"score_linear":112634.0,"score_lmetric":113482},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2038,"cache_hit":112640,"new_prefill":842,"score_linear":-112640.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":111946,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-112640.0,"chosen_score_lmetric":0,"t_first_token":340716.520745258,"t_first_token_unix":1779771581.5346887,"t_done":340716.954896802,"t_done_unix":1779771581.9688401},{"request_id":"1253804:55:1803044:1095","session_id":"1253804","input_length":112634,"t_proxy_recv":340600.742473125,"t_decision_unix":1779771465.7564144,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":112128,"affinity_cache_ratio":0.9955075732016976,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (506 < 8000)","cache_hit":112128,"estimated_new_tokens":506,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1950,"cache_hit":112128,"new_prefill":506,"score_linear":-112128.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3137,"cache_hit":3584,"new_prefill":109050,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":109050,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2030,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2426,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-112128.0,"chosen_score_lmetric":0,"t_first_token":340600.917426684,"t_first_token_unix":1779771465.9313703,"t_done":340725.907556359,"t_done_unix":1779771590.9215},{"request_id":"1270606:9:1847569:1107","session_id":"1270606","input_length":116467,"t_proxy_recv":340749.856065338,"t_decision_unix":1779771614.8700066,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":113152,"affinity_cache_ratio":0.9715370018975332,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3315 < 8000)","cache_hit":113152,"estimated_new_tokens":3315,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2039,"cache_hit":113152,"new_prefill":3315,"score_linear":-113152.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":114931,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-113152.0,"chosen_score_lmetric":0,"t_first_token":340752.10260076,"t_first_token_unix":1779771617.1165442,"t_done":340753.643869714,"t_done_unix":1779771618.6578133},{"request_id":"1270606:10:1851128:1108","session_id":"1270606","input_length":116659,"t_proxy_recv":340761.93227648,"t_decision_unix":1779771626.9462178,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":116224,"affinity_cache_ratio":0.9962711835349181,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (435 < 8000)","cache_hit":116224,"estimated_new_tokens":435,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2045,"cache_hit":116224,"new_prefill":435,"score_linear":-116224.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":115123,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-116224.0,"chosen_score_lmetric":0,"t_first_token":340762.180937952,"t_first_token_unix":1779771627.1948814,"t_done":340762.942865043,"t_done_unix":1779771627.9568086},{"request_id":"1305906:7:1851943:1109","session_id":"1305906","input_length":58042,"t_proxy_recv":340764.476371936,"t_decision_unix":1779771629.4903135,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9438682333482651,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (3258 < 8000)","cache_hit":54784,"estimated_new_tokens":3258,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2045,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":54784,"new_prefill":3258,"score_linear":-54784.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":340765.77118911,"t_first_token_unix":1779771630.785133,"t_done":340768.047411741,"t_done_unix":1779771633.061355},{"request_id":"1305906:8:1866131:1110","session_id":"1305906","input_length":58687,"t_proxy_recv":340812.946526886,"t_decision_unix":1779771677.9604683,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9858401349532264,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (831 < 8000)","cache_hit":57856,"estimated_new_tokens":831,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2045,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2436,"cache_hit":57856,"new_prefill":831,"score_linear":-57856.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":340813.278652253,"t_first_token_unix":1779771678.2925963,"t_done":340815.153774753,"t_done_unix":1779771680.1677182},{"request_id":"1270606:11:1868274:1111","session_id":"1270606","input_length":117828,"t_proxy_recv":340820.228669081,"t_decision_unix":1779771685.2426105,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":116224,"affinity_cache_ratio":0.9863869368910615,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1604 < 8000)","cache_hit":116224,"estimated_new_tokens":1604,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2045,"cache_hit":116224,"new_prefill":1604,"score_linear":-116224.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2438,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":116292,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-116224.0,"chosen_score_lmetric":0,"t_first_token":340821.239611108,"t_first_token_unix":1779771686.2535546,"t_done":340822.98643024,"t_done_unix":1779771688.0003738},{"request_id":"1270606:11:1870810:1113","session_id":"1270606","input_length":118012,"t_proxy_recv":340828.721209738,"t_decision_unix":1779771693.735151,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":117760,"affinity_cache_ratio":0.9978646239365488,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (252 < 8000)","cache_hit":117760,"estimated_new_tokens":252,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":118012,"score_linear":112926.0,"score_lmetric":118012},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2048,"cache_hit":117760,"new_prefill":252,"score_linear":-117760.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2438,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":116476,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-117760.0,"chosen_score_lmetric":0,"t_first_token":340828.895755753,"t_first_token_unix":1779771693.909699,"t_done":340829.411583697,"t_done_unix":1779771694.425527},{"request_id":"1356885:12:1874205:1114","session_id":"1356885","input_length":135435,"t_proxy_recv":340840.184490591,"t_decision_unix":1779771705.1984317,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":134656,"affinity_cache_ratio":0.9942481633255805,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (779 < 8000)","cache_hit":134656,"estimated_new_tokens":779,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":13067,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":135435,"score_linear":112926.0,"score_lmetric":135435},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":134656,"new_prefill":779,"score_linear":-134656.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2048,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2438,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":128779,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-134656.0,"chosen_score_lmetric":0,"t_first_token":340840.852545714,"t_first_token_unix":1779771705.8664894,"t_done":340843.416725421,"t_done_unix":1779771708.4306693},{"request_id":"1340290:17:1875101:1115","session_id":"1340290","input_length":40328,"t_proxy_recv":340843.116037564,"t_decision_unix":1779771708.1299787,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9902797064074589,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (392 < 8000)","cache_hit":39936,"estimated_new_tokens":392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":40328,"score_linear":112926.0,"score_lmetric":40328},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":135435,"ongoing_decode_tokens":135435,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":0,"new_prefill":40328,"score_linear":135435.0,"score_lmetric":40328},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2048,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2438,"cache_hit":39936,"new_prefill":392,"score_linear":-39936.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":39304,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":340843.191475309,"t_first_token_unix":1779771708.2054188,"t_done":340846.324693914,"t_done_unix":1779771711.3386378},{"request_id":"1270606:12:1876851:1116","session_id":"1270606","input_length":118600,"t_proxy_recv":340849.080532484,"t_decision_unix":1779771714.094474,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":117760,"affinity_cache_ratio":0.9929173693086003,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (840 < 8000)","cache_hit":117760,"estimated_new_tokens":840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":118600,"score_linear":112926.0,"score_lmetric":118600},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2048,"cache_hit":117760,"new_prefill":840,"score_linear":-117760.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2439,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":117064,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-117760.0,"chosen_score_lmetric":0,"t_first_token":340849.670795465,"t_first_token_unix":1779771714.6847389,"t_done":340851.839275104,"t_done_unix":1779771716.8532186},{"request_id":"1340290:18:1878955:1117","session_id":"1340290","input_length":40583,"t_proxy_recv":340856.791796304,"t_decision_unix":1779771721.8057375,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9966734839711209,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (135 < 8000)","cache_hit":40448,"estimated_new_tokens":135,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":40583,"score_linear":112926.0,"score_lmetric":40583},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2050,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2439,"cache_hit":40448,"new_prefill":135,"score_linear":-40448.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":39559,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":340856.857015477,"t_first_token_unix":1779771721.870959,"t_done":340857.070641021,"t_done_unix":1779771722.0845842},{"request_id":"1340290:19:1880565:1118","session_id":"1340290","input_length":40676,"t_proxy_recv":340862.064331449,"t_decision_unix":1779771727.0782728,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9943947290785722,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (228 < 8000)","cache_hit":40448,"estimated_new_tokens":228,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":40676,"score_linear":112926.0,"score_lmetric":40676},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2050,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2439,"cache_hit":40448,"new_prefill":228,"score_linear":-40448.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":39652,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":340862.167558695,"t_first_token_unix":1779771727.1815016,"t_done":340862.732020531,"t_done_unix":1779771727.7459643},{"request_id":"1340290:20:1882251:1120","session_id":"1340290","input_length":40883,"t_proxy_recv":340867.875625359,"t_decision_unix":1779771732.8895664,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9893598806349827,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (435 < 8000)","cache_hit":40448,"estimated_new_tokens":435,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":40883,"score_linear":112926.0,"score_lmetric":40883},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119030,"ongoing_decode_tokens":119030,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2050,"cache_hit":0,"new_prefill":40883,"score_linear":119030.0,"score_lmetric":40883},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2439,"cache_hit":40448,"new_prefill":435,"score_linear":-40448.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":39859,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":340867.991969022,"t_first_token_unix":1779771733.005913,"t_done":340868.321249403,"t_done_unix":1779771733.3351934},{"request_id":"1270606:12:1881189:1119","session_id":"1270606","input_length":119030,"t_proxy_recv":340864.480306003,"t_decision_unix":1779771729.4942472,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":118784,"affinity_cache_ratio":0.9979332941275308,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (246 < 8000)","cache_hit":118784,"estimated_new_tokens":246,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":119030,"score_linear":112926.0,"score_lmetric":119030},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2050,"cache_hit":118784,"new_prefill":246,"score_linear":-118784.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2439,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":117494,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-118784.0,"chosen_score_lmetric":0,"t_first_token":340864.845248557,"t_first_token_unix":1779771729.8591926,"t_done":340868.889189453,"t_done_unix":1779771733.9031332},{"request_id":"1340290:21:1883855:1121","session_id":"1340290","input_length":41002,"t_proxy_recv":340873.343426912,"t_decision_unix":1779771738.3573682,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":40448,"affinity_cache_ratio":0.986488463977367,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (554 < 8000)","cache_hit":40448,"estimated_new_tokens":554,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":41002,"score_linear":112926.0,"score_lmetric":41002},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2439,"cache_hit":40448,"new_prefill":554,"score_linear":-40448.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":39978,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":340873.463046495,"t_first_token_unix":1779771738.4769897,"t_done":340873.711566867,"t_done_unix":1779771738.7255104},{"request_id":"1340290:22:1884847:1122","session_id":"1340290","input_length":41909,"t_proxy_recv":340876.815939559,"t_decision_unix":1779771741.8298807,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":40960,"affinity_cache_ratio":0.9773556992531437,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (949 < 8000)","cache_hit":40960,"estimated_new_tokens":949,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":41909,"score_linear":112926.0,"score_lmetric":41909},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2440,"cache_hit":40960,"new_prefill":949,"score_linear":-40960.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":40885,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40960.0,"chosen_score_lmetric":0,"t_first_token":340877.14653031,"t_first_token_unix":1779771742.1604738,"t_done":340877.489405543,"t_done_unix":1779771742.5033486},{"request_id":"1253804:55:1869941:1112","session_id":"1253804","input_length":112926,"t_proxy_recv":340825.855517065,"t_decision_unix":1779771690.8694582,"policy":"unified_v2","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":112640,"affinity_cache_ratio":0.9974673680109098,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (286 < 8000)","cache_hit":112640,"estimated_new_tokens":286,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":112640,"new_prefill":286,"score_linear":-112640.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3146,"cache_hit":3584,"new_prefill":109342,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":3584,"new_prefill":109342,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2048,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2438,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-112640.0,"chosen_score_lmetric":0,"t_first_token":340826.019591072,"t_first_token_unix":1779771691.0335343,"t_done":340879.060588376,"t_done_unix":1779771744.0745323},{"request_id":"1340290:23:1886013:1124","session_id":"1340290","input_length":42086,"t_proxy_recv":340880.846519124,"t_decision_unix":1779771745.8604605,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":41472,"affinity_cache_ratio":0.9854108254526446,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (614 < 8000)","cache_hit":41472,"estimated_new_tokens":614,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63845,"ongoing_decode_tokens":0,"pending_prefill_tokens":5477,"num_requests":1,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":42086,"score_linear":63845.0,"score_lmetric":47563},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2441,"cache_hit":41472,"new_prefill":614,"score_linear":-41472.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":41062,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41472.0,"chosen_score_lmetric":0,"t_first_token":340880.959738615,"t_first_token_unix":1779771745.973682,"t_done":340881.259489322,"t_done_unix":1779771746.2734327},{"request_id":"1270606:12:1886765:1125","session_id":"1270606","input_length":119440,"t_proxy_recv":340883.591032938,"t_decision_unix":1779771748.604974,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119296,"affinity_cache_ratio":0.9987943737441393,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (144 < 8000)","cache_hit":119296,"estimated_new_tokens":144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63845,"ongoing_decode_tokens":63845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":119440,"score_linear":63845.0,"score_lmetric":119440},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":119296,"new_prefill":144,"score_linear":-119296.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2442,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":117904,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119296.0,"chosen_score_lmetric":0,"t_first_token":340883.82305008,"t_first_token_unix":1779771748.8369932,"t_done":340884.882746168,"t_done_unix":1779771749.8966897},{"request_id":"1340290:24:1887125:1126","session_id":"1340290","input_length":42199,"t_proxy_recv":340884.620987069,"t_decision_unix":1779771749.6349285,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9949050925377378,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (215 < 8000)","cache_hit":41984,"estimated_new_tokens":215,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63845,"ongoing_decode_tokens":63845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":42199,"score_linear":63845.0,"score_lmetric":42199},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119440,"ongoing_decode_tokens":119440,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":0,"new_prefill":42199,"score_linear":119440.0,"score_lmetric":42199},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2442,"cache_hit":41984,"new_prefill":215,"score_linear":-41984.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":41175,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":340884.730653587,"t_first_token_unix":1779771749.7445974,"t_done":340884.903270517,"t_done_unix":1779771749.9172142},{"request_id":"1340290:25:1888447:1127","session_id":"1340290","input_length":42444,"t_proxy_recv":340889.015571364,"t_decision_unix":1779771754.0295122,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9891621901800018,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (460 < 8000)","cache_hit":41984,"estimated_new_tokens":460,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":63845,"ongoing_decode_tokens":63845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":0,"new_prefill":42444,"score_linear":63845.0,"score_lmetric":42444},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2442,"cache_hit":41984,"new_prefill":460,"score_linear":-41984.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":41420,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":340889.170519584,"t_first_token_unix":1779771754.1844633,"t_done":340889.47708864,"t_done_unix":1779771754.4910321},{"request_id":"1277428:20:1885419:1123","session_id":"1277428","input_length":63845,"t_proxy_recv":340878.731141272,"t_decision_unix":1779771743.7450826,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9142141123032343,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5477 < 8000)","cache_hit":58368,"estimated_new_tokens":5477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":4608,"new_prefill":59237,"score_linear":108318.0,"score_lmetric":59237},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3147,"cache_hit":58368,"new_prefill":5477,"score_linear":-58368.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":27493,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2441,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":340880.951224002,"t_first_token_unix":1779771745.9651673,"t_done":340891.160683245,"t_done_unix":1779771756.174629},{"request_id":"1340290:26:1889402:1128","session_id":"1340290","input_length":48504,"t_proxy_recv":340892.334077924,"t_decision_unix":1779771757.3480191,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":41984,"affinity_cache_ratio":0.8655780966518225,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (6520 < 8000)","cache_hit":41984,"estimated_new_tokens":6520,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3159,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2442,"cache_hit":41984,"new_prefill":6520,"score_linear":-41984.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":47480,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":340894.221712122,"t_first_token_unix":1779771759.2356558,"t_done":340894.431818813,"t_done_unix":1779771759.4457624},{"request_id":"1270606:13:1890252:1129","session_id":"1270606","input_length":119686,"t_proxy_recv":340895.317615508,"t_decision_unix":1779771760.3315568,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119296,"affinity_cache_ratio":0.9967414735223836,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (390 < 8000)","cache_hit":119296,"estimated_new_tokens":390,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3159,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":119296,"new_prefill":390,"score_linear":-119296.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2454,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":118150,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119296.0,"chosen_score_lmetric":0,"t_first_token":340895.675340038,"t_first_token_unix":1779771760.6892843,"t_done":340896.6092267,"t_done_unix":1779771761.623171},{"request_id":"1340290:27:1892591:1131","session_id":"1340290","input_length":49361,"t_proxy_recv":340902.921935341,"t_decision_unix":1779771767.9358766,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9750207653815766,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1233 < 8000)","cache_hit":48128,"estimated_new_tokens":1233,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3159,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":119828,"ongoing_decode_tokens":119828,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":0,"new_prefill":49361,"score_linear":119828.0,"score_lmetric":49361},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2454,"cache_hit":48128,"new_prefill":1233,"score_linear":-48128.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":48337,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48128.0,"chosen_score_lmetric":0,"t_first_token":340903.291899925,"t_first_token_unix":1779771768.3058434,"t_done":340903.620563079,"t_done_unix":1779771768.6345062},{"request_id":"1270606:14:1892412:1130","session_id":"1270606","input_length":119828,"t_proxy_recv":340902.264670867,"t_decision_unix":1779771767.2786117,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119296,"affinity_cache_ratio":0.9955603031011115,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (532 < 8000)","cache_hit":119296,"estimated_new_tokens":532,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3159,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2051,"cache_hit":119296,"new_prefill":532,"score_linear":-119296.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2454,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":118292,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119296.0,"chosen_score_lmetric":0,"t_first_token":340902.518615775,"t_first_token_unix":1779771767.5325594,"t_done":340904.496745443,"t_done_unix":1779771769.51069},{"request_id":"1340290:28:1893837:1132","session_id":"1340290","input_length":49454,"t_proxy_recv":340907.231430432,"t_decision_unix":1779771772.2453713,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9938933149997978,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (302 < 8000)","cache_hit":49152,"estimated_new_tokens":302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3159,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2052,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2456,"cache_hit":49152,"new_prefill":302,"score_linear":-49152.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":48430,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":340907.341830559,"t_first_token_unix":1779771772.3557744,"t_done":340907.92542719,"t_done_unix":1779771772.9393709},{"request_id":"1305906:9:1893880:1133","session_id":"1305906","input_length":59265,"t_proxy_recv":340907.400985718,"t_decision_unix":1779771772.414927,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9935037543237999,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (385 < 8000)","cache_hit":58880,"estimated_new_tokens":385,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3159,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2052,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":49454,"ongoing_decode_tokens":49454,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2456,"cache_hit":58880,"new_prefill":385,"score_linear":-9426.0,"score_lmetric":385},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9426.0,"chosen_score_lmetric":385,"t_first_token":340907.709663416,"t_first_token_unix":1779771772.7236073,"t_done":340908.405569314,"t_done_unix":1779771773.419513},{"request_id":"1270606:15:1895310:1134","session_id":"1270606","input_length":120058,"t_proxy_recv":340912.067565408,"t_decision_unix":1779771777.0815065,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119808,"affinity_cache_ratio":0.9979176731246564,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (250 < 8000)","cache_hit":119808,"estimated_new_tokens":250,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3159,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2052,"cache_hit":119808,"new_prefill":250,"score_linear":-119808.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2456,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":118522,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119808.0,"chosen_score_lmetric":0,"t_first_token":340912.322587975,"t_first_token_unix":1779771777.3365312,"t_done":340913.315299191,"t_done_unix":1779771778.3292432},{"request_id":"1340290:29:1896119:1136","session_id":"1340290","input_length":49576,"t_proxy_recv":340914.555596571,"t_decision_unix":1779771779.5695379,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9914474745844764,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (424 < 8000)","cache_hit":49152,"estimated_new_tokens":424,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":135905,"ongoing_decode_tokens":135905,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":3159,"cache_hit":0,"new_prefill":49576,"score_linear":135905.0,"score_lmetric":49576},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2052,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2456,"cache_hit":49152,"new_prefill":424,"score_linear":-49152.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":48552,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":340914.66412566,"t_first_token_unix":1779771779.6780698,"t_done":340914.915360311,"t_done_unix":1779771779.9293041},{"request_id":"1356885:13:1895311:1135","session_id":"1356885","input_length":135905,"t_proxy_recv":340912.080089825,"t_decision_unix":1779771777.0940316,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":135168,"affinity_cache_ratio":0.9945770942938081,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (737 < 8000)","cache_hit":135168,"estimated_new_tokens":737,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":13537,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3159,"cache_hit":135168,"new_prefill":737,"score_linear":-135168.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":120058,"ongoing_decode_tokens":0,"pending_prefill_tokens":250,"num_requests":1,"active_p_offloads":0,"cached_blocks":2052,"cache_hit":0,"new_prefill":135905,"score_linear":120058.0,"score_lmetric":136155},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2456,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":129249,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-135168.0,"chosen_score_lmetric":0,"t_first_token":340912.50586262,"t_first_token_unix":1779771777.5198064,"t_done":340914.953931844,"t_done_unix":1779771779.9678762},{"request_id":"1340290:30:1896991:1137","session_id":"1340290","input_length":49695,"t_proxy_recv":340917.529660709,"t_decision_unix":1779771782.5436018,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9890733474192575,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (543 < 8000)","cache_hit":49152,"estimated_new_tokens":543,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2052,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2456,"cache_hit":49152,"new_prefill":543,"score_linear":-49152.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":48671,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":340917.640503458,"t_first_token_unix":1779771782.6544468,"t_done":340919.159058382,"t_done_unix":1779771784.1730044},{"request_id":"1340290:31:1899492:1138","session_id":"1340290","input_length":49927,"t_proxy_recv":340925.913239796,"t_decision_unix":1779771790.9271808,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9947323091713902,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (263 < 8000)","cache_hit":49664,"estimated_new_tokens":263,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2052,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2457,"cache_hit":49664,"new_prefill":263,"score_linear":-49664.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":48903,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":340926.007176548,"t_first_token_unix":1779771791.0211198,"t_done":340926.673144009,"t_done_unix":1779771791.6870875},{"request_id":"1270606:16:1902252:1140","session_id":"1270606","input_length":120572,"t_proxy_recv":340934.42246221,"t_decision_unix":1779771799.4364028,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":119808,"affinity_cache_ratio":0.9936635371396344,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (764 < 8000)","cache_hit":119808,"estimated_new_tokens":764,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2052,"cache_hit":119808,"new_prefill":764,"score_linear":-119808.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50078,"ongoing_decode_tokens":50078,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2457,"cache_hit":0,"new_prefill":120572,"score_linear":50078.0,"score_lmetric":120572},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":119036,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-119808.0,"chosen_score_lmetric":0,"t_first_token":340935.013458153,"t_first_token_unix":1779771800.0274022,"t_done":340935.97814776,"t_done_unix":1779771800.9920926},{"request_id":"1270606:17:1908800:1141","session_id":"1270606","input_length":121006,"t_proxy_recv":340956.009840205,"t_decision_unix":1779771821.0237813,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":120320,"affinity_cache_ratio":0.9943308596267953,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (686 < 8000)","cache_hit":120320,"estimated_new_tokens":686,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2053,"cache_hit":120320,"new_prefill":686,"score_linear":-120320.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50078,"ongoing_decode_tokens":50078,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2457,"cache_hit":0,"new_prefill":121006,"score_linear":50078.0,"score_lmetric":121006},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":119470,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-120320.0,"chosen_score_lmetric":0,"t_first_token":340956.601676827,"t_first_token_unix":1779771821.6156204,"t_done":340957.226219874,"t_done_unix":1779771822.2401636},{"request_id":"1340290:32:1901260:1139","session_id":"1340290","input_length":50078,"t_proxy_recv":340931.323565136,"t_decision_unix":1779771796.3375058,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9917328966811774,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (414 < 8000)","cache_hit":49664,"estimated_new_tokens":414,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2052,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2457,"cache_hit":49664,"new_prefill":414,"score_linear":-49664.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":49054,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":340931.446745525,"t_first_token_unix":1779771796.460689,"t_done":340962.149158465,"t_done_unix":1779771827.163102},{"request_id":"1270606:18:1915824:1142","session_id":"1270606","input_length":121288,"t_proxy_recv":340978.990459698,"t_decision_unix":1779771844.004401,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":120832,"affinity_cache_ratio":0.9962403535386848,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (456 < 8000)","cache_hit":120832,"estimated_new_tokens":456,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2054,"cache_hit":120832,"new_prefill":456,"score_linear":-120832.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2458,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":119752,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-120832.0,"chosen_score_lmetric":0,"t_first_token":340979.365657456,"t_first_token_unix":1779771844.379601,"t_done":340980.24816231,"t_done_unix":1779771845.262106},{"request_id":"1270606:19:1920499:1143","session_id":"1270606","input_length":121560,"t_proxy_recv":340994.144237126,"t_decision_unix":1779771859.1581786,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":121344,"affinity_cache_ratio":0.9982230997038499,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (216 < 8000)","cache_hit":121344,"estimated_new_tokens":216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":121344,"new_prefill":216,"score_linear":-121344.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2458,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":120024,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-121344.0,"chosen_score_lmetric":0,"t_first_token":340994.513017424,"t_first_token_unix":1779771859.5269618,"t_done":340995.501857277,"t_done_unix":1779771860.5158014},{"request_id":"1270606:20:1922639:1144","session_id":"1270606","input_length":121704,"t_proxy_recv":341001.600449196,"t_decision_unix":1779771866.6143901,"policy":"unified_v2","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":121344,"affinity_cache_ratio":0.9970420035495957,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (360 < 8000)","cache_hit":121344,"estimated_new_tokens":360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":121344,"new_prefill":360,"score_linear":-121344.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2458,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1536,"new_prefill":120168,"score_linear":-1536.0,"score_lmetric":0}],"chosen_score_linear":-121344.0,"chosen_score_lmetric":0,"t_first_token":341001.837020833,"t_first_token_unix":1779771866.8509648,"t_done":341003.164677717,"t_done_unix":1779771868.178622},{"request_id":"1305906:10:1928316:1145","session_id":"1305906","input_length":61801,"t_proxy_recv":341020.149716168,"t_decision_unix":1779771885.1636546,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9527353926311872,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2921 < 8000)","cache_hit":58880,"estimated_new_tokens":2921,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2458,"cache_hit":58880,"new_prefill":2921,"score_linear":-58880.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":341021.189969047,"t_first_token_unix":1779771886.2039137,"t_done":341021.545763957,"t_done_unix":1779771886.5597074},{"request_id":"1340290:33:1939781:1146","session_id":"1340290","input_length":53990,"t_proxy_recv":341057.900594562,"t_decision_unix":1779771922.9145358,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":53760,"affinity_cache_ratio":0.9957399518429338,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (230 < 8000)","cache_hit":53760,"estimated_new_tokens":230,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2463,"cache_hit":53760,"new_prefill":230,"score_linear":-53760.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":52966,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":341057.99851909,"t_first_token_unix":1779771923.0124629,"t_done":341059.100076998,"t_done_unix":1779771924.1140206},{"request_id":"1340290:34:1942249:1147","session_id":"1340290","input_length":54164,"t_proxy_recv":341065.738354581,"t_decision_unix":1779771930.7522957,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":53760,"affinity_cache_ratio":0.992541171257662,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (404 < 8000)","cache_hit":53760,"estimated_new_tokens":404,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2463,"cache_hit":53760,"new_prefill":404,"score_linear":-53760.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":53140,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":341065.834571202,"t_first_token_unix":1779771930.8485146,"t_done":341068.317116816,"t_done_unix":1779771933.3310602},{"request_id":"1356885:14:1948151:1148","session_id":"1356885","input_length":136210,"t_proxy_recv":341085.510733794,"t_decision_unix":1779771950.5246751,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":135680,"affinity_cache_ratio":0.9961089494163424,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (530 < 8000)","cache_hit":135680,"estimated_new_tokens":530,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":13842,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3160,"cache_hit":135680,"new_prefill":530,"score_linear":-135680.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2464,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":129554,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-135680.0,"chosen_score_lmetric":0,"t_first_token":341085.839082927,"t_first_token_unix":1779771950.8530266,"t_done":341087.557908813,"t_done_unix":1779771952.5718524},{"request_id":"1340290:35:1964727:1149","session_id":"1340290","input_length":54498,"t_proxy_recv":341140.214175193,"t_decision_unix":1779772005.2281163,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":54272,"affinity_cache_ratio":0.9958530588278469,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (226 < 8000)","cache_hit":54272,"estimated_new_tokens":226,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3161,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2464,"cache_hit":54272,"new_prefill":226,"score_linear":-54272.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":53474,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54272.0,"chosen_score_lmetric":0,"t_first_token":341140.320736411,"t_first_token_unix":1779772005.3346796,"t_done":341144.132985118,"t_done_unix":1779772009.1469283},{"request_id":"1340290:36:1969509:1150","session_id":"1340290","input_length":54811,"t_proxy_recv":341155.978092143,"t_decision_unix":1779772020.9920335,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9995073981500063,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (27 < 8000)","cache_hit":54784,"estimated_new_tokens":27,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3161,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2465,"cache_hit":54784,"new_prefill":27,"score_linear":-54784.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":53787,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":341156.059278914,"t_first_token_unix":1779772021.0732222,"t_done":341156.518481577,"t_done_unix":1779772021.5324252},{"request_id":"1340290:37:1971106:1151","session_id":"1340290","input_length":55358,"t_proxy_recv":341161.121723127,"t_decision_unix":1779772026.1356645,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9896311282922071,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (574 < 8000)","cache_hit":54784,"estimated_new_tokens":574,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3161,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2465,"cache_hit":54784,"new_prefill":574,"score_linear":-54784.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":54334,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":341161.41876985,"t_first_token_unix":1779772026.4327133,"t_done":341161.885369447,"t_done_unix":1779772026.899313},{"request_id":"1340290:38:1972289:1152","session_id":"1340290","input_length":56006,"t_proxy_recv":341165.050479312,"t_decision_unix":1779772030.0644207,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":55296,"affinity_cache_ratio":0.9873227868442667,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (710 < 8000)","cache_hit":55296,"estimated_new_tokens":710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3161,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2466,"cache_hit":55296,"new_prefill":710,"score_linear":-55296.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":54982,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55296.0,"chosen_score_lmetric":0,"t_first_token":341165.364677428,"t_first_token_unix":1779772030.3786209,"t_done":341166.03269841,"t_done_unix":1779772031.0466418},{"request_id":"1340290:39:1973889:1153","session_id":"1340290","input_length":56242,"t_proxy_recv":341170.476670917,"t_decision_unix":1779772035.4906125,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":55808,"affinity_cache_ratio":0.9922833469649017,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (434 < 8000)","cache_hit":55808,"estimated_new_tokens":434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3161,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2467,"cache_hit":55808,"new_prefill":434,"score_linear":-55808.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":55218,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":341170.664863211,"t_first_token_unix":1779772035.6788068,"t_done":341171.334385975,"t_done_unix":1779772036.3483298},{"request_id":"1340290:40:1975278:1154","session_id":"1340290","input_length":56416,"t_proxy_recv":341175.036344766,"t_decision_unix":1779772040.050286,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":56320,"affinity_cache_ratio":0.998298355076574,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (96 < 8000)","cache_hit":56320,"estimated_new_tokens":96,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3161,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2468,"cache_hit":56320,"new_prefill":96,"score_linear":-56320.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":55392,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-56320.0,"chosen_score_lmetric":0,"t_first_token":341175.157669199,"t_first_token_unix":1779772040.1716125,"t_done":341175.646976982,"t_done_unix":1779772040.6609204},{"request_id":"1340290:41:1977055:1156","session_id":"1340290","input_length":57181,"t_proxy_recv":341180.974247688,"t_decision_unix":1779772045.9881895,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":56320,"affinity_cache_ratio":0.9849425508473094,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (861 < 8000)","cache_hit":56320,"estimated_new_tokens":861,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":67793,"ongoing_decode_tokens":0,"pending_prefill_tokens":2769,"num_requests":1,"active_p_offloads":0,"cached_blocks":3161,"cache_hit":0,"new_prefill":57181,"score_linear":67793.0,"score_lmetric":59950},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2468,"cache_hit":56320,"new_prefill":861,"score_linear":-56320.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":56157,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-56320.0,"chosen_score_lmetric":0,"t_first_token":341181.387248264,"t_first_token_unix":1779772046.401192,"t_done":341183.383779392,"t_done_unix":1779772048.3977227},{"request_id":"1277428:21:1976968:1155","session_id":"1277428","input_length":67793,"t_proxy_recv":341180.691641367,"t_decision_unix":1779772045.7055829,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9591550750077442,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2769 < 8000)","cache_hit":65024,"estimated_new_tokens":2769,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":4608,"new_prefill":63185,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3161,"cache_hit":65024,"new_prefill":2769,"score_linear":-65024.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":31441,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2468,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":341181.94807405,"t_first_token_unix":1779772046.9620175,"t_done":341187.879801462,"t_done_unix":1779772052.8937452},{"request_id":"1340290:42:1982198:1157","session_id":"1340290","input_length":57463,"t_proxy_recv":341197.881913241,"t_decision_unix":1779772062.8958545,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9979291022048971,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (119 < 8000)","cache_hit":57344,"estimated_new_tokens":119,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3167,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":57344,"new_prefill":119,"score_linear":-57344.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":56439,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":341197.983435532,"t_first_token_unix":1779772062.997379,"t_done":341198.480191706,"t_done_unix":1779772063.4941356},{"request_id":"1340290:43:1983901:1158","session_id":"1340290","input_length":57640,"t_proxy_recv":341203.273745566,"t_decision_unix":1779772068.2876868,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9948646773074254,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (296 < 8000)","cache_hit":57344,"estimated_new_tokens":296,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3167,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":57344,"new_prefill":296,"score_linear":-57344.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":56616,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":341203.427053692,"t_first_token_unix":1779772068.4409971,"t_done":341203.951638758,"t_done_unix":1779772068.9655824},{"request_id":"1340290:44:1986230:1159","session_id":"1340290","input_length":57809,"t_proxy_recv":341211.356778133,"t_decision_unix":1779772076.3707194,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9919562697849815,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (465 < 8000)","cache_hit":57344,"estimated_new_tokens":465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3167,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2470,"cache_hit":57344,"new_prefill":465,"score_linear":-57344.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":56785,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":341211.513634437,"t_first_token_unix":1779772076.527578,"t_done":341212.234868868,"t_done_unix":1779772077.2488122},{"request_id":"1277428:21:1989132:1160","session_id":"1277428","input_length":69987,"t_proxy_recv":341220.775761372,"t_decision_unix":1779772085.789703,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9729806964150485,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1891 < 8000)","cache_hit":68096,"estimated_new_tokens":1891,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":4608,"new_prefill":65379,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3167,"cache_hit":68096,"new_prefill":1891,"score_linear":-68096.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":33635,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":341221.547726201,"t_first_token_unix":1779772086.5616698,"t_done":341221.902081373,"t_done_unix":1779772086.9160252},{"request_id":"1277428:22:1990436:1161","session_id":"1277428","input_length":70055,"t_proxy_recv":341225.315841004,"t_decision_unix":1779772090.3297822,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":69632,"affinity_cache_ratio":0.993961887088716,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (423 < 8000)","cache_hit":69632,"estimated_new_tokens":423,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":4608,"new_prefill":65447,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3170,"cache_hit":69632,"new_prefill":423,"score_linear":-69632.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":36352,"new_prefill":33703,"score_linear":-36352.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69632.0,"chosen_score_lmetric":0,"t_first_token":341225.430508571,"t_first_token_unix":1779772090.444452,"t_done":341225.849208566,"t_done_unix":1779772090.8631516},{"request_id":"1340290:45:1991694:1162","session_id":"1340290","input_length":58053,"t_proxy_recv":341229.330099972,"t_decision_unix":1779772094.344041,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9966065491878111,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (197 < 8000)","cache_hit":57856,"estimated_new_tokens":197,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3170,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":57856,"new_prefill":197,"score_linear":-57856.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":57029,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":341229.526010946,"t_first_token_unix":1779772094.5399551,"t_done":341229.781196646,"t_done_unix":1779772094.7951403},{"request_id":"1340290:46:1992980:1163","session_id":"1340290","input_length":58122,"t_proxy_recv":341233.553701296,"t_decision_unix":1779772098.5676427,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9954234197033826,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (266 < 8000)","cache_hit":57856,"estimated_new_tokens":266,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3170,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":57856,"new_prefill":266,"score_linear":-57856.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":57098,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":341233.653245014,"t_first_token_unix":1779772098.6671886,"t_done":341234.077773776,"t_done_unix":1779772099.091717},{"request_id":"1340290:47:1995358:1164","session_id":"1340290","input_length":58307,"t_proxy_recv":341241.418144965,"t_decision_unix":1779772106.4320862,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9922650796645343,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (451 < 8000)","cache_hit":57856,"estimated_new_tokens":451,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3170,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":57856,"new_prefill":451,"score_linear":-57856.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":57283,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":341241.563451084,"t_first_token_unix":1779772106.5773945,"t_done":341241.820570695,"t_done_unix":1779772106.8345141},{"request_id":"1340290:48:1996945:1165","session_id":"1340290","input_length":58339,"t_proxy_recv":341246.722709636,"t_decision_unix":1779772111.736651,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9917208042647285,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (483 < 8000)","cache_hit":57856,"estimated_new_tokens":483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3170,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2471,"cache_hit":57856,"new_prefill":483,"score_linear":-57856.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":57315,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":341246.802615653,"t_first_token_unix":1779772111.816559,"t_done":341248.846600857,"t_done_unix":1779772113.8605442},{"request_id":"1356885:15:1998292:1166","session_id":"1356885","input_length":136653,"t_proxy_recv":341251.191816734,"t_decision_unix":1779772116.2057586,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":136192,"affinity_cache_ratio":0.9966264919174844,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (461 < 8000)","cache_hit":136192,"estimated_new_tokens":461,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":14285,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3170,"cache_hit":136192,"new_prefill":461,"score_linear":-136192.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2472,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":129997,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-136192.0,"chosen_score_lmetric":0,"t_first_token":341251.844202662,"t_first_token_unix":1779772116.858146,"t_done":341253.536483853,"t_done_unix":1779772118.5504282},{"request_id":"1340290:48:1999591:1167","session_id":"1340290","input_length":58622,"t_proxy_recv":341255.739252074,"t_decision_unix":1779772120.753193,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9956671556753437,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (254 < 8000)","cache_hit":58368,"estimated_new_tokens":254,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2472,"cache_hit":58368,"new_prefill":254,"score_linear":-58368.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":57598,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":341255.843175277,"t_first_token_unix":1779772120.8571184,"t_done":341256.449551675,"t_done_unix":1779772121.4634953},{"request_id":"1340290:48:2001313:1168","session_id":"1340290","input_length":58808,"t_proxy_recv":341261.201528506,"t_decision_unix":1779772126.2154694,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9925180247585362,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (440 < 8000)","cache_hit":58368,"estimated_new_tokens":440,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2472,"cache_hit":58368,"new_prefill":440,"score_linear":-58368.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":57784,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":341261.36038656,"t_first_token_unix":1779772126.37433,"t_done":341261.8824298,"t_done_unix":1779772126.8963761},{"request_id":"1340290:48:2003195:1169","session_id":"1340290","input_length":58977,"t_proxy_recv":341267.525522831,"t_decision_unix":1779772132.5394642,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58368,"affinity_cache_ratio":0.989673940688743,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (609 < 8000)","cache_hit":58368,"estimated_new_tokens":609,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2472,"cache_hit":58368,"new_prefill":609,"score_linear":-58368.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":57953,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":341267.697656237,"t_first_token_unix":1779772132.7115998,"t_done":341268.580144438,"t_done_unix":1779772133.5940876},{"request_id":"1340290:48:2005449:1170","session_id":"1340290","input_length":59220,"t_proxy_recv":341274.911863651,"t_decision_unix":1779772139.925805,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58880,"affinity_cache_ratio":0.994258696386356,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (340 < 8000)","cache_hit":58880,"estimated_new_tokens":340,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2473,"cache_hit":58880,"new_prefill":340,"score_linear":-58880.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":58196,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":341275.065917857,"t_first_token_unix":1779772140.0798614,"t_done":341275.917319351,"t_done_unix":1779772140.9312627},{"request_id":"1340290:48:2007287:1171","session_id":"1340290","input_length":59487,"t_proxy_recv":341281.042601892,"t_decision_unix":1779772146.0565434,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9897960899019954,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (607 < 8000)","cache_hit":58880,"estimated_new_tokens":607,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2473,"cache_hit":58880,"new_prefill":607,"score_linear":-58880.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":58463,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":341281.244894251,"t_first_token_unix":1779772146.2588377,"t_done":341281.638011168,"t_done_unix":1779772146.6519544},{"request_id":"1340290:48:2008380:1173","session_id":"1340290","input_length":60935,"t_proxy_recv":341285.146049908,"t_decision_unix":1779772150.159991,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":59392,"affinity_cache_ratio":0.9746779355050463,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1543 < 8000)","cache_hit":59392,"estimated_new_tokens":1543,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66506,"ongoing_decode_tokens":0,"pending_prefill_tokens":5066,"num_requests":1,"active_p_offloads":0,"cached_blocks":2474,"cache_hit":59392,"new_prefill":1543,"score_linear":7114.0,"score_lmetric":6609},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":59911,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":7114.0,"chosen_score_lmetric":6609,"t_first_token":341286.717077052,"t_first_token_unix":1779772151.7310207,"t_done":341287.742165967,"t_done_unix":1779772152.7561095},{"request_id":"1305906:11:2008110:1172","session_id":"1305906","input_length":66506,"t_proxy_recv":341284.140424712,"t_decision_unix":1779772149.1543663,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9238264216762397,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (5066 < 8000)","cache_hit":61440,"estimated_new_tokens":5066,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2474,"cache_hit":61440,"new_prefill":5066,"score_linear":-61440.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":341286.09899451,"t_first_token_unix":1779772151.1129384,"t_done":341289.377741759,"t_done_unix":1779772154.391685},{"request_id":"1340290:48:2010180:1174","session_id":"1340290","input_length":61080,"t_proxy_recv":341291.006856263,"t_decision_unix":1779772156.0207891,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9975114603798297,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (152 < 8000)","cache_hit":60928,"estimated_new_tokens":152,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2487,"cache_hit":60928,"new_prefill":152,"score_linear":-60928.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":60056,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":341291.154591031,"t_first_token_unix":1779772156.1685343,"t_done":341292.193522504,"t_done_unix":1779772157.207466},{"request_id":"1340290:48:2012871:1175","session_id":"1340290","input_length":61365,"t_proxy_recv":341299.682708073,"t_decision_unix":1779772164.6966496,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9928786767701459,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (437 < 8000)","cache_hit":60928,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2487,"cache_hit":60928,"new_prefill":437,"score_linear":-60928.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":60341,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":341299.91447503,"t_first_token_unix":1779772164.9284186,"t_done":341300.424542352,"t_done_unix":1779772165.4384859},{"request_id":"1340290:48:2014153:1176","session_id":"1340290","input_length":61518,"t_proxy_recv":341304.079687757,"t_decision_unix":1779772169.0936291,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9904093110959394,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (590 < 8000)","cache_hit":60928,"estimated_new_tokens":590,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2487,"cache_hit":60928,"new_prefill":590,"score_linear":-60928.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":60494,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":341304.232218524,"t_first_token_unix":1779772169.246162,"t_done":341304.720312606,"t_done_unix":1779772169.734256},{"request_id":"1340290:48:2015982:1177","session_id":"1340290","input_length":69113,"t_proxy_recv":341310.177010718,"t_decision_unix":1779772175.1909523,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":61440,"affinity_cache_ratio":0.8889789185826111,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (7673 < 8000)","cache_hit":61440,"estimated_new_tokens":7673,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2488,"cache_hit":61440,"new_prefill":7673,"score_linear":-61440.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":68089,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":341313.428433007,"t_first_token_unix":1779772178.4423764,"t_done":341315.8842513,"t_done_unix":1779772180.8981948},{"request_id":"1340290:48:2018973:1178","session_id":"1340290","input_length":69426,"t_proxy_recv":341319.963624344,"t_decision_unix":1779772184.9775655,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9955924293492352,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (306 < 8000)","cache_hit":69120,"estimated_new_tokens":306,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-69113,"ongoing_decode_tokens":0,"pending_prefill_tokens":-7673,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":69120,"new_prefill":306,"score_linear":-138233.0,"score_lmetric":7367},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":68402,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-138233.0,"chosen_score_lmetric":7367,"t_first_token":341320.096949573,"t_first_token_unix":1779772185.1108932,"t_done":341320.692040996,"t_done_unix":1779772185.7059846},{"request_id":"1340290:48:2020779:1179","session_id":"1340290","input_length":69601,"t_proxy_recv":341326.017844686,"t_decision_unix":1779772191.031786,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9930891797531645,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (481 < 8000)","cache_hit":69120,"estimated_new_tokens":481,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-69113,"ongoing_decode_tokens":0,"pending_prefill_tokens":-7673,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":69120,"new_prefill":481,"score_linear":-138233.0,"score_lmetric":7192},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":68577,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-138233.0,"chosen_score_lmetric":7192,"t_first_token":341326.203926922,"t_first_token_unix":1779772191.2178702,"t_done":341326.779948212,"t_done_unix":1779772191.7938914},{"request_id":"1340290:48:2023753:1180","session_id":"1340290","input_length":69951,"t_proxy_recv":341335.638736298,"t_decision_unix":1779772200.6526775,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":69632,"affinity_cache_ratio":0.9954396649082929,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (319 < 8000)","cache_hit":69632,"estimated_new_tokens":319,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":-69113,"ongoing_decode_tokens":0,"pending_prefill_tokens":-7673,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2504,"cache_hit":69632,"new_prefill":319,"score_linear":-138745.0,"score_lmetric":7354},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":68927,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-138745.0,"chosen_score_lmetric":7354,"t_first_token":341335.873321252,"t_first_token_unix":1779772200.8872647,"t_done":341336.771662426,"t_done_unix":1779772201.785606},{"request_id":"1340290:48:2026751:1181","session_id":"1340290","input_length":70342,"t_proxy_recv":341345.914456546,"t_decision_unix":1779772210.9283981,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":69632,"affinity_cache_ratio":0.989906457024253,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (710 < 8000)","cache_hit":69632,"estimated_new_tokens":710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2504,"cache_hit":69632,"new_prefill":710,"score_linear":-69632.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":69318,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69632.0,"chosen_score_lmetric":0,"t_first_token":341346.150366997,"t_first_token_unix":1779772211.1643105,"t_done":341346.682559745,"t_done_unix":1779772211.6965034},{"request_id":"1340290:48:2028652:1182","session_id":"1340290","input_length":70503,"t_proxy_recv":341352.166020319,"t_decision_unix":1779772217.1799614,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":70144,"affinity_cache_ratio":0.9949080180985206,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (359 < 8000)","cache_hit":70144,"estimated_new_tokens":359,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2505,"cache_hit":70144,"new_prefill":359,"score_linear":-70144.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":69479,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70144.0,"chosen_score_lmetric":0,"t_first_token":341352.349444045,"t_first_token_unix":1779772217.3633876,"t_done":341353.256957413,"t_done_unix":1779772218.270901},{"request_id":"1340290:48:2033065:1183","session_id":"1340290","input_length":70851,"t_proxy_recv":341366.895392764,"t_decision_unix":1779772231.909334,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":70144,"affinity_cache_ratio":0.9900213123315126,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (707 < 8000)","cache_hit":70144,"estimated_new_tokens":707,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2505,"cache_hit":70144,"new_prefill":707,"score_linear":-70144.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":69827,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70144.0,"chosen_score_lmetric":0,"t_first_token":341367.12990491,"t_first_token_unix":1779772232.1438487,"t_done":341367.779350836,"t_done_unix":1779772232.7932944},{"request_id":"1340290:48:2034908:1184","session_id":"1340290","input_length":71067,"t_proxy_recv":341373.113559502,"t_decision_unix":1779772238.1275008,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":70656,"affinity_cache_ratio":0.9942167250622652,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (411 < 8000)","cache_hit":70656,"estimated_new_tokens":411,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":70656,"new_prefill":411,"score_linear":-70656.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":70043,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70656.0,"chosen_score_lmetric":0,"t_first_token":341373.279586563,"t_first_token_unix":1779772238.2935302,"t_done":341373.803223467,"t_done_unix":1779772238.817167},{"request_id":"1258908:2:2035438:1185","session_id":"1258908","input_length":7844,"t_proxy_recv":341374.925388695,"t_decision_unix":1779772239.93933,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":7680,"affinity_cache_ratio":0.9790922998470168,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (164 < 8000)","cache_hit":7680,"estimated_new_tokens":164,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":7680,"new_prefill":164,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":341374.966216203,"t_first_token_unix":1779772239.9801595,"t_done":341376.004617414,"t_done_unix":1779772241.0185611},{"request_id":"1258908:3:1388683:699","session_id":"1258908","input_length":9120,"t_proxy_recv":341376.008133623,"t_decision_unix":1779772241.0220754,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":7680,"affinity_cache_ratio":0.8421052631578947,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1440 < 8000)","cache_hit":7680,"estimated_new_tokens":1440,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1631,"cache_hit":7680,"new_prefill":1440,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":341376.158246972,"t_first_token_unix":1779772241.1721907,"t_done":341376.843351032,"t_done_unix":1779772241.8572955},{"request_id":"1340290:48:2036994:1186","session_id":"1340290","input_length":71240,"t_proxy_recv":341380.060041679,"t_decision_unix":1779772245.073983,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":70656,"affinity_cache_ratio":0.9918023582257159,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (584 < 8000)","cache_hit":70656,"estimated_new_tokens":584,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2506,"cache_hit":70656,"new_prefill":584,"score_linear":-70656.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":70216,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70656.0,"chosen_score_lmetric":0,"t_first_token":341380.246237436,"t_first_token_unix":1779772245.2601807,"t_done":341380.973684707,"t_done_unix":1779772245.9876282},{"request_id":"1340290:48:2040473:1187","session_id":"1340290","input_length":71545,"t_proxy_recv":341391.586035877,"t_decision_unix":1779772256.5999773,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9947305891397022,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (377 < 8000)","cache_hit":71168,"estimated_new_tokens":377,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":71168,"new_prefill":377,"score_linear":-71168.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":70521,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71168.0,"chosen_score_lmetric":0,"t_first_token":341391.822359978,"t_first_token_unix":1779772256.8363035,"t_done":341392.980570368,"t_done_unix":1779772257.994514},{"request_id":"1340290:48:2043771:1188","session_id":"1340290","input_length":71867,"t_proxy_recv":341402.600262246,"t_decision_unix":1779772267.6142037,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9902737000292207,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (699 < 8000)","cache_hit":71168,"estimated_new_tokens":699,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2507,"cache_hit":71168,"new_prefill":699,"score_linear":-71168.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":70843,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71168.0,"chosen_score_lmetric":0,"t_first_token":341402.835326395,"t_first_token_unix":1779772267.8492699,"t_done":341403.419120003,"t_done_unix":1779772268.4330635},{"request_id":"1340290:48:2045403:1189","session_id":"1340290","input_length":72025,"t_proxy_recv":341408.116060491,"t_decision_unix":1779772273.130002,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":71680,"affinity_cache_ratio":0.995209996528983,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (345 < 8000)","cache_hit":71680,"estimated_new_tokens":345,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":71680,"new_prefill":345,"score_linear":-71680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":71001,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":341408.264747284,"t_first_token_unix":1779772273.2786908,"t_done":341408.95731144,"t_done_unix":1779772273.9712548},{"request_id":"1340290:48:2047167:1190","session_id":"1340290","input_length":72211,"t_proxy_recv":341413.879551627,"t_decision_unix":1779772278.8934927,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9926465496946448,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (531 < 8000)","cache_hit":71680,"estimated_new_tokens":531,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2508,"cache_hit":71680,"new_prefill":531,"score_linear":-71680.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":71187,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":341414.073782357,"t_first_token_unix":1779772279.0877259,"t_done":341414.635808303,"t_done_unix":1779772279.6497521},{"request_id":"1340290:48:2048793:1191","session_id":"1340290","input_length":72777,"t_proxy_recv":341419.311072343,"t_decision_unix":1779772284.3250139,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":72192,"affinity_cache_ratio":0.9919617461560658,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (585 < 8000)","cache_hit":72192,"estimated_new_tokens":585,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":72192,"new_prefill":585,"score_linear":-72192.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":71753,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72192.0,"chosen_score_lmetric":0,"t_first_token":341419.690602977,"t_first_token_unix":1779772284.7045543,"t_done":341420.219205199,"t_done_unix":1779772285.2331488},{"request_id":"1340290:48:2050838:1192","session_id":"1340290","input_length":72952,"t_proxy_recv":341426.53490292,"t_decision_unix":1779772291.5488443,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":72704,"affinity_cache_ratio":0.9966005044412765,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (248 < 8000)","cache_hit":72704,"estimated_new_tokens":248,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":72704,"new_prefill":248,"score_linear":-72704.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":71928,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72704.0,"chosen_score_lmetric":0,"t_first_token":341426.724965311,"t_first_token_unix":1779772291.7389088,"t_done":341427.090265436,"t_done_unix":1779772292.104209},{"request_id":"1340290:48:2052523:1193","session_id":"1340290","input_length":73208,"t_proxy_recv":341432.102310933,"t_decision_unix":1779772297.1162522,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":72704,"affinity_cache_ratio":0.9931155065020216,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (504 < 8000)","cache_hit":72704,"estimated_new_tokens":504,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2510,"cache_hit":72704,"new_prefill":504,"score_linear":-72704.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":72184,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72704.0,"chosen_score_lmetric":0,"t_first_token":341432.338935829,"t_first_token_unix":1779772297.3528793,"t_done":341433.130747731,"t_done_unix":1779772298.1446917},{"request_id":"1340290:48:2055365:1194","session_id":"1340290","input_length":73454,"t_proxy_recv":341441.816482715,"t_decision_unix":1779772306.830424,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9967598769297792,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (238 < 8000)","cache_hit":73216,"estimated_new_tokens":238,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":73216,"new_prefill":238,"score_linear":-73216.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":72430,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":341442.052192313,"t_first_token_unix":1779772307.066136,"t_done":341442.482074946,"t_done_unix":1779772307.4960182},{"request_id":"1340290:48:2057042:1195","session_id":"1340290","input_length":73636,"t_proxy_recv":341447.188505798,"t_decision_unix":1779772312.2024472,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9942962681297192,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (420 < 8000)","cache_hit":73216,"estimated_new_tokens":420,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":73216,"new_prefill":420,"score_linear":-73216.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":72612,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":341447.361791626,"t_first_token_unix":1779772312.3757353,"t_done":341447.636948647,"t_done_unix":1779772312.6508923},{"request_id":"1340290:48:2058614:1196","session_id":"1340290","input_length":73653,"t_proxy_recv":341452.332478789,"t_decision_unix":1779772317.34642,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9940667725686666,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (437 < 8000)","cache_hit":73216,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":73216,"new_prefill":437,"score_linear":-73216.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":72629,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":341452.430341304,"t_first_token_unix":1779772317.4442847,"t_done":341454.202133457,"t_done_unix":1779772319.216077},{"request_id":"1340290:48:2061834:1197","session_id":"1340290","input_length":73858,"t_proxy_recv":341462.479745966,"t_decision_unix":1779772327.4936874,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9982398656882125,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (130 < 8000)","cache_hit":73728,"estimated_new_tokens":130,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2512,"cache_hit":73728,"new_prefill":130,"score_linear":-73728.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":72834,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":341462.592111193,"t_first_token_unix":1779772327.6060543,"t_done":341463.150881796,"t_done_unix":1779772328.1648254},{"request_id":"1258908:3:2063441:1198","session_id":"1258908","input_length":9359,"t_proxy_recv":341467.517651009,"t_decision_unix":1779772332.5315921,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.9847205898066033,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (143 < 8000)","cache_hit":9216,"estimated_new_tokens":143,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":9216,"new_prefill":143,"score_linear":-9216.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2512,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":341467.570560201,"t_first_token_unix":1779772332.5845034,"t_done":341468.657410846,"t_done_unix":1779772333.6713543},{"request_id":"1340290:48:2063837:1199","session_id":"1340290","input_length":74011,"t_proxy_recv":341468.873541345,"t_decision_unix":1779772333.887483,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9961762440718271,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (283 < 8000)","cache_hit":73728,"estimated_new_tokens":283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9740,"ongoing_decode_tokens":9740,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":0,"new_prefill":74011,"score_linear":9740.0,"score_lmetric":74011},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2512,"cache_hit":73728,"new_prefill":283,"score_linear":-73728.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":72987,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":341469.021102244,"t_first_token_unix":1779772334.0350456,"t_done":341469.607617463,"t_done_unix":1779772334.6215608},{"request_id":"1258908:4:1553508:958","session_id":"1258908","input_length":9740,"t_proxy_recv":341468.661039193,"t_decision_unix":1779772333.6749809,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.9462012320328542,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (524 < 8000)","cache_hit":9216,"estimated_new_tokens":524,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1634,"cache_hit":9216,"new_prefill":524,"score_linear":-9216.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2512,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":341468.721819462,"t_first_token_unix":1779772333.7357624,"t_done":341470.644618937,"t_done_unix":1779772335.6585624},{"request_id":"1258908:5:1617181:988","session_id":"1258908","input_length":10201,"t_proxy_recv":341470.64798711,"t_decision_unix":1779772335.6619287,"policy":"unified_v2","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9728,"affinity_cache_ratio":0.9536319968630527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (473 < 8000)","cache_hit":9728,"estimated_new_tokens":473,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1635,"cache_hit":9728,"new_prefill":473,"score_linear":-9728.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2512,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":341470.701655616,"t_first_token_unix":1779772335.7155988,"t_done":341472.79514816,"t_done_unix":1779772337.8090916},{"request_id":"1340290:48:2066647:1200","session_id":"1340290","input_length":74557,"t_proxy_recv":341478.307108118,"t_decision_unix":1779772343.3210497,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9888809903831968,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (829 < 8000)","cache_hit":73728,"estimated_new_tokens":829,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2512,"cache_hit":73728,"new_prefill":829,"score_linear":-73728.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":73533,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":341478.703512338,"t_first_token_unix":1779772343.7174559,"t_done":341479.947345496,"t_done_unix":1779772344.961289},{"request_id":"1340290:48:2071598:1201","session_id":"1340290","input_length":74863,"t_proxy_recv":341494.968058049,"t_decision_unix":1779772359.9819994,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":74240,"affinity_cache_ratio":0.991678132054553,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (623 < 8000)","cache_hit":74240,"estimated_new_tokens":623,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2513,"cache_hit":74240,"new_prefill":623,"score_linear":-74240.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":73839,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74240.0,"chosen_score_lmetric":0,"t_first_token":341495.214032287,"t_first_token_unix":1779772360.2279756,"t_done":341496.03509357,"t_done_unix":1779772361.049037},{"request_id":"1340290:48:2076822:1202","session_id":"1340290","input_length":75113,"t_proxy_recv":341512.483870359,"t_decision_unix":1779772377.4978118,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":74752,"affinity_cache_ratio":0.9951939078455128,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (361 < 8000)","cache_hit":74752,"estimated_new_tokens":361,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2514,"cache_hit":74752,"new_prefill":361,"score_linear":-74752.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":74089,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74752.0,"chosen_score_lmetric":0,"t_first_token":341512.724663416,"t_first_token_unix":1779772377.738607,"t_done":341513.170747302,"t_done_unix":1779772378.184691},{"request_id":"1340290:48:2078461:1203","session_id":"1340290","input_length":75299,"t_proxy_recv":341517.770417706,"t_decision_unix":1779772382.7843592,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":74752,"affinity_cache_ratio":0.9927356272991673,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (547 < 8000)","cache_hit":74752,"estimated_new_tokens":547,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2514,"cache_hit":74752,"new_prefill":547,"score_linear":-74752.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":74275,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74752.0,"chosen_score_lmetric":0,"t_first_token":341518.015019722,"t_first_token_unix":1779772383.0289633,"t_done":341518.306428169,"t_done_unix":1779772383.3203723},{"request_id":"1340290:48:2080016:1204","session_id":"1340290","input_length":75338,"t_proxy_recv":341523.065212814,"t_decision_unix":1779772388.0791543,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9990177599617723,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (74 < 8000)","cache_hit":75264,"estimated_new_tokens":74,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2515,"cache_hit":75264,"new_prefill":74,"score_linear":-75264.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":74314,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":341523.168816219,"t_first_token_unix":1779772388.1827598,"t_done":341523.552154903,"t_done_unix":1779772388.566099},{"request_id":"1340290:48:2082644:1206","session_id":"1340290","input_length":75439,"t_proxy_recv":341531.740114292,"t_decision_unix":1779772396.7540555,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9976802449661316,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (175 < 8000)","cache_hit":75264,"estimated_new_tokens":175,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2515,"cache_hit":75264,"new_prefill":175,"score_linear":-75264.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":74415,"score_linear":113947.0,"score_lmetric":135626},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":341531.896977099,"t_first_token_unix":1779772396.9109201,"t_done":341532.895872703,"t_done_unix":1779772397.909816},{"request_id":"1356885:16:2084239:1207","session_id":"1356885","input_length":136991,"t_proxy_recv":341536.83147973,"t_decision_unix":1779772401.845421,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":136704,"affinity_cache_ratio":0.9979049718594653,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (287 < 8000)","cache_hit":136704,"estimated_new_tokens":287,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":14623,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":136704,"new_prefill":287,"score_linear":-136704.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2515,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":0,"new_prefill":136991,"score_linear":114971.0,"score_lmetric":198202},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":130335,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-136704.0,"chosen_score_lmetric":0,"t_first_token":341537.266286669,"t_first_token_unix":1779772402.2802312,"t_done":341540.122430147,"t_done_unix":1779772405.1363745},{"request_id":"1340290:48:2085626:1208","session_id":"1340290","input_length":77337,"t_proxy_recv":341541.817898257,"t_decision_unix":1779772406.8318396,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9731952364327553,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (2073 < 8000)","cache_hit":75264,"estimated_new_tokens":2073,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3172,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2515,"cache_hit":75264,"new_prefill":2073,"score_linear":-75264.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":76313,"score_linear":113947.0,"score_lmetric":137524},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":341542.793366585,"t_first_token_unix":1779772407.8073108,"t_done":341543.308592987,"t_done_unix":1779772408.3225372},{"request_id":"1340290:48:2087584:1209","session_id":"1340290","input_length":78683,"t_proxy_recv":341548.395049136,"t_decision_unix":1779772413.40899,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":77312,"affinity_cache_ratio":0.9825756516655441,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (1371 < 8000)","cache_hit":77312,"estimated_new_tokens":1371,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3172,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2519,"cache_hit":77312,"new_prefill":1371,"score_linear":-77312.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":77659,"score_linear":113947.0,"score_lmetric":138870},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-77312.0,"chosen_score_lmetric":0,"t_first_token":341549.233764968,"t_first_token_unix":1779772414.2477093,"t_done":341550.709756879,"t_done_unix":1779772415.7237005},{"request_id":"1340290:48:2092759:1211","session_id":"1340290","input_length":79023,"t_proxy_recv":341565.675714811,"t_decision_unix":1779772430.689656,"policy":"unified_v2","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9913063285372613,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (687 < 8000)","cache_hit":78336,"estimated_new_tokens":687,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3172,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2521,"cache_hit":78336,"new_prefill":687,"score_linear":-78336.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":1024,"new_prefill":77999,"score_linear":113947.0,"score_lmetric":139210},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":341565.928231432,"t_first_token_unix":1779772430.9421744,"t_done":341567.224309128,"t_done_unix":1779772432.2382526},{"request_id":"1382086:2:2082052:1205","session_id":"1382086","input_length":114971,"t_proxy_recv":341529.724955784,"t_decision_unix":1779772394.738897,"policy":"unified_v2","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":53760,"affinity_cache_ratio":0.467596176427099,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"chosen_no_active_decode (num_req=0 decode_tok=0)","cache_hit":53760,"estimated_new_tokens":61211,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3171,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2515,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1960,"cache_hit":53760,"new_prefill":61211,"score_linear":-53760.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":341572.574526866,"t_first_token_unix":1779772437.5884705,"t_done":341573.746519446,"t_done_unix":1779772438.760463},{"request_id":"1382086:3:2089879:1210","session_id":"1382086","input_length":115184,"t_proxy_recv":341573.763289722,"t_decision_unix":1779772438.7772315,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":114688,"affinity_cache_ratio":0.9956938463675511,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (496 < 8000)","cache_hit":114688,"estimated_new_tokens":496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3172,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2522,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2079,"cache_hit":114688,"new_prefill":496,"score_linear":-114688.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-114688.0,"chosen_score_lmetric":0,"t_first_token":341574.068747859,"t_first_token_unix":1779772439.0826917,"t_done":341574.577343752,"t_done_unix":1779772439.5912874},{"request_id":"1382086:4:2092953:1212","session_id":"1382086","input_length":115246,"t_proxy_recv":341574.595140982,"t_decision_unix":1779772439.609083,"policy":"unified_v2","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":115200,"affinity_cache_ratio":0.9996008538257293,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (46 < 8000)","cache_hit":115200,"estimated_new_tokens":46,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3172,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2522,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2080,"cache_hit":115200,"new_prefill":46,"score_linear":-115200.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-115200.0,"chosen_score_lmetric":0,"t_first_token":341574.756879299,"t_first_token_unix":1779772439.7708228,"t_done":341581.8672739,"t_done_unix":1779772446.881217},{"request_id":"1356885:17:2097011:1213","session_id":"1356885","input_length":137546,"t_proxy_recv":341579.549333042,"t_decision_unix":1779772444.5632741,"policy":"unified_v2","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":137216,"affinity_cache_ratio":0.9976008026405712,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"v2_pd_sep":false,"v2_decision":"local","v2_reason":"new_local_below_threshold (330 < 8000)","cache_hit":137216,"estimated_new_tokens":330,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":122368,"new_prefill":15178,"score_linear":-122368.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1951,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":3172,"cache_hit":137216,"new_prefill":330,"score_linear":-137216.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1636,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2522,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":115246,"ongoing_decode_tokens":115246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2080,"cache_hit":0,"new_prefill":137546,"score_linear":115246.0,"score_lmetric":137546},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":6656,"new_prefill":130890,"score_linear":-6656.0,"score_lmetric":0}],"chosen_score_linear":-137216.0,"chosen_score_lmetric":0,"t_first_token":341580.214685727,"t_first_token_unix":1779772445.2286289,"t_done":341582.916572178,"t_done_unix":1779772447.930516}]
Reference in New Issue View Git Blame Copy Permalink